Simple Kinesis Example

This example demonstrates how to setup a Kinesis producer and consumer to send and receive messages through a Kinesis Data Stream.

Use Cases

  • Decouple message producers from message consumers.
  • This is one way to architect for scale and reliability.
  • Real-time processing of streaming data

Setup

  • sls deploy

Usage

  • To send a message to the producer, get the address from your sls deploy output.
Serverless: Stack update finished...
Service Information
service: aws-node-typescript-kinesis
stage: dev
region: us-east-1
stack: aws-node-typescript-kinesis-dev
resources: 16
api keys:
  None
endpoints:
  POST - https://xxx.execute-api.us-east-1.amazonaws.com/dev/producer
functions:
  producer: aws-node-typescript-kinesis-dev-producer
  consumer: aws-node-typescript-kinesis-dev-consumer
layers:
  None

  • To print out the logs of the Kinesis consumer handler on the terminal sls logs -f consumer -t

  • send a HTTP POST request to the producer lambda

curl -d "{ 'key': 'employee', 'value': 'Bill' }" \
https://xxx.execute-api.us-east-1.amazonaws.com/dev/producer

  • You should see confirmation that the message was sent. {"message":"Message placed in the Event Stream!"}

  • The logs from the consumer will be delayed several seconds.

INFO    Kinesis Message:
          partition key: eb2da704-4972-4bd7-8c25-cce1decce95d
          sequence number: 49608726715828497972227004620876254203171519877947064322
          kinesis schema version: 1.0
          data: { 'key': 'employee', 'value': 'Bill' }

Acknowledgements

Adapted from Miguel Frazao's SQS Standard example.