While this question has already been answered, it might be a good idea for future readers to consider using the Kinesis Client Library (KCL) for Python
instead of using boto
directly. It simplifies consuming from the stream when you have multiple consumer instances, and/or changing shard configurations.
https://aws.amazon.com/blogs/aws/speak-to-kinesis-in-python/
A more complete enumeration of what the KCL provides
- Connects to the stream
- Enumerates the shards
- Coordinates shard associations with other workers (if any)
- Instantiates a record processor for every shard it manages
- Pulls data records from the stream
- Pushes the records to the corresponding record processor
- Checkpoints processed records (it uses DynamoDB so your code doesn't have to manually persist the checkpoint value)
- Balances shard-worker associations when the worker instance count changes
- Balances shard-worker associations when shards are split or merged
The items in bold are the ones that I think are where the KCL really provides non-trivial value over boto. But depending on your usecase boto may be much much much simpler.