top | item 8589052

(no title)

ewencp | 11 years ago

In particular, http://kafka.apache.org/documentation.html#intro_consumers addresses the concept of consumer groups and what ordering is guaranteed. One thing that might be worth noting for the grandparent is that Kafka consumers have an offset commit API that gives some control over how failures are handled. If a consumer dies before committing an offset but after reading data from the broker, a fresh consumer that joins the consumer group can see the same data once the system determines the original has died; that ensures all data will be processed, even in the event of consumer failures.

Kinesis provides the same ordering guarantees. They use different terminology (Kafka topics == Kinesis streams; Kafka partitions == Kinesis shards) but have the same system interface. The details of the APIs used for consumption differ, but they provide the same basic functionality of Kafka's "consumer groups".

discuss

order

No comments yet.