[miguno]

PS: Not fully sure what could have caused your Kafka woes. Certainly all what you described is supported, and it also 'should' normally be easy to use as a user/developer.

For example, with Kafka Streams, any app you build with it just needs to set “processing.guarantee” to “exactly_once” in its configuration, and regardless of what happens to the app or its environment it will not lose messages (on write) or miss messages (on read) from Kafka.

Consider asking your question with a few more details in the Kafka user mailing list [1], or in the Confluent Community Slack [2] if you prefer chatting.

[1] https://kafka.apache.org/contact [2] https://launchpass.com/confluentcommunity