[krsoninikhil]

I love sqlite backed system, one less component to worry about. But when using Tansu with sqlite storage, what are my options for horizonal scaling and keeping Tansu HA?

Also, are there any benchmark on how Tansu with S3 storage would perform in comparison to Kafka or something like WarpStream?

[shortishly]

You could use the proxy to spread topics over a number of brokers. The broker and proxy share a number of services and layers, that could be used to route:

https://blog.tansu.io/articles/route-layer-service

My itch for SQLite was smaller scale (and reproducible) environments, e.g., development, test/integration (with a single file to reset the environment). PostgreSQL was intended for "larger scale", with (database level) partitioning of Kafka records on each topic/partition, and replication for leader/follower setups, which might work better for HA. S3 for environments where latency is less of any issue (though with the SlateDB/S3 engine that might change).

S3: Not yet. I've been working through tuning each engine, S3 is next on the list.