top | item 45761142

(no title)

guhcampos | 4 months ago

It is a *very bad* replacement for an MQ system, for the simple reason you can't quickly and effortlessly scale int/out consumers.

discuss

saberience|4 months ago

Why can't you? In my experience, scaling Kafka was far easier than scaling our RabbitMQ cluster. We started running into issues when our RabbitMQ cluster hit 25k TPS, our Kafka cluster of equivalent resources didn't break a sweat at 500k TPS.

otterley|4 months ago

Scaling consumers, not throughput. And it’s both directions (in and out, not just out).

guhcampos|4 months ago

How did you handle re-partitioning and rebalancing every time you scaled your cluster in or out?

jdswain|4 months ago

What sort of systems do you work on to require this kind of traffic volume? I've worked on one project that I'd consider relatively high volume (UK Post Office Horizon Online) and we were only targeting 500 TPS.