[miguno]

> Bookies can seamlessly be added.

Kafka brokers can seamlessly be added, too.

> I remember adding brokers to Kafka and taking advantage of them on existing topics meant repartitioning which if I recall correctly breaks the golden ordering contract that most devs bank on.

Adding brokers to Kafka does not require repartitioning. It requires data rebalancing ('migrate' some data to the new brokers), which does not break any ordering contract. I suppose the words sound sufficiently similar that they are easy to be mixed up. :)

(For what it's worth, BookKeeper requires the same data rebalancing process.)

> The data written to the partition will always be in the order for that partition itself.

Yes, for Kafka, all log segments that make up a topic-partition are always stored -- or, when rebalancing, moved -- in unison on the same broker. Or, brokers (plural) when we factor in replication. Kafka's approach has downsides but also upsides: data is always stored in a contiguous manner, and can thus also be read by consumers in a contiguous manner, which is very fast.

In comparison, BookKeeper has segmented storage, too. But here the segments -- called ledgers -- of the same topic-partition are spread across different BK bookies. Also, because of how BookKeeper's protocol works (https://bookkeeper.apache.org/docs/4.10.0/development/protoc...), what bookies store are actually not contiguous 'ledgers', but in fact 'fragments of ledgers' (see link). As mentioned elsewhere in this discussion, one downside of this approach is that BK suffers proverbially from data fragmentation. (Remember Windows 95 disk fragmentation? Quite similar.)

No approach is universally better than the other one. As often, design decisions were made to achieve different trade-offs.

[ckdarby]

Maybe we're both mixed up.

Let's say we have a cluster with a sort of "global" topic called "incoming-events" and it has 10 partitions.

We'll most likely eventually end up with a "hot" write partition because rarely do I see a perfect even distribution in event streams.

I'd like to seamlessly add capacity to remove this hot spot.

With Pulsar you're using BK which means just spinning up more bookies which will take on new segments and then the rebalancement will move some segments off.

With Kafka I don't know what the option is aside from spinning up a larger and larger broker & rebalancing to the larger box. What I typically see in companies are they repartition from 10 to 20 to avoid expensive one off boxes.

Nobody likes non-uniform resources because it is a nightmare to manage. Imagine a k8s deployment where you have replica:10 but have to handle a custom edge case for resource allocation on one pod different than the other 9 brokers.

(Just assumed 10 partitions to 10 pods)

[miguno]

Both Kafka and Pulsar have this kind of bottleneck in your scenario -- say, one "hot" write partition.

If one Kafka broker or BookKeeper bookie node cannot keep up with the write load (e.g. network or disk too slow, CPU util too high), you must add more partitions. For Kafka, for the reasons you already mentioned. For BookKeeper (and Pulsar), because only a single ledger of a topic-partition is open for writes at any given time.

[matteomerli]

> Kafka brokers can seamlessly be added, too.

You can add as man as you want, but they take no traffic.