[loire280]

In fact, a properly-configured Kafka cluster on minimal hardware will saturate its network link before it hits CPU or disk bottlenecks.

[theK]

Isn't that true for everything on the cloud? I thought we are long into the era where your disk comes over the network there.

[EdwardDiego]

Depends on how you configure the clients, ask me how I know that using a K8s pod id in a consumer group id is a really bad idea - or how setting batch size to 1 and linger to 0 is a really bad idea - the former blows up disk (all those unique consumer groups cause the backing topic to consume a lot of space, as the topic is by default only compacted) and the latter thrashes request handler CPU time.

[j45]

But it can do so many processes a second I’ll be able to scale to the moon before I ever launch.

[altcognito]

This doesn't even make sense. How do you know what the network links or the other bottlenecks are like? There are a grandiose number of assumptions being made here.

[loire280]

There is a finite and relatively narrow range of ratios of CPU, memory, and network throughput in both modern cloud offerings and bare hardware configurations.

Obviously it's possible to build, for example, a machine with 2 cores, a 10Gbps network link, and a single HDD that would falsify my statement.

[UltraSane]

A network link can be anything from 1Gbps to 800Gbps.