[Joeri]

Kafka isn’t a queue, it’s a distributed log. A partitioned topic can take very large volumes of message writes, persist them indefinitely, deliver them to any subscriber in-order and at-least-once (even for subscribers added after the message was published), and do all of that distributed and HA.

If you need all those things, there just are not a lot of options.

[diarrhea]

Perhaps the best terse summary of Kafka I have come across yet.

[holografix]

Finally a valuable answer thank you

[iamcreasy]

Why the delineation? As long as you have a queue - you can publishing whatever you want in it, right? Be it log or something else.

[HarHarVeryFunny]

Why do you say log rather than just publish and subscribe?

[exitb]

Clients don’t have to subscribe to latest messages, but rather can request any available offset range.

[halifaxbeard]

the log stays on kafka for replay until your per log retention settings delete it

[HarHarVeryFunny]

The way people choose to use feedback on HN never fails to suprise me - we've got a generally intelligent user base here, but the most common type of feedback voting isn't because something is wrong but rather a childish "I don't like it - I want to suppress this comment".

In this case it's something different - this was an honest question, and received two useful replies, so why downvote?! The mental model of people using Kafka is useful to know - in this case the published data being more log-like than stream-like since it's retained per a TTL policy, with each "subscriber" having their own controllable read index.

[edem]

What do you think about Temporal?

[cyberpunk]

Okay for small numbers of high value jobs (e.g uber trips or food deliveries etc), prohibitively expensive for anything you need even a few k/sec of.

[mannycalavera42]

HERO