[fatal94]

Sure, if you're working on a small homelab with minimal to no processing volume.

The second you approach any kind of scale, this falls apart and/or you end up with a more expensive and worse version of Kafka.

[devnull3]

I think there is a wide spectrum between small-homelab and google scale.

I was surprised how far sqlite goes with some sharding on modern SSDs for those in-between scale services/saas

[fatal94]

What you're doing is fine for a homelab, or learning. But barring any very specific reason other than just not liking Kafka, its bad. The second that pattern needs to be fanned out to support even 50+ producers/consumers, the overhead and complexity needed to manage already-solved problems becomes a very bad design choice.

Kafka already solves this problem and gives me message durability, near infinite scale out, sharding, delivery guarantees, etc out of the box. I do not care to develop, reshard databases or production-alize this myself.

[raverbashing]

"Any kind of scale" No, there's a long way of better and more straightforward solutions than the simple SELECT

(SELECT * from EVENTS where TIMESTAMP > LAST_TS LIMIT 50) for example