top | item 40724408

(no title)

burutthrow1234 | 1 year ago

I think "message queues" have become pretty commoditized. You can buy Confluent or RedPanda or MSK as a service and never have to administer Kafka yourself.

Change Data Capture (CDC) has also gotten really good and mainstream. It's relatively easy to write your data to a RDBMS and then capture the change data and propagate it to other systems. This pattern means people aren't writing about Kafka, for instance, because the message queue is just the backbone that the CDC system uses to relay messages.

These architectures definitely still exist and they mostly satisfy organizational constraints - if you have a write-once, read-many queue like Kafka you're exposing an API to other parts of the organization. A lot of companies use this pattern to shuffle data between different teams.

A small team owning a lot of microservices feels like resume-driven developnent. But in companies with 100+ engineers it makes sense.

discuss

order

morbicer|1 year ago

This. You don't need Google scale for Kafka to make sense. Just few acquisitions and a need to fan out some data to multiple products. For example you have SCIM hooks that will write to Kafka so all parts of org can consume the updates. Or customer provisioning.