RabbitMQ 4.0

[fidotron]

I've regarded RabbitMQ as a secret weapon in plain sight for years. The killer reason people don't use it more is it "doesn't scale" to truly enormous sizes, but for anyone with less than a million users it's great.

Too many people end up with their own half rolled pubsub via things like grpc, and they'd be far better off using this, particularly in the early stages of development.

[leetharris]

This is how I feel about NATS: https://nats.io/

It's an infinitely more friendly version of Kafka, pub/sub, etc that is extremely lightweight. Yet every environment trends towards Kafka because it was "chosen" by the big corps.

[ansd]

Modern RabbitMQ scales well. RabbitMQ Streams provide a throughput of millions of messages per second. With native support for MQTT, more than a million users can connect to RabbitMQ: https://www.rabbitmq.com/blog/2023/03/21/native-mqtt

The new Khepri metadata store in RabbitMQ 4.0 allows for even better scalability.

[brightball]

Is there a big advantage over Redis? Just coming from the Ruby world I know Sidekiq is extremely popular as a Redis backed queue. I know there's a RabbitMQ backed queue called Sneakers that's gained a lot of popularity as well though.

Just wondering what the biggest selling points are for somebody making a decision?

[8n4vidtmkvmk]

I think people don't use it more because people don't really know what it is. From their website:

> RabbitMQ is a reliable and mature messaging and streaming broker, which is easy to deploy on cloud environments, on-premises, and on your local machine.

What does that mean? "Messaging and streaming broker"? I understand the need for worker queues to process videos, images, emails and such but I can't easily tell if that's what this is.

Also, what are the benefits of this over just processing incomplete records straight out of my database? i.e. using MySQL as a queue.

[stackskipton]

Also want to give a shoutout to BeanStalkd: https://github.com/beanstalkd/beanstalkd

If you are looking at RabbitMQ with "Maybe this is too much". Beanstalkd likely has features you need with almost none of the setup. Just don't web expose it ;)

[datavirtue]

The real reason people don't use it is because they don't know about it or understand it. Then they apply the "it doesn't scale" retroactively.

You have to read a lot of docs or you WILL hold RabbitMQ wrong.

[smetj]

Agreed. I ran a log ingestion infra 8 years ago doing 20k msg/s sustained on RabbitMQ ... back then we went through a lot of instabilities though they settled over time with new releases. Great times. Besides a quality product the development/release process was very professional and mature.

The biggest issue back then was finding a quality client implementation for the language you were using. Not sure what the status of that is these days.

[latenightcoding]

It isn't more popular because it's not easy to use it properly.

I haven't touched it in years so I can't expand, but when I did, I had to write so many wrappers and add extra logic to use it properly.

[jimbokun]

For me the killer feature of Kafka is that topics are persistent until the data expires. Meaning different readers can be working at different offsets. And you can rewind or fast forward the offset you are reading, which can really be a life saver when things go sideways.

Does RabbitMQ have equivalent features?

[nesarkvechnep]

Erlang is a secret weapon.

[joshlemer]

Do you have any recommended resources to learn how to apply these tools (RabbitMQ, Nats, etc) to typical enterprise services? Common patterns/usecases/best practices and things?

[bankcust08385]

The anti-pattern to be avoided is cobbling together a nonperformant grand centralized ESB and making it a SPoF and bottleneck for everything, but it depends entirely on the use-case. MQTT scales to millions of devices for low data rates. ZK works well for little bits of coherent distributed cluster metadata. Kafka has its niches. ZMQ helps in others.

[saberience]

I've seen it used for a company with way, way in excess of a million users. We used it for a system with 100M+ users for our login systems and in general all of our main account systems relied on it. Most of the brokers were always running at 15k to 25k messages per second.

I loved it and the only issues we had were due to our fuckups.

[sevenf0ur]

How do you manage schema for exchanges, queues, and routing? The pattern seems to be for each client to set up the schema for the parts they touch before hand, but that doesn't scale well. The schema ends up siloed in each project and nobody knows what the current state should be.

[Joel_Mckay]

Probably using it wrong if complaining about AMQP queue scale limits...

Perhaps people are still thinking in single point of ingress design paradigms. Admittedly RabbitMQ can be a pain to administer for Juniors, but then again so are the other options. =3

[matrix2003]

If you like RabbitMQ, check out NATS!

I can’t speak to the new version, but it comes with support for even more messaging patterns out of the box.

[hk1337]

I know this is said a lot about things people don't like or think doesn't scale but I think I a lot of people don't set it up and use properly and it doesn't scale doing their half-baked implementation.

[heipei]

I could say the same thing about NSQ which is a distributed message queue with very simple semantics and a great HTTP API for message publishing and control actions. What it doesn't offer natively is HA though.

[dyeje]

If you’re not worried about scale, I’d just use a database backed queue.

[rodrigobellusci]

Do you know of any book or video to get started with Rabbit?

Earlier this year I tried setting it up as a websockets message broker in a Spring Boot app but failed miserably. I ended up using Spring’s simple broker.

[hztar]

whut.. transferred TB of data per hour with Rabbit.. does it not scale anymore?

[rhodin]

This release includes a new (native, no longer a plugin) AMQP 1.0 implementation, new quorum queue features, and a new schema data store (Khepri)

[sebazzz]

AMQP 1.0 is great - then you can, behind the right abstraction layer, use it as drop-in replacement for Azure Service Bus or similar.

[depr]

RabbitMQ is developed by VMware which was acquired by Broadcom. I hope they will remain unaffected.

[tony-allan]

https://github.com/rabbitmq/rabbitmq-server/releases/tag/v4....

[raminf]

RabbitMQ can also act as a native MQTT broker. For edge applications/devices, you can also use MQTT over websockets: https://www.rabbitmq.com/docs/web-mqtt

Edit: FWIW, NATS also supports MQTT: https://docs.nats.io/running-a-nats-service/configuration/mq...

[PHGamer]

interesting so there is no more free support for rabbitmq is what im seeing here for the most part.

[Jenk]

I lost a lot of respect for the RabbitMQ maintainers when they refused to honor the semantic versioning scheme in package managers like nuget/maven/etc. "Safely upgrading" was impossible. 3.5 => 3.6 saw the removal of an argument.

They didn't lose my respect for the removal of the argument, however, they lost my respect for whatabouting the conversation calling SemVer a "no true scotsman" fallacy, then trying to claim that removing a redundant argument is not a breaking change, and other reality-warping nonsense, before blocking myself and other complainants from their issues - and even deleting some of their own comments to mop up some of their own terrible reasoning.

I'm sure there is no love lost on their side, either. Personal rant over.

[sebazzz]

> I lost a lot of respect for the RabbitMQ maintainers when they refused to honor the semantic versioning scheme in package managers like nuget/maven/etc. "Safely upgrading" was impossible. 3.5 => 3.6 saw the removal of an argument.

Well, at least AMQP 1.0 is now supported so I expect that for most things you are able to use any client now.

[57546gwg]

Based. fuck semver.

[edweis]

For what reason should we move from SNS/SQS to RabbitMQ? Our SaaS processes ~20 events/second.

[declan_roberts]

That's like $0.50/day in aws costs. Absolutely no reason to switch if it's working.

[stackskipton]

You TRULY NEED to be multicloud? That would be only reason. Otherwise, assuming SNS/SQS is meeting your needs well, I wouldn't even consider it. You are paying ~20 USD per month with no maintenance costs. That's hell of a deal.

[pantsforbirds]

I probably wouldn't swap unless you have a specific complaint with SNS/SQS. I do like how easy it is to spin up Rabbit locally via docker-compose for testing, but I don't think that convenience is worth refactoring a significant portion of your code base.

[BurnGpuBurn]

Link without description or anything, nice. After reading the website for 30 seconds, my question would be: What is RabbitMQ?

[bdcravens]

You could say the same thing of any product update link, like the latest versions of Rails, Postgresql, Mongodb, etc. A general assumption is that there are many products that the community is already familiar with, so that an introduction is unnecessary.

[phishhook]

Would be keen to know if this passes Jepsen style tests now

[mrbluecoat]

Any pros/cons compared to EMQX, NanoMQ, or FlashMQ?

[383toast]

does celery work with this out of the box?

[jihadjihad]

It should--RabbitMQ is the default broker

https://docs.celeryq.dev/en/stable/getting-started/backends-...

[okr]

The linked github page gives me 404. But here are the release notes:

https://github.com/rabbitmq/rabbitmq-server/blob/main/releas...

[justinclift]

Looks like 4.0.0 was pulled and replaced with a 4.0.1 release.