[tejinderss]

At my current place, using Redis with celery is becoming bottleneck for number of concurrent connections it can hold. We are using 1000 workers and start seeing issues (ceiling is 10k connections in Redis); apparently Celery creates huge number of connections. We are considering moving to RabbitMQ for same reason.

[zo1]

Celery is an over engineered pile.. Rather move lower down the stack, with your first step being Kombu which powers celery under the hood. It's oddly "configurable" so if you need to optimize and adjust connections this is where you should go, and it's pretty interchangeable between Redis and AMQP. Really, almost a drop in replacement that should just be a config change.