top | item 39776382

(no title)

supz_k | 1 year ago

Slightly off-topic: Until last week, we used Redis for Laravel queues and cache in our blogging platform. We decided to get rid of Redis and use the database. The reason was that we are planning to allow self-hosting of our software so removing a dependency is a huge win to reduce complexity (didn't know about the license change then). There are a lot of arguments against using a relational DB for queues, but from our testing, it just worked! So, we just went with it in production. Surprisingly, there are no noticeable performance issues so far.

We initially used Redis because, well, Laravel recommends it. But, what I learned is that Redis is not a requirement until you absolutely need it.

discuss

order

crizzlenizzle|1 year ago

Almost three years ago, but the last time we used database as engine for Laravel’s queue subsystem it exploded due to some database table locks under high load. We switched to redis and things just worked well.

jrochkind1|1 year ago

if you have very high volume, there can be some tricks to using an rdbms as a queue system.

For Rails use, 37Signals recently released the open source solid queue package, meant for use with either msyql or postgres. It makes some odd choices, like separating things into different tables depending on state. And it also makes careful use of the not-necessarily-super-well-known (but standard, and in both mysql and postgres) `FOR UPDATE SKIP LOCKED` clause.

These choices were to get good performance at high volumes, which they think they have done and tested.

So it can be done! Probably. And indeed solid queue is MIT-licensed, there are not barriers to looking at the choices they made and copying them in whatever language/platform you want.

But it's not necessarily trivial. At non-huge volumes though it's probably not an issue.

https://github.com/rails/solid_queue