Dragonfly – Performant in-memory data store

[chocolatkey]

I apologize for the somewhat off-topic nature off this comment, but it really irritates me when I see the website for a product with competition, go to their sitemap, and see this blatant SEO gaming. For example, I went to their FAQ (https://www.dragonflydb.io/faq) expecting things about their database, instead I get a bunch of copycat answers to weird questions intended to drive traffic to their site when people ask questions about other databases. I see many application sites doing this nowadays, and it makes me not want to use them. I imagine with GPT this is very easy to generate now

[Veen]

> I imagine with GPT this is very easy to generate now

That's exactly what they are doing, but if they had any brains, they'd have a human read the answers before they publish:

https://www.dragonflydb.io/faq/is-memcached-good

> As an AI language model, I am unable to make subjective judgments on whether Memcached is "good" or not. However, I can provide information about the features, benefits, and limitations of Memcached.

[OPoncz]

All, My name is Oded Poncz, I am the CEO of DragonflyDB. Our goal with the FAQ was to aggragate valuable information for our community. We will take all the valuable feedback from this thread and improve. Thank you all for the feedback.

[prakis]

Its a bit surprising, the faq is all about Redis and other tools but not about Dragonfly.

[spacechild1]

Strange, your link to the FAQ gives me a 404, but existing links to specific FAQ sections, posted in the comments below, still work. On the website I can only find this: https://www.dragonflydb.io/docs/about/faq. Maybe they took down the original FAQ?

[romange]

You are absolutely right, and I apologize for having this experience. SEO is part of the game, but it should not be in FAQ section.

[pearjuice]

I don't know how well the results hold up after a year, but according to Redis the Dragonfly performance test is biased and with Redis configured properly it reached higher throughput than Dragonfly. YMMV but just putting this up here. Personally I never used Dragonfly so I wonder if the "marketing metrics" actually hold up in production.

https://redis.com/blog/redis-architecture-13-years-later/

[gadelat]

Redis Cluster has lot of limitations though. It's unusable for multi-key operations, no scan, no transactions, single database only, client has to support it, the way it works makes it unusable when connecting to it outside the network (see https://redis.io/docs/management/scaling/) etc. At that point RedisCluster is not Redis anymore and it's disingenuous to call it that. I would rather have slightly lower performance and not have to deal with those limitations AND not have tot deal with orchestration.

[youngtaff]

Yeh but isn’t the Redis one just a biased?

What might have been interesting would be to test on a range of cores / clusters, and consider the overhead of managing 1VM vs 64VMs etc.

[pella]

The v1.3.0 version was released 3 days ago.

https://github.com/dragonflydb/dragonfly/releases/tag/v1.3.0

If you want to integrate it, you also need to check the Software License.

https://github.com/dragonflydb/dragonfly/blob/main/LICENSE.m...

Dragonfly Business Source License 1.1

License: BSL 1.1

Licensor: DragonflyDB, Ltd.

Licensed Work: Dragonfly including the software components, or any portion of them, and any modification.

Change Date: March 15, 2028

Change License: Apache License, Version 2.0, as published by the Apache Foundation.

Additional Use Grant: You may make use of the Licensed Work (i) only as part of your own product or service, provided it is not an in-memory data store product or service; and (ii) provided that you do not use, provide, distribute, or make available the Licensed Work as a Service. A “Service” is a commercial offering, product, hosted, or managed service, that allows third parties (other than your own employees and contractors acting on your behalf) to access and/or use the Licensed Work or a substantial set of the features or functionality of the Licensed Work to third parties as a software-as-a-service, platform-as-a-service, infrastructure-as-a-service or other similar services that compete with Licensor products or services.

...

[tebbers]

Sounds fair enough to me. Look what AWS did to Elasticsearch (amongst many others).

[nly]

Does it really stop anything?

Amazon etc can just "take inspiration" from it and develop something from scratch that's almost the same

[dvh]

And people are still mocking Stallman.

[warkanlock]

Weirdly, in-memory data storage is barely the bottleneck most of the time.

Why do we need faster storage? We need better models & domains that improve our developer experience. Not something faster. We don't need cars that go 650km/h. We need better roads and superior security controls

Blazingly fast wasn't enough; now we add "on Earth" to be impactful enough to capture our attention

[quags]

I use redis with rspamd and have systems that scan millions of mails per day and redis is a bottle neck for some of the modules. I’ve been impressed with dragonfly for the reputation set which has stopped the timeouts I was seeing and was able to replace a multi redis set up using rspamd internal hash/sharding to a single dragonflydb. I plan to replace more over but need stream support which is coming. The dev is also responsive on their discord channel.

[jayd16]

> Weirdly, in-memory data storage is barely the bottleneck most of the time.

Yeah but sometimes it is and its a way better dev experience to scale up than out.

[electic]

There was a great HN post of DragonFly vs. some other databases: https://news.ycombinator.com/item?id=31796311.

I am not sure if the latest version improves performance but those are some bold claims of Dragonfly.

[pella]

other related thread in the HN (today)

"KeyDB – A Multithreaded Fork of Redis (keydb.dev)"

https://news.ycombinator.com/item?id=36018149

[FrostKiwi]

I instantly confused this with DragonFlyBSD.

[isubasinghe]

That is a huge claim, almost screams red flag.

[dangoodmanUT]

I never trust metrics from a company anymore, see the bloodbath comment thread on the recent Kafka vs Redpanda for proof.

[avinassh]

Their blog post goes into detail on how they are benchmarking and they use the benchmarking tool developed by Redis [1]

> We ran our tests on AWS Graviton2 EC2 instances, which are network-optimized and provide the best performance for web applications. We used memtier_benchmark — a widely used benchmarking tool developed by Redis Ltd. — to test throughput and latency, and Prometheus to monitor and visualize memory usage.

It is easy to reproduce and check the claims. Redis also countered with a blog post [2]

[1] - https://www.dragonflydb.io/blog/scaling-performance-redis-vs...

[2] - https://redis.com/blog/redis-architecture-13-years-later/

[monlockandkey]

How does performance look like on a vCPU machine Vs Redis? Cause this looks interesting.

I'm most interested in latency to store the coordinates of thousands of players in real time. Well definitely check this out

[rickstanley]

Sorry, but I find "Redis API compatible" checked for Redis itself funny. :)

[jgalt212]

Once one moves to dragonfly cloud, I guess the "performant" part of the value prop falls by the wayside. i.e. you need your in-memory data store on the same machine (or on the same rack) otherwise really un-peformant solutions can out-perform dragonfly cloud.

[sgt]

> high-performance, low-complexity, and built for scale.

So, exactly like redis?