Kronotop: Redis-compatible, transactional document store backed by FoundationDB

[NathanFlurry]

What fascinates me most is not the protocol itself, but the potential to add a thin scripting layer (similar to Redis' EVAL or the newer modules API) to enable complex logic with atomic transactions and high QPS.

In the past, we relied heavily on using EVAL[SHA] with 200+ loc Lua scripts in order to implement high throughput, atomic transactions for realtime systems. We also used the JSON & Redis Query Language (previous named "full-text search") to build a more maintainable & strongly consistent system than using raw key-values and manually building secondary indexes.

We’ve since migrated to a native FoundationDB and SQLite hybrid setup, but this approach would have been really helpful for early-stage prototyping with a higher performance ceiling (thanks to FDB sharding) than a single-node Redis with AOF.

Related: Redis Cluster is a world of pain when handling clustering keys and cross-node queries and orchestration. DragonflyDB is chasing after the market of companies considering sharding Redis because of performance issues by providing better single-node performance. There's probably an alternative approach that could work by using an architecture like this.

[veggieroll]

This is exactly what [Tarantool](https://www.tarantool.io/en/doc/latest/platform/app/lua_tuto...) does. It's a shame I think that it never got much popular attention outside of Russia. I'm not sure why.

[danpalmer]

The awesome thing about Redis is that it's less a "cache" and more a set of fast network-shared, generic data structures. To have that then also be transactional and durable would be really handy sometimes.

[robertlagrant]

> more a set of fast network-shared, generic data structures

Exactly. It's a distributed list, map, etc that is often used as a cache, and sometimes as a queue, but it's bigger than all that.

[hobobaggins]

If only it wasn't Java.. but it looks very cool anyway.

[anonzzzies]

I was discussing this with a colleague; Java is supposed to be some of the most performant runtimes and yet, every time when someone introduces anything in Java, literally everyone tech in the company goes 'aw shit no Java please'. You can show me a running top processlist without the names of the processes and i'll point out the Java processes to you. And it seems that companies can make viable businesses around porting to Rust/Go/C++ of (Apache) Java projects because they are so resource intensive and slow (unless you spin up 10000 nodes). Why is anyone using it still outside legacy?

[tasn]

One thing I wish redis had is the ability to extend itself beyond its memory capacity. So if I run out of memory, just move some stuff to disk. Or if I'm using persisted redis, just free up some memory.

Maybe this is it.

[NathanFlurry]

Redis on Flash [1] is one of the key ways Redis gets people on their enterprise plan (esp before the SSPL relicense). I've spoken with their enterprise team before; it's ungodly expensive, even compared to AWS MemoryDB (wish I had the numbers on hand).

[1] https://redis.io/resources/building-large-databases-redis-en...

[adulau]

If you're looking for a Redis-compatible key-value store, kvrocks[1][2] is an excellent choice. We've used it in many projects, and it's proven to be very stable. Since it's based on RocksDB, it doesn't have the memory limitations of Redis and the license delirium ;-)

[1] https://kvrocks.apache.org/ [2] https://github.com/apache/kvrocks

[remram]

Other API-compatible reimplementations (others?):

https://github.com/dragonflydb/dragonfly DragonflyDB (not open source, BuSL-1.1) with more performance

https://github.com/apache/kvrocks Apache Kvrocks (Apache-2.0) uses disk-based NoSQL database to lower memory usage

[cluckindan]

The naming implies it’s a KDE desktop process monitoring tool with history graphs.

[dangoodmanUT]

Damnit this has been in my ideas list for a long time

[dangoodmanUT]

Oh it’s in Java we’re good

[ddorian43]

Much easier to do OLAP. OLTP is way harder.