(no title)
gkapur | 2 years ago
- What underlying data flow technology is this based on? Is it Timely dataflow a la Materialize, something else a la RisingWave, etc.? - What can the materialized views not have? Ie. Does it support window functions? Unions? User-defined functions? - Finally I’m curious about how you think of the landscape versus RisingWave, Readyset, Materialize, etc.!
ntur1337|2 years ago
- Actually, Epsio has its own dataflow engine. It's based on much of the same theory as these other technologies, we just found that their implementation doesn't quite fit our use case neatly.
Just as an example, Timely/Differential dataflow requires saving state in memory. While this results in extremely low latency (beneficial for its original purpose), the trade-off is the high memory footprint required to save state for complex SQL queries (JOINS, DISTINCT ON, etc.). We found this to be quite costly for the use case of replacing heavy SQL queries in PG/MySQL.
We actually recently published a blog post on how we see the streaming landscape -https://www.epsio.io/blog/epsios-diff-in-the-streaming-lands... , welcome to take a look for our full thoughts :)
- Regarding SQL support, we support today DISTINCT [ON], UNIONs, JOINs, CTEs, ORDER BYs, GROUP BYs, and some additional operators/functions (full list is in this link: https://docs.epsio.io/sql-support/queries/)
Window functions are supposed to be released soon :)