I'm sure there's a lot of really cool local-first databases out there, but SQLite has the benefit of being incredibly widely battle-tested, with literally billions of installations worldwide. It has received thorough security research and fuzzing (it's part of Chrome's attack surface after all). And there's tons of resources online to help people understand how to use it. Although I'm sure there are alternatives that serve certain use cases better it's hard to imagine anything coming close for ours.
That said, the storage engine we've built is not that heavily dependent on SQLite specifically. Any database that uses a write-ahead log like SQLite does should be possible to adapt to it in the future. So maybe we'll eventually open it up to a variety of choices, or even let you bring your own as a Wasm module.
Oh, I've been informed that DuckDB uses SQLite under the hood, so maybe compiling DuckDB to Wasm and running it on top of this will be possible, we'll see.
Correct. DuckDB is really interesting technology, but it's not a direct successor to SQLite for transactional workloads. It's also very new: there's a LOT of new code in DuckDB on top of the (heavily fuzzed) SQLite parts.
(I use it personally, but it's not the same thing as what we're building with D1)
Also, for OLAP workloads there's Workers Analytics Engine. Analytics Engine is arbitrarily scalable as opposed to a D1-like solution - I can almost guarantee I've inserted more data into its internal variant (which is built on extremely similar underlying infrastructure) than just about any customer would think of doing, and it handles it like a champ.
Basically, we already have a product for doing DuckDB-like things, and the D1 architecture isn't great for high volume OLAP workloads.
Disclaimer: I'm an engineer at Cloudflare, but not on a developer platform team. I'm not speaking for our developer platform strategy or anything, I'm just commenting on how it looks from where I sit.
kentonv|2 years ago
I'm sure there's a lot of really cool local-first databases out there, but SQLite has the benefit of being incredibly widely battle-tested, with literally billions of installations worldwide. It has received thorough security research and fuzzing (it's part of Chrome's attack surface after all). And there's tons of resources online to help people understand how to use it. Although I'm sure there are alternatives that serve certain use cases better it's hard to imagine anything coming close for ours.
That said, the storage engine we've built is not that heavily dependent on SQLite specifically. Any database that uses a write-ahead log like SQLite does should be possible to adapt to it in the future. So maybe we'll eventually open it up to a variety of choices, or even let you bring your own as a Wasm module.
kentonv|2 years ago
rektide|2 years ago
elithrar|2 years ago
(I use it personally, but it's not the same thing as what we're building with D1)
piperswe|2 years ago
Basically, we already have a product for doing DuckDB-like things, and the D1 architecture isn't great for high volume OLAP workloads.
Disclaimer: I'm an engineer at Cloudflare, but not on a developer platform team. I'm not speaking for our developer platform strategy or anything, I'm just commenting on how it looks from where I sit.