The first prototype implementation of dqlite was in Go, leveraging the hashicorp/raft implementation of the Raft algorithm. The project was later rewritten entirely in C because of performance problems due to the way Go interoperates with C: Go considers a function call into C that lasts more than ~20 microseconds as a blocking system call, in that case it will put the goroutine running that C call in waiting queue and resuming it will effectively cause a context switch, degrading performance (since there were a lot of them happening). See also this issue in the Go bug tracker.
The added benefit of the rewrite in C is that it's now easy to embed dqlite into project written in effectively any language, since all major languages have provisions to create C bindings.
They could also use D or Rust for this. If borrow-checker is too much, Rust can still do automatic, memory management with other benefits remaining. Both also support letting specific modules be unsafe where performance is critical.
I don't want to flame, but I did find it curious they went with C rather than Rust. In my experience the transition is straightforward and the string handling (particularly with unicode encodings) is way better (in addition to the normal ownership benefits), and the result (an easily linkable library exposing a C ABI) is roughly the same.
There's one more big distinction, rqlite's replication is command based [0] where as dqlite is/was WAL frame-based -- so basically one ships the command and the other ships WAL frames. This distinction means that non-deterministic commands (ex. `RANDOM()`) will work differently.
It looks like dqlite's documentation has changed -- for some reason frames are no longer mentioned anywhere[2]. So maybe this isn't the case any more, but this was once the biggest differentiator for me.
This is developed by LXD team for it's cluster. It's used by us in production as a part of LXD cluster. Initially there were some issues but now it can support thousands of nodes in a cluster easily in our regression tests.
It's good they made it as a separate project can be used independent of LXD containers.
In includes an answer about the difference with rqlite.
To me reading the docs it seems like dqlite has been developed by the team who develops LXD at Canonical as LXD is listed as the biggest user of the project and it says on the authors github that he works at Canonical at/with LXD/LXC.
Interesting project, good luck to the author/authors if you read this!
The one annoying thing about SQLite is that there is no easy way to change the table structure. Adding/Removing/Renaming columns is super complicated and afaik there is no good command line tool that does it for you.
That is the primary reason why I do not consider it for new projects. It's just to slow to iterate on.
That's a hell of a reason not to use sqlite. Staging data in a temporary table while a table is dropped, recreated, and then data is reinserted is not much of an inconvenience.
I really like the design of this website. It's simple, information-rich, fast, and doesn't contain a ridiculous number of images or dynamic components. It's a shame that I can only say this for a select few websites these days.
Yet it's still loaded with hundreds of KB of custom fonts, because the designers would rather I look at a blank page for a couple seconds than gaze upon their design with a typeface that isn't exactly the same one they have on their computer.
My resolution, window size, color settings, text zoom, font rendering, etc, are almost certainly different, too, but at least they've made the page more than twice as slow by forcing the correct font.
What can this be used for (example use cases)? Is there 24/7 support available? How long has it been around and is there a commitment to long term releases?
I used Dqlite for a side project[1], which replicates some of the features of LXD. Was relatively easy to use, but Dqlite moves at some pace and trying to keep up is quite "interesting". Anyway once I do end up getting time, I'm sure it'll be advantageous to what I'm doing.
Oh I had no idea somebody was using it in the wild! It has been unstable until now, we just released v1.0.0 yesterday. So no more public API breakage from now on.
Hey, free-ekanayaka, a few more questions for your FAQ if you're still paying attention:
Does this store the entire log for all time? When you bring up a new node, does it replay the entire history? If not, how do you bring up a new node without data?
How does backup/restore work?
How do upgrades work? Is the shared WAL low-level enough that it's 100% stable/compatible between sqlite/dqlite versions? If not, what happens if half your cluster is on the old version while you're upgrading, and sees things it doesn't understand yet?
Is it possible to encrypt node/node traffic? Or can you easily send the node-node traffic over a proxy, like Envoy? How about over a unix domain socket or "@named" unix domain socket (which we use for Envoy here at Square)
Linux does support async I/O to disks using various other interfaces/approaches - it’s just that the classic approach of select()’ing on non-blocking FDs doesn’t work for disk:
The support of async disk I/O in Linux differs depending on kernel version and file system type. But it is possible to get 100% async I/O with the is_submit(), and dqlite will leverage that if detected.
There is now a new async I/O API available in Linux (I'm not remembering the name right now, but it was developed by folks at Facebook). It looks promising so I'll check it at some point. (dqlite author here)
It's not a buzz term. It's really fully async disk I/O. Dqlite does not use SQLite's stock vfs implementation for writing to disk, as it's an entirely different model (based on raft).
[+] [-] bb88|6 years ago|reply
https://github.com/canonical/dqlite/blob/master/doc/faq.md
Why C?
The first prototype implementation of dqlite was in Go, leveraging the hashicorp/raft implementation of the Raft algorithm. The project was later rewritten entirely in C because of performance problems due to the way Go interoperates with C: Go considers a function call into C that lasts more than ~20 microseconds as a blocking system call, in that case it will put the goroutine running that C call in waiting queue and resuming it will effectively cause a context switch, degrading performance (since there were a lot of them happening). See also this issue in the Go bug tracker.
The added benefit of the rewrite in C is that it's now easy to embed dqlite into project written in effectively any language, since all major languages have provisions to create C bindings.
[+] [-] nickpsecurity|6 years ago|reply
[+] [-] MuffinFlavored|6 years ago|reply
Does WebAssembly or any of its runtimes provide a way to do this?
[+] [-] SomeOldThrow|6 years ago|reply
[+] [-] emmelaich|6 years ago|reply
Found the answer on Reddit:
> rqlite is a full RDBMS application, but dqlite is a library you must link with other code. It's like the difference between MySQL and libsqlite3.so.
* https://www.reddit.com/r/golang/comments/8a8h8y/dqlite_distr...
[+] [-] hardwaresofton|6 years ago|reply
It looks like dqlite's documentation has changed -- for some reason frames are no longer mentioned anywhere[2]. So maybe this isn't the case any more, but this was once the biggest differentiator for me.
[0]: https://github.com/rqlite/rqlite/#limitations
[1]: https://webcache.googleusercontent.com/search?q=cache:p1XBgh...
[2]: https://github.com/canonical/dqlite/commit/35ea7cd56e93a36c5...
[+] [-] otoolep|6 years ago|reply
Yes, exactly, Dqlite is a library, rqlite is a full application.
[+] [-] dragonsh|6 years ago|reply
It's good they made it as a separate project can be used independent of LXD containers.
[+] [-] unknown|6 years ago|reply
[deleted]
[+] [-] Scorpiion|6 years ago|reply
In includes an answer about the difference with rqlite.
To me reading the docs it seems like dqlite has been developed by the team who develops LXD at Canonical as LXD is listed as the biggest user of the project and it says on the authors github that he works at Canonical at/with LXD/LXC.
Interesting project, good luck to the author/authors if you read this!
[+] [-] otoolep|6 years ago|reply
Yes, good luck to the creators of this project, it looks very interesting and I've been watching it for a few years now.
[+] [-] FreeHugs|6 years ago|reply
That is the primary reason why I do not consider it for new projects. It's just to slow to iterate on.
[+] [-] Dowwie|6 years ago|reply
[+] [-] masklinn|6 years ago|reply
sqlite supports ADD COLUMN and RENAME COLUMN DDLs.
Dropping columns is not supported, nor is adding some of the more complex column, that does require going through full table rewriting.
[+] [-] mysterydip|6 years ago|reply
[+] [-] crusader1099|6 years ago|reply
[+] [-] timClicks|6 years ago|reply
[+] [-] ken|6 years ago|reply
My resolution, window size, color settings, text zoom, font rendering, etc, are almost certainly different, too, but at least they've made the page more than twice as slow by forcing the correct font.
[+] [-] JacKTrocinskI|6 years ago|reply
[+] [-] free-ekanayaka|6 years ago|reply
I added answers to your other questions to the FAQ:
https://github.com/canonical/dqlite/blob/master/doc/faq.md#w...
https://github.com/canonical/dqlite/blob/master/doc/faq.md#w...
https://github.com/canonical/dqlite/blob/master/doc/faq.md#i...
[+] [-] tootie|6 years ago|reply
[+] [-] nilsjuenemann|6 years ago|reply
[+] [-] free-ekanayaka|6 years ago|reply
[+] [-] nightfly|6 years ago|reply
[+] [-] stickupkid|6 years ago|reply
[+] [-] hamandcheese|6 years ago|reply
https://github.com/rqlite/rqlite
[+] [-] duiker101|6 years ago|reply
The main differences from rqlite are:
- Embeddable in any language that can interoperate with C
- Full support for transactions
- No need for statements to be deterministic (e.g. you can use time())
- Frame-based replication instead of statement-based replication
[1] https://github.com/canonical/dqlite/blob/master/doc/faq.md
[+] [-] stickupkid|6 years ago|reply
1. https://github.com/spoke-d/thermionic
[+] [-] free-ekanayaka|6 years ago|reply
[+] [-] bryanlarsen|6 years ago|reply
[+] [-] free-ekanayaka|6 years ago|reply
[+] [-] zellyn|6 years ago|reply
Does this store the entire log for all time? When you bring up a new node, does it replay the entire history? If not, how do you bring up a new node without data?
How does backup/restore work?
How do upgrades work? Is the shared WAL low-level enough that it's 100% stable/compatible between sqlite/dqlite versions? If not, what happens if half your cluster is on the old version while you're upgrading, and sees things it doesn't understand yet?
Is it possible to encrypt node/node traffic? Or can you easily send the node-node traffic over a proxy, like Envoy? How about over a unix domain socket or "@named" unix domain socket (which we use for Envoy here at Square)
Looks awesome, by the way!
[+] [-] continuations|6 years ago|reply
I thought Linux didn't support real async disk IO. Is that not the case?
If Linux has no real async disk IO, how does Dqlite achieve fully async disk IO?
[+] [-] rrdharan|6 years ago|reply
https://blog.cloudflare.com/io_submit-the-epoll-alternative-...
[+] [-] free-ekanayaka|6 years ago|reply
There is now a new async I/O API available in Linux (I'm not remembering the name right now, but it was developed by folks at Facebook). It looks promising so I'll check it at some point. (dqlite author here)
[+] [-] brennebeck|6 years ago|reply
http://man7.org/linux/man-pages/man2/io_submit.2.html
[+] [-] rockwotj|6 years ago|reply
https://www.sqlite.org/asyncvfs.html
[+] [-] free-ekanayaka|6 years ago|reply
[+] [-] tyingq|6 years ago|reply
[+] [-] unknown|6 years ago|reply
[deleted]
[+] [-] weitzj|6 years ago|reply
[+] [-] yeasayer|6 years ago|reply
[+] [-] free-ekanayaka|6 years ago|reply
[+] [-] bbmario|6 years ago|reply
That's interesting. Didn't know about it.
[+] [-] jchanimal|6 years ago|reply