When is the right time to "level up" from "I'm good with just plain old Postgres" to QuestDB, InfluxDB, Patroni, etc.?
> Unfortunately, automatic failover is solved neither by PostgreSQL nor TimescaleDB, but there are 3rd-party solutions like Patroni that add support for that functionality. PostgreSQL describes the process of failover as STONITH (Shoot The Other Node In The Head), meaning that the primary node has to be shot down once it starts to misbehave.
Does QuestDB do "Raft consensus"? I don't see Raft mentioned in the article.
Aren't all distributed databases basically really clever wrappers around write-ahead log + really tight timestamp/clock syncing?
> Aren't all distributed databases basically really clever wrappers around write-ahead log + really tight timestamp/clock syncing?
As far as I know, the second requirement is often solved differently. Google’s Spanner has tight clock synchronisation via GPS and/or atomic clocks, and will even report uncertainties. Knowing these uncertainties allows it to simply wait them out before committing, for example.
But in general, exact time keeping and clock syncing is often too hard and costly. Luckily, it’s often not required and one can do with logical clocks, such as version vectors or Lamport time stamps. These order events by causality (A before B, B before A, A and B happened concurrently), which eventually allows the WAL to be sorted deterministically.
Things like multi leader with async replication will inevitably run into conflicts though. These will need some sort of resolution (manually or automatically via CRDTs). There’s no way around it due to the builtin, inherent possibility of concurrent writes.
Note that concurrent in these scenarios has essentially nothing to do with time. It’s not about “happened at the same time”. It’s a question of “did A know about B?”. No? Then A can’t be causally dependent on B and they are concurrent events. Exactly like two “parallel” branches in git. They’ll need to be merged later on, and conflicts will need to be resolved.
Lastly, if we can deterministically order events, every node can reach the same conclusions. This is equivalent to consensus.
So my take would be: distributed databases are often about a log of (write) events, and some consensus mechanism to agree upon the exact order in that log. Logical clocks are a good solution for that, but physical clocks ca be made to work as well (Google Spanner).
This is all taken from the book “Designing Data Intensive Applications”, a great read!
You don’t choose a database to “level up”. It’s a tool.
Use the right tool for the right job.
I’ve migrated rdbms to wide column databases like Cassandra or dynamo because we had specific requirements that rdbms were not fulfilling.
I’ve also migrated from document database to rdbms because the document store didn’t meet our specific requirements.
I wouldn’t just use any random database because I want to appear cool (?) because I know Cassandra or how to use a vector database. That’s not the point.
There's a lot of use cases for which Postgres works very well at scale, and the main benefit of a solution like these specialized ones is more of a convenience layer.
> failover as STONITH (Shoot The Other Node In The Head)
What functional consensus protocol doesn't mandate attempted murder? When a node becomes incoherent it can't be relied upon to notice that it has done so and bow out gracefully. Like cancer, there is always a change that 'cell death' will fail and leave you in a pathological state.
TiDB and CRDB handles all these scenarios. They are designed for synchronized distributed replication from the ground up and a tremendous amount of engineering work has gone into these systems.
Is there any book/textbook course out there that goes through how to write a database or dbms from scratch up to something useful, think something like nand to tetris style? I have been looking but there is not much on this topic out there I feel like.
> we chose our goal to be achieving multi-master replication with Async consistency. We believe that this approach strikes the best balance of fault tolerance and transaction throughput.
What was the most interesting thing that you learned while implementing the WAL? Have you thought about how WAL is going to work in the multi-master setup?
We write to WAL and then register the transaction in the transaction sequence registry.
If a concurrent transaction registered between the start and the end of the transaction, we update the current uncommitted transaction data with concurrent transactions and re-try registering it in the sequencer again.
To scale to multi-master we will move the transaction sequence registry to a service with a consensus algorithm.
MuffinFlavored|2 years ago
> Unfortunately, automatic failover is solved neither by PostgreSQL nor TimescaleDB, but there are 3rd-party solutions like Patroni that add support for that functionality. PostgreSQL describes the process of failover as STONITH (Shoot The Other Node In The Head), meaning that the primary node has to be shot down once it starts to misbehave.
Does QuestDB do "Raft consensus"? I don't see Raft mentioned in the article.
Aren't all distributed databases basically really clever wrappers around write-ahead log + really tight timestamp/clock syncing?
diarrhea|2 years ago
As far as I know, the second requirement is often solved differently. Google’s Spanner has tight clock synchronisation via GPS and/or atomic clocks, and will even report uncertainties. Knowing these uncertainties allows it to simply wait them out before committing, for example.
But in general, exact time keeping and clock syncing is often too hard and costly. Luckily, it’s often not required and one can do with logical clocks, such as version vectors or Lamport time stamps. These order events by causality (A before B, B before A, A and B happened concurrently), which eventually allows the WAL to be sorted deterministically.
Things like multi leader with async replication will inevitably run into conflicts though. These will need some sort of resolution (manually or automatically via CRDTs). There’s no way around it due to the builtin, inherent possibility of concurrent writes.
Note that concurrent in these scenarios has essentially nothing to do with time. It’s not about “happened at the same time”. It’s a question of “did A know about B?”. No? Then A can’t be causally dependent on B and they are concurrent events. Exactly like two “parallel” branches in git. They’ll need to be merged later on, and conflicts will need to be resolved.
Lastly, if we can deterministically order events, every node can reach the same conclusions. This is equivalent to consensus.
So my take would be: distributed databases are often about a log of (write) events, and some consensus mechanism to agree upon the exact order in that log. Logical clocks are a good solution for that, but physical clocks ca be made to work as well (Google Spanner).
This is all taken from the book “Designing Data Intensive Applications”, a great read!
moomoo11|2 years ago
Use the right tool for the right job.
I’ve migrated rdbms to wide column databases like Cassandra or dynamo because we had specific requirements that rdbms were not fulfilling.
I’ve also migrated from document database to rdbms because the document store didn’t meet our specific requirements.
I wouldn’t just use any random database because I want to appear cool (?) because I know Cassandra or how to use a vector database. That’s not the point.
omneity|2 years ago
There's a lot of use cases for which Postgres works very well at scale, and the main benefit of a solution like these specialized ones is more of a convenience layer.
hinkley|2 years ago
What functional consensus protocol doesn't mandate attempted murder? When a node becomes incoherent it can't be relied upon to notice that it has done so and bow out gracefully. Like cancer, there is always a change that 'cell death' will fail and leave you in a pathological state.
Andys|2 years ago
franckpachot|2 years ago
gregwebs|2 years ago
marsupialtail_2|2 years ago
The key difference here is that these systems don't store data, so fault tolerance means recovering within a query instead of not losing data.
Raminj95|2 years ago
gavinray|2 years ago
You write a database in Java while having the principles explained along the way.
https://www.amazon.com/Database-Design-Implementation-Data-C...
_georgesim_|2 years ago
foodoos|2 years ago
"SLOG: Serializable, Low-latency, Geo-replicated Transactions"
https://par.nsf.gov/servlets/purl/10126332
hartem_|2 years ago
olluk|2 years ago