top | item 44749845

(no title)

_benedict | 7 months ago

Do you anywhere elaborate what you mean by leaderless, and how this affects the semantics and guarantees you offer?

So far as I understand both Kafka and Pulsar use (leader-based) consensus protocols to deliver some of their features and guarantees, so to match these you must either have developed a leaderless consensus protocol, or modify the guarantees you offer, or else have a leader-based consensus protocol you utilise still?

From one of your other answers, you mention you rely on Apache Bookkeeper, which appears to be leader-based?

I ask because I am aware of only one industry leaderless consensus protocol under development (and I am working on it), and it is always fun to hear about related work.

discuss

order

upghost|7 months ago

Whoa a leaderless consensus protocol sounds pretty revolutionary!! So many question -- do you have any resources on this you could share?

_benedict|7 months ago

Revolutionary may be an overstatement, it just affords different system characteristics. There's plenty of literature on the topic though, starting generally with EPaxos[1]. The protocol that we are developing is for Apache Cassandra, is called Accord[2], and forms the basis of our new distributed transaction feature [3]. I will note that the whitepaper linked in [3] is a bit out of date, and there was a bug in the protocol specification at that time. We hope to publish an updated paper in a proper venue in the near future.

[1] https://www.cs.cmu.edu/~dga/papers/epaxos-sosp2013.pdf [2] https://github.com/apache/cassandra-accord [3] https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-15...