I have some very unfortunate news to share with the Cap'n Proto and Sandstorm communities.
Ian Denhardt (zenhack on HN), a lead contributor to the Go implementation, suddenly and unexpectedly passed away a few weeks ago. Before making a request to the community, I want to express how deeply saddened I am by this loss. Ian and I collaborated extensively over the past three years, and we had become friends.
As the de facto project lead, it now befalls me to fill Ian's very big shoes. Please, if you're able to contribute to the project, I could really use the help. And if you're a contributor or maintainer of some other implementation (C++, Rust, etc.), I would *REALLY* appreciate it if we could connect. I'm going to need to surround myself with very smart people if I am to continue Ian's work.
RIP Ian, and thank you. I learned so much working with you.
I've had a couple people suddenly taken from me, and it is soul crushing. Every time it happens it reminds me of how fragile life is, and how quickly things can change. I've started trying to enjoy the small things in life more, and while I don't neglect the future, I also try to enjoy the present.
He has left an amazing legacy that has touched a lot of people. RIP Ian.
I find it surprising how few protocols (besides Cap'n Proto) have promise pipelining. The only other example I can think of is 9p, but that's not a general purpose protocol.
As neat as it is I guess it's hard optimize the backend for it compared to explicitly grouping the queries. I imagine a bespoke RPC call that results in a single SQL query is better than several pipelined but separate RPC calls, for example.
But even still, you would think it would be more popular.
> I find it surprising how few protocols (besides Cap'n Proto) have promise pipelining.
Pipelining is a bad idea. It reifies object instances, and thus makes robust implementation much harder. You no longer make stateless calls, but you are running functions with particular object instances.
And you immediately start getting problems. Basically, Client Joe calls Service A and then pass the promised result of the call to Service B. So that Service B will have to do a remote call to Service A to retrieve the result of the promise.
This creates immediate complications with security boundaries (what is your delegation model?). But what's even worse, it removes the backpressure. Client Joe can make thousands of calls to Service A, and then pass the not-yet-materialized results to Service B. Which will then time out because Service A is being DDoS-ed.
Or more specifically, it seems to have client-chosen file descriptors, so the client can open a file, then immediately send a read on that file, and if the open fails, the read will also fail (with EBADF). Awesome!
This is great, but "promise pipelining" also needs support in the client.
Are there 9p clients which support promise pipelining? For example, if the user issues several walks, they're all sent before waiting for the reply to the first walk?
Also, it only has promise pipelining for file descriptors. That gives you a lot, definitely, but if for example you wanted to read every file in a directory, you'd want to be able to issue a read and then walk to the result of that read. Which 9p doesn't seem to support. (I actually support this in my own remote syscall protocol library thing, rsyscall :) )
There is also CapnP’s moral ancestor CapTP[1]/VatTP aka Pluribus developed to accompany Mark Miller’s E language (yes, it’s a pun, there is also a gadget called an “unum” in there). For deeper genealogy—including a reference to Barbara Liskov for promise pipelining and a number of other relevant ideas in the CLU extension Argus—see his thesis[2].
(If I’m not misremembering, Mark Miller later wrote the promise proposal for JavaScript, except the planned extension for RPC never materialized and instead we got async/await, which don’t seem compatible with pipelining.)
The more recent attempts to make a distributed capability system in the image of E, like Spritely Goblins[3] and the OCapN effort[4], also try for pipelining, so maybe if you hang out on cap-talk[5] you’ll hear about a couple of other protocols that do it, if not ones with any real-world usage.
(And I again reiterate that, neat as it is, promise pipelining seems to require programming with actual explicit promises, and at this point it’s well-established how gnarly that can get.)
One idea that I find interesting and little-known from the other side—event loops and cooperatively concurrent “active objects”—is “causality IDs”[6] from DCOM/COM+ as a means of controlling reentrancy, see CoGetCurrentLogicalThreadId[7] in the Microsoft documentation and the discussion of CALLTYPE_TOPLEVEL_CALLPENDING in Effective COM[8]—I think they later tried to sell this as a new feature in Win8/UWP’s ASTAs[9]?
Without knowing how exactly capnproto promise pipelining works, when I thought about it, I was concerned about cases like reading a directory and stating everything in it, or getting back two response values and wanting to pass only one to the next call. The latter could be made to work, I guess, but the former depends on eg the number of values in the result list.
Congrats on the release! It must be very exciting after 10 years :)
If you don't mind the question: will there be more work on implementations for other languages in the future? I really like the idea of the format, but the main languages in our stack aren't supported in a way I'd use in a product.
This is indeed the main weakness of Cap'n Proto. I only really maintain the C++ implementation. Other implementations come from various contributors which can lead to varying levels of completeness and quality.
Unfortunately I can't really promise anything new here. My work on Cap'n Proto is driven by the needs of my main project, the Cloudflare Workers runtime, which is primarily C++. We do interact with Go and Rust services, and the respective implementations seem to get the job done there.
Put another way, Cap'n Proto is an open source project, and I hope it is useful to people, but it is not a product I'm trying to sell, so I am not particularly focused on trying to get everyone to adopt it. As always, contributions are welcome.
The one case where I might foresee a big change is if we (Cloudflare) decided to make Cap'n Proto be a public-facing feature of the Workers platform. Then we'd have a direct need to really polish it in many languages. That is certainly something we discuss from time to time but there are no plans at present.
There are people who have tried to write the RPC layer without it simply being a wrapper around the C++ implementation, but it's a LOT of code to rewrite for not a lot of direct benefit.
Feel free to take a crack at it. People would likely be rather cooperative about it. However, know that it's just simply a lot of work.
If any cloudflare employees end up here who helped decide on Capn Proto over other stuff (e.g. protobuf), what considerations went into that choice? I'm curious if the reasons will be things important to me, or things that you don't need to worry about unless you deal with huge scale.
To this day, Cloudflare's data pipeline (which produces logs and analytics from the edge) is largely based on Cap'n Proto serialization. I haven't personally been much involved with that project.
As for Cloudflare Workers, of course, I started the project, so I used my stuff. Probably not the justification you're looking for. :)
That said, I would argue the extreme expressiveness of Cap'n Proto's RPC protocol compared to alternatives has been a big help in implementing sandboxing in the Workers Runtime, as well as distributed systems features like Durable Objects. https://blog.cloudflare.com/introducing-workers-durable-obje...
I don't work at Cloudflare but follow their work and occasionally work on performance sensitive projects.
If I had to guess, they looked at the landscape a bit like I do and regarded Cap'n Proto, flatbuffers, SBE, etc. as being in one category apart from other data formats like Avro, protobuf, and the like.
So once you're committed to record'ish shaped (rather than columnar like Parquet) data that has an upfront parse time of zero (nominally, there could be marshalling if you transmogrify the field values on read), the list gets pretty short.
Cap'n Proto was originally made for https://sandstorm.io/. That work (which Kenton has presumably done at Cloudflare since he's been employed there) eventually turned into Cloudflare workers.
To summarize something from a little over a year after I joined there: Cloudflare was building out a way to ship logs from its edge to a central point for customer analytics and serving logs to enterprise customers. As I understood it, the primary engineer who built all of that out, Albert Strasheim, benchmarked the most likely serialization options available and found Cap'n Proto to be appreciably faster than protobuf. It had a great C++ implementation (which we could use from nginx, IIRC with some lua involved) and while the Go implementation, which we used on the consuming side, had its warts, folks were able to fix the key parts that needed attention.
Anyway. Cloudflare's always been pretty cost efficient machine wise, so it was a natural choice given the performance needs we had. In my time in the data team there, Cap'n Proto was always pretty easy to work with, and sharing proto definitions from a central schema repo worked pretty well, too. Thanks for your work, Kenton!
I'm excited by Cap'n Proto's participation in the OCAPN standardization effort. Can you speak to if that's going to be part of the Cap'n Proto 2.0 work?
While I never used Cap'n Proto, I want to thank kentonv for the extremely informative FAQ answer [1] on why required fields are problematic in a protocol
I link it to people all the time, especially when they ask why protobuf 3 doesn't have required fields.
This is some very valuable perspective. Personally, I previously also struggled to understand why. For me, the thing that clicked was to understand protobuf and Cap'n proto as serialization formats that need to work across API boundaries and need to work with different versions of their schema in a backwards- and forwards-compatible way; do not treat them as in-memory data structures that represent the world from the perspective of a single process running a single version without no compatibility concerns. Thus, the widely repeated mantra of "making illegal states unrepresentable" does not apply.
>To help you safely add and remove required fields, Typical offers an intermediate state between optional and required: asymmetric. An asymmetric field in a struct is considered required for the writer, but optional for the reader. Unlike optional fields, an asymmetric field can safely be promoted to required and vice versa.
Can't we extend this argument to eliminating basically all static typing? And frankly that'd not even be wrong, and is why Alan Kay defined OOP as one that's dynamically typed and late bound, and we went against it anyway to keep relearning the same lessons over and over.
> The right answer is for applications to do validation as-needed in application-level code.
It would've been nice to include a parameter to switch "required message validation" on and off, instead of relying on application code. Internally in an application, we can turn this off, the message bus can turn it off, but in general, developers would really benefit from this being on.
A gotcha along the same path. Deserialization of things not needed as what you get with generated clients. An aspect of interfaces in Go I really like. Remotely type what I use. Skip the rest. Not fun to have incidents caused by changes to a contract that is not even used by a service. Also hard to find.
Avro solves this problem completely, and more elegantly with its schema resolution mechanism. Exchanging schemas at the beginning of a connection handshake is hardly burdensome
Great achievement. To be honest I wouldn't recommend Capnp. The C++ API is very awkward.
The zero copy parsing is less of a benefit than you'd expect - pretty unlikely you're going to want to keep your data as a Capnp data structure because of how awkward it is to use. 99% of the time you'll just copy it into your own data structures anyway.
There's also more friction with the rest of the world which has more or less settled on Protobuf as the most popular binary implementation of this sort of idea.
I only used it for serialisation. Maybe the RPC stuff is more compelling.
I really wish Thrift had taken off instead of Protobuf/gRPC. It was so much better designed and more flexible than anything I've seen before or since. I think it died mainly due to terrible documentation. I guess it also didn't have a big name behind it.
I do agree that the API required for zero-copy turns out a bit awkward, particularly on the writing side. The reading side doesn't look much different. Meanwhile zero-copy is really only a paradigm shift in certain scenarios, like when used with mmap(). For network communications it doesn't change much unless you are doing something hardcore like RDMA. I've always wanted to add an optional alternative API to Cap'n Proto that uses "plain old C structures" (or something close to it) with one-copy serialization (just like protobuf) for the use cases where zero-copy doesn't really matter. But haven't gotten around to it yet...
That said I personally have always been much more excited about the RPC protocol than the serialization. I think the RPC protocol is actually a paradigm shift for almost any non-trivial use case.
I find MessagePack to be pretty great if you don't need schema. JSON serialization is unreasonably fast in V8 though and even message pack can't beat it; though it's often faster in other languages and saves on bytes.
I always liked the idea of capnp, but it bothers me that what is ultimately a message encoding protocol has an opinion on how I should architect my server.
FWIW, gRPC certainly has this problem too, but it’s very clearly distinct from protobuf, although pb has gRPC-related features.
That entanglement makes me lean towards flatbuffers or even protobuf every time I weigh them against capnp, especially since it means that fb and pb have much simpler implementations, and I place great value on simplicity for both security and maintenance reasons.
I think the lack of good third-party language implementations speaks directly to the reasonability of that assessment. It also makes the bus factor and longevity story very poor. Simplicity rules.
We have a great plethora of binary serialization libraries now, but I've noticed none of them offer the following:
* Specification of the number of bits I want to cap out a field at during serialization, ie: `int` that only uses 3 bits.
* Delta encoding for serialization and deserialization, this would further decrease the size of each message if there is an older message that I can use as the initial message to delta encode/decode from.
this changed my world how i think about computing on the web.
if there was just a good enough js library as for lua and you could directly send capnp messages to workerd instead of always going through files. I guess one day i have to relearn c++ and understand how the internals actually work.
If you are interested in learning more about binary serialization (for Cap'n Proto and others), I wrote two free-to-read papers extensively looking at the scene from a space-efficiency point of view that people here might be interested in:
Not something that aims to compete with Cap'n Proto (different use cases), but I've been also working on a binary serialization format that is pure JSON-compatible with a focus on space-efficiency: https://www.jsonbinpack.org.
It’s a testament to the subtlety of software engineering that even after four tries (protobuf 1-3, capn proto 1) there are still breaking changes that need to be made to the solution of what on the surface appears to be a relatively constrained problem.
Since then, we extended and improved the support: added it for export (initially, it was only for import) and improved the performance.
About strange stuff in the library - it uses a non-obvious approach for exception handling, and their C++ code feels like they too much focusing on some non-orthodox approaches.
Some of these “high perf” RPC libraries never get one key point, if I really need something to be fast, the most important aspect is that it must be simple.
[+] [-] omginternets|2 years ago|reply
Ian Denhardt (zenhack on HN), a lead contributor to the Go implementation, suddenly and unexpectedly passed away a few weeks ago. Before making a request to the community, I want to express how deeply saddened I am by this loss. Ian and I collaborated extensively over the past three years, and we had become friends.
As the de facto project lead, it now befalls me to fill Ian's very big shoes. Please, if you're able to contribute to the project, I could really use the help. And if you're a contributor or maintainer of some other implementation (C++, Rust, etc.), I would *REALLY* appreciate it if we could connect. I'm going to need to surround myself with very smart people if I am to continue Ian's work.
RIP Ian, and thank you. I learned so much working with you.
------
P.S: I can be reached in the following places
- https://github.com/lthibault
- https://matrix.to/#/#go-capnp:matrix.org
- Telegram: @lthibault
- gmail: louist87
[+] [-] freedomben|2 years ago|reply
He has left an amazing legacy that has touched a lot of people. RIP Ian.
[+] [-] jcalabro|2 years ago|reply
[+] [-] pja|2 years ago|reply
[+] [-] CodesInChaos|2 years ago|reply
https://capnproto.org/news/2013-12-13-promise-pipelining-cap...
[+] [-] jayd16|2 years ago|reply
But even still, you would think it would be more popular.
[+] [-] cyberax|2 years ago|reply
Pipelining is a bad idea. It reifies object instances, and thus makes robust implementation much harder. You no longer make stateless calls, but you are running functions with particular object instances.
And you immediately start getting problems. Basically, Client Joe calls Service A and then pass the promised result of the call to Service B. So that Service B will have to do a remote call to Service A to retrieve the result of the promise.
This creates immediate complications with security boundaries (what is your delegation model?). But what's even worse, it removes the backpressure. Client Joe can make thousands of calls to Service A, and then pass the not-yet-materialized results to Service B. Which will then time out because Service A is being DDoS-ed.
[+] [-] catern|2 years ago|reply
Or more specifically, it seems to have client-chosen file descriptors, so the client can open a file, then immediately send a read on that file, and if the open fails, the read will also fail (with EBADF). Awesome!
This is great, but "promise pipelining" also needs support in the client. Are there 9p clients which support promise pipelining? For example, if the user issues several walks, they're all sent before waiting for the reply to the first walk?
Also, it only has promise pipelining for file descriptors. That gives you a lot, definitely, but if for example you wanted to read every file in a directory, you'd want to be able to issue a read and then walk to the result of that read. Which 9p doesn't seem to support. (I actually support this in my own remote syscall protocol library thing, rsyscall :) )
[+] [-] mananaysiempre|2 years ago|reply
(If I’m not misremembering, Mark Miller later wrote the promise proposal for JavaScript, except the planned extension for RPC never materialized and instead we got async/await, which don’t seem compatible with pipelining.)
The more recent attempts to make a distributed capability system in the image of E, like Spritely Goblins[3] and the OCapN effort[4], also try for pipelining, so maybe if you hang out on cap-talk[5] you’ll hear about a couple of other protocols that do it, if not ones with any real-world usage.
(And I again reiterate that, neat as it is, promise pipelining seems to require programming with actual explicit promises, and at this point it’s well-established how gnarly that can get.)
One idea that I find interesting and little-known from the other side—event loops and cooperatively concurrent “active objects”—is “causality IDs”[6] from DCOM/COM+ as a means of controlling reentrancy, see CoGetCurrentLogicalThreadId[7] in the Microsoft documentation and the discussion of CALLTYPE_TOPLEVEL_CALLPENDING in Effective COM[8]—I think they later tried to sell this as a new feature in Win8/UWP’s ASTAs[9]?
[1] http://erights.org/elib/distrib/captp/index.html
[2] http://erights.org/talks/thesis/index.html
[3] https://spritely.institute/goblins/
[4] https://github.com/ocapn/ocapn
[5] https://groups.google.com/g/captalk/
[6] https://learn.microsoft.com/openspecs/windows_protocols/ms-d...
[7] https://learn.microsoft.com/windows/win32/api/combaseapi/nf-...
[8] https://archive.org/details/effectivecom50wa00boxd/page/150
[9] https://devblogs.microsoft.com/oldnewthing/20210224-00/?p=10...
[+] [-] dan-robertson|2 years ago|reply
[+] [-] dontlaugh|2 years ago|reply
[+] [-] giovannibonetti|2 years ago|reply
[1] https://redis.com/ebook/part-2-core-concepts/chapter-4-keepi...
[+] [-] Timon3|2 years ago|reply
If you don't mind the question: will there be more work on implementations for other languages in the future? I really like the idea of the format, but the main languages in our stack aren't supported in a way I'd use in a product.
[+] [-] kentonv|2 years ago|reply
Unfortunately I can't really promise anything new here. My work on Cap'n Proto is driven by the needs of my main project, the Cloudflare Workers runtime, which is primarily C++. We do interact with Go and Rust services, and the respective implementations seem to get the job done there.
Put another way, Cap'n Proto is an open source project, and I hope it is useful to people, but it is not a product I'm trying to sell, so I am not particularly focused on trying to get everyone to adopt it. As always, contributions are welcome.
The one case where I might foresee a big change is if we (Cloudflare) decided to make Cap'n Proto be a public-facing feature of the Workers platform. Then we'd have a direct need to really polish it in many languages. That is certainly something we discuss from time to time but there are no plans at present.
[+] [-] bsder|2 years ago|reply
Feel free to take a crack at it. People would likely be rather cooperative about it. However, know that it's just simply a lot of work.
[+] [-] hiddencost|2 years ago|reply
[+] [-] AceJohnny2|2 years ago|reply
[+] [-] maccam912|2 years ago|reply
[+] [-] kentonv|2 years ago|reply
To this day, Cloudflare's data pipeline (which produces logs and analytics from the edge) is largely based on Cap'n Proto serialization. I haven't personally been much involved with that project.
As for Cloudflare Workers, of course, I started the project, so I used my stuff. Probably not the justification you're looking for. :)
That said, I would argue the extreme expressiveness of Cap'n Proto's RPC protocol compared to alternatives has been a big help in implementing sandboxing in the Workers Runtime, as well as distributed systems features like Durable Objects. https://blog.cloudflare.com/introducing-workers-durable-obje...
[+] [-] coolsunglasses|2 years ago|reply
If I had to guess, they looked at the landscape a bit like I do and regarded Cap'n Proto, flatbuffers, SBE, etc. as being in one category apart from other data formats like Avro, protobuf, and the like.
So once you're committed to record'ish shaped (rather than columnar like Parquet) data that has an upfront parse time of zero (nominally, there could be marshalling if you transmogrify the field values on read), the list gets pretty short.
https://capnproto.org/news/2014-06-17-capnproto-flatbuffers-... goes into some of the trade-offs here.
Cap'n Proto was originally made for https://sandstorm.io/. That work (which Kenton has presumably done at Cloudflare since he's been employed there) eventually turned into Cloudflare workers.
Another consideration: https://github.com/google/flatbuffers/issues/2#issuecomment-...
[+] [-] hblanks|2 years ago|reply
Anyway. Cloudflare's always been pretty cost efficient machine wise, so it was a natural choice given the performance needs we had. In my time in the data team there, Cap'n Proto was always pretty easy to work with, and sharing proto definitions from a central schema repo worked pretty well, too. Thanks for your work, Kenton!
[+] [-] matlin|2 years ago|reply
[+] [-] dannyobrien|2 years ago|reply
https://github.com/ocapn/ocapn
[+] [-] kentonv|2 years ago|reply
For my part, I'm a fan of OCapN, but I am not sure how much time I can personally commit to it, with everything on my plate.
I wish I had better news here. This was a tragic loss for all of us.
[+] [-] dtech|2 years ago|reply
I link it to people all the time, especially when they ask why protobuf 3 doesn't have required fields.
[1] https://capnproto.org/faq.html#how-do-i-make-a-field-require...
[+] [-] kccqzy|2 years ago|reply
[+] [-] oftenwrong|2 years ago|reply
https://github.com/stepchowfun/typical#asymmetric-fields-can...
>To help you safely add and remove required fields, Typical offers an intermediate state between optional and required: asymmetric. An asymmetric field in a struct is considered required for the writer, but optional for the reader. Unlike optional fields, an asymmetric field can safely be promoted to required and vice versa.
[+] [-] 3cats-in-a-coat|2 years ago|reply
[+] [-] throw14082020|2 years ago|reply
> The right answer is for applications to do validation as-needed in application-level code.
It would've been nice to include a parameter to switch "required message validation" on and off, instead of relying on application code. Internally in an application, we can turn this off, the message bus can turn it off, but in general, developers would really benefit from this being on.
[1] https://capnproto.org/faq.html#how-do-i-make-a-field-require...
[+] [-] AtNightWeCode|2 years ago|reply
A gotcha along the same path. Deserialization of things not needed as what you get with generated clients. An aspect of interfaces in Go I really like. Remotely type what I use. Skip the rest. Not fun to have incidents caused by changes to a contract that is not even used by a service. Also hard to find.
[+] [-] kaba0|2 years ago|reply
[+] [-] nly|2 years ago|reply
[+] [-] alphanullmeric|2 years ago|reply
[+] [-] IshKebab|2 years ago|reply
The zero copy parsing is less of a benefit than you'd expect - pretty unlikely you're going to want to keep your data as a Capnp data structure because of how awkward it is to use. 99% of the time you'll just copy it into your own data structures anyway.
There's also more friction with the rest of the world which has more or less settled on Protobuf as the most popular binary implementation of this sort of idea.
I only used it for serialisation. Maybe the RPC stuff is more compelling.
I really wish Thrift had taken off instead of Protobuf/gRPC. It was so much better designed and more flexible than anything I've seen before or since. I think it died mainly due to terrible documentation. I guess it also didn't have a big name behind it.
[+] [-] kentonv|2 years ago|reply
That said I personally have always been much more excited about the RPC protocol than the serialization. I think the RPC protocol is actually a paradigm shift for almost any non-trivial use case.
[+] [-] mgaunard|2 years ago|reply
It has established itself as the de-facto standard, with a few other places using SBE instead.
In any case the main problems with binary serialization are:
- schemas and message version management
- delta-encoding
If you ignore these, flat binary serialization is trivial.
No library provides a good solution that covers the two points above.
[+] [-] Rapzid|2 years ago|reply
[+] [-] nvarsj|2 years ago|reply
[+] [-] binary132|2 years ago|reply
FWIW, gRPC certainly has this problem too, but it’s very clearly distinct from protobuf, although pb has gRPC-related features.
That entanglement makes me lean towards flatbuffers or even protobuf every time I weigh them against capnp, especially since it means that fb and pb have much simpler implementations, and I place great value on simplicity for both security and maintenance reasons.
I think the lack of good third-party language implementations speaks directly to the reasonability of that assessment. It also makes the bus factor and longevity story very poor. Simplicity rules.
[+] [-] batch12|2 years ago|reply
https://memory-alpha.fandom.com/wiki/Captain_Proton
[+] [-] shdh|2 years ago|reply
* Specification of the number of bits I want to cap out a field at during serialization, ie: `int` that only uses 3 bits.
* Delta encoding for serialization and deserialization, this would further decrease the size of each message if there is an older message that I can use as the initial message to delta encode/decode from.
[+] [-] jFriedensreich|2 years ago|reply
https://github.com/cloudflare/workerd/blob/main/src/workerd/...
this changed my world how i think about computing on the web.
if there was just a good enough js library as for lua and you could directly send capnp messages to workerd instead of always going through files. I guess one day i have to relearn c++ and understand how the internals actually work.
[+] [-] insanitybit|2 years ago|reply
[+] [-] jviotti|2 years ago|reply
- https://arxiv.org/abs/2201.02089
- https://arxiv.org/abs/2201.03051
Not something that aims to compete with Cap'n Proto (different use cases), but I've been also working on a binary serialization format that is pure JSON-compatible with a focus on space-efficiency: https://www.jsonbinpack.org.
Benchmarks here: https://arxiv.org/abs/2211.12799
[+] [-] s17n|2 years ago|reply
[+] [-] unknown|2 years ago|reply
[deleted]
[+] [-] unknown|2 years ago|reply
[deleted]
[+] [-] Xeoncross|2 years ago|reply
[+] [-] zX41ZdbW|2 years ago|reply
It was contributed by Marek Vavruša from CloudFlare in 2017: https://github.com/ClickHouse/ClickHouse/pull/1387
Since then, we extended and improved the support: added it for export (initially, it was only for import) and improved the performance.
About strange stuff in the library - it uses a non-obvious approach for exception handling, and their C++ code feels like they too much focusing on some non-orthodox approaches.
[+] [-] up2isomorphism|2 years ago|reply