(no title)
sewen | 11 months ago
The storage engine is pretty tightly integrated with the log, but the programming model allows you to attach quasi arbitrary state to keys.
So see whether this fits your use case, would be great to better understand the data and structure you are working with. Do you have a link where we could look at this?
ALLTaken|11 months ago
Happy to explain in more detail =) But it's not public yet.
I'm working on the consensus module optimised for hypergraphs with a succinct data-structure. The edges serve as an order-free index (FIT). Achieving max-flow, flow-matching, graph-reduction via circuits is amongst the goals.
Targeting low-latency/hig-performance distributed inference enabling layer-combination of distinct models and resumable multi-use computations as a sort of distributed compute cache.
A data-structure follows a data-format for persistence, resumeability and achieving service resilience. But although I've learned quite a bit about state management, it's still a topic I have much respect for and think using restate.dev maybe better than re-inventing the wheel. I didn't have in mind to also build a cellular automaton for state management, it maybe trivial, but I currently don't feel like having the capacity for it. Restate looks like a great production ready solution before delaying a release.
I intend to open-source it once it's mature. (But I believe binary distribution will be the more popular choice.)