If have a little free time someday I'm going to spend a day polishing what I was building and Open-source it. It was a kind of investigation into a few different ideas, it was the synced data store which was the more interesting thing that came out of it.
This is the very rough code behind the PouchDB/Yjs datastore. Effectively each Pouch/Couch document is actually "managed" by Yjs, all changes/operations via it. It then saves the binary Yjs blob as an attachment on the Pouch document with the current Yjs state exported as JSON for the main Pouch document. This gives you all the indexing/search you get with Pouch/Couch but with automatic merging of conflicting edits.
Ultimately though I don't think PouchDB is a good platform for this, building something that is native Yjs would be much better. If anyone is interested I would love to hear from them though!
I'm also interested in following updates to your approach here.
Something that stands out immediately to me is that reliance on binary attachments. In my own CouchDB ecosystem work binary attachments have turned out to be just about the worst part of the ecosystem. PouchDB stores them pretty reliably, but every other CouchDB ecosystem database (Couchbase, Cloudant) including different versions of CouchDB itself (1.x is different from 2.x is different from 3.x in all sorts of ways) all have very different behavior when synchronizing attachments, the allowed size of attachments, the allowed types of attachments, the allowed characters in attachment names, and in general the sync protocol itself is prone to failures/timeouts with large attachments that are tough to work around because the break in the middle of replications. The number of times I've had to delete an attachment that PouchDB stored just fine to get a sync operation to complete with another server has been way too many already.
I've had to build bespoke attachment sync tools because I haven't been able to rely on attachments working in the CouchDB ecosystem.
Probably more evidence to back up your gut instinct that a native Yjs-oriented store may be better in the long term.
(Admittedly, too, I've been thinking that I need to replace the CouchDB ecosystem as a whole. PouchDB is great, but the flux I've seen in the Apache CouchDB project and the issues I've had with the managed service providers especially Cloudant after IBM makes it really hard to recommend the ecosystem. Overall it seems unhealthy/in-decline, which is sad when the core sync infrastructure seems so nice to work with when it works.)
samwillis|3 years ago
This is the very rough code behind the PouchDB/Yjs datastore. Effectively each Pouch/Couch document is actually "managed" by Yjs, all changes/operations via it. It then saves the binary Yjs blob as an attachment on the Pouch document with the current Yjs state exported as JSON for the main Pouch document. This gives you all the indexing/search you get with Pouch/Couch but with automatic merging of conflicting edits.
https://gist.github.com/samwillis/1465da23194d1ad480a5548458...
and a tread about it:
https://discuss.yjs.dev/t/distributed-offline-editing-with-c...
Ultimately though I don't think PouchDB is a good platform for this, building something that is native Yjs would be much better. If anyone is interested I would love to hear from them though!
WorldMaker|3 years ago
Something that stands out immediately to me is that reliance on binary attachments. In my own CouchDB ecosystem work binary attachments have turned out to be just about the worst part of the ecosystem. PouchDB stores them pretty reliably, but every other CouchDB ecosystem database (Couchbase, Cloudant) including different versions of CouchDB itself (1.x is different from 2.x is different from 3.x in all sorts of ways) all have very different behavior when synchronizing attachments, the allowed size of attachments, the allowed types of attachments, the allowed characters in attachment names, and in general the sync protocol itself is prone to failures/timeouts with large attachments that are tough to work around because the break in the middle of replications. The number of times I've had to delete an attachment that PouchDB stored just fine to get a sync operation to complete with another server has been way too many already.
I've had to build bespoke attachment sync tools because I haven't been able to rely on attachments working in the CouchDB ecosystem.
Probably more evidence to back up your gut instinct that a native Yjs-oriented store may be better in the long term.
(Admittedly, too, I've been thinking that I need to replace the CouchDB ecosystem as a whole. PouchDB is great, but the flux I've seen in the Apache CouchDB project and the issues I've had with the managed service providers especially Cloudant after IBM makes it really hard to recommend the ecosystem. Overall it seems unhealthy/in-decline, which is sad when the core sync infrastructure seems so nice to work with when it works.)