top | item 36605060

(no title)

stu_k | 2 years ago

Has anyone built a mobile app on top of SQLite that can work offline, but then sync to a server when it gets connectivity? It feels like this could be built with a similar approach to this distributed SQLite, you'd "just" need more robust conflict handling.

discuss

order

maxpert|2 years ago

Great that you brought it up. I will fill in the perspective of what I am doing for solving this in Marmot (https://github.com/maxpert/marmot). Today Marmot already records changes via installing triggers to record changes of a table, hence all the offline changes (while Marmot is not running) are never lost. Today when Marmot comes up after a long off-time (depending upon max_log_size configuration), it realizes that and tries to catch up changes via restoring a snapshot and then applying rest of logs from NATS (JetStream) change logs. I am working on change that will be publishing those change logs to NATS before it restores snapshots, and once it reapplies those changes after restoring snapshot everyone will have your changes + your DB will be up to date. Now in this case one of the things that bothers people is the fact that if two nodes coming up with conflicting rows the last writer wins.

For that I am also exploring on SQLite-Y-CRDT (https://github.com/maxpert/sqlite-y-crdt) which can help me treat each row as document, and then try to merge them. I personally think CRDT gets harder to reason sometimes, and might not be explainable to an entry level developers. Usually when something is hard to reason and explain, I prefer sticking to simplicity. People IMO will be much more comfortable knowing they can't use auto incrementing IDs for particular tables (because two independent nodes can increment counter to same values) vs here is a magical way to merge that will mess up your data.

qweqwe14|2 years ago

Not 100% sure if it uses SQLite, but there's an app called "Flaming Durtles" on Android. It's a frontend for WaniKani (Japanese learning app). The app downloads all the lessons from WK's servers, and you can do them even when offline. When you go online, it uploads them to the backend and syncs everything else. In my experience, it works flawlessly.

It's actually a 3rd party app, WK provides an API so that people can make apps that interface with the system.

Forum thread: https://community.wanikani.com/t/android-flaming-durtles-and...

Source code (may be outdated): https://github.com/ejplugge/com.the_tinkering.wk

nijave|2 years ago

Not sure how it's implemented but seems like a pretty common pattern on Android. For instance, Gmail and Mint apps both allow offline changes that get synced. Not sure if they use SQLite but afaik that's sort of the defacto standard for Android app storage.

pritambarhate|2 years ago

we have done it a few times. It is a lot more work and a lot of edge cases come up, especially if the same data can be written by multiple people. But if data or use case is such that only one person can write to same data and you limit only one logged in client at a time, it's much easier to do.