[ncruces]

I need to bring writes to my version of the VFS.

I'm still waiting on how they'll prevent accidental corruption from multiple writers; there's a PR implementing write leases, not sure if that's the direction they'll take.

That said, pausing local polling when writes are enabled - i.e. assuming you're the only writer - makes sense, it's a good idea; hadn't occurred to me.

Ideally, I'd like to offer durability on fullfsync. I think this is feasible. In a concurrent system (single host), while a writer is waiting for durability confirmation, readers can continue reading the previous state, and the next writer can read the committed - but not yet durable - data and queue its writes to be batched. You can have as many pending writes as you're willing to have connections.

[benbjohnson]

Litestream author here. Currently we're handling the "single writer" issue outside of Litestream. We have controls in our internal systems that make it work well. But yes, the lease PR is the direction we're looking at going.

I'm not sure you can have readers see something separate than writers. When SQLite promotes a read lock to a write lock under WAL then it checks if any of the data has changed and then fails the transaction if it has.

[ncruces]

You're correct that it's hard to have readers see something separate from writers. It actually depends on an experimental feature that might just go away at any time.

In journal mode (which is what the VFS uses), the sequence for a write transaction is: (1) xLock(shared); (2) xRead(change-counter); (3) xLock(reserved) ...

The read of the change counter with the shared lock complicates things, because at that point we don't know if there will be a write upgrade to reserved.

However, the experimental feature makes it like this: (1) xFileControl(experimental-pragma); (2) xLock(shared); (3) xRead(change-counter); (4) xLock(reserved) ...

So you can distinguish shared locks that will be immediately upgraded.

https://sqlite.org/forum/forumpost/c4ca8e7f4a887aa4