I built this to have a dedicated wire-protocol client for postgres logical replication.
General-purpose SQL clients either don't implement the replication protocol at all, or bury it behind abstractions designed for other use cases. Replication has a bit different mechanics - it's a stateful binary stream requiring LSN tracking, standby heartbeats, and feedback to prevent WAL bloat. Bolting that onto a query-focused client has its own challenges.
This is just the transport - raw XLogData frames and LSNs. Use pg_replicate, as an example, if you need "replicate to BigQuery." Use this if you're building replication infrastructure.
What it does:
- Explicit LSN control - start/stop at specific WAL positions for deterministic recovery
- Automatic standby feedback - no more forgotten heartbeats filling your disk with WAL
- Bounded channels - backpressure propagates to Postgres naturally
- Pure Rust, no libpq
What it doesn't do: pgoutput decoding (intentionally). That belongs in a higher layer.
Simplest way of using this:
while let Some(event) = client.recv().await? {
match event {
ReplicationEvent::XLogData { wal_end, data, .. } => {
process(&data);
client.update_applied_lsn(wal_end);
}
_ => {}
}
}
I learned about this tonight when Claude Code picked up your library for my application that uses logical replication. Looking forward to putting it through its paces.
Nice one, great to see this addition to the Rust ecosystem!
Reading through the README, this piqued my curiosity:
> Small or fast transactions may share the same WAL position.
I don't think that's true; each data change and each commit (whether explicit or not) has its own dedicated LSN.
> LSNs should be treated as monotonic but not dense.
That's not correct; commit LSNs are monotonically increasing, and within a transaction, event LSNs are monotonically increasing. I.e. the tuple commit-LSN/event-LSN is monotonically increasing, but not LSNs per se. You can run multiple concurrent transactions to observe this.
sacs0ni|1 month ago
This is just the transport - raw XLogData frames and LSNs. Use pg_replicate, as an example, if you need "replicate to BigQuery." Use this if you're building replication infrastructure.
What it does:
- Explicit LSN control - start/stop at specific WAL positions for deterministic recovery
- Automatic standby feedback - no more forgotten heartbeats filling your disk with WAL
- Bounded channels - backpressure propagates to Postgres naturally
- Pure Rust, no libpq
What it doesn't do: pgoutput decoding (intentionally). That belongs in a higher layer. Simplest way of using this:
while let Some(event) = client.recv().await? { match event { ReplicationEvent::XLogData { wal_end, data, .. } => { process(&data); client.update_applied_lsn(wal_end); } _ => {} } }
nkmnz|1 month ago
Would I use this if I host my own postgres and want to use replication for „real time backups“ into a hot standby?
malodyets|1 month ago
jsjfusua|1 month ago
[deleted]
gunnarmorling|1 month ago
Reading through the README, this piqued my curiosity:
> Small or fast transactions may share the same WAL position.
I don't think that's true; each data change and each commit (whether explicit or not) has its own dedicated LSN.
> LSNs should be treated as monotonic but not dense.
That's not correct; commit LSNs are monotonically increasing, and within a transaction, event LSNs are monotonically increasing. I.e. the tuple commit-LSN/event-LSN is monotonically increasing, but not LSNs per se. You can run multiple concurrent transactions to observe this.
sacs0ni|1 month ago
To demo your point I created a gist, for myself and others to see the (commit-LSN, event-LSN) ordering in action:
https://gist.github.com/vnvo/a8cf59fc3cd8719dbea56d3bb5201f9...
I'll update the readme to reflect this more accurately. Appreciate you taking the time to point it out.