Yes. Postgres stores the checkpoint on the source database replication slot. You can configure Debezium Engine to send checkpoints to the file /dev/null to disable the default external system checkpoint behavior.
Debezium relies on Kafka Connect's `OffsetBackingStore` contract [1] for persisting and restoring offset positions (that's subject to abstraction from Connect's SPIs at some point). While acknowledged LSNs are committed to the source DB in case of Postgres indeed, the offset store is still needed in most cases, e.g. for persisting other information, e.g. the fact whether or not an initial snapshot has happened yet.
> I don’t want to manage that in data producers anymore.
Could you elaborate a bit? The outbox pattern typically _is_ implemented in source applications, so I'm curious what alternative you have in mind? Stream processing e.g. with Flink actually can be an alternative, creating external data representations after the fact, though challenges arise e.g. in terms of transactional consistency.
aynyc|2 years ago
gunnarmorling|2 years ago
> I don’t want to manage that in data producers anymore.
Could you elaborate a bit? The outbox pattern typically _is_ implemented in source applications, so I'm curious what alternative you have in mind? Stream processing e.g. with Flink actually can be an alternative, creating external data representations after the fact, though challenges arise e.g. in terms of transactional consistency.
[1] https://debezium.io/documentation/reference/stable/developme...