Re: Data loss on logical replication, 12.12 to 14.5, ALTER SUBSCRIPTION

Hello.

> None of these entries are from the point mentioned by you [1]
> yesterday where you didn't find the corresponding data in the
> subscriber. How did you identify that the entries corresponding to
> that timing were missing?

Some of the before the interval, some after... But the source database
was generating a lot of WAL during logical replication
- some of these log entries from time AFTER completion of initial sync
of B but (probably) BEFORE finishing B table catch up (entering
streaming mode).

Just to clarify, tables A, B and C are updated in the same
transaction, something like:

BEGIN;
UPDATE A SET x = x +1 WHERE id = :id;
INSERT INTO B(current_time, :id);
INSERT INTO C(current_time, :id);
COMMIT;

Other (non-mentioned) tables also included into this transaction, but
only B missed small amount of data.

So, shortly the story looks like:

* initial sync of A (and other tables) started and completed, they are
in streaming mode
* B and C initial sync started (by altering PUBLICATION and SUBSCRIPTION)
* B sync completed, but new changes are still applying to the tables
to catch up primary
* logical replication apply worker is restarted because IO error on WAL receive
* Postgres killed
* Postgres restarted
* C initial sync restarted
* logical replication apply worker few times restarted because IO
error on WAL receive
* finally every table in streaming mode but with small gap in B

Thanks,
Michail.