Re: Missing rows after migrating from postgres 11 to 12 with logical replication

On 12/21/20 12:26 PM, Lars Vonk wrote:
> Hi Adrian,
> 
> Thanks for taking the time to reply!

> 
>     2) Are the logs still available for inspection to see if there where
>     any
>     errors thrown?
> 
> 
> Yes, and we dug into those. And we also found some indications that 
> something went wrong.
> 

>     4) How did you determine the rows where missing?
> 
> 
> We were alerted by a bug later that day and found that some rows were 
> missing in the new primary. We did a compare based on primary key and 
> found that several tables were missing rows. Before the switch we 
> unfortunately only checked max(id) and did some counts on tables and 
> those all checked out. We didn't do a count on all tables...
> 
> So to come back at the logs:
> 
> We dug a little deeper and we did found ERROR logs around the time we 
> ran the initial copies. During a period of several hours that day we see 
> a couple of messages like:
> 
>     ERROR: requested WAL segment 00000001000001F10000001D has already
>     been removed

What was being run when the above ERROR was triggered?


> 
> Regards,
> Lars
> 
> On Sun, Dec 20, 2020 at 6:58 PM Adrian Klaver <adrian.klaver@aklaver.com 



-- 
Adrian Klaver
adrian.klaver@aklaver.com