Re: Skipping logical replication transactions on subscriber side

On Thu, Oct 28, 2021 at 10:48 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
>
> On Thu, Oct 28, 2021 at 1:05 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >
> > On Thu, Oct 28, 2021 at 7:49 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> > >
> > > >
> > > > Either from the error messages in the server log or from the new view
> > > > we are planning to add. I think such a case is possible during the
> > > > initial synchronization phase where apply worker went ahead then
> > > > tablesync worker by skipping to apply the changes on the corresponding
> > > > table. After that it is possible, that table sync worker failed during
> > > > copy and apply worker fails during the processing of some other rel.
> > >
> > > Does it mean that if both initial copy for the corresponding table by
> > > table sync worker and applying changes for other rels by apply worker
> > > fail, we skip both by specifying LSN?
> > >
> >
> > Yes.
> >
> > > If so, can't we disable the
> > > initial copy for the table and skip only the changes for other rels
> > > that cannot be applied?
> > >
> >
> > But anyway you need some way to skip changes via a particular
> > tablesync worker so that it can mark the relation in 'ready' state.
>
> Right.
>
> >  I
> > think one can also try to use disable_on_error option in such
> > scenarios depending on how we expose it. Say, if the option means that
> > all workers (apply or table sync) should be disabled on an error then
> > it would be a bit tricky but if we can come up with a way to behave
> > differently for different workers then it is possible to disable one
> > set of workers and skip the changes in another set of workers.
>
> Yes, I would prefer to skip individual transactions in question rather
> than skip changes until the particular LSN. It’s not advisable to use
> LSN to skip changes since it has a risk of skipping unrelated changes
> too.
>

Fair enough but I think providing LSN is also useful if user can
identify the same easily as otherwise there might be more
administrative work to make replication progress.

--
With Regards,
Amit Kapila.