Re: [HACKERS] Interval for launching the table sync worker

On 06/04/17 16:44, Masahiko Sawada wrote:
> On Thu, Apr 6, 2017 at 9:06 PM, Kyotaro HORIGUCHI
> <horiguchi.kyotaro@lab.ntt.co.jp> wrote:
>> At Thu, 06 Apr 2017 17:02:14 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp> wrote
in<20170406.170214.263553093.horiguchi.kyotaro@lab.ntt.co.jp>
 
>>> At Thu, 6 Apr 2017 16:15:33 +0900, Masahiko Sawada <sawada.mshk@gmail.com> wrote in
<CAD21AoCrcwi3SwKKOW_efwW0finxyycLgsbw09n5uy2sxneO_A@mail.gmail.com>
>>>> On Thu, Apr 6, 2017 at 3:45 PM, Kyotaro HORIGUCHI
>>>> <horiguchi.kyotaro@lab.ntt.co.jp> wrote:
>>>>> I was thinking the same.
>>>>>
>>>>> At Thu, 6 Apr 2017 11:33:22 +0900, Masahiko Sawada <sawada.mshk@gmail.com> wrote in
<CAD21AoDCnyRJDUY=ESVVe68AukvOP2dFomTeBFpAd1TiFbjsGg@mail.gmail.com>
>>>>>> Hi all,
>>>>>>
>>>>>> While testing table sync worker for logical replication I noticed that
>>>>>> if the table sync worker of logical replication failed to insert the
>>>>>> data for whatever reason, the table sync worker process exits with
>>>>>> error. And then the main apply worker launches the table sync worker
>>>>>> again soon without interval. This routine is executed at very high
>>>>>> frequency without interval.
>>>>>>
>>>>>> Should we do put a interval (wal_retrieve_interval or make a new GUC
>>>>>> parameter?) for launching the table sync worker?
>>
>> Hmm. I was playing with something wrong. Now I see the invervals
>> 5 seconds. No problem.
> 
> Yeah, this issue happens only in the table sync worker.
> 
>>
>>>>> After introducing encoding conversion, untranslatable characters
>>>>> in a published table causes this situation.
>>>>
>>>> I think it's better to make a new GUC parameter for the table sync
>>>> worker. Having multiple behaviors in wal_retrieve_retry_interval is
>>>> not good idea. Thought?
>>
>> So, this is working, sorry.
>>
>>> I prefer subscription option than GUC. Something like following.
>>>
>>> CREATE SUBSCRIPTION s1 CONNECTION 'blah'
>>>        PUBLICATION p1 WITH (noreconnect = true);
>>>
>>> Stored in pg_subscription?

I don't think that's a very good solution, you'd lose replication on
every network glitch, upstream server restart, etc.

>>>
>>>>> Interval can reduce
>>>>> the frequence of reconnecting, but I think that walreciever
>>>>> should refrain from reconnecting on unrecoverable(or repeating)
>>>>> error in walsender.
>>>>>
>>>>
>>>> Yeah, that's also considerable issue.
>>>
>>> But not to do now.
>>
> 
> I've added this as an open item, and sent a patch for this.
> 

I am not exactly sure what's the open item from this thread. To use the
wal_retrieve_interval to limit table sync restarts?

--  Petr Jelinek                  http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training &
Services