Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

Hi,

This patch is not applying on the HEAD. Please rebase and share the
updated patch.

Thanks and Regards
Shlok Kyal

On Wed, 10 Jan 2024 at 14:55, Peter Smith <smithpb2250@gmail.com> wrote:
>
> Oops - now with attachments
>
> On Mon, Aug 21, 2023 at 5:56 PM Peter Smith <smithpb2250@gmail.com> wrote:
>>
>> Hi Melih,
>>
>> Last week we revisited your implementation of design#2. Vignesh rebased it, and then made a few other changes.
>>
>> PSA v28*
>>
>> The patch changes include:
>> * changed the logic slightly by setting recv_immediately(new variable), if this variable is set the main apply
workerloop will not wait in this case. 
>> * setting the relation state to ready immediately if there are no more incremental changes to be synced.
>> * receive the incremental changes if applicable and set the relation state to ready without waiting.
>> * reuse the worker if the worker is free before trying to start a new table sync worker
>> * restarting the tablesync worker only after wal_retrieve_retry_interval
>>
>> ~
>>
>> FWIW, we just wanted to share with you the performance measurements seen using this design#2 patch set:
>>
>> ======
>>
>> RESULTS (not busy tests)
>>
>> ------
>> 10 empty tables
>>                 2w      4w      8w      16w
>> HEAD:           125     119     140     133
>> HEAD+v28*:      92      93      123     134
>> %improvement:   27%     22%     12%     -1%
>> ------
>> 100 empty tables
>>                 2w      4w      8w      16w
>> HEAD:           1037    843     1109    1155
>> HEAD+v28*:      591     625     2616    2569
>> %improvement:   43%     26%     -136%   -122%
>> ------
>> 1000 empty tables
>>                 2w      4w      8w      16w
>> HEAD:           15874   10047   9919    10338
>> HEAD+v28*:      33673   12199   9094    9896
>> %improvement:   -112%   -21%    8%      4%
>> ------
>> 2000 empty tables
>>                 2w      4w      8w      16w
>> HEAD:           45266   24216   19395   19820
>> HEAD+v28*:      88043   21550   21668   22607
>> %improvement:  -95%     11%     -12%    -14%
>>
>> ~~~
>>
>> Note - the results were varying quite a lot in comparison to the HEAD
>> e.g. HEAD results are very consistent, but the v28* results observed are not
>> HEAD 1000 (2w): 15861, 15777, 16007, 15950, 15886, 15740, 15846, 15740, 15908, 15940
>> v28* 1000 (2w):  34214, 13679, 8792, 33289, 31976, 56071, 57042, 56163, 34058, 11969
>>
>> ------
>> Kind Regards,
>> Peter Smith.
>> Fujitsu Australia