Hi Peter,
Peter Smith <
smithpb2250@gmail.com>, 11 Tem 2023 Sal, 05:59 tarihinde şunu yazdı:
> Even if patches 0003 and 0002 are to be combined, I think that should
> not happen until after the "reuse" design is confirmed which way is
> best.
>
> e.g. IMO it might be easier to compare the different PoC designs for
> patch 0002 if there is no extra logic involved.
>
> PoC design#1 -- each tablesync decides for itself what to do next
> after it finishes
> PoC design#2 -- reuse tablesync using a "pool" of available workers
Right. I made a patch 0003 to change 0002 so that tables will be assigned to sync workers by apply worker.
It's a rough POC and ignores some edge cases. But this is what I think how apply worker would take the responsibility of table assignments. Hope the implementation makes sense and I'm not missing anything that may cause degraded perforrmance.
PoC design#1 --> apply only patch 0001 and 0002
PoC design#2 --> apply all patches, 0001, 0002 and 0003
Here are some quick numbers with 100 empty tables.
+--------------+----------------+----------------+----------------+
| | 2 sync workers | 4 sync workers | 8 sync workers |
+--------------+----------------+----------------+----------------+
| POC design#1 | 1909.873 ms | 986.261 ms | 552.404 ms |
+--------------+----------------+----------------+----------------+
| POC design#2 | 4962.208 ms | 1240.503 ms | 1165.405 ms |
+--------------+----------------+----------------+----------------+
| master | 2666.008 ms | 1462.012 ms | 986.848 ms |
+--------------+----------------+----------------+----------------+
Seems like design#1 is better than both design#2 and master overall. It's surprising to see that even master beats design#2 in some cases though. Not sure if that is expected or there are some places to improve design#2 even more.
What do you think?
PS: I only attached the related patches and not the whole patch set. 0001 and 0002 may contain some of your earlier reviews, but I'll send a proper updated set soon.
Thanks,
--