Re: [POC] Fast COPY FROM command for the table with foreign partitions

On 2/9/21 9:35 AM, tsunakawa.takay@fujitsu.com wrote:
>     From: tsunakawa.takay@fujitsu.com <tsunakawa.takay@fujitsu.com>
>> From: Andrey Lepikhov <a.lepikhov@postgrespro.ru>
>> Also, I might defer working on the extended part (v9 0003 and 0004) and further
>> separate them in a different thread, if it seems to take longer.
> 
> I reviewed them but haven't rebased them (it seems to take more labor.)
> Andrey-san, could you tell us:
> 
> * Why is a separate FDW connection established for each COPY?  To avoid using the same FDW connection for multiple
foreigntable partitions in a single COPY run?
 
With separate connection you can init a 'COPY FROM' session for each 
foreign partition just one time on partition initialization.
> 
> * In what kind of test did you get 2-4x performance gain?  COPY into many foreign table partitions where the input
rowsare ordered randomly enough that many rows don't accumulate in the COPY buffer?
 
I used 'INSERT INTO .. SELECT * FROM generate_series(1, N)' to generate 
test data and HASH partitioning to avoid skews.

-- 
regards,
Andrey Lepikhov
Postgres Professional