Re: [POC] Fast COPY FROM command for the table with foreignpartitions

Thank you for the answer,

02.06.2020 05:02, Etsuro Fujita пишет:
> I think I also thought something similar to this before [1].  Will take a look.

> [1] https://www.postgresql.org/message-id/23990375-45a6-5823-b0aa-a6a7a6a957f0%40lab.ntt.co.jp
> 
I have looked into the thread.
My first version of the patch was like your idea. But when developing 
the “COPY FROM” code, the following features were discovered:
1. Two or more partitions can be placed at the same node. We need to 
finish COPY into one partition before start COPY into another partition 
at the same node.
2. On any error we need to send EOF to all started "COPY .. FROM STDIN" 
operations. Otherwise FDW can't cancel operation.

Hiding the COPY code under the buffers management machinery allows us to 
generalize buffers machinery, execute one COPY operation on each buffer 
and simplify error handling.

As i understand, main idea of the thread, mentioned by you, is to add 
"COPY FROM" support without changes in FDW API.
It is possible to remove BeginForeignCopy() and EndForeignCopy() from 
the patch. But it is not trivial to change ExecForeignInsert() for the 
COPY purposes.
All that I can offer in this place now is to introduce one new 
ExecForeignBulkInsert(buf) routine that will execute single "COPY FROM 
STDIN" operation, send tuples and close the operation. We can use the 
ExecForeignInsert() routine for each buffer tuple if 
ExecForeignBulkInsert() is not supported.

One of main questions here is to use COPY TO machinery for serializing a 
tuple. It is needed (if you will take a look into the patch) to 
transform the CopyTo() routine to an iterative representation: 
start/next/finish. May it be acceptable?

In the attachment there is a patch with the correction of a stupid error.

-- 
Andrey Lepikhov
Postgres Professional
https://postgrespro.com
The Russian Postgres Company