Re: An idea for parallelizing COPY within one backend

On Feb 27, 2008, at 9:11 AM, Florian G. Pflug wrote:

> Dimitri Fontaine wrote:
>> Of course, the backends still have to parse the input given by  
>> pgloader, which only pre-processes data. I'm not sure having the  
>> client prepare the data some more (binary format or whatever) is a  
>> wise idea, as you mentionned and wrt Tom's follow-up. But maybe I'm  
>> all wrong, so I'm all ears!
>
> As far as I understand, pgloader starts N threads or processes that  
> open up N individual connections to the server. In that case, moving  
> then text->binary conversion from the backend into the loader won't  
> give any
> additional performace I'd say.
>
> The reason that I'd love some within-one-backend solution is that  
> I'd allow you to utilize more than one CPU for a restore within a  
> *single* transaction. This is something that a client-side solution  
> won't be able to deliver, unless major changes to the architecture  
> of postgres happen first...

It seems like multiple backends should be able to take advantage of  
2PC for transaction safety.

Cheers,
M