Re: An idea for parallelizing COPY within one backend

Поиск
Список
Период
Сортировка
От Florian G. Pflug
Тема Re: An idea for parallelizing COPY within one backend
Дата
Msg-id 47C56F7E.3030301@phlo.org
обсуждение исходный текст
Ответ на Re: An idea for parallelizing COPY within one backend  (Dimitri Fontaine <dfontaine@hi-media.com>)
Ответы Re: An idea for parallelizing COPY within one backend  ("A.M." <agentm@themactionfaction.com>)
Список pgsql-hackers
Dimitri Fontaine wrote:
> Of course, the backends still have to parse the input given by pgloader, which 
> only pre-processes data. I'm not sure having the client prepare the data some 
> more (binary format or whatever) is a wise idea, as you mentionned and wrt 
> Tom's follow-up. But maybe I'm all wrong, so I'm all ears!

As far as I understand, pgloader starts N threads or processes that open 
up N individual connections to the server. In that case, moving then 
text->binary conversion from the backend into the loader won't give any
additional performace I'd say.

The reason that I'd love some within-one-backend solution is that I'd 
allow you to utilize more than one CPU for a restore within a *single* 
transaction. This is something that a client-side solution won't be able 
to deliver, unless major changes to the architecture of postgres happen 
first...

regards, Florian Pflug



В списке pgsql-hackers по дате отправления:

Предыдущее
От: Alvaro Herrera
Дата:
Сообщение: Re: Required make version
Следующее
От: cristianopintado@gmail.com
Дата:
Сообщение: select avanced