Re: [WIP] Allow pg_upgrade to copy segments of the same relfilenode in parallel

Поиск
Список
Период
Сортировка
От Andres Freund
Тема Re: [WIP] Allow pg_upgrade to copy segments of the same relfilenode in parallel
Дата
Msg-id 20220322003431.tk7ec5hm336kkbsc@alap3.anarazel.de
обсуждение исходный текст
Ответ на [WIP] Allow pg_upgrade to copy segments of the same relfilenode in parallel  (Jaime Casanova <jcasanov@systemguards.com.ec>)
Ответы Re: [WIP] Allow pg_upgrade to copy segments of the same relfilenode in parallel  (Jaime Casanova <jcasanov@systemguards.com.ec>)
Список pgsql-hackers
Hi,

On 2022-02-01 21:57:00 -0500, Jaime Casanova wrote:
> This patch adds a new option (-J num, --jobs-per-disk=num) in 
> pg_upgrade to speed up copy mode. This generates upto ${num} 
> processes per tablespace to copy segments of the same relfilenode 
> in parallel.
> 
> This can help when you have many multi gigabyte tables (each segment 
> is 1GB by default) in different tablespaces (each tablespace in a 
> different disk) and multiple processors.
> 
> In a customer's database (~20Tb) it went down from 6h to 4h 45min.
> 
> It lacks documentation and I need help with WIN32 part of it, I created
> this new mail to put the patch on the next commitfest.

The patch currently fails on cfbot due to warnings, likely related due to the
win32 issue: https://cirrus-ci.com/task/4566046517493760?logs=mingw_cross_warning#L388

As it's a new patch submitted to the last CF, hasn't gotten any review yet and
misses some platform support, it seems like there's no chance it can make it
into 15?

Greetings,

Andres Freund



В списке pgsql-hackers по дате отправления:

Предыдущее
От: Tomas Vondra
Дата:
Сообщение: Re: PATCH: generate fractional cheapest paths in generate_orderedappend_path
Следующее
От: Andres Freund
Дата:
Сообщение: Re: [PATCH] Proof of concept for GUC improvements