Max write throughput for single COPY

Поиск
Список
Период
Сортировка
От Joe Wildish
Тема Max write throughput for single COPY
Дата
Msg-id 140B5BD1-A9B4-4D4D-B6A9-17185A9BA32C@lateraljoin.com
обсуждение исходный текст
Список pgsql-performance
Hello,

I am trying to speed up the initial logical replication sync process.  The database being replicated is dominated by
onetable that is 750GB (heap).  The process quickly boils down to a single COPY writing into the subscriber.  We have
droppedall indexes and key constraints in the subscriber but we are seeing the sync take >24 hours before we have to
killthe process (to avoid so much WAL being reserved on the publisher). 

I haven’t done a huge amount of performance tuning at the Linux level before as I’m used to working with cloud-managed
installationswhere you obviously don’t have access to the underlying host.  However, in this case, the subscriber
instanceis not a cloud-managed one. 

Can anyone give comment on what might be a reasonable throughput in MB/s for a single COPY operation?

The material I’ve read on I/O talks about saturating the device … I’m pretty sure that a single COPY operation is not
capableof doing this.  It’s therefore one thing to see the advertised top-line figures about IOPS and throughput, vs
whatyou can actually do with the single COPY.  I’d be interested in hearing what other people are able to get as a
throughputfigure for COPY. 

-Joe





В списке pgsql-performance по дате отправления:

Предыдущее
От: MichaelDBA
Дата:
Сообщение: Re: How to analyze of short but heavy intermittent slowdown on BIND on production database (or BIND vs log_lock_waits)
Следующее
От: spiral
Дата:
Сообщение: Advice on best way to store a large amount of data in postgresql