Обсуждение: PGbackrest very slow!
[global]
backup-standby=y
repo1-path=/postgresql/backup
repo1-retention-full=2
start-fast=y
[PG]
pg1-path=/postgresql/data
pg2-host=node2
pg2-host-user=postgres
pg2-path=/postgresql/data
pg3-host=node3
pg3-host-user=postgres
pg3-path=/postgresql/data
=================================================
I run the backuplike this
/usr/bin/pgbackrest --log-level-console=detail --stanza=PG --type=full backup
The backup of 50GB took 40 min.
It is not a network issue. I tested to transfer 2GB between two host and it took 4 sec.
I think backup-standby=y slows down the backup out of all minds.
Br Pär
Hi Pär, On Thu, Oct 23, 2025 at 10:59 AM Pär Mattsson <par.x.mattsson@gmail.com> wrote: > > Hi! > I have 3 node postgres cluster with repmgr and pgpool. > > Pgbackrest is configed like this > We use pgbackrest with this config > ======================================= It is a quite basic config, > It is not a network issue. I tested to transfer 2GB between two host and it took 4 sec. > > > I think backup-standby=y slows down the backup out of all minds. > so if you are sure it is not a network issue, your guess could be right. You can try to compare with backup from primary and if it is faster, it could be archiving process on standby which slows you down. You can try to improve it with more processes [1] and with better zst compression [2]. Async archiving [3] could also help > > Br Pär > > [1] https://pgbackrest.org/configuration.html#section-general/option-process-max [2] https://pgbackrest.org/configuration.html#section-general/option-compress [3] https://pgbackrest.org/configuration.html#section-archive/option-archive-async best regards, Ilya Kosmodemiansky CEO, Founder Data Egret GmbH Your remote PostgreSQL DBA team T.: +49 6821 919 3297 ik@dataegret.com
> On Oct 23, 2025, at 2:59 AM, Pär Mattsson <par.x.mattsson@gmail.com> wrote: > > ... > The backup of 50GB took 40 min. > It is not a network issue. I tested to transfer 2GB between two host and it took 4 sec. > > I think backup-standby=y slows down the backup out of all minds. > > Br Pär pgbackrest defaults to gzip compression, which is slow--try lz4 and see what happens
Skickat: Thursday, October 23, 2025 3:14:20 PM
Till: Pär Mattsson <par.x.mattsson@gmail.com>
Kopia: pgsql-admin <pgsql-admin@postgresql.org>
Ämne: Re: PGbackrest very slow!
>
> ...
> The backup of 50GB took 40 min.
> It is not a network issue. I tested to transfer 2GB between two host and it took 4 sec.
>
> I think backup-standby=y slows down the backup out of all minds.
>
> Br Pär
pgbackrest defaults to gzip compression, which is slow--try lz4 and see what happens
> On Oct 23, 2025, at 2:59 AM, Pär Mattsson <par.x.mattsson@gmail.com> wrote:
>
> ...
> The backup of 50GB took 40 min.
> It is not a network issue. I tested to transfer 2GB between two host and it took 4 sec.
>
> I think backup-standby=y slows down the backup out of all minds.
>
> Br Pär
pgbackrest defaults to gzip compression, which is slow--try lz4 and see what happens
On Thu, Oct 23, 2025 at 10:21:37AM -0400, Ron Johnson wrote: > On Thu, Oct 23, 2025 at 9:14 AM Scott Ribe <scott_ribe@elevated-dev.com> > wrote: > > > > On Oct 23, 2025, at 2:59 AM, Pär Mattsson <par.x.mattsson@gmail.com> > > wrote: > > > > > > ... > > > The backup of 50GB took 40 min. > > > It is not a network issue. I tested to transfer 2GB between two host and > > it took 4 sec. > > > > > > I think backup-standby=y slows down the backup out of all minds. > > > > > > Br Pär > > > > pgbackrest defaults to gzip compression, which is slow--try lz4 and see > > what happens > > > > lz4 is faster, but not 1440x faster. > But it might be 24x faster, using 2s/GB. :) --Ken
On Thu, 23 Oct 2025 at 19:22, Ron Johnson<ronljohnsonjr@gmail.com> wrote:On Thu, Oct 23, 2025 at 9:14 AM Scott Ribe <scott_ribe@elevated-dev.com> wrote:> On Oct 23, 2025, at 2:59 AM, Pär Mattsson <par.x.mattsson@gmail.com> wrote:
>
> ...
> The backup of 50GB took 40 min.
> It is not a network issue. I tested to transfer 2GB between two host and it took 4 sec.
>
> I think backup-standby=y slows down the backup out of all minds.
>
> Br Pär
pgbackrest defaults to gzip compression, which is slow--try lz4 and see what happenslz4 is faster, but not 1440x faster.--Death to <Redacted>, and butter sauce.Don't boil me, I'm still alive.<Redacted> lobster!
Skickat: Thursday, October 23, 2025 7:54:40 PM
Till: ronljohnsonjr@gmail.com <ronljohnsonjr@gmail.com>; Pgsql-admin <pgsql-admin@lists.postgresql.org>
Ämne: Re: PGbackrest very slow!
On Thu, 23 Oct 2025 at 19:22, Ron Johnson<ronljohnsonjr@gmail.com> wrote:On Thu, Oct 23, 2025 at 9:14 AM Scott Ribe <scott_ribe@elevated-dev.com> wrote:> On Oct 23, 2025, at 2:59 AM, Pär Mattsson <par.x.mattsson@gmail.com> wrote:
>
> ...
> The backup of 50GB took 40 min.
> It is not a network issue. I tested to transfer 2GB between two host and it took 4 sec.
>
> I think backup-standby=y slows down the backup out of all minds.
>
> Br Pär
pgbackrest defaults to gzip compression, which is slow--try lz4 and see what happenslz4 is faster, but not 1440x faster.--Death to <Redacted>, and butter sauce.Don't boil me, I'm still alive.<Redacted> lobster!
compress-type=lz4
compress-level= -- based on the CPU CORE
Thanks & Regards
Dinesh Nair
Sent: Thursday, October 23, 2025 7:21 PM
To: Scott Ribe <scott_ribe@elevated-dev.com>
Cc: pgsql-admin <pgsql-admin@postgresql.org>
Subject: Re: PGbackrest very slow!
You don't often get email from par.x.mattsson@gmail.com. Learn why this is important |
Skickat: Thursday, October 23, 2025 3:14:20 PM
Till: Pär Mattsson <par.x.mattsson@gmail.com>
Kopia: pgsql-admin <pgsql-admin@postgresql.org>
Ämne: Re: PGbackrest very slow!
>
> ...
> The backup of 50GB took 40 min.
> It is not a network issue. I tested to transfer 2GB between two host and it took 4 sec.
>
> I think backup-standby=y slows down the backup out of all minds.
>
> Br Pär
pgbackrest defaults to gzip compression, which is slow--try lz4 and see what happens
[global]
repo1-path=/postgresql/backup
repo1-retention-full=2
start-fast=y
process-max=4
When I deleted backup-standby=y and increased process-max=4 , then the backup took 7 min of 50GB.
I will test all your tips! Thanks alot!
Br Pär
Hi,Kindly try changing below mentioned parameters.Parallel-process= -- based on the CPU CORE
compress-type=lz4
compress-level= -- based on the CPU COREThanks & Regards
Dinesh Nair
From: Pär Mattsson <par.x.mattsson@gmail.com>
Sent: Thursday, October 23, 2025 7:21 PM
To: Scott Ribe <scott_ribe@elevated-dev.com>
Cc: pgsql-admin <pgsql-admin@postgresql.org>
Subject: Re: PGbackrest very slow!
You don't often get email from par.x.mattsson@gmail.com. Learn why this is importantCaution: This email was sent from an external source. Please verify the sender’s identity before clicking links or opening attachments.Thanks I will test that!Br PärMvh PärFrån: Scott Ribe <scott_ribe@elevated-dev.com>
Skickat: Thursday, October 23, 2025 3:14:20 PM
Till: Pär Mattsson <par.x.mattsson@gmail.com>
Kopia: pgsql-admin <pgsql-admin@postgresql.org>
Ämne: Re: PGbackrest very slow!> On Oct 23, 2025, at 2:59 AM, Pär Mattsson <par.x.mattsson@gmail.com> wrote:
>
> ...
> The backup of 50GB took 40 min.
> It is not a network issue. I tested to transfer 2GB between two host and it took 4 sec.
>
> I think backup-standby=y slows down the backup out of all minds.
>
> Br Pär
pgbackrest defaults to gzip compression, which is slow--try lz4 and see what happens