Обсуждение: pgbackrest archive.info got deleted unexpected

@font-face {font-family:"Cambria Math"; panose-1:2 4 5 3 5 4 6 3 2 4;}@font-face {font-family:DengXian; panose-1:2 1 6 0 3 1 1 1 1 1;}@font-face {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4;}@font-face {font-family:"\@DengXian"; panose-1:2 1 6 0 3 1 1 1 1 1;}p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0in; font-size:11.0pt; font-family:"Calibri",sans-serif;}span.EmailStyle17 {mso-style-type:personal-compose; font-family:"Calibri",sans-serif; color:windowtext;}.MsoChpDefault {mso-style-type:export-only; font-family:"Calibri",sans-serif;}div.WordSection1 {page:WordSection1;}

Hi,

   We use pgbackrest 2.34 on RHEL8.4 to backup PGv13 database to AWS S3 buckets , we found after some time successful backup , suddenly got below error:

2022-07-23 10:30:04.743 P00   INFO: archive-push:async command begin 2.38: [/pgdata/pg_wal] --archive-async --archive-push-queue-max=10240MB --compress-type=none --exec-id=813450-3a783f75 --log-level-console=off --log-level-stderr=off --pg1-path=/pgdata --process-max=2 --repo1-path=/pgbackup --repo1-s3-bucket=xxxxxxxxx --repo1-s3-endpoint=s3-fips.us-west-2.amazonaws.com --repo1-s3-key-type=auto --repo1-s3-region=us-west-2 --repo1-type=s3 --spool-path=/pgarchive/pgbackrest --stanza=xxxxxxx

2022-07-23 10:30:04.744 P00   INFO: push 1 WAL file(s) to archive: 00000001000000DB000000A4

2022-07-23 10:30:04.907 P00  ERROR: [103]: unable to find a valid repository:

                                    repo1: [FileMissingError] unable to load info file '/pgbackup/archive/xxxxxx/archive.info' or '/pgbackup/archive/xxxxxx/archive.info.copy':

                                    FileMissingError: unable to open missing file '/pgbackup/archive/xxxxxx/archive.info' for read

                                    FileMissingError: unable to open missing file '/pgbackup/archive/xxxxxx/archive.info.copy' for read

                                    HINT: archive.info cannot be opened but is required to push/get WAL segments.

                                    HINT: is archive_command configured correctly in postgresql.conf?

                                    HINT: has a stanza-create been performed?

                                    HINT: use --no-archive-check to disable archive checks during backup if you have an alternate archiving scheme.

 

Then we login to check both archive.info and archive.info.copy got disappeared in S3 bucket , we do not have job or command to manually delete that file, and we do not recreate standza either, not sure why suddenly missed the archive.info.  without the archive.info, we have to recreate pgbackrest stanza repo again, but all old backup pieces got invalid too.

 

Could you help figure out how that happened ? we faced this issue several times .

On 8/3/22 05:23, James Pang (chaolpan) wrote:
> 
>     We use pgbackrest 2.34 on RHEL8.4 to backup PGv13 database to AWS S3 
> buckets , we found after some time successful backup , suddenly got 
> below error:
> 
> 2022-07-23 10:30:04.743 P00   INFO: archive-push:async command begin 
> 2.38: [/pgdata/pg_wal] --archive-async --archive-push-queue-max=10240MB 
> --compress-type=none --exec-id=813450-3a783f75 --log-level-console=off 
> --log-level-stderr=off --pg1-path=/pgdata --process-max=2 
> --repo1-path=/pgbackup --repo1-s3-bucket=xxxxxxxxx 
> --repo1-s3-endpoint=s3-fips.us-west-2.amazonaws.com 
> --repo1-s3-key-type=auto --repo1-s3-region=us-west-2 --repo1-type=s3 
> --spool-path=/pgarchive/pgbackrest --stanza=xxxxxxx
> 
> 2022-07-23 10:30:04.744 P00   INFO: push 1 WAL file(s) to archive: 
> 00000001000000DB000000A4
> 
> 2022-07-23 10:30:04.907 P00  ERROR: [103]: unable to find a valid 
> repository:
> 
>                                      repo1: [FileMissingError] unable to 
> load info file '/pgbackup/archive/xxxxxx/archive.info' or 
> '/pgbackup/archive/xxxxxx/archive.info.copy':
> 
>                                      FileMissingError: unable to open 
> missing file '/pgbackup/archive/xxxxxx/archive.info' for read
> 
>                                      FileMissingError: unable to open 
> missing file '/pgbackup/archive/xxxxxx/archive.info.copy' for read
> 
>                                      HINT: archive.info cannot be opened 
> but is required to push/get WAL segments.
> 
>                                      HINT: is archive_command configured 
> correctly in postgresql.conf?
> 
>                                      HINT: has a stanza-create been 
> performed?
> 
>                                      HINT: use --no-archive-check to 
> disable archive checks during backup if you have an alternate archiving 
> scheme.
> 
> Then we login to check both archive.info and archive.info.copy got 
> disappeared in S3 bucket , we do not have job or command to manually 
> delete that file, and we do not recreate standza either, not sure why 
> suddenly missed the archive.info. without the archive.info, we have to 
> recreate pgbackrest stanza repo again, but all old backup pieces got 
> invalid too.
> 
> Could you help figure out how that happened ? we faced this issue 
> several times .

These files do not get updated after stanza-create. Do you have any kind 
of auto-expiration of old files in the bucket? e.g., 
https://aws.amazon.com/blogs/aws/amazon-s3-object-expiration.

Regards,
-David