Обсуждение: database restoration problem- data became incorrect caused by incorrect date in the server

Поиск
Список
Период
Сортировка

database restoration problem- data became incorrect caused by incorrect date in the server

От
Khangelani Gama
Дата:

Hi all

 

I am a junior DBA:

 

Please give an advice on how I should solve the following problem,

 

We are still using 7.3.4 PostgreSQL but will be upgrading very soon, database is running on CentOS 5 O/S .

 

Problem: - it’s about the wrong data in the system:

 

Everything was fine until a certain process(for financial period processing) that normally generates many transactions in the database took place when the server lost the date. The date changed from 07-04-2010 to 15-08-2003…hence the process ran until it finished which leaded to the generation of the wrong data.

 

When we thought of restoring from a successful dump file(backup file) for the 07-04-2010 so that we could restore it to have everything before the date changed, we were not lucky because the only latest successful dump file was on the 03/04/2010.

 

 

Question:

How can we removed everything that got processed on the wrong date and have everything restored so that it’s in the correct state as at 07/04/2010… I tried to search for a solution but I can’t find anything.

 

 

 

 

Your help would be much appreciated

 

Thanks

Khangelani

 

 

 

 

 

Confidentiality Notice:http://ucs.co.za/conf.html

 

 



The contents of and attachments to this e-mail are intended for the addressee only, and may contain the confidential information of UCS Group and/or its subsidiaries. Any review, use or dissemination thereof by anyone other than the intended addressee is prohibited. If you are not the intended addressee please notify the writer immediately and destroy the e-mail. UCS Group Limited and its subsidiaries distance themselves from and accept no liability for unauthorised use of their e-mail facilities or e-mails sent other than strictly for business purposes.

Re: database restoration problem- data became incorrect caused by incorrect date in the server

От
Khangelani Gama
Дата:

To add on to previous comments:

 

There is only one database in the server and the backup server also has incorrect data because all the wrong data replicated to the backup server.

 

 

 

 

From: pgsql-admin-owner@postgresql.org [mailto:pgsql-admin-owner@postgresql.org] On Behalf Of Khangelani Gama
Sent: Friday, April 09, 2010 2:22 PM
To: pgsql-admin@postgresql.org
Subject: [ADMIN] database restoration problem- data became incorrect caused by incorrect date in the server

 

Hi all

 

I am a junior DBA:

 

Please give an advice on how I should solve the following problem,

 

We are still using 7.3.4 PostgreSQL but will be upgrading very soon, database is running on CentOS 5 O/S .

 

Problem: - it’s about the wrong data in the system:

 

Everything was fine until a certain process(for financial period processing) that normally generates many transactions in the database took place when the server lost the date. The date changed from 07-04-2010 to 15-08-2003…hence the process ran until it finished which leaded to the generation of the wrong data.

 

When we thought of restoring from a successful dump file(backup file) for the 07-04-2010 so that we could restore it to have everything before the date changed, we were not lucky because the only latest successful dump file was on the 03/04/2010.

 

 

Question:

How can we removed everything that got processed on the wrong date and have everything restored so that it’s in the correct state as at 07/04/2010… I tried to search for a solution but I can’t find anything.

 

 

 

 

Your help would be much appreciated

 

Thanks

Khangelani

 

 

 

 

 

Confidentiality Notice:http://ucs.co.za/conf.html

 

 

 


The contents of and attachments to this e-mail are intended for the addressee only, and may contain the confidential information of UCS Group and/or its subsidiaries. Any review, use or dissemination thereof by anyone other than the intended addressee is prohibited. If you are not the intended addressee please notify the writer immediately and destroy the e-mail. UCS Group Limited and its subsidiaries distance themselves from and accept no liability for unauthorised use of their e-mail facilities or e-mails sent other than strictly for business purposes.



The contents of and attachments to this e-mail are intended for the addressee only, and may contain the confidential information of UCS Group and/or its subsidiaries. Any review, use or dissemination thereof by anyone other than the intended addressee is prohibited. If you are not the intended addressee please notify the writer immediately and destroy the e-mail. UCS Group Limited and its subsidiaries distance themselves from and accept no liability for unauthorised use of their e-mail facilities or e-mails sent other than strictly for business purposes.

Re: database restoration problem- data became incorrect caused by incorrect date in the server

От
"Kevin Grittner"
Дата:
Khangelani Gama <Khangelani.Gama@ucs-software.co.za> wrote:

> We are still using 7.3.4 PostgreSQL

Do you really mean that, or is that a typo?

What sort of backups do you have?  (pg_dump, file system copy, etc.)

-Kevin

Re: Re: database restoration problem- data became incorrect caused by incorrect date in the server

От
Khangelani Gama
Дата:
Hi

I really mean it....the backups we have uses pg_dump command. The automatic backup dumps file happen daily and gets
savedin a certain directory 

But the issue now is that we don't have any backup file for us to restore the database but exclude every data that got
createwhen the date in server was incorrect. 

Thanks



-----Original Message-----
From: pgsql-admin-owner@postgresql.org [mailto:pgsql-admin-owner@postgresql.org] On Behalf Of Kevin Grittner
Sent: Friday, April 09, 2010 4:14 PM
To: pgsql-admin@postgresql.org; Khangelani Gama
Subject: [ADMIN] Re: database restoration problem- data became incorrect caused by incorrect date in the server

Khangelani Gama <Khangelani.Gama@ucs-software.co.za> wrote:

> We are still using 7.3.4 PostgreSQL

Do you really mean that, or is that a typo?

What sort of backups do you have?  (pg_dump, file system copy, etc.)

-Kevin

--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin

The contents of and attachments to this e-mail are intended for the addressee only, and may contain the confidential
informationof UCS Group and/or its subsidiaries.  Any review, use or dissemination thereof by anyone other than the
intendedaddressee is prohibited.  If you are not the intended addressee please notify the writer immediately and
destroythe e-mail.  UCS Group Limited and its subsidiaries distance themselves from and accept no liability for
unauthoriseduse of their e-mail facilities or e-mails sent other than strictly for business purposes. 

Re: Re: database restoration problem- data became incorrect caused by incorrect date in the server

От
"Kevin Grittner"
Дата:
Khangelani Gama <Khangelani.Gama@ucs-software.co.za> wrote:

> the backups we have uses pg_dump command. The automatic
> backup dumps file happen daily and gets saved in a certain
> directory

No off-site copies or archives to fall back on?  If not, my ideas
are:

(1)  I hope you made a file copy as soon as you found the problem.
If not, stop the database and make one immediately.  If the old
versions of the rows haven't yet been vacuumed away, it might be
possible to fish some or all of the corrupted data out of the
database.  This is very tricky work, though, and you might want to
contract with an expert.  (I don't do that anymore, but there are
several good companies with people who do.)

(2)  Are there any source documents you can fall back on to
reconstruct the data?

(3)  Is there any way to "reverse" the particular calculations or
operations which mangled the data?

Once you do get upgraded, you should look into Point In Time
Recovery (PITR) backups.  Had you been running this, you could have
replayed the transactions right up to the moment of corruption.
Well, if you kept a base backup from before the corruption and all
the WAL files since that base backup.

-Kevin

Re: Re: database restoration problem- data became incorrect caused by incorrect date in the server

От
Khangelani Gama
Дата:
Thanks for the ideas

-----Original Message-----
From: Kevin Grittner [mailto:Kevin.Grittner@wicourts.gov]
Sent: Friday, April 09, 2010 8:03 PM
To: pgsql-admin@postgresql.org; Khangelani Gama
Subject: RE: [ADMIN] Re: database restoration problem- data became incorrect caused by incorrect date in the server

Khangelani Gama <Khangelani.Gama@ucs-software.co.za> wrote:

> the backups we have uses pg_dump command. The automatic
> backup dumps file happen daily and gets saved in a certain
> directory

No off-site copies or archives to fall back on?  If not, my ideas
are:

(1)  I hope you made a file copy as soon as you found the problem.
If not, stop the database and make one immediately.  If the old
versions of the rows haven't yet been vacuumed away, it might be
possible to fish some or all of the corrupted data out of the
database.  This is very tricky work, though, and you might want to
contract with an expert.  (I don't do that anymore, but there are
several good companies with people who do.)

(2)  Are there any source documents you can fall back on to
reconstruct the data?

(3)  Is there any way to "reverse" the particular calculations or
operations which mangled the data?

Once you do get upgraded, you should look into Point In Time
Recovery (PITR) backups.  Had you been running this, you could have
replayed the transactions right up to the moment of corruption.
Well, if you kept a base backup from before the corruption and all
the WAL files since that base backup.

-Kevin

The contents of and attachments to this e-mail are intended for the addressee only, and may contain the confidential
informationof UCS Group and/or its subsidiaries.  Any review, use or dissemination thereof by anyone other than the
intendedaddressee is prohibited.  If you are not the intended addressee please notify the writer immediately and
destroythe e-mail.  UCS Group Limited and its subsidiaries distance themselves from and accept no liability for
unauthoriseduse of their e-mail facilities or e-mails sent other than strictly for business purposes.