Re: CRITICAL HELP NEEDED! DEAD DB!

Cott Lang <cott@internetstaff.com> writes:
> Sep 24 10:22:37 snafu postgres[18306]: [2-1] LOG:  database system was
> interrupted while in recovery at 2004-09-24 10:21:41 MST
> Sep 24 10:22:37 snafu postgres[18306]: [2-2] HINT:  This probably means
> that some data is corrupted and you will have to use the last backup for
> recovery.
> Sep 24 10:22:37 snafu postgres[18306]: [3-1] LOG:  checkpoint record is
> at 9A/C2022368
> Sep 24 10:22:37 snafu postgres[18306]: [4-1] LOG:  redo record is at
> 9A/C2022368; undo record is at 0/0; shutdown FALSE
> Sep 24 10:22:37 snafu postgres[18306]: [5-1] LOG:  next transaction ID:
> 197841225; next OID: 715436086
> Sep 24 10:22:37 snafu postgres[18306]: [6-1] LOG:  database system was
> not properly shut down; automatic recovery in progress
> Sep 24 10:22:37 snafu postgres[18306]: [7-1] LOG:  redo starts at
> 9A/C20223B0
> Sep 24 10:22:37 snafu postgres[18306]: [8-1] PANIC:  btree_insert_redo:
> failed to add item
> Sep 24 10:22:37 snafu postgres[18299]: [2-1] LOG:  startup process (PID
> 18306) was terminated by signal 6
> Sep 24 10:22:37 snafu postgres[18299]: [3-1] LOG:  aborting startup due
> to startup process failure

> Any suggestions to recover?!  I'm dead in the water!  Please!!!

I think your only chance is pg_resetxlog.  Be aware that you won't
necessarily have a consistent database afterwards --- in particular,
whichever index that failure is about is certainly broken.  I'd
recommend a dump and reload, plus as much manual verification of data
consistency as you can manage.

How did you get into this state, anyway?
        regards, tom lane