Re: Checkpoint Err on Startup of Rsynced System

Jeff Janes,

Ok.  I checked this further and just found that the pg_xlog area is
symlinked to another area.. and indeed that other area was not being
rsynced (!) and I thought it was.  So, I just fixed this, re-ran it and
now it is working.  Now I believe I have a stable postgres running on M2.

So, thanks Jeff for mentioning pg_xlog.  Thanks to others as well for
your input.

--Jim Longwill

On 06/01/2016 08:44 AM, Jeff Janes wrote:
> On Tue, May 31, 2016 at 10:13 AM, Jim Longwill <longwill@psmfc.org> wrote:
>> I am trying to setup a 2nd, identical, db server (M2) for development and
>> I've run into a problem with starting up the 2nd Postgres installation.
>>
>> Here's what I've done:
>>    1) did a 'clone' of 1st (production) machine M1 (so both machines on Cent
>> OS 7.2)
>>    2) setup an rsync operation, did a complete 'rsync' from M1 to M2
>>    3) did a final 'CHECKPOINT' command on M1 postgres
>>    4) shutdown postgres on M1 with 'pg_ctl stop'
>>    5) did final 'rsync' operation  (then restarted postgres on M1 with
>> 'pg_ctl start')
>>    6) tried to startup postgres on M2
>>
>> It won't start, & in the log file gives the error message:
>> ...
>> < 2016-05-31 09:02:52.337 PDT >LOG:  invalid primary checkpoint record
>> < 2016-05-31 09:02:52.337 PDT >LOG:  invalid secondary checkpoint record
>> < 2016-05-31 09:02:52.337 PDT >PANIC:  could not locate a valid checkpoint
>> record
>> < 2016-05-31 09:02:53.184 PDT >LOG:  startup process (PID 26680) was
>> terminated by signal 6: Aborted
>> < 2016-05-31 09:02:53.184 PDT >LOG:  aborting startup due to startup process
>> failure
>>
>> I've tried several times to do this but always get this result.  So, do I
>> need to do a new 'initdb..' operation on machine M2 + restore from M1
>> backups?  Or is there another way to fix this?
>
> It sounds like you did not include pg_xlog in your rsync.  What you
> have done is basically a cold backup.  Cold backups must include
> pg_xlog, at least if you want them to work without WAL archival.  If
> you look farther up in the log, it should tell you what xlog file it
> needs, and you can copy that from M1 if it is still there.
>
> Cheers,
>
> Jeff

--
--o--o--o--o--o--o--o--o--o--o--o--o--
Jim Longwill
PSMFC Regional Mark Processing Center
Ph:503-595-3146; FAX:503-595-3446
JLongwill@psmfc.org
--o--o--o--o--o--o--o--o--o--o--o--o--