Re: BUG #13368: standby cluster immediately promotes after pg_basebackup from previously promoted master

On Wed, Jun 3, 2015 at 1:04 AM, Fujii Masao wrote:

On Mon, Jun 1, 2015 at 5:19 PM, Michael Paquier
>> Some testing shows us that in some cases, when pg_ctl promote is called
>> multiple
>> times, a promote file is left in the PGDATA directory, even though the
>> cluster
>> has been succesfully promoted and is accepting read/write queries.
>
> This is not surprising, pg_ctl bases its analysis that a node needs to
> be promoted if recovery.conf exists or not, and there is an interval
> of time between which recovery.conf is removed and the promotion is
> actually triggered, so you can create a promote file even after even
> sending SIGUSR1 to the standby's postmaster
>
>> We will try to workaround this issue by ensuring we do not send multiple
>> promote request using pg_ctl to the same cluster.
>
> Well, we could for example have the server switch promote to
> promote_done in CheckForStandbyTrigger() and then unlink it when
> recovery.conf is switched to .done. Opinions are welcome on the
> matter.

Or we can just always remove the signal file at the end of recovery.
That filename switch seems unnecessary.

Well, by doing so, in the event of a crash during recovery the promote signal file would be present in PGDATA, and this would enforce a promotion at the next startup of the node. I don't think that this is a good idea. In the case of a promoted node crash a user may want to look at his node back in a recovery state.

Also, this intermediate promote file, let's say promote.detected, would be useful for external tools to let them know that the promotion has been acknoledged (you can already know it if your tool knows that a promote has been triggered, that promote has been removed by the server and if recovery.conf is still present). That's not something you would want on back branches btw as this changes how promotion bevahes seen from an external point of view. But that would be a patch simple enough (got a WIP for people wondering).

An open question would be what to do with pg_ctl promote if a promote file already exists. I think that we should ignore the creation of the promote file but still kick the signal SIGUSR1.
 

In addition to that change, we should make pg_basebackup skip
the signal file?