Re: Why does replication need the old history file?

>> The related source code comment says
>>
>>         /*
>>          * Get any missing history files. We do this always, even when we're
>>          * not interested in that timeline, so that if we're promoted to
>>          * become the master later on, we don't select the same timeline that
>>          * was already used in the current master. This isn't bullet-proof -
>>          * you'll need some external software to manage your cluster if you
>>          * need to ensure that a unique timeline id is chosen in every case,
>>          * but let's avoid the confusion of timeline id collisions where we
>>          * can.
>>          */
>>         WalRcvFetchTimeLineHistoryFiles(startpointTLI, primaryTLI);
> 
> So this seems to be something we're doing "just in case" which is
> preventing a useful way to spin up large master/replica clusters from
> PITR backup.  Might this be something we want to change, and simply
> warn that we can't find the history file instead of ERROR?

Having experimented with this further, it seems to me that either we
have a functionality bug here, or a documentation bug.  Currently our
docs do not treat the history files as critical, so much so that authors
of backup tools, like Barman, treat them as expireable.  But replication
apparently regards them as being as essiential as CLOG files.  The
result is that, in some cases, a Barman restored database can never
participate in replication again.

So either we should remove the requirement to have old history files to
start replication, *or* we should document somewhere to never, ever
delete history files and to retain them permanently for backups.  I know
what I'd prefer, but I'd like to hear the case for what could go wrong
if we don't retain the old history file requirement.

BTW, if retaining history files forever is a requirement, then maybe
they should go in their own directory in the future.

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com