Re: Standby trying "restore_command" before local WAL

On 08/08/2018 04:08 PM, David Steele wrote:
> On 8/7/18 12:05 PM, Stephen Frost wrote:
>>>
>>> All I'm saying is that (assuming my understanding of RestoreArchivedFile is
>>> correct) we can't just do that in the current restore_command. We do need a
>>> way to ask the archive for some metadata/checksums, and restore_command is
>>> too late.
>>
>> Yeah, I see what you mean there.  An alternative might be to *not*
>> unlink the file, in case the restore command wants to check if it's
>> valid and matches what's in the archive, and instead restore the
>> requested file somewhere else and then perform an unlink/mv after
>> the restore command has been run.
> I don't see any cases (in master, at least) where RestoredArchivedFile()
> would unlink an existing WAL segment.  If you look at the call sites
> they all pass "RECOVERYHISTORY" or "RECOVERYXLOG"  to the recovername
> parameter.
> 
> RestoreArchivedFile() does overwrite the existing file after we decide
> that we prefer the restored version over the one in pg_wal, but that
> happens after restore_command returns.
> 
> So as far as I can see, it could be possible for the restore_command to
> do something clever here.
> 


Oh, I see - I really was reading RestoredArchivedFile() wrong ...

Yes, it seems possible to do this in restore_command. Currently it'd 
require constructing the local segment path (replacing RECOVERYXLOG with 
the actual segment filename), but that's solvable by adding a new 
pattern similar to %p.

I'm not sure if packing all this into a single restore_command is a good 
idea, because I'm pretty sure it'll lead to abhorrently monstrous bash 
commands jammed into the restore_command. But then again, we kinda 
already have that, and all archival systems already provide simple and 
correct commands doing the right thing. And extending that would not be 
a big deal.

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services