Re: [RFC] Incremental backup v2: add backup profile to base backup

On Fri, Oct 3, 2014 at 1:08 PM, Marco Nenciarini
<marco.nenciarini@2ndquadrant.it> wrote:
>>> Any comment will be appreciated. In particular I'd appreciate comments
>>> on correctness of relnode files detection and LSN extraction code.
>>
>> I didn't look at it in detail, but one future problem comes to mind:
>> Once you implement the server-side code that only sends a file if its
>> LSN is higher than the cutoff point that the client gave, you'll have to
>> scan the whole file first, to see if there are any blocks with a higher
>> LSN. At least until you find the first such block. So with a file-level
>> implementation of this sort, you'll have to scan all files twice, in the
>> worst case.
>>
>
> It's true. To solve this you have to keep a central maxLSN directory,
> but I think it introduces more issues than it solves.

I see that as a worthy optimization on the server side, regardless of
whether file or block-level backups are used, since it allows
efficient skipping of untouched segments (common for append-only
tables).

Still, it would be something to do after it works already (ie: it's an
optimization)