Re: increasing the default WAL segment size

Hi Robert,

On 3/24/17 3:00 PM, Robert Haas wrote:
> On Wed, Mar 22, 2017 at 6:05 PM, David Steele <david@pgmasters.net> wrote:
>>> Wait, really?  I thought you abandoned this approach because there's
>>> then no principled way to handle WAL segments of less than the default
>>> size.
>>
>> I did say that, but I thought I had hit on a compromise.
>>
>> But, as I originally pointed out the hex characters in the filename are not
>> aligned correctly for > 8 bits (< 16MB segments) and using different
>> alignments just made it less consistent.
>
> I don't think I understand what the compromise is.  Are you saying we
> should have one rule for segments < 16MB and another rule for segments
>> 16MB?  I think using two different rules for file naming depending
> on the segment size will be a negative for both tool authors and
> ordinary users.

Sorry for the confusion, I meant to say that if we want to keep LSNs in 
the filenames and not change alignment for the current default, then we 
would need to drop support for segment sizes < 16MB (more or less what I 
said below).  Bad editing on my part.

>> It would be OK if we were willing to drop the 1,2,4,8 segment sizes because
>> then the alignment would make sense and not change the current 16MB
>> sequence.
>
> Well, that is true.  But the thing I'm trying to do here is to keep
> this patch down to what actually needs to be changed in order to
> accomplish the original purchase, not squeeze more and more changes
> into it.

Attached is a patch to be applied on top of Beena's v8 patch that 
preserves LSNs in the file naming for all segment sizes.  It's not quite 
complete because it doesn't modify the lower size limit everywhere, but 
I think it's enough so you can see what I'm getting at.  This passes 
check-world and I've poked at it in other segment sizes as well manually.

Behavior for the current default of 16MB is unchanged, and all other 
sizes go through a logical progression.

1GB:
000000010000000000000040
000000010000000000000080
0000000100000000000000C0
000000010000000100000000

256GB:

000000010000000000000010
000000010000000000000020
000000010000000000000030
...
0000000100000000000000E0
0000000100000000000000F0
000000010000000100000000

64GB:

000000010000000100000004
000000010000000100000008
00000001000000010000000C
...
0000000100000001000000F8
0000000100000001000000FC
000000010000000100000000

I believe that maintaining an easy correspondence between LSN and 
filename is important.  The cluster will not always be up to help with 
these calculations and tools that do the job may not be present or may 
have issues.

I'm happy to merge this with Beena's patch (and tidy my patch up) if 
this looks like an improvement to everyone.

-- 
-David
david@pgmasters.net