Re: fallocate / posix_fallocate for new WAL file creation (etc...)

On Tue, May 28, 2013 at 10:36 AM, Greg Smith <greg@2ndquadrant.com> wrote:
> On 5/28/13 11:12 AM, Jon Nelson wrote:
>>
>> It opens a new file, fallocates 16MB, calls fdatasync.
>
>
> Outside of the run for performance testing, I think it would be good at this
> point to validate that there is really a 16MB file full of zeroes resulting
> from these operations.  I am not really concerned that posix_fallocate might
> be slower in some cases; that seems unlikely.  I am concerned that it might
> result in a file that isn't structurally the same as the 16MB of zero writes
> implementation used now.

util-linux comes with fallocate which (might?) suffice for testing in
that respect, no?
If that is a real concern, it could be made part of the autoconf
testing, perhaps.

> The timing program you're writing has some aspects that are similar to the
> contrib/pg_test_fsync program.  You might borrow some code from there
> usefully.

Thanks! If it looks like what I'm attaching will not do, then I'll
look at that as a possible next step.

> To clarify the suggestion I was making before about including performance
> test results:  that doesn't necessarily mean the testing code must run using
> only the database.  That's better if possible, but as Robert says it may not
> be for some optimizations.  The important thing is to have something
> measuring the improvement that a reviewer can duplicate, and if that's a
> standalone benchmark problem that's still very useful.  The main thing I'm
> wary of is any "this should be faster" claims that don't come with any
> repeatable measurements at all.  Very often theories about the fastest way
> to do something don't match what's actually seen in testing.

Ack.
A note: The attached test program uses *fsync* instead of *fdatasync*
after calling fallocate (or writing out 16MB of zeroes), per an
earlier suggestion.

--
Jon