Re: Scaling XLog insertion (was Re: Moving more work outside WALInsertLock)

On 07.03.2012 17:28, Tom Lane wrote:
> Simon Riggs<simon@2ndQuadrant.com>  writes:
>> On Wed, Mar 7, 2012 at 3:04 PM, Tom Lane<tgl@sss.pgh.pa.us>  wrote:
>>> Alvaro Herrera<alvherre@commandprompt.com>  writes:
>>>> So they are undoubtely rare.  Not sure if as rare as Higgs bosons.
>
>>> Even if they're rare, having a major performance hiccup when one happens
>>> is not a side-effect I want to see from a patch whose only reason to
>>> exist is better performance.
>
>> I agree the effect you point out can exist, I just don't want to slow
>> down the main case as a result.
>
> I don't see any reason to think that what I suggested would slow things
> down, especially not if the code were set up to fall through quickly in
> the typical case where no page boundary is crossed.  Integer division is
> not slow on any machine made in the last 15 years or so.

Agreed. I wasn't worried about the looping with extra-large records, but 
might as well not do it.

Here's an updated patch. It now only loops once per segment that a 
record crosses. Plus a lot of other small cleanup.

I've been doing some performance testing with this, using a simple C 
function that just inserts a dummy WAL record of given size. I'm not 
totally satisfied. Although the patch helps with scalability at 3-4 
concurrent backends doing WAL insertions, it seems to slow down the 
single-client case with small WAL records by about 5-10%. This is what 
Robert also saw with an earlier version of the patch 
(http://archives.postgresql.org/pgsql-hackers/2011-12/msg01223.php). I 
tested this with the data directory on a RAM drive, unfortunately I 
don't have a server with a hard drive that can sustain the high 
insertion rate. I'll post more detailed results, once I've refined the 
tests a bit.

--   Heikki Linnakangas  EnterpriseDB   http://www.enterprisedb.com