Re: Group commit, revised

On 31.01.2012 01:35, Simon Riggs wrote:
> The plan here is to allow WAL flush and clog updates to occur
> concurrently. Which allows the clog contention and update time to be
> completely hidden behind the wait for the WAL flush. That is only
> possible if we have the WALwriter involved since we need two processes
> to be actively involved.
>
> ...
>
> The theory behind this is clear, but needs some explanation.
>
> There are 5 actions that need to occur at commit
> 1) insert WAL record
> 2) optionally flush WAL record
> 3) mark the clog AND set LSN from (1) if we skipped (2)
> 4) optionally wait for sync rep
> 5) remove the proc from the procarray
>> ...
>
> Notice that step (2) and step (3) are actually independent of each other.
>
> So an improved design for commit is to
> 2) request flush up to LSN, but don't wait
> 3) mark the clog and set LSN
> 4) wait for LSN once, either for walwriter or walsender to release us

That seems like a pretty marginal gain. If you're bound by the speed of 
fsyncs, this will reduce the latency by the time it takes to mark the 
clog, which is tiny in comparison to all the other stuff that needs to 
happen, like, flushing the WAL. And that's ignoring any additional 
overhead caused by the signaling between processes. If you're bound by 
CPU capacity, this doesn't help at all because it just moves the work 
around.

Anyway, this is quite different from the original goal and patch for 
group commit, so can we please leave this for 9.3, and move on with the 
review of pending 9.2 patches.

--   Heikki Linnakangas  EnterpriseDB   http://www.enterprisedb.com