Re: Bgwriter LRU cleaning: we've been going at this all wrong

On Tue, 26 Jun 2007, Tom Lane wrote:

> It should try to be behind the sweep, ie, so far ahead that it's lapped 
> the clock sweep and is trailing along right behind it, cleaning buffers 
> immediately after their usage_count falls to zero.  All the rest of the 
> buffer arena is either clean or has positive usage_count.

I've said before here that something has to fundamentally change with the 
LRU writer for it to ever be really useful, because most of the time it's 
executing over pages with a positive usage_count as you say here.  One 
idea I threw out before was to have it premptively lower the usage counts 
as it scans ahead of the sweep point and then add the pages to the free 
list, which you rightly had some issues with.  This suggestion of a change 
so you'd expect it to follow right behind the sweep point sounds like a 
better plan that should result in even less client back-end writes, and I 
really like a plan that finally casts the LRU writer control parameter in 
a MB/s context.

(Some pointers to your comments when we've gone over this neighborhood 
before: http://archives.postgresql.org/pgsql-hackers/2007-03/msg00642.php 
http://archives.postgresql.org/pgsql-hackers/2007-04/msg00799.php )

I broke Itagaki-san's patch into two pieces when I was doing the review 
cleanup on it specifically to make it easier to tinker with this part 
without losing some of its other neat features.  Heikki, did you do 
anything with that LRU adjustment patch since I sent it out: 
http://archives.postgresql.org/pgsql-patches/2007-05/msg00142.php

I already fixed the race condition bug you found in my version of the 
code.

Unless someone else has a burning desire to implement Tom's idea faster 
than me, I should be to build this new implementation myself in the next 
couple of days.  I still have the test environment leftover from the last 
time I worked on this code, and I think everybody else who could handle 
this job has more important higher-level things they could be working on 
instead.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD