Re: Experimental patch for inter-page delay in VACUUM

Jan Wieck wrote:
> Bruce Momjian wrote:
> 
> > Jan Wieck wrote:
> >> Bruce Momjian wrote:
> >> > I would be interested to know if you have the background write process
> >> > writing old dirty buffers to kernel buffers continually if the sync()
> >> > load is diminished.  What this does is to push more dirty buffers into
> >> > the kernel cache in hopes the OS will write those buffers on its own
> >> > before the checkpoint does its write/sync work.  This might allow us to
> >> > reduce sync() load while preventing the need for O_SYNC/fsync().
> >> 
> >> I tried that first. Linux 2.4 does not, as long as you don't tell it by 
> >> reducing the dirty data block aging time with update(8). So you have to 
> >> force it to utilize the write bandwidth in the meantime. For that you 
> >> have to call sync() or fsync() on something.
> >> 
> >> Maybe O_SYNC is not as bad an option as it seems. In my patch, the 
> >> checkpointer flushes the buffers in LRU order, meaning it flushes the 
> >> least recently used ones first. This has the side effect that buffers 
> >> returned for replacement (on a cache miss, when the backend needs to 
> >> read the block) are most likely to be flushed/clean. So it reduces the 
> >> write load of backends and thus the probability that a backend is ever 
> >> blocked waiting on an O_SYNC'd write().
> >> 
> >> I will add some counters and gather some statistics how often the 
> >> backend in comparision to the checkpointer calls write().
> > 
> > OK, new idea.  How about if you write() the buffers, mark them as clean
> > and unlock them, then issue fsync().  The advantage here is that we can
> 
> Not really new, I think in my first mail I wrote that I simplified this 
> new mdfsyncrecent() function by calling sync() instead ... other than 
> that the code I posted worked exactly that way.

I am confused --- I was suggesting we call fsync after we write a few
blocks for a given table, and that was going to happen between
checkpoints.  Is the sync() happening then or only at checkpoint time.

Sorry I am lost but there seems to be an email delay in my receiving the
replies.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
359-1001+  If your life is a hard drive,     |  13 Roberts Road +  Christ can be your backup.        |  Newtown Square,
Pennsylvania19073