Re: Experimental patch for inter-page delay in VACUUM

scott.marlowe wrote:
> On Tue, 4 Nov 2003, Tom Lane wrote:
> 
> > Jan Wieck <JanWieck@Yahoo.com> writes:
> > > What still needs to be addressed is the IO storm cause by checkpoints. I 
> > > see it much relaxed when stretching out the BufferSync() over most of 
> > > the time until the next one should occur. But the kernel sync at it's 
> > > end still pushes the system hard against the wall.
> > 
> > I have never been happy with the fact that we use sync(2) at all.  Quite
> > aside from the "I/O storm" issue, sync() is really an unsafe way to do a
> > checkpoint, because there is no way to be certain when it is done.  And
> > on top of that, it does too much, because it forces syncing of files
> > unrelated to Postgres.
> > 
> > I would like to see us go over to fsync, or some other technique that
> > gives more certainty about when the write has occurred.  There might be
> > some scope that way to allow stretching out the I/O, too.
> > 
> > The main problem with this is knowing which files need to be fsync'd.
> 
> Wasn't this a problem that the win32 port had to solve by keeping a list 
> of all files that need fsyncing since Windows doesn't do sync() in the 
> classical sense?  If so, then could we use that code to keep track of the 
> files that need fsyncing?

Yes, I have that code from SRA.  They used threading, so they recorded
all the open files in local memory and opened/fsync/closed them for
checkpoints.  We have to store the file names in a shared area, perhaps
an area of shared memory with an overflow to a disk file.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
359-1001+  If your life is a hard drive,     |  13 Roberts Road +  Christ can be your backup.        |  Newtown Square,
Pennsylvania19073