Re: Load distributed checkpoint

On Fri, 8 Dec 2006, Takayuki Tsunakawa wrote:

> Though I'm not sure, isn't it the key to use O_SYNC so that write()s
> transfer data to disk?

If disk writes near checkpoint time aren't happening fast enough now, I 
doubt forcing a sync after every write will make that better.

> If PostgreSQL is to use these, writing of dirty buffers has to be
> improved. To decrease the count of I/O, pages adjacent on disk that
> are also adjacent on memory must be written with one write().

Sorting out which pages are next to one another on disk is one of the jobs 
the file system cache does; bypassing it will then make all that 
complicated sorting logic the job of the database engine.  And unlike the 
operating system, the engine doesn't even really know anything about the 
filesystem or physical disks involved, so what are the odds it's going to 
do a better job?  That's the concern with assuming direct writes are the 
solution here--you have to be smarter than the OS is for that to be an 
improvement, and you have a lot less information to make your decisions 
with than it does.

I like Itagaki's idea of some automatic tuning of the checkpoint timeout 
and wal segment parameters to help out with checkpoint behavior; that's an 
idea that would help a lot of people.  I'm not so sure that trying to make 
PostgreSQL take over operations that it's relying on the OS to handle 
intelligently right now will be as helpful, and it's a big programming job 
to even try.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD