Re: pgsql-server/src backend/storage/buffer/bufmgr ...

Jan Wieck wrote:
> Tom Lane wrote:
>
> > wieck@svr1.postgresql.org (Jan Wieck) writes:
> >>     Added GUC variable bgwriter_flush_method controlling the action
> >>     done by the background writer between writing dirty blocks and
> >>     napping.
> >
> >>     none (default)   no action
> >>     sync             bgwriter calls smgrsync() causing a sync(2)
> >
> > Why would that be useful at all?  I thought the purpose of the bgwriter
> > was to avoid I/O storms, not provoke them.
>
> Calling sync(2) every time the background writer naps means calling it
> every couple hundred milliseconds. That can hardly be called an IO
> storm, it's more like a constant breeze.

Have you tested this option?  It seems like sub-second sync would kill
performance.

> So far nobody bothered to make any other proposal how to cause the
> kernel to actually do some writing at all. A lot of people babble about
> fsync(), fdatasync() and fadvise and whatnot. A week ago I posted the
> proposal for this and got exactly zero response.

I assumed all Unixes flush dirty pages at least every 30 seconds, so if
checkpoints are every 2-3 minutes, most of the dirty pages should
already be flushed.

Perhaps instead of tieing sync to the background writer sleeps, we should
have a sync_frequency that could be set to sync every 15 or 30 seconds.
Is there any value in doing it more frequently than that?

> Before this, the bgwriter did only write the dirty blocks, so that the
> checkpoint (doing the sync() call) still caused all the physical IO to
> happen at once. Sure, with the bgwriter doing the major write IO, he'd
> know what files have been written to and can do fsync() and fdatasync()
> on the. But even with that, the checkpoint doing sync() will be in
> danger to cause a lot of unexpected IO from wherenot, making the time
> the checkpoint takes totally unpredictable.
>
> The whole point of the bgwriter is to give responsetimes a better
> variance, I never claimed that it will improve performance.

Uh, our goal is better performance overall.  If this new options causes
dismal performance when enabled, who cares how fast the checkpoints are? :-)

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073