Re: [Lsf-pc] Linux kernel impact on PostgreSQL performance

On 01/15/2014 07:50 AM, Dave Chinner wrote:
> However, the first problem is dealing with the IO storm problem on
> fsync. Then we can measure the effect of spreading those writes out
> in time and determine what triggers read starvations (if they are
> apparent). The we can look at whether IO scheduling tweaks or
> whether blk-io throttling solves those problems. Or whether
> something else needs to be done to make it work in environments
> where problems are manifesting.
>
> FWIW [and I know you're probably sick of hearing this by now], but
> the blk-io throttling works almost perfectly with applications that
> use direct IO.....

For checkpoint writes, direct I/O actually would be reasonable. 
Bypassing the OS cache is a good thing in that case - we don't want the 
written pages to evict other pages from the OS cache, as we already have 
them in the PostgreSQL buffer cache.

Writing one page at a time with O_DIRECT from a single process might be 
quite slow, so we'd probably need to use writev() or asynchronous I/O to 
work around that.

We'd still need to issue an fsync() to flush any already-written pages 
from the OS cache to disk, though.

- Heikki