Re: a question about Direct I/O and double buffering

On 4/5/07, Xiaoning Ding <dingxn@cse.ohio-state.edu> wrote:
> Alex Deucher wrote:
> > On 4/5/07, Erik Jones <erik@myemma.com> wrote:
> >>
> >> On Apr 5, 2007, at 1:22 PM, Xiaoning Ding wrote:
> >>
> >> Erik Jones wrote:
> >> On Apr 5, 2007, at 12:09 PM, Xiaoning Ding wrote:
> >> Hi,
> >>
> >> A page may be double buffered in PG's buffer pool and in OS's buffer
> >> cache.
> >> Other DBMS like DB2 and Oracle has provided Direct I/O option to
> >> eliminate
> >> double buffering. I noticed there were discusses on the list. But
> >> I can not find similar option in PG. Does PG support direct I/O now?
> >>
> >> The tuning guide of PG usually recommends a small shared buffer pool
> >> (compared
> >> to the size of physical memory).  I think it is to avoid swapping. If
> >> there were
> >> swapping, OS kernel may swap out some pages in PG's buffer pool even PG
> >> want to keep them in memory. i.e. PG would loose full control over
> >> buffer pool.
> >> A large buffer pool is not good because it may
> >> 1. cause more pages double buffered, and thus decrease the efficiency of
> >> buffer
> >> cache and buffer pool.
> >> 2. may cause swapping.
> >> Am I right?
> >>
> >> If PG's buffer pool is small compared with physical memory, can I say
> >> that the
> >> hit ratio of PG's buffer pool is not so meaningful because most misses
> >> can be
> >> satisfied by OS Kernel's buffer cache?
> >>
> >> Thanks!
> >> To the best of my knowledge, Postgres itself does not have a direct IO
> >> option (although it would be a good addition).  So, in order to use
> >> direct
> >> IO with postgres you'll need to consult your filesystem docs for how
> >> to set
> >> the forcedirectio mount option.  I believe it can be set dynamically,
> >> but if
> >> you want it to be permanent you'll to add it to your fstab/vfstab file.
> >>
> >> I use Linux.  It supports direct I/O on a per-file basis only.  To
> >> bypass OS
> >> buffer cache,
> >> files should be opened with O_DIRECT option.  I afraid that I have to
> >> modify
> >> PG.
> >>
> >> Xiaoning
> >> Looks like it.  I just did a cursory search of the archives and it seems
> >> that others have looked at this before so you'll probably want to start
> >> there if your up to it.
> >>
> >
> > Linux used to have (still does?) a RAW interface which might also be
> > useful.  I think the original code was contributed by oracle so they
> > could support direct IO.
> >
> > Alex
> I am more concerned with reads , and how to do direct I/O under Linux here.
> Reading raw devices in linux bypasses OS buffer cache. But how can you
> mount a raw device( it is a character device) as a file system?
>

In this case, I guess you'd probably have to do it within pg itself.

Alex