Re: Why we are going to have to go DirectIO

(2013/12/04 11:28), Tatsuo Ishii wrote:
>> Magnus Hagander <magnus@hagander.net> writes:
>>> On Tue, Dec 3, 2013 at 11:44 PM, Josh Berkus <josh@agliodbs.com> wrote:
>>>> Would certainly be nice.  Realistically, getting good automated
>>>> performace tests will require paying someone like Greg S., Mark or me
>>>> for 6 solid months to develop them, since worthwhile open source
>>>> performance test platforms currently don't exist.  That money has never
>>>> been available; maybe I should do a kickstarter.
>>
>>> So in order to get *testing* we need to pay somebody. But to build a great
>>> database server, we can rely on volunteer efforts or sponsorship from
>>> companies who are interested in moving the project forward?
>>
>> And even more to the point, volunteers to reinvent the kernel I/O stack
>> can be found on every street corner?  And those volunteers won't need any
>> test scaffolding to be sure that *their* version never has performance
>> regressions?  (Well, no, they won't, because no such thing will ever be
>> built.  But we do need better test scaffolding for real problems.)
>
> Can we avoid the Linux kernel problem by simply increasing our shared
> buffer size, say up to 80% of memory?
It will be swap more easier.

I think that we should use latest system-calls in Linux which are like 
posix_fadvise(), fallocate() and sync_file_range() etc, when we use linux 
buffered IO. Hoevere, PostgreSQL doesn't use these system-call a lots. 
Especially, I think that checkpoint algorithm is very ugly..

Regards,
-- 
Mitsumasa KONDO
NTT Open Source Software Center