how to demonstrate the effect of direct I/O ?

Hi all,

I've been running a lot of benchmarks recently (I'll publish the results
once I properly analyze them). One thing I'd like to demonstrate is the
effect of direct I/O when the wal_fsync_method is set to
open_sync/open_datasync.

I.e. I'd like to see cases when this improves/hurts performance
(compared to fsync/fdatasync) and if/how this works on SSD compared to
old-fashioned HDD. But no matter what, I see no significant differences
in performance.

This is what pg_test_fsync gives on the SSD (Intel 320):

        open_datasync                   12492.192 ops/sec
        fdatasync                       11646.257 ops/sec
        fsync                            9839.101 ops/sec
        fsync_writethrough                            n/a
        open_sync                       10420.971 ops/sec

and this is what I get on the HDD (7.2k SATA)

        open_datasync                     120.041 ops/sec
        fdatasync                         120.042 ops/sec
        fsync                              48.000 ops/sec
        fsync_writethrough                            n/a
        open_sync                          48.116 ops/sec

I can post the rest of the pg_test_fsync output if needed.

What should I do to see the effect of direct I/O? I'm wondering if I
need something like a RAID array or a controller with write cache to see
the difference.

All this was run on a kernel 3.1.5 using an ext4 filesystem.

thanks
Tomas