Re: [PoC] Non-volatile WAL buffer

Hi,

Let me share some numbers from a few more tests. I've been experimenting 
with two optimization ideas - alignment and non-temporal writes.

The first idea (alignment) is not entirely unique to PMEM - we have a 
bunch of places where we align stuff to cacheline, and the same thing 
does apply to PMEM. The cache lines are 64B, so I've tweaked the WAL 
format to align records accordingly - the header sizes are a multiple of 
64B, and the space is reserved in 64B chunks. It's a bit crude, but good 
enough for experiments, I think. This means the WAL format would not be 
compatible, and there's additional overhead (not sure how much).


The second idea is somewhat specific to PMEM - the pmem_memcpy provided 
by libpmem allows specifying flags, determining whether the data should 
go to CPU cache or not, whether it should be flushed, etc. So far the 
code was using

     pmem_memcpy(..., PMEM_F_MEM_NOFLUSH);

following the idea that caching data in CPU cache and then flushing it 
in larger chunks is more efficient. I heard some recommendations to use 
non-temporal writes (which should not use CPU cache), so I tested that 
switching to

     pmem_memcpy(..., PMEM_F_NON_TEMPORAL);

The experimental patches doing these things are attached, as usual.

The results are a bit better than for the preceding patches, but only by 
a couple percent. That's a bit disappointing. Attached is a PDF with 
charts for the three WAL segment sizes as before.


It's possible the patches are introducing some internal bottleneck, so I 
plan to focus on profiling and optimizing them next. I'd welcome some 
feedback with ideas what might be wrong, of course ;-)


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company