Re: Performance Improvement by reducing WAL for Update Operation

On Fri, Dec 6, 2013 at 3:39 PM, Haribabu kommi
<haribabu.kommi@huawei.com> wrote:
> On 06 December 2013 12:29 Amit Kapila wrote:
>> >> Note -
>> >> a. Performance is data is taken on my laptop, needs to be tested on
>> >> some better m/c b. Attached Patch is just a prototype of chunkwise
>> >> concept, code needs to be improved and decode
>> >>     handling/test is pending.
>> >
>> > I ran the performance test on linux machine and attached the results
>> in the mail.
>>
>
> I ran the performance test on above patches including another patch which
> Does snappy hash instead of normal hash in LZ algorithm. The performance
> Readings and patch with snappy hash not including new data in compression
> are attached in the mail.
  Thanks for taking the data.

> The chunk wise approach is giving good performance in most of the scenarios.
  Agreed, summarization of data for LZ/Chunkwise encoding especially for  non-compressible (hundred tiny fields, all
changed/halfchanged) or less  compressible data (hundred tiny fields, half nulled) w.r.t CPU
 
usage is as below:
  a. For hard disk, there is an overhead of 7~16% with LZ delta encoding      and there is an overhead of 5~8% with
Chunkwise encoding.
 
  b. For Tempfs (which means operate on RAM as disk), there is an
overhead of 19~26%      with LZ delta encoding and there is an overhead of 9~18% with
Chunk wise encoding.

There might be some variation of data (in your last mail the overhead
for chunkwise method for Tempfs was < 12%),
but in general the data suggests that chunk wise encoding has less
overhead than LZ encoding for non-compressible data
and for others it is better or equal.

Now, I think we have below options for this patch:
a. If the performance overhead for worst case is acceptable (we can
try to reduce some more, but don't think it will be something
drastic),   then this can be done without any flag/option.
b. Have it with table level option to enable/disable WAL compression
c. Drop this patch, as for worst cases there is some performance overhead.
d. Go back and work more on it, if there is any further suggestions
for improvement.

With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com