Re: Improve compression speeds in pg_lzcompress.c

On 7 January 2013 13:36, Greg Stark <stark@mit.edu> wrote:
> On Mon, Jan 7, 2013 at 10:21 AM, John R Pierce <pierce@hogranch.com> wrote:
>> On 1/7/2013 2:05 AM, Andres Freund wrote:
>>>
>>> I think there should be enough bits available in the toast pointer to
>>> indicate the type of compression. I seem to remember somebody even
>>> posting a patch to that effect?
>>> I agree that it's probably too late in the 9.3 cycle to start with this.
>>
>>
>> so an upgraded database would have old toasted values in the old compression
>> format, and new toasted values in the new format in an existing table?
>> that's kind of ugly.
>
> I haven't looked at the patch. It's not obvious to me from the
> description that the output isn't backwards compatible. The way the LZ
> toast compression works the output is self-describing. There are many
> different outputs that would decompress to the same thing and the
> compressing code can choose how hard to look for earlier matches and
> when to just copy bytes wholesale but the decompression will work
> regardless.

Good point, and a great reason to use this patch rather than LZ4 for 9.3

We could even have tuning parameters for toast compression, as long as
we keep the on disk format identical.

-- Simon Riggs                   http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services