Re: Proposed patch to change TOAST compression strategy

On 2/18/2008 5:33 AM, Gregory Stark wrote:
> "Tom Lane" <tgl@sss.pgh.pa.us> writes:
>
>> * Adds an early-failure path to the compressor as suggested by Jan:
>> if it's been unable to find even one compressible substring in the
>> first 1KB (parameterizable), assume we're looking at incompressible
>> input and give up.  (There are various ways this could be done, but
>> this way seems to add the least overhead to the compressor's inner
>> loop.)
>
> I'm not sure how to test the rest of it, but this bit seems testable. I fear
> this may be too conservative. Even nigh incompressible data will find a few
> backreferences.

One could argue that storing JPEG in a bytea column and not configuring
the column to skip compression is a pilot error. But I guess this
happens much more often than accidentally finding some data that has
zero backref within the first KB and changes pattern thereafter. Do you
have any example for data that is like that?


Jan

--
Anyone who trades liberty for security deserves neither
liberty nor security. -- Benjamin Franklin