Re: jsonb format is pessimal for toast compression

On 08/12/2014 10:58 AM, Robert Haas wrote:
> What would really be ideal here is if the JSON code could inform the
> toast compression code "this many initial bytes are likely
> incompressible, just pass them through without trying, and then start
> compressing at byte N", where N is the byte following the TOC.  But I
> don't know that there's a reasonable way to implement that.
>

Sorry, being late for the party.

Anyhow, this strikes me as a good basic direction of thought. But I 
think we should put the burden on the data type, not on toast. To do 
that data types could have an optional toast_hint_values() function, 
which the toast code can call with the actual datum at hand and its 
default parameter array. The hint values function then can modify that 
parameter array, telling toast how much to skip, how hard to try (or not 
at all) and so on. A data type specific function should know much better 
how to figure out how compressible a particular datum may be.

Certainly nothing for 9.4, but it might require changing the toast API 
in a different way than just handing it an oid and hard-coding the 
JASONBOID case into toast for 9.4. If we are going to change the API, we 
might as well do it right.


Regards,
Jan

-- 
Jan Wieck
Senior Software Engineer
http://slony.info