Re: Add ZSON extension to /contrib/

On 5/27/21 4:15 AM, Andrew Dunstan wrote:
> 
> On 5/26/21 5:29 PM, Bruce Momjian wrote:
>> On Tue, May 25, 2021 at 01:55:13PM +0300, Aleksander Alekseev wrote:
>>> Hi hackers,
>>>
>>> Back in 2016 while being at PostgresPro I developed the ZSON extension [1]. The
>>> extension introduces the new ZSON type, which is 100% compatible with JSONB but
>>> uses a shared dictionary of strings most frequently used in given JSONB
>>> documents for compression. These strings are replaced with integer IDs.
>>> Afterward, PGLZ (and now LZ4) applies if the document is large enough by common
>>> PostgreSQL logic. Under certain conditions (many large documents), this saves
>>> disk space, memory and increases the overall performance. More details can be
>>> found in README on GitHub.
>> I think this is interesting because it is one of the few cases that
>> allow compression outside of a single column.  Here is a list of
>> compression options:
>>
>>     https://momjian.us/main/blogs/pgblog/2020.html#April_27_2020
>>     
>>     1. single field
>>     2. across rows in a single page
>>     3. across rows in a single column
>>     4. across all columns and rows in a table
>>     5. across tables in a database
>>     6. across databases
>>
>> While standard Postgres does #1, ZSON allows 2-5, assuming the data is
>> in the ZSON data type.  I think this cross-field compression has great
>> potential for cases where the data is not relational, or hasn't had time
>> to be structured relationally.  It also opens questions of how to do
>> this cleanly in a relational system.
>>
> 
> I think we're going to get the best bang for the buck on doing 2, 3, and
> 4. If it's confined to a single table then we can put a dictionary in
> something like a fork.

Agreed.

> Maybe given partitioning we want to be able to do multi-table
> dictionaries, but that's less certain.
> 

Yeah. I think it'll have many of the same issues/complexity as global
indexes, and the gains are likely limited. At least assuming the
partitions are sufficiently large, but tiny partitions are inefficient
in general, I think.


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company