Re: Add LZ4 compression in pg_dump

On Wed, Mar 01, 2023 at 04:52:49PM +0100, Tomas Vondra wrote:
> Thanks. That seems correct to me, but I find it somewhat confusing,
> because we now have
> 
>  DeflateCompressorInit vs. InitCompressorGzip
> 
>  DeflateCompressorEnd vs. EndCompressorGzip
> 
>  DeflateCompressorData - The name doesn't really say what it does (would
>                          be better to have a verb in there, I think).
> 
> I wonder if we can make this somehow clearer?

To move things along, I updated Georgios' patch:

Rename DeflateCompressorData() to DeflateCompressorCommon();
Rearrange functions to their original order allowing a cleaner diff to the prior code;
Change pg_fatal() to an assertion+comment;
Update the commit message and fix a few typos;

> Also, InitCompressorGzip says this:
> 
>    /*
>     * If the caller has defined a write function, prepare the necessary
>     * state. Avoid initializing during the first write call, because End
>     * may be called without ever writing any data.
>     */
>     if (cs->writeF)
>         DeflateCompressorInit(cs);
>
> Does it actually make sense to not have writeF defined in some cases?

InitCompressor is being called for either reading or writing, either of
which could be null:

src/bin/pg_dump/pg_backup_custom.c:     ctx->cs = AllocateCompressor(AH->compression_spec,
src/bin/pg_dump/pg_backup_custom.c-                                                              NULL,
src/bin/pg_dump/pg_backup_custom.c-                                                              _CustomWriteFunc);
--
src/bin/pg_dump/pg_backup_custom.c:     cs = AllocateCompressor(AH->compression_spec,
src/bin/pg_dump/pg_backup_custom.c-                                                     _CustomReadFunc, NULL);

It's confusing that the comment says "Avoid initializing...".  What it
really means is "Initialize eagerly...".  But that makes more sense in
the context of the commit message for this bugfix than in a comment.  So
I changed that too.

+       /* If deflation was initialized, finalize it */
                    
 
+       if (cs->private_data)
                    
 
+               DeflateCompressorEnd(AH, cs);
                    
 

Maybe it'd be more clear if this used "if (cs->writeF)", like in the
init function ?

-- 
Justin