Re: Quick-and-dirty compression for WAL backup blocks

On Mon, 6 Jun 2005, Tom Lane wrote:

> Junji TERAMOTO <teramoto.junji@lab.ntt.co.jp> writes:
>
>> In "whole buffer log", there is a page header that includes offset of
>> "hole" (lower and upper). If we use that information, we don't need
>> any overhead, do we?
>
> No, because the WAL code cannot assume that all pages follow the
> convention that pd_lower and pd_upper represent the boundaries of
> free space.  (As a counterexample: index metapages don't always
> do that.)  I think the transformation has to be guaranteed lossless,
> which means that at a minimum you'd need to check whether the data
> in between pd_lower and pd_upper really is zeroes.  So the irreducible
> minimum overhead is 1 bit to tell whether you compressed or not.

Vacuum doesn't zero out the free space between lower and upper, it's 
just marked as unused, so a lossless compression becomes less efficient 
on tables that have free space released by vacuum in them.

How about adding a flag to XLogRecData to indicate if the space between 
pd_lower and pd_upper is meaningful or not? The XLogInsert caller probably 
knows that. That way you could completely skip over the free space if 
it's not meaningful, saving even more cycles.

- Heikki