Re: Proposal for CSN based snapshots

On 06/03/2014 05:53 PM, Greg Stark wrote:
> On Tue, Jun 3, 2014 at 2:55 PM, Robert Haas <robertmhaas@gmail.com> wrote:
>> That doesn't address Bruce's concern about CLOG disk consumption.
>
> Well we would only need the xid->lsn mapping for transactions since
> globalxmin. Anything older we would just need the committed bit. So we
> could maintain two structures, one like our current clog going back
> until the freeze_max_age and one with 32-bits (or 64 bits?) per xid
> but only going back as far as the globalxmin. There are a myriad of
> compression techniques we could use on a sequence of mostly similar
> mostly increasing numbers in a small range too. But I suspect they
> wouldn't really be necessary.

Yeah, that seems like a better design, after all.

Attached is a new patch. It now keeps the current pg_clog unchanged, but
adds a new pg_csnlog besides it. pg_csnlog is more similar to
pg_subtrans than pg_clog: it's not WAL-logged, is reset at startup, and
segments older than GlobalXmin can be truncated.

This addresses the disk space consumption, and simplifies pg_upgrade.

There are no other significant changes in this new version, so it's
still very much WIP. But please take a look!

- Heikki