Re: POC: Cleaning up orphaned files using undo logs

On 07/08/2019 13:52, Dilip Kumar wrote:
> I have one more problem related to compression of the command id
> field.  Basically, the problem is that we don't set the command id in
> the WAL and we will always store FirstCommandId in the undo[1].   So
> suppose there were 2 operations under a different CID then during DO
> time both the undo record will store the CID field in their respective
> undo records but during REDO time, all the commands will store the
> same CID(FirstCommandId) so as per the compression logic the
> subsequent record for the same transaction will not store the CID
> field.  I am not sure what is the best way to handle this but I have
> few ideas.
> 
> 1) Don't compress the CID field ever.
> 2) Write CID in WAL,  but just for compressing the CID field in undo
> (which may not necessarily go to disk) we don't want to add extra 4
> bytes in the WAL.

Most transactions have only a few commands, so you could optimize for 
that. If you use some kind of a variable-byte encoding for it, it could 
be a single byte or even just a few bits, for the common cases.

For the first version, I'd suggest keeping it simple, though, and 
optimize later.

- Heikki