Re: Detecting File Damage & Inconsistencies

On Tue, Jun 22, 2021 at 6:32 AM Craig Ringer
<craig.ringer@enterprisedb.com> wrote:

> IIRC the restart_lsn horizon already ensures that we can't miss the xl_xact_assignment at the start of a txn. We
wouldensure that the desired info is available throughout decoding of the txn, including at commit record processing
time,by adding it to the toplevel ReorderBufferTxn. 
>
> The only advantage I can see to annotating the commit record instead is that we don't have to spend a few bytes per
reorder-bufferedtopxid to track this info between start of decoding for the tx and processing of the commit record. I
don'tthink that's worth caring about.The advantages that having it earlier would give us are much more significant. 
>
> A few examples:
>
> * Skip reorder buffering of non-target transactions early, so we can decode the WAL stream to find the target
transactionsmuch faster using less memory and I/O; 
>
> * Read the database change stream and use the session info to stream info into an intrusion detection system and/or
auditengine in real time, using txn streaming to avoid the need to create huge reorder buffers; 
>
> * Re-decode the WAL stream to identify a target txn you know was aborted, and commit it instead, so you can recover
datafrom aborted txns from the WAL stream using logical decoding. (Only possible if the catalog_xmin hasn't advanced
pastthat point already though) 
>
> So yeah. I think it'd be better to log the info you want at start-of-txn unless there's a compelling reason not so,
andI don't see one yet. 

AFAIK, XLOG_XACT_ASSIGNMENT does not occur for normal top-level
transactions, only for subxids.

I don't really want to add an extra record just for this because it
will slow down applications and it won't get turned on as often.

Thoughts?

--
Simon Riggs                http://www.EnterpriseDB.com/