Re: WAL format

"Kevin Grittner" <Kevin.Grittner@wicourts.gov> writes:
> Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> wrote:
>> In particular I wonder why we bother with the page headers.
> Since we re-use the file for a new segment, without overwriting the
> old contents, it seems like we would need to do *something* to
> reliably determine when we've hit the end of a segment and have
> moved into old data from a previous use of the file.  Would your
> proposed changes cover that adequately?

AFAICT the proposal would make us 100% dependent on the record CRC
to detect when a record has been torn (ie, only the first few sectors
made it to disk).  I'm a bit nervous about that from a reliability
standpoint --- with a 32-bit CRC you've got a 1-in-4-billion chance
of accepting bad data.  Checking the page headers too gives us many
more bits that have to be as-expected to consider the data good.

Since the records are fed to XLogInsert as units, it seems like the
actual problem might be addressable by hooking in the sync-rep data
sending at that level, rather than looking at the WAL page buffers
as I gather it must be doing now.
        regards, tom lane