Re: Bug in walreceiver

On 13.01.2011 10:28, Fujii Masao wrote:
> When the master shuts down or crashes, there seems to be
> the case where walreceiver exits without flushing WAL which
> has already been written. This might lead startup process to
> replay un-flushed WAL and break a Write-Ahead-Logging rule.

Hmm, that can happen at a crash even with no replication involved. If 
you "kill -9 postmaster", and some WAL had been written but not fsync'd, 
on crash recovery we will happily recover the unsynced WAL. We could 
prevent that by fsyncing all WAL before applying it - presumably 
fsyncing a file that has already been flushed is quick. But is it worth 
the trouble?

> walreceiver.c
>>         /* Wait a while for data to arrive */
>>         if (walrcv_receive(NAPTIME_PER_CYCLE,&type,&buf,&len))
>>         {
>>             /* Accept the received data, and process it */
>>             XLogWalRcvProcessMsg(type, buf, len);
>>
>>             /* Receive any more data we can without sleeping */
>>             while (walrcv_receive(0,&type,&buf,&len))
>>                 XLogWalRcvProcessMsg(type, buf, len);
>>
>>             /*
>>              * If we've written some records, flush them to disk and let the
>>              * startup process know about them.
>>              */
>>             XLogWalRcvFlush();
>>         }
>
> The problematic case happens when the latter walrcv_receive
> emits ERROR. In this case, the WAL received by the former
> walrcv_receive is not guaranteed to have been flushed yet.
>
> The attached patch ensures that all WAL received is flushed to
> disk before walreceiver exits. This patch should be backported
> to 9.0, I think.

Yeah, we probably should do that, even though it doesn't completely 
close the window tahat unsynced WAL is replayed.

--   Heikki Linnakangas  EnterpriseDB   http://www.enterprisedb.com