Обсуждение: low wal_retrieve_retry_interval causes missed signals on Windows

I discussed this elsewhere [0], but I thought it deserved its own thread.

After setting wal_retrieve_retry_interval to 1ms in the tests, I noticed
that some of the archiving tests began consistently failing on Windows.  I
believe the problem is that WaitForWALToBecomeAvailable() depends on the
call to WaitLatch() for wal_retrieve_retry_interval to ensure that signals
are dispatched (i.e., pgwin32_dispatch_queued_signals()).  With a low retry
interval, WaitForWALToBecomeAvailable() might skip the call to WaitLatch(),
and the signals are never processed.

The attached patch fixes this by always calling WaitLatch(), even if
wal_retrieve_retry_interval milliseconds have already elapsed and the
timeout is 0.

[0] https://postgr.es/m/20221231235019.GA1223171%40nathanxps13

-- 
Nathan Bossart
Amazon Web Services: https://aws.amazon.com

Hi,

On 2023-01-10 22:11:16 -0800, Nathan Bossart wrote:
> The attached patch fixes this by always calling WaitLatch(), even if
> wal_retrieve_retry_interval milliseconds have already elapsed and the
> timeout is 0.

It doesn't seem right to call WaitLatch() just for that purpose - nor
necessarily a complete fix.

Given that we check for interrupts in other parts of recovery with
HandleStartupProcInterrupt(), which doesn't interact with latches, isn't the
actual bug that HandleStartupProcInterrupt() doesn't contain the same black
magic that CHECK_FOR_INTERRUPTS() contains on windows?  Namely this stuff:


#ifndef WIN32
...
#else
#define INTERRUPTS_PENDING_CONDITION() \
    (unlikely(UNBLOCKED_SIGNAL_QUEUE()) ? pgwin32_dispatch_queued_signals() : 0, \
     unlikely(InterruptPending))
#endif

/* Service interrupt, if one is pending and it's safe to service it now */
#define CHECK_FOR_INTERRUPTS() \
do { \
    if (INTERRUPTS_PENDING_CONDITION()) \
        ProcessInterrupts(); \
} while(0)


Looks like we have that bug in quite a few places... Some are "protected" by
unconditional WaitLatch() calls, but at least pgarch.c, checkpointer.c via
CheckpointWriteDelay() seem borked.


Greetings,

Andres Freund

On Wed, Jan 11, 2023 at 12:48:36PM -0800, Andres Freund wrote:
> Given that we check for interrupts in other parts of recovery with
> HandleStartupProcInterrupt(), which doesn't interact with latches, isn't the
> actual bug that HandleStartupProcInterrupt() doesn't contain the same black
> magic that CHECK_FOR_INTERRUPTS() contains on windows?  Namely this stuff:

Yeah, this seems like a more comprehensive fix.  I've attached a patch that
adds this Windows signaling stuff to the HandleXXXInterrupts() functions in
the files you listed.  Is this roughly what you had in mind?  If so, I'll
look around for anywhere else it is needed.

-- 
Nathan Bossart
Amazon Web Services: https://aws.amazon.com

Hi,

On 2023-01-11 15:26:45 -0800, Nathan Bossart wrote:
> On Wed, Jan 11, 2023 at 12:48:36PM -0800, Andres Freund wrote:
> > Given that we check for interrupts in other parts of recovery with
> > HandleStartupProcInterrupt(), which doesn't interact with latches, isn't the
> > actual bug that HandleStartupProcInterrupt() doesn't contain the same black
> > magic that CHECK_FOR_INTERRUPTS() contains on windows?  Namely this stuff:
> 
> Yeah, this seems like a more comprehensive fix.  I've attached a patch that
> adds this Windows signaling stuff to the HandleXXXInterrupts() functions in
> the files you listed.  Is this roughly what you had in mind?  If so, I'll
> look around for anywhere else it is needed.

Yes, that's what I roughly was thinking of. Although seeing the diff, I think
it might be worth introducing a helper function that'd containing at least
pgwin32_dispatch_queued_signals() and ProcessProcSignalBarrier(). It's a bit
complicated by ProcessProcSignalBarrier() only being applicable to shared
memory connected processes - excluding e.g. pgarch.

Greetings,

Andres Freund

On Wed, Jan 11, 2023 at 04:40:14PM -0800, Andres Freund wrote:
> On 2023-01-11 15:26:45 -0800, Nathan Bossart wrote:
>> On Wed, Jan 11, 2023 at 12:48:36PM -0800, Andres Freund wrote:
>> > Given that we check for interrupts in other parts of recovery with
>> > HandleStartupProcInterrupt(), which doesn't interact with latches, isn't the
>> > actual bug that HandleStartupProcInterrupt() doesn't contain the same black
>> > magic that CHECK_FOR_INTERRUPTS() contains on windows?  Namely this stuff:
>> 
>> Yeah, this seems like a more comprehensive fix.  I've attached a patch that
>> adds this Windows signaling stuff to the HandleXXXInterrupts() functions in
>> the files you listed.  Is this roughly what you had in mind?  If so, I'll
>> look around for anywhere else it is needed.
> 
> Yes, that's what I roughly was thinking of. Although seeing the diff, I think
> it might be worth introducing a helper function that'd containing at least
> pgwin32_dispatch_queued_signals() and ProcessProcSignalBarrier(). It's a bit
> complicated by ProcessProcSignalBarrier() only being applicable to shared
> memory connected processes - excluding e.g. pgarch.

As of d75288f, the archiver should be connected to shared memory, so we
might be in luck.  I guess we'd need to watch out for this if we want to
back-patch it beyond v14.  I'll work on a patch...

-- 
Nathan Bossart
Amazon Web Services: https://aws.amazon.com