Re: [GENERAL] the database system is shutting down - terminating walsender process due to replication timeout

Zarko Aleksic <zarko.aleksic@live.com> writes:
> I'm looking for a bit of help understanding a particular behavior we are seeing with our PostgreSQL 9.6. After
issuinga service shutdown command with "systemctl stop" on RHEL 7 our PostgreSQL instance started behaving weirdly. For
thefirst time it wouldn't shutdown so easily / quickly. 
> From the logs we could see that standby nodes that were trying to connect were rejected due to database being
shutdown.After wal_sender_timeout and wal_receiver_timeout (default 60s) were reached the database finally shut down.
Itseems that walsender process was preventing the shutdown of the master database - until timeout was reached, a
behaviorwe didn't experience before. 

9.6.what?

There were several possibly-relevant bug fixes in 9.6.3 and 9.6.4,
notably this one:

Author: Tom Lane <tgl@sss.pgh.pa.us>
Branch: master Release: REL_10_BR [fca85f8ef] 2017-06-30 12:00:15 -0400
Branch: REL9_6_STABLE Release: REL9_6_4 [e9d4aa594] 2017-06-30 12:00:03 -0400
Branch: REL9_5_STABLE Release: REL9_5_8 [446914f6b] 2017-06-30 12:00:03 -0400
Branch: REL9_4_STABLE Release: REL9_4_13 [5aa8db014] 2017-06-30 12:00:03 -0400
   Fix walsender to exit promptly if client requests shutdown.      It's possible for WalSndWaitForWal to be asked to
waitfor WAL that doesn't   exist yet.  That's fine, in fact it's the normal situation if we're caught   up; but when
theclient requests shutdown we should not keep waiting.   The previous coding could wait indefinitely if the source
serverwas idle.      In passing, improve the rather weak comments in this area, and slightly   rearrange some related
codefor better readability.      Back-patch to 9.4 where this code was introduced.      Discussion:
https://postgr.es/m/14154.1498781234@sss.pgh.pa.us

I think that would only apply if the walsender's client had tried to
disconnect at the same time you were doing the master-server shutdown,
but maybe that's what happened.

There is still work going on around the walsender timeout, so maybe
what you hit is an as-yet-unresolved bug, but in any case you should
be keeping up with minor releases.
        regards, tom lane


--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general