While troubleshooting one of our production replication clusters (phys. streaming replication using the patroni framework)
I stumbled over a - at least for me - strange phenomenon in the postgres logs of the two cluster members:
*** node-01 ***
[postgres@db-node-01 main]$ grep 'LOG: database' postgresql-2022-06-05.log
time=2022-06-05 18:25:26 CEST, pid=1720 LOG: database system is shut down
time=2022-06-05 18:25:29 CEST, pid=3252374 LOG: database system was shut down at 2022-06-05 18:25:23 CEST
time=2022-06-05 18:25:31 CEST, pid=3252371 LOG: database system is ready to accept read only connections
time=2022-06-05 18:29:11 CEST, pid=3252371 LOG: database system is ready to accept connections
time=2022-06-05 18:32:01 CEST, pid=1816 LOG: database system was interrupted while in recovery at log time 2022-06-05 18:29:11 CEST
time=2022-06-05 18:32:03 CEST, pid=1813 LOG: database system is ready to accept read only connections
time=2022-06-05 19:00:26 CEST, pid=1813 LOG: database system is ready to accept connections
*** node-02 ***
[postgres@db-node-02 main]$ grep 'LOG: database' postgresql-2022-06-05.log
time=2022-06-05 18:25:26 CEST, pid=1720 LOG: database system is shut down
time=2022-06-05 18:25:29 CEST, pid=3252374 LOG: database system was shut down at 2022-06-05 18:25:23 CEST
time=2022-06-05 18:25:31 CEST, pid=3252371 LOG: database system is ready to accept read only connections
time=2022-06-05 18:29:11 CEST, pid=3252371 LOG: database system is ready to accept connections
time=2022-06-05 18:32:01 CEST, pid=1816 LOG: database system was interrupted while in recovery at log time 2022-06-05 18:29:11 CEST
time=2022-06-05 18:32:03 CEST, pid=1813 LOG: database system is ready to accept read only connections
time=2022-06-05 19:00:26 CEST, pid=1813 LOG: database system is ready to accept connections
The output is by no means complete - I only kept the duplicate entries.
My question:
How is it possible that the error logs are 'merged' across the two database nodes? Are the message/error-logs also replicated? Is this the intended behaviour?
Thank you in advance for shedding light on this.
KR p.