pgsql: Fix old-fd issues using global barriers everywhere.

Fix old-fd issues using global barriers everywhere.

Commits 4eb21763 and b74e94dc introduced a way to force every backend to
close all relation files, to fix an ancient Windows-only bug.

This commit extends that behavior to all operating systems and adds
a couple of extra barrier points, to fix a totally different class of
bug: the reuse of relfilenodes in scenarios that have no other kind of
cache invalidation to prevent file descriptor mix-ups.

In all releases, data corruption could occur when you moved a database
to another tablespace and then back again.  Despite that, no back-patch
for now as the infrastructure required is too new and invasive.  In
master only, since commit aa010514, it could also happen when using
CREATE DATABASE with a user-supplied OID or via pg_upgrade.

Author: Andres Freund <andres@anarazel.de>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/20220209220004.kb3dgtn2x2k2gtdm%40alap3.anarazel.de

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/e2f65f42555ff531c6d7c8f151526b4ef7c016f8

Modified Files
--------------
src/backend/commands/dbcommands.c            |   9 +-
src/backend/commands/tablespace.c            |  11 +-
src/include/pg_config_manual.h               |  11 --
src/test/recovery/Makefile                   |   2 +-
src/test/recovery/t/032_relfilenode_reuse.pl | 233 +++++++++++++++++++++++++++
5 files changed, 241 insertions(+), 25 deletions(-)