Re: Improve TAP tests of pg_upgrade for cross-version tests

On Sun, Oct 02, 2022 at 10:02:37AM -0700, Andres Freund wrote:
> This fails tests widely, and has so for a while:
> https://cirrus-ci.com/build/4862820121575424
> https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest/39/3649
>
> Note that it causes timeouts, which end up chewing up a cfbot "slot" for an
> hour...

Sorry for kicking the can down the road for a too-long time.  Attached
is an updated patch that I have strimmed down to the minimum that
addresses all the issues I want fixed as of the scope of this thread:
- The filtering of the dumps is reduced to a minimum, removing only
comments and empty lines.
- The configuration of the old and new nodes is tweaked so as it is
abvle to handle upgrade from nodes older than 10.
- pg_dumpall is tweaked to use --extra-float-digits=0 for the old
nodes older than 11 to minimize the amount of diffs generated.

That's quite nice in itself, as it becomes possible to use much more
dump patterns loaded as part of the tests.  More filtering rules could
be used, like the part about procedures and functions that I have sent
in the previous versions, but what I have here is enough to make the
test complete with all the versions supported by Cluster.pm.  I am
thinking to get this stuff applied soon before moving on to the PATH
issue.

There is still one issue reported upthread by Justin about the fact
that we can use unexpected commands depending on the node involved.
For example, something like that would build a PATH so as psql from
the old node is used, not from the new node:
$newnode->command_ok(['psql', '-X', '-f', 'blah.sql']);

So with the current Cluster.pm, we could fail upgrade_adapt.sql if the
version upgraded from does not support psql's \if, and I don't think
that we should use a full path to the binary either.  I am not
completely done analyzing that and this deserves a separate thread, as
it impacts all the commands used in TAP tests manipulating nodes from
multiple versions.
--
Michael