Re: Rewriting the test of pg_upgrade as a TAP test

On Tue, Apr 4, 2017 at 7:38 AM, Stephen Frost <sfrost@snowman.net> wrote:
> Not good if it lowers the coverage, but hopefully that's fixable. Have you
> analyzed where we're reducing coverage..?

The current set of tests is just running pg_upgrade using the same
version for the source and target instances. Based on that I am not
lowering what is happening in this set of tests. Just doing some
cleanup.

> As for what I'm remembering, there's this:
> https://www.postgresql.org/message-id/5669acd9-efdc-2a0f-afea-10ba6003a050@dunslane.net
>
> Of course, it's possible I misunderstood..

This invokes directly pg_upgrade, so that's actually a third,
different way to test pg_upgrade on top of the two existing methods
that are used in vcregress.pl and pg_upgrade's test.sh

> That seems focused on upgrading and I'd really like to see a general way to
> do this with the TAP structure, specifically so we can test pg_dump and psql
> against older versions.  Having the ability to then be run under the
> coverage testing would be fantastic and would help a great deal with the
> coverage report.

I don't disagree with that. What we need first is some logic to store
in a temporary directory the installation of all the previous major
versions that we have. For example use a subfolder in tmp_install
tagged with the major version number, and then when the TAP test
starts we scan for all the versions present in tmp_install and test
the upgrade with a full grid. One issue though is that we add
$(bindir) in PATH and that there is currently no logic to change PATH
automatically depending on the major/minor versions you are working
on.

So in short I don't think that this lack of infrastructure should be a
barrier for what is basically a cleanup but... I just work here.
-- 
Michael