Re: can we add subscription TAP test option "vcregress subscriptioncheck" for MSVC builds?

On Sat, Oct 16, 2021 at 6:35 PM Andrew Dunstan <andrew@dunslane.net> wrote:
>
> > The problems with having "vcregress checkworld" are: 1) required code
> > modifications are more as the available "vcregress" test functions,
> > which required pre-started pg instance, can't be used directly. 2) it
> > looks like spinning up a separate postgres instance for all module
> > tests takes time on Windows which might make the test time longer. If
> > we were to have "vcregress installcheckworld", we might have to add
> > new code for converting some of the existing functions to not use a
> > pre-started pg instance.
> >
> > IMHO, we can just have "vcregress subscriptioncheck" and let users
> > decide which tests they want to run.
> >
> > I would like to hear more opinions on this.
> >
>
> My opinion hasn't changed. There is already a way to spell this and I'm
> opposed to adding more such specific tests to vcregress.pl. Every such
> addition we make increases the maintenance burden.

Thanks for your opinion. IIUC, the subscription tests can be run with
setting environment variables PROVE_FLAGS, PROVE_TESTS and the
"vcregress taptest" command right? I failed to set the environment
variables appropriately and couldn't run. Can you please let me know
the right way to run the test?

If any test can be run with a set of environment flags and "vcregress
taptest" command, then in the first place, it doesn't make sense to
have recoverycheck, upgragecheck and so on. Another thing is that the
list of "vcregress" commands cover almost all the tests core, tap,
bin, isolation, contrib tests except, subscription tests. If we add
"vcregress subscrtptioncheck", the list of tests that can be run with
the "vcregress" command will be complete without having to depend on
the environment variables.

IMHO, we can have "vcregress subscriptioncheck" to make it complete
and easier for the user to run those tests. However, let's hear what
other hackers have to say about this.

Another thing I noticed is that we don't have any mention of
"vcregress taptest" command in the docs [1], if I read the docs
correctly. How about we have it along with a  sample example on how to
run a specific TAP tests with it in the docs?

[1] - https://www.postgresql.org/docs/current/install-windows-full.html

Regards,
Bharath Rupireddy.