Re: [HACKERS] psql & regress tests

Bruce Momjian <pgman@candle.pha.pa.us> writes:
>> It wouldn't be particularly "dangerous" if we assume that no one else is
>> accessing the regression database.  What it *would* be is less useful at
>> catching problems.  Standalone mode wouldn't test the cross-backend
>> interlocking code at all.

> Any modifications to shared pg_ tables would be a problem.  Also, pg_log
> and pg_variable locking is not happening in there either, is it?

Good point --- it wouldn't be just that database, but the whole
installation (data directory) that would have to be unused.  You really
wouldn't dare even have a postmaster running, at least not in the same
data directory.  But that could be made safe by using a nonstandard
location for the data directory for regression.

However, this is all beside the main point: we want the regress tests
to run in an environment as close as possible to the way Postgres is
normally used.  The more we hack up a special regress-test environment,
the less the tests reflect reality.

Aside from the cross-backend synchronization issue, I forgot to point
out a really obvious benefit: doing it the current way allows the regress
tests to help check the backend's frontend communication code, and
libpq, and psql itself.  We'd need some other way of testing all that
code if we switched to a standalone-backend regression test set.

What I *would* like to see is more support for running regress tests on
a not-yet-installed build, so people can test a fresh build before they
blow away their working installation.  This requires doing an initdb
into a temporary directory, starting a postmaster therein (using a
nonstandard port number), and running the tests there.  This is doable
by hand, of course, but it's tedious and error-prone even for an expert;
I think it's out of the question for a novice installer.  We ought to
offer a canned script to do it that way.
        regards, tom lane