Re: tests and meson - test names and file locations

On 11.08.22 06:04, Andres Freund wrote:
> At the moment there's three suites differentiating by the type of test:
> 'pg_regress', 'isolation' and 'tap'. There's also a separate "axis" of suites,
> describing what's being tested, e.g. 'main', 'test_decoding', 'recovery' etc.
> 
> That currently works out to each test having two suites, although I've
> wondered about adding a 'contrib' suite as well.

I'm not sure what the value of these suite names would be.  I don't 
usually find myself wanting to run, say, just all tap tests.

Perhaps suites would be useful to do things like select slow tests 
(numeric_big) or handle the ssl, ldap, kerberos tests.

> = Log and Data locations =
> 
> To make things like the selection of log files for a specific test easier,
> I've so far set it up so that test data and logs are stored in a separate
> directory from the sources.
> 
> testrun/<main|recovery|...>/<testname>/<log|tmp_check|results...>
> 
> The runner now creates a test.start at the start of a test and either
> test.success or test.failure at the end. That should make it pretty easy for
> e.g. the buildfarm and CI to make the logs for a failed test easily
> accessible. I've spent far too much time going through the ~hundred logs in
> src/test/recovery/ that the buildfarm displays as one thing.

I don't really understand which problem this solves and how.  Sure, the 
test output is somewhat complex, but I know where it is and I've never 
found myself wishing it to be somewhere else.