Re: buildfarm + meson

On 2023-02-23 06:27:23 -0500, Andrew Dunstan wrote:

On 2023-02-22 We 20:20, Andres Freund wrote:

There is work to do to make sure we pick up the right log files, and maybe
adjust a module or two. I have adopted a design where instead of trying to
know a lot about the testing regime the client needs to know a lot less.
Instead, it gets meson to tell it the set of tests. I will probably work on
enabling some sort of filter, but I think this makes things more
future-proof. I have stuck with the design of making testing fairly
fine-grained, so each suite runs separately.

I don't understand why you'd want to run each suite separately. Serially
executing the test takes way longer than doing so in parallel. Why would we
want to enforce that?

Particularly because with meson the tests log files and the failed tests can
directly be correlated? And it should be easy to figure out which log files
need to be kept, you can just skip the directories in testrun/ that contain
test.success.

We can revisit that later. For now I'm more concerned with getting a working
setup.

My fear is that this ends up being entrenched in the design and hard to change
later.

The requirements of the buildfarm are a bit different from those of a
developer, though. Running things in parallel can make things faster, but
that can also increase the compute load.

Sure, I'm not advocating to using a [high] concurrency by default.

Perhaps the latest version will be more to your taste. This is now working on my MSVC test rig (WS2019, VS2019, Strawberry Perl), including TAP tests. I do get a whole lot of annoying messages like this:

Unknown TAP version. The first line MUST be `TAP version <int>`. Assuming version 12.

Anyway, I think this is ready for any brave soul who wants to take it for a test run, not on a reporting animal just yet, though. To activate it you need the config to have 'using_meson => 1' and a meson_opts section - see the sample file. You can get the dev/meson version at <https://github.com/PGBuildFarm/client-code/archive/refs/heads/dev/meson.zip>

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com