On Thu, Oct 19, 2023 at 6:09 PM Andres Freund <andres@anarazel.de> wrote:
> Hm. Did you not run into simmilar issues with make check-world? I found the
> concurrency of that to be even more variable over a run.
I did not, but I didn't generally run that in parallel, either, mostly
for fear of being unable to see failures properly in the output. I
used -j8 when building, though.
> But perhaps there's something else wrong here? Perhaps we should deal with
> this in Cluster.pm to some degree? Controlling this from the level of meson
> (or make/prove for that matter) doesn't really work well, because different
> tests start differently many postgres instances.
I'm not sure, but I'm open to however anybody would like to improve things.
> How many cores does your machine have? I've run the tests in a loop on my m1
> mac mini in the past without running into this issue. It has "only" 8 cores
> though, whereas I infer, from you mentioning -j8, that you have more cores?
System Information shows "Total Number of Cores: 8" but sysctl hw.ncpu
returns 16. No real idea what is fastest, I haven't been real
scientific about choosing values for -j.
> > My next thought was that there ought to be some environmental variable
> > that I could set to control this behavior. But I can't find a list of
> > environment variables that affect meson behavior anywhere. I guess the
> > authors don't believe in environment variable as a control mechanism.
>
> They indeed do not like them - but there is one in this
> case: MESON_TESTTHREADS
>
> There's even documentation for it: https://mesonbuild.com/Unit-tests.html#parallelism
I mean, I probably glanced at that page at some point, but it's hardly
obvious that there's a mention of an environment variable buried
somewhere in the middle of the page. Most of the code you see looking
at the page is Python, and the other environment variables mentioned
seem to be ones that it sets, rather than ones that you can set. They
really ought to have a documentation page somewhere that lists all of
the environment variables that the user can set, and maybe another one
that lists all the ones that the tool itself sets before running
subprocesses. You can't expect people to navigate through every page
of the documentation and read every word on the page carefully to find
stuff like this.
--
Robert Haas
EDB: http://www.enterprisedb.com