Re: improving speed of make check-world

On 8/31/14 5:36 AM, Fabien COELHO wrote:
> Running "make -j2 check-world" does not work because "initdb" is not
> found by "pg_regress". but "make -j1 check-world" does work fine. It
> seems that some dependencies might be missing and there is a race
> condition between temporary install and running some checks?? Maybe it
> is not expected to work anyway? See below suggestions to make it work.

Here is an updated patch that fixes this problem.

The previous problem was simply a case were the make rules were not
parallel-safe.

For recursive make, we (through magic) set up targets like this:

    check: check-subdir1-check check-subdir2-check

And with my old patch we added

    check: temp-install

So the aggregate prerequisites were in effect something like

    check: check-subdir1-check check-subdir2-check temp-install

And so there was nothing stopping a parallel make to descend into the
subdirectories before the temp install was set up.

What we need is additional prerequisites like

    check-subdir1-check check-subdir2-check: temp-install

I have hacked this directly into the $(recurse) function, which is ugly.
 This could possibly be improved somehow, but the effect would be the
same in any case.

With this, I can now run things like

    make -C src/pl check -j3
    make -C src/bin check -j8

A full make check-world -j$X is still prone to fail because some test
suites can't run in parallel with others, but that's a separate problem
to fix.


Note:  We have in the meantime added logic to pg_regress to clean up the
temporary installation after the run.  This changes that because
pg_regress is no longer responsible for the temporary installation.
pg_regress still cleans up the temporary data directory, so you still
get quite a bit of space savings.  But the temporary installation is not
cleaned up.  But since we would now only use a single temporary
installation, the disk space usage still stays in the same order of
magnitude.