On Thu, 4 Apr 2024 at 23:06, Daniel Gustafsson <daniel@yesql.se> wrote:
>
> > On 4 Apr 2024, at 23:02, Jelte Fennema-Nio <postgres@jeltef.nl> wrote:
>
> > How about we make it meson/make targets, so they are simply cached
> > just like any of our other build artefacts are cached. Then only clean
> > builds are impacted, not every test run.
>
> They already are (well, make not meson yet), they're just not hooked up to any
> top-level commands. Running "make ssfiles-clean ssfiles" in src/test/ssl
> regenerates all the files from the base config files that define their
> contents.
Okay turns out even generating them in parallel isn't any faster than
running that sequentially. I guess it's because of the strong random
generation being the slow part. Command I used was the following and
took ~5 seconds on my machine:
make -C src/test/ssl sslfiles-clean && make -C src/test/ssl sslfiles -j20
And I think that's actually a good thing, because that would mean
total build time is pretty much not impacted if we'd include it as
part of the regular clean build. Since building these certs are
bottlenecked on randomness, not on CPU (as pretty much all of our
other build artifacts are). So they should pipeline pretty very well
with the other items, assuming build concurrency is set slightly
higher than the number of cores.
As a proof of concept I ran the above command in a simple bash loop constantly:
while true; do make -C src/test/ssl sslfiles-clean && make -C
src/test/ssl sslfiles -j20; done
And then ran a clean (parallel) build in another shell:
ninja -C build clean && ninja -C build
And total build time went from 41 to 43 seconds. To be clear, that's
while constantly running the ssl file creation. If I only run the
creation once, there's no noticeable increase in build time at all.