Re: Reducing buildfarm disk usage: remove temp installs when done

On 2015-01-18 17:48:11 -0500, Tom Lane wrote:
> One of the biggest causes of buildfarm run failures is "out of disk
> space".  That's not just because people are running buildfarm critters
> on small slow machines; it's because "make check-world" is an enormous
> space hog.  Some numbers from current HEAD:
> 
> clean source tree:        120MB
> built source tree:        400MB
> tree after make check-world:    3GB
> 
> (This is excluding ~250MB for one's git repo.)
> 
> The reason for all the bloat is the temporary install trees that we
> create, which tend to eat up about 100MB apiece, and there are dozens
> of them (eg, one per testable contrib module).  Those don't get removed
> until the end of the test run, so the usage is cumulative.
> 
> The attached proposed patch removes each temp install tree as soon as
> we're done with it, in the normal case where no error was detected.
> This brings the peak space usage down from ~3GB to ~750MB.

I was wondering before if we couldn't always do the the temp
installation into $top_builddir/tmp_install or something like it. With
an additional small ugly hacking ontop we could even avoid reinstalling
for every target in check-world.

Greetings,

Andres Freund

-- Andres Freund                       http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services