Re: Reducing buildfarm disk usage: remove temp installs when done

On 1/19/15 1:07 PM, Andres Freund wrote:
> On 2015-01-18 17:48:11 -0500, Tom Lane wrote:
>> One of the biggest causes of buildfarm run failures is "out of disk
>> space".  That's not just because people are running buildfarm critters
>> on small slow machines; it's because "make check-world" is an enormous
>> space hog.  Some numbers from current HEAD:
>>
>> clean source tree:        120MB
>> built source tree:        400MB
>> tree after make check-world:    3GB
>>
>> (This is excluding ~250MB for one's git repo.)
>>
>> The reason for all the bloat is the temporary install trees that we
>> create, which tend to eat up about 100MB apiece, and there are dozens
>> of them (eg, one per testable contrib module).  Those don't get removed
>> until the end of the test run, so the usage is cumulative.
>>
>> The attached proposed patch removes each temp install tree as soon as
>> we're done with it, in the normal case where no error was detected.
>> This brings the peak space usage down from ~3GB to ~750MB.
>
> I was wondering before if we couldn't always do the the temp
> installation into $top_builddir/tmp_install or something like it. With
> an additional small ugly hacking ontop we could even avoid reinstalling
> for every target in check-world.

FWIW, if anyone's going to do some serious tinkering in here; it'd be really nice to create a separate utility for
managingtemporary installs. That would make it trivial for PGXN modules to use something other than pg_regress for
theirtest framework.
 
-- 
Jim Nasby, Data Architect, Blue Treble Consulting
Data in Trouble? Get it in Treble! http://BlueTreble.com