Re: buildfarm: strange OOM failures on markhor (running CLOBBER_CACHE_RECURSIVELY)

On 17.5.2014 23:09, Andres Freund wrote:
> Hi,
>
> On 2014-05-17 22:33:31 +0200, Tomas Vondra wrote:
>> Anyway, the main difference between the analyze snapshot seems to be this:
>>
>>   init:  CacheMemoryContext: 67100672 total in 17 blocks; ...
>>   350MB: CacheMemoryContext: 134209536 total in 25 blocks; ...
>>   400MB: CacheMemoryContext: 192929792 total in 32 blocks; ...
>>   500MB: CacheMemoryContext: 293593088 total in 44 blocks; ...
>>   600MB: CacheMemoryContext: 411033600 total in 58 blocks; ...
>
> Hm, so something is definitely leaking memory inside CacheMemoryContext
> itself. Is that happening constantly or just with individual tests?

I think it's happening constantly - I've been eyeballing the VM for some
time, and the backends usually hoover around ~220 MBs, executing SELECTs
etc. And then it executes CREATE INDEX and it starts growing.

Attached is a log for a CREATE INDEX session at the monent it had ~1.3GB
allocated (create-index-5.log.gz). It has ~217k copies of

pg_attrdef_adrelid_adnum_index: 3072 total in 2 blocks; 1968 free (0
chunks); 1104 used

And also another log (AFAIK from another CREATE INDEX session), with
~11k copies of

pg_namespace_oid_index: 1024 total in 1 blocks; 88 free (0 chunks); 936 used

>> Not sure if there's something wrong with the SELECT memory context. It
>> has ~1500 of nested nodes like these:
>>
>>    SQL function data: 24576 total in 2 blocks; ...
>>   ExecutorState: 24576 total in 2 blocks; ...
>>     SQL function data: 24576 total in 2 blocks; ...
>>     ExprContext: 8192 total in 1 blocks; ...
>>
>> But maybe it's expected / OK.
>
> I'd guess that's a recursive function call. Any chance you know what's
> been executing at that point? I'd bet it's been the 'errors' check. That
> has:
> -- Check that stack depth detection mechanism works and
> -- max_stack_depth is not set too high
> create function infinite_recurse() returns int as
> 'select infinite_recurse()' language sql;
> \set VERBOSITY terse
> select infinite_recurse();
> ERROR:  stack depth limit exceeded
>
> which'd pretty much produce a tree of executions like yours.

Yeah, I was thinking it's something like that. I'm not sure what tests
were running at that that particular moment, so maybe it really was the
'errors' check.

Tomas