Re: excessive amounts of consumed memory (RSS), triggering OOM killer

On 2.12.2014 01:33, Tom Lane wrote:
> Tomas Vondra <tv@fuzzy.cz> writes:
>> On 2.12.2014 00:31, Andrew Dunstan wrote:
>>> Doesn't this line:
>>> TopMemoryContext: 136614192 total in 16678 blocks; 136005936 free
>>> (500017 chunks); 608256 used
>>> look pretty suspicious?
> 
>> It certainly looks a bit large, especially considering this is a fresh
>> cluster with a single DB, containing a single table (created by user),
>> no doubt about that.
>> OTOH, it's "just" 130MB, and the RSS values are ~6GB when the OOM hits.
> 
> Yeah, but what was that 130MB being used for?  It's not generally
> considered good practice to put large or random stuff in TopMemoryContext
> --- arguably, any code doing so is broken already, because almost by
> definition such allocations won't be reclaimed on error.
> 
> What I suspect you're looking at here is the detritus of creation of
> a huge number of memory contexts. mcxt.c keeps its own state about
> existing contents in TopMemoryContext. So, if we posit that those
> contexts weren't real small, there's certainly room to believe that
> there was some major memory bloat going on recently.

Aha! MemoryContextCreate allocates the memory for the new context from
TopMemoryContext explicitly, so that it survives resets of the parent
context. Is that what you had in mind by keeping state about existing
contexts?

That'd probably explain the TopMemoryContext size, because array_agg()
creates separate context for each group. So if you have 1M groups, you
have 1M contexts. Although I don's see how the size of those contexts
would matter?

Maybe we could move this info (list of child contexts) to the parent
context somehow, so that it's freed when the context is destroyed? Aside
from the regular blocks, of course. But maybe it's not worth the effort,
because you can only have so many memory contexts created concurrently.
For AllocSet it's at most RAM/1kB (because that's the minimum block
size) before the OOM intervenes. And in this case the contexts have 8kB
initial size, so the number of contexts is even lower. So maybe it's not
worth the effort.

Also, this explains the TopMemoryContext size, but not the RSS size (or
am I missing something)?

regards
Tomas