Re: Out of Memory errors are frustrating as heck!

Gunther <raj@gusw.net> writes:
> So what I am wondering now, is there seems to be an EXPLOSION of memory 
> consumption near the time of the crash. That ExecutorState has 
> 2,238,648,944 but just until the very last second(s) the RES memory as 
> per top was 1.5 GB I swear.

That's not hugely surprising really, especially in a complex query.
It could be performing some preliminary join that doesn't leak, and
then when it starts to perform the join that does have the leak,
kaboom.  Also, given that you seem to be invoking multi-batch joins,
maybe the preliminary phase is fine and there's only a leak when
reading back a batch.

Anyway, the upshot is that you need to investigate what's happening
while the memory consumption is increasing.  The behavior before
that starts to happen isn't going to be very interesting.  It might
be a bit tricky to catch that if it only takes a few seconds to blow
up, but you could try "c 10000" or so to step through a lot of
AllocSetAlloc calls, repeating till the bad stuff starts to happen,
and then going back to looking at just where the calls are coming
from.

            regards, tom lane