Re: Excessive memory usage in multi-statement queries w/ partitioning

Horiguchi-san,

Thanks for the comment.  My reply is a bit late now, but....

On Wed, Jul 10, 2019 at 5:39 PM Kyotaro Horiguchi
<horikyota.ntt@gmail.com> wrote:
> At Wed, 10 Jul 2019 16:35:18 +0900, Amit Langote <amitlangote09@gmail.com> wrote:
> -     * Switch to appropriate context for constructing querytrees (again,
> -     * these must outlive the execution context).
> +     * Switch to appropriate context for constructing query and plan trees
> +     * (again, these must outlive the execution context).  Normally, it's
> +     * MessageContext, but if there are more queries to plan, we use a
> +     * temporary child context that will be reset after executing this
> +     * query.  We avoid that overhead of setting up a separate context
> +     * for the common case of having just a single query.
>
> Might be stupid, but I feel uneasy that "usually it must live in
> MessageContxt, but not necessarily if there is succeeding
> query".. *I* need more explanation why it is safe to use that
> short-lived context.

So the problem we're trying solve with this is that memory consumed
when parsing/planning individual statements pile up in a single
context (currently, MessageContext), which can lead to severe bloat
especially if the planning of individual statements consumes huge
amount of memory.  The solution is to use a sub-context of
MessageContext for each statement that's reset when its execution is
finished.  I think it's safe to use a shorter-lived context for each
statement because the memory of a given statement should not need to
be referenced when its execution finishes.  Do you see any problem
with that assumption?

Thanks,
Amit