Re: Memory management with C aggregate

Tom Lane wrote:
> Ian Burrell <imb@rentrak.com> writes:
> 
>>We have user-defined aggregates written in C running inside the server. 
>>  We are running into some memory management issues and wondering what 
>>is the best way to solve the problem.
> 
> 
>>The state of the aggregates is a structure with a pointer to allocated 
>>memory.  The structure and memory are allocated in the 
>>PortalMemoryContext and freed in the ffunc.  We just discovered that the 
>>ffunc function can be called multiple times with the same state on 
>>certain queries.  The double free causes a crash.
> 
> 
> AFAICS the ffunc should be called only once per aggregated group.
> Perhaps your code is confused about the handling of groups?  If so,
> the double free is hardly your only worry --- you'll be computing wrong
> answers anyway till you fix that.
> 

The aggregate is in a subquery which might make a difference.  The ffunc 
is only called multiple times when a nested loop is used in the query 
plan.  With other query plans, the ffunc is only called once.  The ffunc 
seems to be called once for every combination.

I can't get the following query to use a nested loop, but our query is 
similar, but with a compound key and a custom aggregate.  If x has N 
rows, y has M, with N groups, then the ffunc can be called N*M times, M 
times for each N group.

SELECT x.key_no, s.agg
FROM x
INNER JOIN (        SELECT y.key_no, custom_agg(num) AS agg        FROM y        GROUP BY key_no
) s
USING (key_no)
ORDER BY key_no

I'll try to come up with a test case that illustrates the problem.
 - Ian