Re: Memory leak in Pl/Python

Andrey Zhidenkov <andrey.zhidenkov@gmail.com> writes:
> I see memory consumption in htop and pg_activity tools.

"top" can be pretty misleading if you don't know how to interpret its
output, specifically that you have to discount whatever it shows as
SHR space.  That just represents the amount of the shared memory block
that this process has touched so far in its lifetime; even if it appears
to be growing, it's not a leak.  That growth will stop eventually, once
the process has touched every available shared buffer.  RES minus SHR
is a fairer estimate of the process's own memory consumption.

I tried to reduce your example to a self-contained test case, thus:

create extension if not exists plpythonu;
create table test (test text);
create or replace
function test() returns bigint as $$
plpy.execute("insert into test(test) values ('test')")
return 1
$$ language plpythonu;
do $$
begin for i in 1..10000000 loop   perform test(); end loop;
end;
$$;

I do not see any significant leakage with this example.  There is some
memory growth, approximately 4 bytes per plpy.execute(), due to having to
keep track of a subtransaction XID for each uncommitted subtransaction.
That's not plpython's fault --- it would happen with any PL that executes
each SQL command as a separate subtransaction, which is probably all of
them other than plpgsql.  And it really ought to be negligible anyway in
any sane usage.

It's possible that you're seeing some other, larger memory consumption;
for instance, if there were triggers or foreign keys on the "test" table
then perhaps there would be an opportunity for leakage in those.
But without a self-contained test case or any indication of the rate
of leakage you're seeing, it's hard to guess about the problem.
        regards, tom lane