Re: Memory leak with PL/Python trigger

On Fri, Jul 31, 2015 at 12:13 PM, Haribabu Kommi
<kommi.haribabu@gmail.com> wrote:
>
>
> On Fri, Jul 31, 2015 at 9:15 AM, Behn, Edward (EBEHN) <EBEHN@arinc.com>
> wrote:
>>
>> I believe that I have discovered a memory leak bug that affects PostgreS=
QL
>> 9.5 Alpha -1. (It also affects 9.4.1.)
>>
>>
>>
>> The leak appears in the server process when a PL/Python trigger function
>> is used to populate a composite type that contains a field that is a dom=
ain
>> type. This leak is worse if the domain type has a constraint.
>>
>> The attached files create a database that has this property (Schema.sql)
>> and add rows to the table causing the leak (Test.py).
>>
>>
>>
>> If the trigger function is implemented in PL/pgSQL, the leak disappears.
>>
>>
>>
>> After 1,000,000 rows, the server process has a memory footprint of 544 M=
iB
>> (~570 bytes/row) if the constraint is present on the domain type.
>>
>>
>>
>> After 1,000,000 rows, the server process has a memory footprint of 144 M=
iB
>> (~150 bytes/row) if the constraint is not present on the domain type.
>>
>>
>>
>> I am running PostgreSQL 9.5 Alpha-1 on Fedora 21. My installation is =E2=
=80=9Cout
>> of the box=E2=80=9D from the YUM repository. I have not altered any conf=
iguration
>> files. I have tested the code as user/role postgres.
>>
>>
>>
>>
>
> Thanks for the defect details.
> I am able to reproduce the issue on Head.
>
> This is because of many unnecessary "Expr Context" that are created in
> function "domain_check_input"
> using CreateStandaloneExprContext function under TopMemoryContext.
>
> This memory context is reset for future use by storing the context pointe=
r
> in "fcinfo->flinfo->fn_extra".
> But this pointer always set as NULL for every call. Because of this reaso=
n
> the memory was leaked.

Adding to this point, In "PLyObject_ToComposite" function the context point=
er
which was allocated earlier is freed without removing the context.
Because of this
reason for the next record, it again allocates the context and thus it
leading to a
memory leak.

Instead of reset the context in "domain_check_input" function, we can free =
that
context, which can solve this problem. But I am not sure, is it right fix?

Instead of the above fix, either we need to teach plpython functions to tak=
ing
care of reusing the pointer or something?

Regards,
Hari Babu
Fujitsu Australia