Re: Perl function leading to out of memory error

On 19/02/13 22:05, Jan Strube wrote:
> we have a Java daemon that´s repeatedly calling a Perl function inside
> our database (version 9.1.8). The function is called about 200 times per
> second. While the Java program is running you can watch the memory usage
> of the postmaster grow continuously until after a few hours we get an
> out of memory error from Postgres. In the log you see a lot of
> "ExprContext..." messages. When the daemon is killed the memory is
> freed.The daemon is using a single database connection during it´s runtime.
>
> You can see the function definition here:
> http://www.deriva.de/tmp/get_comment.txt

A minor point, but are you aware that Perl has a switch syntax
(given/where keywords) which could replace the "for($table) /regex/ &&
do { ... last; }" stuff?

perldoc -f given


On a more useful point, can I suggest that you extract your program into
a standalone script (mock up something to provide the elog/spi_exec
calls). Then you can run that thousands of times to check if it is
leaking memory. You can also run it up against something such as
Devel::Leak:
http://search.cpan.org/~srezic/Devel-Leak/Leak.pm

As another posted commented, you may well be leaking the %data
structure, since it's being referenced from inside a closure. Perl isn't
totally stupid, and should clean up %data once $get_value goes out of
scope, but it's worth checking you aren't keeping a reference somewhere.

Cheers,
Toby