Re: Leaky Perl / DBIx / Postgres 9.0.1 Trio

Hi,

On 04/04/12 19:21, Eliot Gable wrote:
> I have a Perl daemon handling some events on a system which inserts or
> updates rows in Postgres, and something is causing memory usage to
> grow. I see the RSS memory size allocated to the Postgres connection
> continual going up (slowly) and the RSS size allocated to the Perl
> daemon also continually going up (slowly). The Perl daemon is using
> DBIx to make insert and update queries to the Postgres DB.

I believe DBIx is more of a namespace than a specific module - do you
mean DBIx::Class or DBI perhaps? If the former, the IRC channel
#dbix-class on irc.perl.org tends to be very helpful in tracing problems
like this, particularly if you can provide a reproducible test case.

> There is only one connection open between the Perl daemon and the
> Postgres DB. That connection stays open until the daemon exits or is
> killed. When the daemon is killed, the Postgres back end is terminated
> and all of its memory is released.
>
> Any suggestions on how to go about troubleshooting this would be
> useful. Most specifically, I would like to know how to find out what
> the Postgres process for that connection is doing with all that
> memory. Is there a way I can see what the memory is allocated for?
> Would running GDB on the Postgres process help? Is there any logging I
> can turn on which might help shed some light on what's using the
> memory? Anything in particular anyone knows about which I should look
> for which might cause both Perl and Postgres memory usage to climb
> continually?

On the Perl side, Devel::Size and possible Test::MemoryGrowth may be of
some use in tracing where the memory is going. Does the memory usage hit
a ceiling at any point, or does it carry on until all virtual memory is
exhausted? You could try running the postgres and perl daemons with
lower ulimit -m / ulimit -v settings if the increase in memory is too
slow to test this in a reasonable timeframe.

Other than that, maybe try looking for statement handles that aren't
closed or long-running transactions?

best regards,

Tom