Leaky Perl / DBIx / Postgres 9.0.1 Trio
I have a Perl daemon handling some events on a system which inserts or
updates rows in Postgres, and something is causing memory usage to grow. I
see the RSS memory size allocated to the Postgres connection continual
going up (slowly) and the RSS size allocated to the Perl daemon also
continually going up (slowly). The Perl daemon is using DBIx to make insert
and update queries to the Postgres DB.
There is only one connection open between the Perl daemon and the Postgres
DB. That connection stays open until the daemon exits or is killed. When
the daemon is killed, the Postgres back end is terminated and all of its
memory is released.
Any suggestions on how to go about troubleshooting this would be useful.
Most specifically, I would like to know how to find out what the Postgres
process for that connection is doing with all that memory. Is there a way I
can see what the memory is allocated for? Would running GDB on the Postgres
process help? Is there any logging I can turn on which might help shed some
light on what's using the memory? Anything in particular anyone knows about
which I should look for which might cause both Perl and Postgres memory
usage to climb continually?
Thanks in advance for any suggestions.
--
Eliot Gable
"We do not inherit the Earth from our ancestors: we borrow it from our
children." ~David Brower
"I decided the words were too conservative for me. We're not borrowing from
our children, we're stealing from them--and it's not even considered to be
a crime." ~David Brower
"Esse oportet ut vivas, non vivere ut edas." (Thou shouldst eat to live;
not live to eat.) ~Marcus Tullius Cicero
Hi,
On 04/04/12 19:21, Eliot Gable wrote:
I have a Perl daemon handling some events on a system which inserts or
updates rows in Postgres, and something is causing memory usage to
grow. I see the RSS memory size allocated to the Postgres connection
continual going up (slowly) and the RSS size allocated to the Perl
daemon also continually going up (slowly). The Perl daemon is using
DBIx to make insert and update queries to the Postgres DB.
I believe DBIx is more of a namespace than a specific module - do you
mean DBIx::Class or DBI perhaps? If the former, the IRC channel
#dbix-class on irc.perl.org tends to be very helpful in tracing problems
like this, particularly if you can provide a reproducible test case.
There is only one connection open between the Perl daemon and the
Postgres DB. That connection stays open until the daemon exits or is
killed. When the daemon is killed, the Postgres back end is terminated
and all of its memory is released.Any suggestions on how to go about troubleshooting this would be
useful. Most specifically, I would like to know how to find out what
the Postgres process for that connection is doing with all that
memory. Is there a way I can see what the memory is allocated for?
Would running GDB on the Postgres process help? Is there any logging I
can turn on which might help shed some light on what's using the
memory? Anything in particular anyone knows about which I should look
for which might cause both Perl and Postgres memory usage to climb
continually?
On the Perl side, Devel::Size and possible Test::MemoryGrowth may be of
some use in tracing where the memory is going. Does the memory usage hit
a ceiling at any point, or does it carry on until all virtual memory is
exhausted? You could try running the postgres and perl daemons with
lower ulimit -m / ulimit -v settings if the increase in memory is too
slow to test this in a reasonable timeframe.
Other than that, maybe try looking for statement handles that aren't
closed or long-running transactions?
best regards,
Tom
On Wed, Apr 4, 2012 at 2:29 PM, Tom Molesworth <tom@audioboundary.com>wrote:
I believe DBIx is more of a namespace than a specific module - do you mean
DBIx::Class or DBI perhaps? If the former, the IRC channel #dbix-class on
irc.perl.org tends to be very helpful in tracing problems like this,
particularly if you can provide a reproducible test case.
It is using DBIx::Class. I will check on the IRC channel, too; thanks.
On the Perl side, Devel::Size and possible Test::MemoryGrowth may be of
some use in tracing where the memory is going. Does the memory usage hit a
ceiling at any point, or does it carry on until all virtual memory is
exhausted? You could try running the postgres and perl daemons with lower
ulimit -m / ulimit -v settings if the increase in memory is too slow to
test this in a reasonable timeframe.
The memory usage eventually exhausts all virtual memory on the system.
Other than that, maybe try looking for statement handles that aren't
closed or long-running transactions?
I checked for long running transactions, but did not find any. Is there a
way in Postgres to check for unclosed statement handles? If such a method
exists, it would be far easier than looking through the DBIx::Class code
for unclosed statement handles.
Thanks for your suggestions.
--
Eliot Gable
-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160
I have a Perl daemon handling some events on a system
which inserts or updates rows in Postgres, and something
is causing memory usage to grow.
The most likely culprit is the part of the chain you did not
list in the subject, DBD::Pg. Do you know what version of
it you are running? There have been various memory leak
problems fixed over the years. The latest is 2.19.2. Your
first solution should be to upgrade that if needed.
- --
Greg Sabino Mullane greg@turnstep.com
End Point Corporation http://www.endpoint.com/
PGP Key: 0x14964AC8 201204041558
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-----BEGIN PGP SIGNATURE-----
iEYEAREDAAYFAk98qdYACgkQvJuQZxSWSshRZwCg9Nk045WbJuYv6hGeHXHQq+ZG
haoAoMml2Nkn5GL1sMhMD1ofklVKYBC6
=UmNz
-----END PGP SIGNATURE-----