PostgreSQL running out of file handles

Started by Christopher Kings-Lynneover 20 years ago6 messages
#1Christopher Kings-Lynne
chriskl@familyhealth.com.au

Hi,

A few days back the load increased on our database server to the point
where it could not get enough file handles. This causes the backends to
crash, get restarted only to crash again, on and on.

We fixed it by bumping kern.maxfiles, but was just wondering if this is
a scenario that PostgreSQL should handle more gracefully?

Chris

#2Alvaro Herrera
alvherre@surnet.cl
In reply to: Christopher Kings-Lynne (#1)
Re: PostgreSQL running out of file handles

On Fri, May 13, 2005 at 09:45:15AM +0800, Christopher Kings-Lynne wrote:

A few days back the load increased on our database server to the point
where it could not get enough file handles. This causes the backends to
crash, get restarted only to crash again, on and on.

We fixed it by bumping kern.maxfiles, but was just wondering if this is
a scenario that PostgreSQL should handle more gracefully?

We certainly go great lengths to prevent this problem ... what version
is this?

--
Alvaro Herrera (<alvherre[a]surnet.cl>)
"The important things in the world are problems with society that we don't
understand at all. The machines will become more complicated but they won't
be more complicated than the societies that run them." (Freeman Dyson)

#3Christopher Kings-Lynne
chriskl@familyhealth.com.au
In reply to: Alvaro Herrera (#2)
Re: PostgreSQL running out of file handles

A few days back the load increased on our database server to the point
where it could not get enough file handles. This causes the backends to
crash, get restarted only to crash again, on and on.

We fixed it by bumping kern.maxfiles, but was just wondering if this is
a scenario that PostgreSQL should handle more gracefully?

We certainly go great lengths to prevent this problem ... what version
is this?

7.4.6

Chris

#4Tom Lane
tgl@sss.pgh.pa.us
In reply to: Christopher Kings-Lynne (#1)
Re: PostgreSQL running out of file handles

Christopher Kings-Lynne <chriskl@familyhealth.com.au> writes:

A few days back the load increased on our database server to the point
where it could not get enough file handles. This causes the backends to
crash, get restarted only to crash again, on and on.

We fixed it by bumping kern.maxfiles, but was just wondering if this is
a scenario that PostgreSQL should handle more gracefully?

I suppose you are running on some BSD variant? BSD is notorious for
promising more than it can deliver with respect to number of open files
per process. This is a kernel bug, not a Postgres bug.

You can adjust Postgres' max_files_per_process setting to compensate for
the kernel's lying about its capabilities.

(Postgres is in fact one of the most robust applications I know of
in terms of not going belly-up in response to EMFILE or ENFILE.
However, if there are not any spare descriptors to close, there's
not a lot we can do except fail.)

regards, tom lane

#5Christopher Kings-Lynne
chriskl@familyhealth.com.au
In reply to: Tom Lane (#4)
Re: PostgreSQL running out of file handles
Show quoted text

I suppose you are running on some BSD variant? BSD is notorious for
promising more than it can deliver with respect to number of open files
per process. This is a kernel bug, not a Postgres bug.

You can adjust Postgres' max_files_per_process setting to compensate for
the kernel's lying about its capabilities.

(Postgres is in fact one of the most robust applications I know of
in terms of not going belly-up in response to EMFILE or ENFILE.
However, if there are not any spare descriptors to close, there's
not a lot we can do except fail.)

regards, tom lane

#6Christopher Kings-Lynne
chriskl@familyhealth.com.au
In reply to: Tom Lane (#4)
Re: PostgreSQL running out of file handles

I suppose you are running on some BSD variant? BSD is notorious for
promising more than it can deliver with respect to number of open files
per process. This is a kernel bug, not a Postgres bug.

Good guess. Freebsd 4.8 or so.

Chris