8.1.2 postmaster died

Started by Ed L.over 19 years ago9 messagesgeneral
Jump to latest
#1Ed L.
pgsql@bluepolka.net

One of our 8.1.2 postmasters on HPUX 11.23 ia64 just received a
SIGKILL signal from unknown origins. After reviewing all
command history files for the DBA and root, I do not believe
anyone manually sent it, and we have no scripts etc that would
do that, at least that we can find or imagine. The machine had
ample RAM available.

Any ideas where could this SIGKILL would have come from?

Thanks,
Ed

#2Talha Khan
talha.amjad@gmail.com
In reply to: Ed L. (#1)
Re: 8.1.2 postmaster died

Hi Ed,

You need to look through the log files for details.

Regards
Talha Khan

Show quoted text

On 11/8/06, Ed L. <pgsql@bluepolka.net> wrote:

One of our 8.1.2 postmasters on HPUX 11.23 ia64 just received a
SIGKILL signal from unknown origins. After reviewing all
command history files for the DBA and root, I do not believe
anyone manually sent it, and we have no scripts etc that would
do that, at least that we can find or imagine. The machine had
ample RAM available.

Any ideas where could this SIGKILL would have come from?

Thanks,
Ed

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org/

#3Shoaib Mir
shoaibmir@gmail.com
In reply to: Talha Khan (#2)
Re: 8.1.2 postmaster died

To be specific you need to look at the syslogs to see if it was actually
some OOM killer or not.

Thanks,
-------
Shoaib Mir
EnterpriseDB (www.enterprisedb.com)

Show quoted text

On 11/9/06, Talha Khan <talha.amjad@gmail.com> wrote:

Hi Ed,

You need to look through the log files for details.

Regards
Talha Khan

On 11/8/06, Ed L. < pgsql@bluepolka.net> wrote:

One of our 8.1.2 postmasters on HPUX 11.23 ia64 just received a
SIGKILL signal from unknown origins. After reviewing all
command history files for the DBA and root, I do not believe
anyone manually sent it, and we have no scripts etc that would
do that, at least that we can find or imagine. The machine had
ample RAM available.

Any ideas where could this SIGKILL would have come from?

Thanks,
Ed

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org/

#4Ed Loehr
ed@LoehrTech.com
In reply to: Shoaib Mir (#3)
Re: 8.1.2 postmaster died

On Wednesday November 8 2006 12:30 pm, Shoaib Mir wrote:

To be specific you need to look at the syslogs to see if it
was actually some OOM killer or not.

I checked all db logs + syslog, of course. All they show is the
process receiving SIGKILL (followed by all others being shutdown
as a result).

Is there any circumstance at all under which pgsql itself would
issue a sigkill to one of the backends?

Ed

#5Talha Khan
talha.amjad@gmail.com
In reply to: Ed Loehr (#4)
Re: 8.1.2 postmaster died

Hi Ed,

I don't think its pgsql thats sending the SIGKILL its being done by the OS.
The reason can only be confirmed after looking at your log files but the
probability is that your OS ran out of memory and in order to protect itself
it had to kill the processes that were taking a lot of memory thus ended up
killing pgsql processes. Its just a hunch that i have but the reason can
only be confirmed after seeing your logs.

Regards
Talha Khan

Show quoted text

On 11/9/06, Ed Loehr <ed@loehrtech.com> wrote:

On Wednesday November 8 2006 12:30 pm, Shoaib Mir wrote:

To be specific you need to look at the syslogs to see if it
was actually some OOM killer or not.

I checked all db logs + syslog, of course. All they show is the
process receiving SIGKILL (followed by all others being shutdown
as a result).

Is there any circumstance at all under which pgsql itself would
issue a sigkill to one of the backends?

Ed

#6Shoaib Mir
shoaibmir@gmail.com
In reply to: Talha Khan (#5)
Re: 8.1.2 postmaster died

Have a look at "16.4.3. Linux Memory Overcommit" on
http://www.postgresql.org/docs/8.1/interactive/kernel-resources.html

Thanks,
------
Shoaib Mir
EnterpriseDB (www.enterprisedb.com)

Show quoted text

On 11/9/06, Talha Khan <talha.amjad@gmail.com> wrote:

Hi Ed,

I don't think its pgsql thats sending the SIGKILL its being done by the
OS. The reason can only be confirmed after looking at your log files but the
probability is that your OS ran out of memory and in order to protect itself
it had to kill the processes that were taking a lot of memory thus ended up
killing pgsql processes. Its just a hunch that i have but the reason can
only be confirmed after seeing your logs.

Regards
Talha Khan

On 11/9/06, Ed Loehr <ed@loehrtech.com> wrote:

On Wednesday November 8 2006 12:30 pm, Shoaib Mir wrote:

To be specific you need to look at the syslogs to see if it
was actually some OOM killer or not.

I checked all db logs + syslog, of course. All they show is the
process receiving SIGKILL (followed by all others being shutdown
as a result).

Is there any circumstance at all under which pgsql itself would
issue a sigkill to one of the backends?

Ed

#7Martijn van Oosterhout
kleptog@svana.org
In reply to: Shoaib Mir (#6)
Re: 8.1.2 postmaster died

On Thu, Nov 09, 2006 at 11:52:48AM +0500, Shoaib Mir wrote:

Have a look at "16.4.3. Linux Memory Overcommit" on
http://www.postgresql.org/docs/8.1/interactive/kernel-resources.html

If you look at the beginning of the thread you'dve seen we're talking
about HPUX here...

Maybe they have an OOM killer too? Although maybe it's quotas?

Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/

Show quoted text

From each according to his ability. To each according to his ability to litigate.

#8Shoaib Mir
shoaibmir@gmail.com
In reply to: Martijn van Oosterhout (#7)
Re: 8.1.2 postmaster died

Look at the database server logs and see what actually was happening just
before the server crashed... do you have the auto vacuuming running at
backend? or was there some client making connection just when the server did
crash? .... your db logs can actually help you here.

Thank you,
-----------
Shoaib Mir
EnterpriseDB (www.enterprisedb.com)

Show quoted text

On 11/9/06, Martijn van Oosterhout <kleptog@svana.org> wrote:

On Thu, Nov 09, 2006 at 11:52:48AM +0500, Shoaib Mir wrote:

Have a look at "16.4.3. Linux Memory Overcommit" on
http://www.postgresql.org/docs/8.1/interactive/kernel-resources.html

If you look at the beginning of the thread you'dve seen we're talking
about HPUX here...

Maybe they have an OOM killer too? Although maybe it's quotas?

Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/

From each according to his ability. To each according to his ability to

litigate.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFFU0RoIB7bNG8LQkwRAjRIAJwJig0SLNyUtfthMw0rPM5olfXuVwCfYoX5
OiFfQVVyp55mDo4MDfD88v8=
=7Enb
-----END PGP SIGNATURE-----

#9Tom Lane
tgl@sss.pgh.pa.us
In reply to: Ed Loehr (#4)
Re: 8.1.2 postmaster died

Ed Loehr <ed@loehrtech.com> writes:

Is there any circumstance at all under which pgsql itself would
issue a sigkill to one of the backends?

No. Other signals yes, but not SIGKILL.

regards, tom lane