Very frequent "Too many clients" eventually crashes postmaster
Is there a way to eliminate postmaster crash/core ?
A surge in short-term connections eventually crashes PG on "no free slots in PMChildFlags array".
Happens on AIX 7.2
=>grep "2020-05-02 08:16" * | grep clients | wc -l
0
=>grep "2020-05-02 08:17" * | grep clients | wc -l
0
=>grep "2020-05-02 08:18" * | grep clients | wc -l
309
=>grep "2020-05-02 08:19" * | grep clients | wc -l
603
=>grep "2020-05-02 08:20" * | grep clients | wc -l
288
=>grep "2020-05-02 08:21" * | grep clients | wc -l
230
2020-05-02 08:21:27.653 CDTFATAL: no free slots in PMChildFlags array
"Abraham, Danny" <danny_abraham@bmc.com> writes:
Is there a way to eliminate postmaster crash/core ?
A surge in short-term connections eventually crashes PG on "no free slots in PMChildFlags array".
Update to a current release, maybe? We fixed a bug with symptoms like
that last fall [1]https://git.postgresql.org/gitweb/?p=postgresql.git&a=commitdiff&h=3887e9455.
regards, tom lane
[1]: https://git.postgresql.org/gitweb/?p=postgresql.git&a=commitdiff&h=3887e9455
On 5/3/20 7:28 AM, Abraham, Danny wrote:
Is there a way to eliminate postmaster crash/core ?
A surge in short-term connections eventually crashes PG on "no free slots in PMChildFlags array".
Happens on AIX 7.2
=>grep "2020-05-02 08:16" * | grep clients | wc -l
0
=>grep "2020-05-02 08:17" * | grep clients | wc -l
0
=>grep "2020-05-02 08:18" * | grep clients | wc -l
309
=>grep "2020-05-02 08:19" * | grep clients | wc -l
603
=>grep "2020-05-02 08:20" * | grep clients | wc -l
288
=>grep "2020-05-02 08:21" * | grep clients | wc -l
2302020-05-02 08:21:27.653 CDTFATAL: no free slots in PMChildFlags array
Well:
backend/storage/ipc/pmsignal.c
/* Out of slots ... should never happen, else postmaster.c messed up */
elog(FATAL, "no free slots in PMChildFlags array");
return 0; /* keep
compiler quiet */
So:
1) What are your connection settings?
2) What is generating the client traffic?
3) Is there a pooler in front of the server?
--
Adrian Klaver
adrian.klaver@aklaver.com
Will setting max_parallel_workers_per_gather to 0
(In fact, no parallel queries)
Solve this issue?
I can sustain the slowness.
We are on 11.5, and the fix is on 11.6.
-----Original Message-----
From: Tom Lane <tgl@sss.pgh.pa.us>
Sent: Sunday, May 03, 2020 5:37 PM
To: Abraham, Danny <danny_abraham@bmc.com>
Cc: pgsql-general@postgresql.org
Subject: [EXTERNAL] Re: Very frequent "Too many clients" eventually crashes postmaster
"Abraham, Danny" <danny_abraham@bmc.com> writes:
Is there a way to eliminate postmaster crash/core ?
A surge in short-term connections eventually crashes PG on "no free slots in PMChildFlags array".
Update to a current release, maybe? We fixed a bug with symptoms like that last fall [1]https://urldefense.proofpoint.com/v2/url?u=https-3A__git.postgresql.org_gitweb_-3Fp-3Dpostgresql.git-26a-3Dcommitdiff-26h-3D3887e9455&d=DwIFAg&c=UrUhmHsiTVT5qkaA4d_oSzcamb9hmamiCDMzBAEwC7E&r=rZ6QRG5QsrF-veoN0_ehDrT6IsQzka7-fVfEhCfor3I&m=LoZHLvaRhw48Dvx9n1OtgDV_SA0hse7hNTIl0fZh0RQ&s=u3c5XpUBsGhiPxGk5KFwOO_Aj-ToS4YN4pPBuVVZRGI&e=.
regards, tom lane