PostgreSQL 9.4.13 is facing issue in shutting down

Started by Abhijit Gharamiover 8 years ago6 messagesbugsgeneral
Jump to latest
#1Abhijit Gharami
abhijitgharami@gmail.com
bugsgeneral

Hi,

Recently we have updated our PostgreSQL version from 9.4.12 to 9.4.13. With
version 9.4.13 while trying to stop the PostgreSQL server, database is
facing issues in shutting down. The problem probably occurs because of the
WAL sender process not getting terminated on shutdown request.

ps output:
postgre+ 7915 0.0 1.9 589772 39528 ? S< 09:11 0:00
/..../bin/postgres -D /.../postgresql -h 0.0.0.0 -p 5432
postgre+ 8617 0.0 0.4 590760 8344 ? S<s 09:13 0:00 postgres:
wal sender process replicator xx.xx.xx.xxx(45948) streaming 0/4000090

Sequence of events in log:
FATAL: terminating connection due to administrator command
LOG: received fast shutdown request
LOG: aborting any active transactions
LOG: autovacuum launcher shutting down
LOG: shutting down
LOG: database system is shut down
FATAL: the database system is shutting down
FATAL: the database system is shutting down
FATAL: the database system is shutting down
FATAL: the database system is shutting down
FATAL: the database system is shutting down
FATAL: the database system is shutting down
FATAL: the database system is shutting down
FATAL: the database system is shutting down

We are using pg_ctl fast mode to shutdown the postgres server.

Regards,
Abhijit

#2Abhijit Gharami
abhijitgharami@gmail.com
In reply to: Abhijit Gharami (#1)
bugsgeneral
Fwd: PostgreSQL 9.4.13 is facing issue in shutting down

Hi,

Recently we have updated our PostgreSQL version from 9.4.12 to 9.4.13. With
version 9.4.13 while trying to stop the PostgreSQL server, database is
facing issues in shutting down. The problem probably occurs because of the
WAL sender process not getting terminated on shutdown request.

ps output:
postgre+ 7915 0.0 1.9 589772 39528 ? S< 09:11 0:00
/..../bin/postgres -D /.../postgresql -h 0.0.0.0 -p 5432
postgre+ 8617 0.0 0.4 590760 8344 ? S<s 09:13 0:00 postgres:
wal sender process replicator xx.xx.xx.xxx(45948) streaming 0/4000090

Sequence of events in log:
FATAL: terminating connection due to administrator command
LOG: received fast shutdown request
LOG: aborting any active transactions
LOG: autovacuum launcher shutting down
LOG: shutting down
LOG: database system is shut down
FATAL: the database system is shutting down
FATAL: the database system is shutting down
FATAL: the database system is shutting down
FATAL: the database system is shutting down
FATAL: the database system is shutting down
FATAL: the database system is shutting down
FATAL: the database system is shutting down
FATAL: the database system is shutting down

We are using pg_ctl fast mode to shutdown the postgres server.

Regards,
Abhijit

#3Michael Paquier
michael@paquier.xyz
In reply to: Abhijit Gharami (#2)
bugsgeneral
Re: Fwd: PostgreSQL 9.4.13 is facing issue in shutting down

On Tue, Aug 22, 2017 at 1:58 PM, Abhijit Gharami
<abhijitgharami@gmail.com> wrote:

Recently we have updated our PostgreSQL version from 9.4.12 to 9.4.13. With
version 9.4.13 while trying to stop the PostgreSQL server, database is
facing issues in shutting down. The problem probably occurs because of the
WAL sender process not getting terminated on shutdown request.
ps output:
postgre+ 7915 0.0 1.9 589772 39528 ? S< 09:11 0:00
/..../bin/postgres -D /.../postgresql -h 0.0.0.0 -p 5432
postgre+ 8617 0.0 0.4 590760 8344 ? S<s 09:13 0:00 postgres:
wal sender process replicator xx.xx.xx.xxx(45948) streaming 0/4000090

One change introduced between 9.4.12 and 9.4.13 is the way WAL senders
are stopped at shutdown. The checkpointer will first send a signal to
all the WAL senders after all the normal backends have exited before
beginning to issue the checkpoint shutdown. This will switch the WAL
senders to a stopping state where all new queries are blocked, and all
WAL senders exit after they have sent any remaining WAL to the
clients, including the checkpoint shutdown record. So, what is the
client behind this WAL sender? It seems that this WAL sender is
waiting for some confirmation activity.
--
Michael

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

#4Abhijit Gharami
abhijitgharami@gmail.com
In reply to: Michael Paquier (#3)
bugsgeneral
Re: [BUGS] Fwd: PostgreSQL 9.4.13 is facing issue in shutting down

The client is a Hot Standby server.

Regards,
Abhijit

On Tue, Aug 22, 2017 at 11:05 AM, Michael Paquier <michael.paquier@gmail.com

Show quoted text

wrote:

On Tue, Aug 22, 2017 at 1:58 PM, Abhijit Gharami
<abhijitgharami@gmail.com> wrote:

Recently we have updated our PostgreSQL version from 9.4.12 to 9.4.13.

With

version 9.4.13 while trying to stop the PostgreSQL server, database is
facing issues in shutting down. The problem probably occurs because of

the

WAL sender process not getting terminated on shutdown request.
ps output:
postgre+ 7915 0.0 1.9 589772 39528 ? S< 09:11 0:00
/..../bin/postgres -D /.../postgresql -h 0.0.0.0 -p 5432
postgre+ 8617 0.0 0.4 590760 8344 ? S<s 09:13 0:00

postgres:

wal sender process replicator xx.xx.xx.xxx(45948) streaming 0/4000090

One change introduced between 9.4.12 and 9.4.13 is the way WAL senders
are stopped at shutdown. The checkpointer will first send a signal to
all the WAL senders after all the normal backends have exited before
beginning to issue the checkpoint shutdown. This will switch the WAL
senders to a stopping state where all new queries are blocked, and all
WAL senders exit after they have sent any remaining WAL to the
clients, including the checkpoint shutdown record. So, what is the
client behind this WAL sender? It seems that this WAL sender is
waiting for some confirmation activity.
--
Michael

#5Peter Eisentraut
peter_e@gmx.net
In reply to: Abhijit Gharami (#1)
bugsgeneral
Re: PostgreSQL 9.4.13 is facing issue in shutting down

On 8/21/17 06:26, Abhijit Gharami wrote:

Recently we have updated our PostgreSQL version from 9.4.12 to 9.4.13.
With version 9.4.13 while trying to stop the PostgreSQL server, database
is facing issues in shutting down. The problem probably occurs because
of the WAL sender process not getting terminated on shutdown request. 

This is probably the same issue as reported here:
/messages/by-id/1434ad79-ab0d-d6a9-4ea2-1afb1c06adaf@2ndquadrant.it

You can try the patch attached there, or downgrade to 9.4.12 while this
issue is being investigated.

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin

#6Michael Paquier
michael@paquier.xyz
In reply to: Abhijit Gharami (#4)
bugsgeneral
Re: Fwd: PostgreSQL 9.4.13 is facing issue in shutting down

On Tue, Aug 22, 2017 at 3:37 PM, Abhijit Gharami
<abhijitgharami@gmail.com> wrote:

The client is a Hot Standby server.

Marco, just added in CC, has taken the time to reproduce the problem
and has found the cause of the problem. He has also posted a patch on
this thread:
/messages/by-id/1434ad79-ab0d-d6a9-4ea2-1afb1c06adaf@2ndquadrant.it
--
Michael

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs