BUG #14320: systemd terminates postgresql hot standby instance 90 seconds after start
The following bug has been logged on the website:
Bug reference: 14320
Logged by: Andreas Papst
Email address: andreas.papst@univie.ac.at
PostgreSQL version: 9.4.9
Operating system: Debian GNU/Linux 8.5 (jessie)
Description:
Hey,
when starting a hot standby database insstance with
`systemctl start postgresql',
anything works perfect: WAL files are beeing applied, r/o connections are
able to query the database, but after 90 seconds (systemd
DefaultTimeoutStartSec) postgresql is beeing shut down:
2016-09-01 10:30:15.235 CEST @ (/26671) LOG: received smart shutdown
request
2016-09-01 10:30:15.330 CEST @ (/26699) FATAL: terminating walreceiver
process
due to administrator command
2016-09-01 10:30:16.967 CEST @ (/26675) LOG: shutting down
2016-09-01 10:30:17.748 CEST @ (/26675) LOG: database system is shut down
and the systemctl start command returns too.
I guess this is due to a missing step in starting up postgresql in hot
standby configuration, where this process should be detached/daemonized
(calling setsid).
To proof it, I sent a signal with
`systemctl reload postgresql´ which caused the systemctl start command to
finish and kept postgresql running.
I've tried to find the respective lines in the code but I totally get lost.
So please if somebody can fix it.
Thank you
--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs
On 9/9/16 6:50 AM, andreas.papst@univie.ac.at wrote:
when starting a hot standby database insstance with
`systemctl start postgresql',
anything works perfect: WAL files are beeing applied, r/o connections are
able to query the database, but after 90 seconds (systemd
DefaultTimeoutStartSec) postgresql is beeing shut down:
This is all quite plausible, but it depends on how your systemd service
for postgresql is configured. So please post that if you want us to
analyze it.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs
Hey Peter,
thank you for your offer.
# cat /etc/debian_version
8.5
# dpkg -l | grep postgres
-------------------------------------------------------------------
ii pgdg-keyring 2014.1 all
keyring for apt.postgresql.org
ii postgresql-9.4 9.4.9-1.pgdg80+1
amd64 object-relational SQL database, version 9.4 server
ii postgresql-9.5 9.5.4-1.pgdg80+1
amd64 object-relational SQL database, version 9.5 server
ii postgresql-client-9.4 9.4.9-1.pgdg80+1
amd64 front-end programs for PostgreSQL 9.4
ii postgresql-client-9.5 9.5.4-1.pgdg80+1
amd64 front-end programs for PostgreSQL 9.5
ii postgresql-client-common 175.pgdg80+1 all
manager for multiple PostgreSQL client versions
ii postgresql-common 175.pgdg80+1 all
PostgreSQL database-cluster manager
ii postgresql-contrib-9.4 9.4.9-1.pgdg80+1
amd64 additional facilities for PostgreSQL
ii postgresql-contrib-9.5 9.5.4-1.pgdg80+1
amd64 additional facilities for PostgreSQL
-------------------------------------------------------------------
attached files:
systemctl cat postgresql.service
systemctl cat postgresql@9.4-at_run.service
/etc/postgresql/9.4/at_run/postgresql.conf
/etc/postgresql/9.4/at_run/start.conf
/usr/bin/pg_ctlcluster
Best regards
Andreas
On 09/13/2016 05:52 PM, Peter Eisentraut wrote:
On 9/9/16 6:50 AM, andreas.papst@univie.ac.at wrote:
when starting a hot standby database insstance with
`systemctl start postgresql',
anything works perfect: WAL files are beeing applied, r/o connections are
able to query the database, but after 90 seconds (systemd
DefaultTimeoutStartSec) postgresql is beeing shut down:This is all quite plausible, but it depends on how your systemd service
for postgresql is configured. So please post that if you want us to
analyze it.
--
________________________________________________________________________
Andreas Papst |
Vienna University Computer Center | E-Mail: andreas.papst@univie.ac.at
Universitaetsstrasse 7 | Phone: +43 1 4277 / 140 36
A-1010 Vienna, Austria, Europe | Fax.: +43 1 4277 / 8 140 36
Attachments:
pg_ctlclustertext/plain; charset=UTF-8; name=pg_ctlclusterDownload
postgresql.conftext/plain; charset=UTF-8; name=postgresql.confDownload
postgresql.servicetext/plain; charset=UTF-8; name=postgresql.serviceDownload
postgresql@9.4-at_run.servicetext/plain; charset=UTF-8; name="postgresql@9.4-at_run.service"Download
start.conftext/plain; charset=UTF-8; name=start.confDownload
Something unusual is going on. Try running the commands in the unit
file by hand and see whether they succeed or print additional
diagnostics. Check the systemd logs, if any. The reason the process
ends up getting killed is that systemd thinks it has not started
successfully.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs
Hello,
On 09/15/2016 02:19 AM, Peter Eisentraut wrote:
Something unusual is going on. Try running the commands in the unit
file by hand and see whether they succeed or print additional
diagnostics. Check the systemd logs, if any. The reason the process
ends up getting killed is that systemd thinks it has not started
successfully.
yeah, when systemd's startup process is running into it's timeout
it terminates postgres process. If I change the timeout threshold
to unlimited, postgres operates for ever.
And as I wrote, when I send Postgres a signal, i.e. `systemctl reload
postgresql´, postgres process detaches, the blocked startup script
is freed and returns success to systemd.
No more logs are available.
Thanks
Andreas
--
________________________________________________________________________
Andreas Papst |
Vienna University Computer Center | E-Mail: andreas.papst@univie.ac.at
Universitaetsstrasse 7 | Phone: +43 1 4277 / 140 36
A-1010 Vienna, Austria, Europe | Fax.: +43 1 4277 / 8 140 36
--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs