Postmaster fails to shut down right after crash restart
Hackorum builds and tests every patch posted to the lists, not only commitfest submissions. This is Hackorum's own CI rather than the PostgreSQL project's, and it is still under testing - please report anything that looks wrong.
You can run a PostgreSQL built from this patch straight from Docker, with no checkout and no build:
docker run --rm -p 5432:5432 ghcr.io/hackorum-dev/postgres-patch:t51491psql -h localhost -U postgresBuilt from patchset v1 (message #1), August 17, 2026 at 09:34 AM.
Hello,
While developing a patch and running regression tests I noticed that the
postmaster could fail to shut down right after crash restart. It could
get stuck in the PM_WAIT_BACKENDS state forever.
As far as I understand, the problem occurs when a shutdown signal is
received before getting PMSIGNAL_RECOVERY_STARTED from the startup
process. In that case the FatalError flag is not cleared, and the
postmaster is stuck in PM_WAIT_BACKENDS waiting for the checkpointer,
which ignores SIGTERM.
To easily reproduce the problem I added pg_usleep in xlogrecovery.c just
before SendPostmasterSignal(PMSIGNAL_RECOVERY_STARTED). See the patch
attached.
Then I run a script that simulates a crash and does pg_ctl stop:
$ ./init.sh
[...]
$ ./stop-after-crash.sh
waiting for server to start.... done
server started
waiting for server to shut
down............................................................... failed
pg_ctl: server does not shut down
Some processes are still alive:
$ ps uf -C postgres
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
sergey 279874 0.0 0.0 222816 28560 ? Ss 14:25 0:00
/home/sergey/pgwork/devel/install/bin/postgres -D data
sergey 279887 0.0 0.0 222772 5664 ? Ss 14:25 0:00 \_
postgres: io worker 0
sergey 279888 0.0 0.0 222772 5664 ? Ss 14:25 0:00 \_
postgres: io worker 1
sergey 279889 0.0 0.0 222772 5664 ? Ss 14:25 0:00 \_
postgres: io worker 2
sergey 279891 0.0 0.0 222884 8480 ? Ss 14:25 0:00 \_
postgres: checkpointer
Here is an excerpt from the debug log:
postmaster[279874] LOG: all server processes terminated; reinitializing
startup[279890] LOG: database system was interrupted; last known up at
2025-04-24 14:25:58 MSK
startup[279890] LOG: database system was not properly shut down;
automatic recovery in progress
postmaster[279874] DEBUG: postmaster received shutdown request signal
postmaster[279874] LOG: received fast shutdown request
postmaster[279874] DEBUG: updating PMState from PM_STARTUP to
PM_STOP_BACKENDS
postmaster[279874] DEBUG: sending signal 15/SIGTERM to background
writer process with pid 279892
postmaster[279874] DEBUG: sending signal 15/SIGTERM to checkpointer
process with pid 279891
postmaster[279874] DEBUG: sending signal 15/SIGTERM to startup process
with pid 279890
postmaster[279874] DEBUG: sending signal 15/SIGTERM to io worker
process with pid 279889
postmaster[279874] DEBUG: sending signal 15/SIGTERM to io worker
process with pid 279888
postmaster[279874] DEBUG: sending signal 15/SIGTERM to io worker
process with pid 279887
postmaster[279874] DEBUG: updating PMState from PM_STOP_BACKENDS to
PM_WAIT_BACKENDS
startup[279890] LOG: invalid record length at 0/175A4D8: expected at
least 24, got 0
postmaster[279874] DEBUG: postmaster received pmsignal signal
startup[279890] LOG: redo is not required
checkpointer[279891] LOG: checkpoint starting: end-of-recovery
immediate wait
checkpointer[279891] LOG: checkpoint complete: wrote 0 buffers (0.0%),
wrote 3 SLRU buffers; 0 WAL file(s) added, 0 removed, 0 recycled;
write=0.007 s, sync=0.002 s, total=0.026 s; sync files=2, longest=0.001
s, average=0.001 s; distance=0 kB, estimate=0 kB; lsn=0/175A4D8, redo
lsn=0/175A4D8
startup[279890] DEBUG: exit(0)
postmaster[279874] DEBUG: updating PMState from PM_WAIT_BACKENDS to
PM_WAIT_BACKENDS
checkpointer[279891] DEBUG: checkpoint skipped because system is idle
checkpointer[279891] DEBUG: checkpoint skipped because system is idle
I don't know how to fix this, but thought it's worth reporting.
Best regards,
--
Sergey Shinderuk https://postgrespro.com/