[PATCH] Fix orphaned backend processes on Windows using Job Objects
Greetings,
When the postmaster exits unexpectedly on Windows (crash, kill, debugger
abort), backend processes continue running. Windows lacks any equivalent
to Unix's getppid() orphan detection. These orphaned backends hold locks
and shared memory, preventing clean restart. This leads to a delay in
restarts and manual killing of orphans.
The problem is easy to reproduce. Start postgres, open a transaction
with LOCK TABLE, then kill the postmaster with taskkill /F. The backend
continues running and restart fails. Manual cleanup is required.
Current approaches (inherited event handles, shared memory flags) depend
on the postmaster running code during exit. A segfault or kill bypasses
all of that.
My proposed solution is to use Windows Job Objects with KILL_ON_JOB_CLOSE.
We just need to call CreateJobObject() in PostmasterMain(), configure
with JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE, and assign the postmaster.
Children inherit membership automatically. When the job handle closes on
postmaster exit, the kernel terminates all children atomically. This is
kernel-enforced with no polling and no race conditions.
Job creation can fail if postgres runs under an existing job (service
managers, debuggers). Windows 7 disallows nested jobs. We detect this
with IsProcessInJob(), and if AssignProcessToJobObject() returns
ERROR_ACCESS_DENIED, we log and continue without orphan protection.
KILL_ON_JOB_CLOSE doesn't interfere with clean shutdown. Normal shutdown
signals backends via SetEvent, they exit, postmaster exits, job closes.
Nothing left to kill. The flag only fires during crashes when backends
are still running - exactly when forced termination is correct.
The code is ~200 lines in pg_job_object.c, less than win32/signal.c
(~500 lines). It fails gracefully and works regardless of how postgres
is started, unlike service manager approaches. This avoids polling
unreliability.
The patch has been tested on Windows 10/11 with both MSVC and MinGW
builds. Nested jobs fail gracefully as expected. Clean shutdown is
unaffected. Crash tests with taskkill /F, debugger abort, and access
violations all correctly terminate children immediately with zero orphans.
This patch does not include automated tests because the core
functionality (orphan prevention on crash) requires simulating process
termination, which is difficult to test reliably in CI.
Patch attached. Can add documentation if this approach is approved.
Thoughts?
Bryan Green
Attachments:
0001-Use-Windows-Job-Objects-to-prevent-orphaned-child-pr.patchtext/plain; charset=UTF-8; name=0001-Use-Windows-Job-Objects-to-prevent-orphaned-child-pr.patchDownload
From 4aea4fd2761e33244cbd55c21a9fbe64897fa732 Mon Sep 17 00:00:00 2001
From: Bryan Green <dbryan.green@gmail.com>
Date: Thu, 30 Oct 2025 11:14:55 -0600
Subject: [PATCH] Use Windows Job Objects to prevent orphaned child processes.
When the postmaster exits on Windows, backends can continue running
because Windows lacks Unix's getppid() orphan detection. Orphaned
backends hold locks and shared memory, preventing clean restart.
Create a Job Object at postmaster startup and assign the postmaster
to it. Children inherit job membership automatically. Configure with
KILL_ON_JOB_CLOSE so the kernel terminates all children when the job
handle closes on postmaster exit.
This is more reliable than existing approaches (inherited handles,
shared memory flags) because it's kernel-enforced with no polling.
Job creation is allowed to fail non-fatally. Some environments run
PostgreSQL under an existing job, and Windows 7 disallows nested jobs.
In such cases we log a message and proceed without orphan protection.
Author: Bryan Green
---
doc/src/sgml/runtime.sgml | 21 +++
src/backend/postmaster/postmaster.c | 12 ++
src/backend/storage/ipc/Makefile | 4 +
src/backend/storage/ipc/meson.build | 7 +
src/backend/storage/ipc/pg_job_object.c | 176 ++++++++++++++++++++++++
src/include/storage/pg_job_object.h | 35 +++++
6 files changed, 255 insertions(+)
create mode 100644 src/backend/storage/ipc/pg_job_object.c
create mode 100644 src/include/storage/pg_job_object.h
diff --git a/doc/src/sgml/runtime.sgml b/doc/src/sgml/runtime.sgml
index 0c60bafac6..95784e9236 100644
--- a/doc/src/sgml/runtime.sgml
+++ b/doc/src/sgml/runtime.sgml
@@ -1501,6 +1501,27 @@ $ <userinput>cat /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages</userinp
</para>
</sect2>
+
+ <sect2 id="windows-process-management">
+ <title>Process Management on <systemitem class="osname">Windows</systemitem></title>
+
+ <para>
+ On <systemitem class="osname">Windows</systemitem>,
+ <productname>PostgreSQL</productname> uses Job Objects to
+ manage child processes. If the postmaster exits unexpectedly
+ (due to a crash or external termination), the operating system
+ automatically terminates all backend processes. This prevents
+ orphaned backends that could hold locks or corrupt shared memory.
+ </para>
+
+ <para>
+ In some cases, Job Object creation may fail (for example, when
+ <productname>PostgreSQL</productname> is already running under a
+ job-aware service manager). The server will log a message and continue
+ to run without orphan protection. This is safe but means that backends
+ may need to be manually terminated if the postmaster crashes.
+ </para>
+ </sect2>
</sect1>
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 00de559ba8..14697b95e2 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -113,6 +113,7 @@
#include "storage/fd.h"
#include "storage/io_worker.h"
#include "storage/ipc.h"
+#include "storage/pg_job_object.h"
#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "tcop/backend_startup.h"
@@ -1005,6 +1006,17 @@ PostmasterMain(int argc, char *argv[])
*/
CreateSharedMemoryAndSemaphores();
+#ifdef WIN32
+ /*
+ * On Windows, create a job object to prevent orphaned backends.
+ * If postmaster crashes, Windows will automatically kill all
+ * child processes in the job.
+ *
+ * We do this after port binding so that if job creation fails,
+ * it's not fatal - we can still run (just without orphan protection).
+ */
+ pg_create_job_object();
+#endif
/*
* Estimate number of openable files. This must happen after setting up
* semaphores, because on some platforms semaphores count as open files.
diff --git a/src/backend/storage/ipc/Makefile b/src/backend/storage/ipc/Makefile
index 9a07f6e1d9..0604f4f382 100644
--- a/src/backend/storage/ipc/Makefile
+++ b/src/backend/storage/ipc/Makefile
@@ -28,4 +28,8 @@ OBJS = \
standby.o \
waiteventset.o
+ifeq ($(PORTNAME), win32)
+OBJS += pg_job_object.o
+endif
+
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/ipc/meson.build b/src/backend/storage/ipc/meson.build
index b1b73dac3b..85ba13b70d 100644
--- a/src/backend/storage/ipc/meson.build
+++ b/src/backend/storage/ipc/meson.build
@@ -21,3 +21,10 @@ backend_sources += files(
'waiteventset.c',
)
+
+# Windows-specific files
+if host_system == 'windows'
+ backend_sources += files(
+ 'pg_job_object.c',
+ )
+endif
diff --git a/src/backend/storage/ipc/pg_job_object.c b/src/backend/storage/ipc/pg_job_object.c
new file mode 100644
index 0000000000..c84aeed9cd
--- /dev/null
+++ b/src/backend/storage/ipc/pg_job_object.c
@@ -0,0 +1,176 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_job_object.c
+ * Windows Job Object support for preventing orphaned backends
+ *
+ * On Unix, backends can detect when the postmaster dies via getppid().
+ * Windows has no equivalent mechanism. We solve this by using Job Objects,
+ * a Windows kernel feature that groups processes and can automatically
+ * terminate all members when the job handle closes.
+ *
+ * By configuring JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE, we ensure that if
+ * the postmaster exits (cleanly or via crash), Windows immediately kills
+ * all backends. This prevents orphaned processes that hold locks and
+ * prevent clean restart.
+ *
+ * The job object handle is stored in a static variable and never explicitly
+ * closed. This is intentional - we rely on Windows closing it automatically
+ * when the postmaster process exits, which triggers the child termination.
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/storage/ipc/pg_job_object.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#ifdef WIN32
+
+#include "postmaster/postmaster.h"
+#include "storage/ipc.h"
+#include "storage/pg_job_object.h"
+
+static HANDLE pg_job_object = NULL;
+
+
+/*
+ * pg_create_job_object
+ *
+ * Create job object for this PostgreSQL instance and configure it to
+ * kill all children when the postmaster exits.
+ *
+ * Failure is not fatal - we log a warning and continue. PostgreSQL will
+ * run without orphan protection, which is no worse than current behavior.
+ */
+void
+pg_create_job_object(void)
+{
+ JOBOBJECT_EXTENDED_LIMIT_INFORMATION limit_info;
+ char job_name[128];
+ DWORD error;
+
+ snprintf(job_name, sizeof(job_name), "PostgreSQL_Port_%d_PID_%lu",
+ PostPortNumber, GetCurrentProcessId());
+
+ pg_job_object = CreateJobObjectA(NULL, job_name);
+
+ if (pg_job_object == NULL)
+ {
+ error = GetLastError();
+ ereport(LOG,
+ (errmsg("could not create job object \"%s\": error code %lu",
+ job_name, error),
+ errdetail("Orphaned process cleanup will not be available.")));
+ return;
+ }
+
+ elog(DEBUG1, "created job object \"%s\"", job_name);
+
+ /*
+ * Set KILL_ON_JOB_CLOSE. When the job handle closes (either explicit
+ * close or process termination), all processes in the job are terminated.
+ *
+ * This is the critical flag that prevents orphaned backends.
+ */
+ memset(&limit_info, 0, sizeof(limit_info));
+ limit_info.BasicLimitInformation.LimitFlags = JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE;
+
+ if (!SetInformationJobObject(pg_job_object,
+ JobObjectExtendedLimitInformation,
+ &limit_info,
+ sizeof(limit_info)))
+ {
+ error = GetLastError();
+ ereport(WARNING,
+ (errmsg("could not configure job object: error code %lu", error),
+ errdetail("Job object created but KILL_ON_JOB_CLOSE not set."),
+ errhint("Orphaned processes may occur if postmaster crashes.")));
+ CloseHandle(pg_job_object);
+ pg_job_object = NULL;
+ return;
+ }
+
+ if (!AssignProcessToJobObject(pg_job_object, GetCurrentProcess()))
+ {
+ error = GetLastError();
+
+ /*
+ * ERROR_ACCESS_DENIED means we're already in a job. This can happen
+ * when PostgreSQL runs under a job-aware supervisor (Windows service
+ * on older Windows, or any process manager using nested jobs).
+ *
+ * On Windows 8+, we could use nested jobs, but for simplicity we
+ * just skip job creation. The parent job should handle cleanup.
+ */
+ if (error == ERROR_ACCESS_DENIED)
+ {
+ ereport(LOG,
+ (errmsg("postmaster is already in a job object"),
+ errdetail("This can occur when PostgreSQL is run under a job-aware supervisor."),
+ errhint("Automatic orphan cleanup will not be available.")));
+ }
+ else
+ {
+ ereport(WARNING,
+ (errmsg("could not assign postmaster to job object: error code %lu", error)));
+ }
+
+ CloseHandle(pg_job_object);
+ pg_job_object = NULL;
+ return;
+ }
+
+ elog(LOG, "PostgreSQL job object configured successfully - orphaned process prevention enabled");
+}
+
+
+/*
+ * pg_destroy_job_object
+ *
+ * Explicitly close the job object handle. This will trigger KILL_ON_JOB_CLOSE,
+ * terminating all backends.
+ *
+ * Note: In most cases we don't call this - we rely on Windows closing the
+ * handle automatically when the postmaster exits. Explicit close is only
+ * needed if we want to control the exact timing of backend termination.
+ */
+void
+pg_destroy_job_object(void)
+{
+ if (pg_job_object != NULL)
+ {
+ elog(DEBUG1, "closing job object - all child processes will terminate");
+ CloseHandle(pg_job_object);
+ pg_job_object = NULL;
+ }
+}
+
+
+/*
+ * pg_is_in_job_object
+ *
+ * Check if current process is in the PostgreSQL job object.
+ * Used primarily for testing and verification.
+ */
+bool
+pg_is_in_job_object(void)
+{
+ BOOL in_job = FALSE;
+
+ if (pg_job_object == NULL)
+ return false;
+
+ if (!IsProcessInJob(GetCurrentProcess(), pg_job_object, &in_job))
+ {
+ elog(DEBUG1, "IsProcessInJob failed: error code %lu", GetLastError());
+ return false;
+ }
+
+ return (bool) in_job;
+}
+
+#endif /* WIN32 */
diff --git a/src/include/storage/pg_job_object.h b/src/include/storage/pg_job_object.h
new file mode 100644
index 0000000000..67de54be5b
--- /dev/null
+++ b/src/include/storage/pg_job_object.h
@@ -0,0 +1,35 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_job_object.h
+ * Windows Job Object support for preventing orphaned backends
+ *
+ * When the postmaster crashes on Windows, child processes continue running
+ * because Windows has no equivalent to Unix's parent death detection. Job
+ * Objects solve this by allowing the kernel to terminate all children when
+ * the job handle closes (which happens automatically on process exit).
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/storage/pg_job_object.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_JOB_OBJECT_H
+#define PG_JOB_OBJECT_H
+
+#ifdef WIN32
+
+extern void pg_create_job_object(void);
+extern void pg_destroy_job_object(void);
+extern bool pg_is_in_job_object(void);
+
+#else /* !WIN32 */
+
+#define pg_create_job_object() ((void) 0)
+#define pg_destroy_job_object() ((void) 0)
+#define pg_is_in_job_object() (false)
+
+#endif /* WIN32 */
+
+#endif /* PG_JOB_OBJECT_H */
--
2.46.0.windows.1
Hi,
On 2025-11-03 09:12:03 -0600, Bryan Green wrote:
We just need to call CreateJobObject() in PostmasterMain(), configure
with JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE, and assign the postmaster.
Children inherit membership automatically. When the job handle closes on
postmaster exit, the kernel terminates all children atomically. This is
kernel-enforced with no polling and no race conditions.
What happens if a postmaster child exits irregularly? Is postmaster terminated
as well?
The patch has been tested on Windows 10/11 with both MSVC and MinGW
builds. Nested jobs fail gracefully as expected. Clean shutdown is
unaffected. Crash tests with taskkill /F, debugger abort, and access
violations all correctly terminate children immediately with zero orphans.This patch does not include automated tests because the core
functionality (orphan prevention on crash) requires simulating process
termination, which is difficult to test reliably in CI.
Why is it difficult to test in CI? We do some related tests in
013_crash_restart.pl, it doesn't seem like it ought to be hard to also add
tests for postmaster?
Greetings,
Andres Freund
On 11/3/2025 9:19 AM, Andres Freund wrote:
Hi,
On 2025-11-03 09:12:03 -0600, Bryan Green wrote:
We just need to call CreateJobObject() in PostmasterMain(), configure
with JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE, and assign the postmaster.
Children inherit membership automatically. When the job handle closes on
postmaster exit, the kernel terminates all children atomically. This is
kernel-enforced with no polling and no race conditions.What happens if a postmaster child exits irregularly? Is postmaster terminated
as well?
No, Job Objects are unidirectional. KILL_ON_JOB_CLOSE only acts when the
postmaster (which holds the job handle) exits. Backend crashes are
handled through PostgreSQL's existing crash recovery mechanism - the
postmaster detects the crash via WaitForMultipleObjects() and initiates
recovery as normal.
The Job Object only takes action when the job handle closes, which
happens when the postmaster exits. It's analogous to a Unix process
group - sending SIGTERM to the group leader kills the group, but
children dying doesn't affect the parent.
The patch has been tested on Windows 10/11 with both MSVC and MinGW
builds. Nested jobs fail gracefully as expected. Clean shutdown is
unaffected. Crash tests with taskkill /F, debugger abort, and access
violations all correctly terminate children immediately with zero orphans.This patch does not include automated tests because the core
functionality (orphan prevention on crash) requires simulating process
termination, which is difficult to test reliably in CI.Why is it difficult to test in CI? We do some related tests in
013_crash_restart.pl, it doesn't seem like it ought to be hard to also add
tests for postmaster?
Fair point. I was hesitant because testing the actual orphan prevention
requires killing the postmaster while backends are active, which seemed
fragile. But you're right that we already test similar scenarios.
I can add a test to 013_crash_restart.pl (or a new Windows-specific test
file) that:
1. Starts server with active backend
2. Kills postmaster ungracefully (taskkill /F)
3. Verifies backend process terminates automatically
4. Confirms clean restart
Would that be sufficient, or do you have other test scenarios in mind?
Show quoted text
Greetings,
Andres Freund
On 2025-11-03 09:25:11 -0600, Bryan Green wrote:
On 11/3/2025 9:19 AM, Andres Freund wrote:
Hi,
On 2025-11-03 09:12:03 -0600, Bryan Green wrote:
We just need to call CreateJobObject() in PostmasterMain(), configure
with JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE, and assign the postmaster.
Children inherit membership automatically. When the job handle closes on
postmaster exit, the kernel terminates all children atomically. This is
kernel-enforced with no polling and no race conditions.What happens if a postmaster child exits irregularly? Is postmaster terminated
as well?No, Job Objects are unidirectional.
Great.
The patch has been tested on Windows 10/11 with both MSVC and MinGW
builds. Nested jobs fail gracefully as expected. Clean shutdown is
unaffected. Crash tests with taskkill /F, debugger abort, and access
violations all correctly terminate children immediately with zero orphans.This patch does not include automated tests because the core
functionality (orphan prevention on crash) requires simulating process
termination, which is difficult to test reliably in CI.Why is it difficult to test in CI? We do some related tests in
013_crash_restart.pl, it doesn't seem like it ought to be hard to also add
tests for postmaster?Fair point. I was hesitant because testing the actual orphan prevention
requires killing the postmaster while backends are active, which seemed
fragile. But you're right that we already test similar scenarios.I can add a test to 013_crash_restart.pl (or a new Windows-specific test
file) that:
1. Starts server with active backend
2. Kills postmaster ungracefully (taskkill /F)
3. Verifies backend process terminates automatically
4. Confirms clean restartWould that be sufficient, or do you have other test scenarios in mind?
That's pretty much what I had in mind.
Greetings,
Andres Freund
On 11/3/2025 9:29 AM, Andres Freund wrote:
On 2025-11-03 09:25:11 -0600, Bryan Green wrote:
On 11/3/2025 9:19 AM, Andres Freund wrote:
Hi,
On 2025-11-03 09:12:03 -0600, Bryan Green wrote:
We just need to call CreateJobObject() in PostmasterMain(), configure
with JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE, and assign the postmaster.
Children inherit membership automatically. When the job handle closes on
postmaster exit, the kernel terminates all children atomically. This is
kernel-enforced with no polling and no race conditions.What happens if a postmaster child exits irregularly? Is postmaster terminated
as well?No, Job Objects are unidirectional.
Great.
The patch has been tested on Windows 10/11 with both MSVC and MinGW
builds. Nested jobs fail gracefully as expected. Clean shutdown is
unaffected. Crash tests with taskkill /F, debugger abort, and access
violations all correctly terminate children immediately with zero orphans.This patch does not include automated tests because the core
functionality (orphan prevention on crash) requires simulating process
termination, which is difficult to test reliably in CI.Why is it difficult to test in CI? We do some related tests in
013_crash_restart.pl, it doesn't seem like it ought to be hard to also add
tests for postmaster?Fair point. I was hesitant because testing the actual orphan prevention
requires killing the postmaster while backends are active, which seemed
fragile. But you're right that we already test similar scenarios.I can add a test to 013_crash_restart.pl (or a new Windows-specific test
file) that:
1. Starts server with active backend
2. Kills postmaster ungracefully (taskkill /F)
3. Verifies backend process terminates automatically
4. Confirms clean restartWould that be sufficient, or do you have other test scenarios in mind?
That's pretty much what I had in mind.
Greetings,
Andres Freund
I've implemented the test in 013_crash_restart.pl.
The test passes on Windows 10/11 with both MSVC and MinGW builds.
Backends are typically terminated within 100-200ms after postmaster
kill, confirming the Job Object KILL_ON_JOB_CLOSE mechanism works as
intended.
Updated patch (v2) attached.
--
Bryan
Attachments:
v2-0001-Use-Windows-Job-Objects-to-prevent-orphaned-child.patchtext/plain; charset=UTF-8; name=v2-0001-Use-Windows-Job-Objects-to-prevent-orphaned-child.patchDownload
From 8e43c7656980a8bcf86f9b592b21de4636502076 Mon Sep 17 00:00:00 2001
From: Bryan Green <dbryan.green@gmail.com>
Date: Thu, 30 Oct 2025 11:14:55 -0600
Subject: [PATCH v2] Use Windows Job Objects to prevent orphaned child
processes.
When the postmaster exits on Windows, backends can continue running
because Windows lacks Unix's getppid() orphan detection. Orphaned
backends hold locks and shared memory, preventing clean restart.
Create a Job Object at postmaster startup and assign the postmaster
to it. Children inherit job membership automatically. Configure with
KILL_ON_JOB_CLOSE so the kernel terminates all children when the job
handle closes on postmaster exit.
This is more reliable than existing approaches (inherited handles,
shared memory flags) because it's kernel-enforced with no polling.
Job creation is allowed to fail non-fatally. Some environments run
PostgreSQL under an existing job, and Windows 7 disallows nested jobs.
In such cases we log a message and proceed without orphan protection.
Author: Bryan Green
---
doc/src/sgml/runtime.sgml | 21 +++
src/backend/postmaster/postmaster.c | 12 ++
src/backend/storage/ipc/Makefile | 4 +
src/backend/storage/ipc/meson.build | 7 +
src/backend/storage/ipc/pg_job_object.c | 176 +++++++++++++++++++++++
src/include/storage/pg_job_object.h | 35 +++++
src/test/recovery/t/013_crash_restart.pl | 111 ++++++++++++++
7 files changed, 366 insertions(+)
create mode 100644 src/backend/storage/ipc/pg_job_object.c
create mode 100644 src/include/storage/pg_job_object.h
diff --git a/doc/src/sgml/runtime.sgml b/doc/src/sgml/runtime.sgml
index 0c60bafac6..26d2ff64a5 100644
--- a/doc/src/sgml/runtime.sgml
+++ b/doc/src/sgml/runtime.sgml
@@ -1501,6 +1501,27 @@ $ <userinput>cat /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages</userinp
</para>
</sect2>
+
+ <sect2 id="windows-process-management">
+ <title>Process Management on <systemitem class="osname">Windows</systemitem></title>
+
+ <para>
+ On <systemitem class="osname">Windows</systemitem>,
+ <productname>PostgreSQL</productname> uses Job Objects to
+ manage child processes. If the postmaster exits unexpectedly
+ (due to a crash or external termination), the operating system
+ automatically terminates all backend processes. This prevents
+ orphaned backends that could hold locks or corrupt shared memory.
+ </para>
+
+ <para>
+ In some cases, Job Object creation may fail (for example, when
+ <productname>PostgreSQL</productname> is already running under a
+ job-aware service manager). The server will log a message and continue
+ to run without orphan protection. This is safe but means that backends
+ may need to be manually terminated if the postmaster crashes.
+ </para>
+ </sect2>
</sect1>
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 00de559ba8..14697b95e2 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -113,6 +113,7 @@
#include "storage/fd.h"
#include "storage/io_worker.h"
#include "storage/ipc.h"
+#include "storage/pg_job_object.h"
#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "tcop/backend_startup.h"
@@ -1005,6 +1006,17 @@ PostmasterMain(int argc, char *argv[])
*/
CreateSharedMemoryAndSemaphores();
+#ifdef WIN32
+ /*
+ * On Windows, create a job object to prevent orphaned backends.
+ * If postmaster crashes, Windows will automatically kill all
+ * child processes in the job.
+ *
+ * We do this after port binding so that if job creation fails,
+ * it's not fatal - we can still run (just without orphan protection).
+ */
+ pg_create_job_object();
+#endif
/*
* Estimate number of openable files. This must happen after setting up
* semaphores, because on some platforms semaphores count as open files.
diff --git a/src/backend/storage/ipc/Makefile b/src/backend/storage/ipc/Makefile
index 9a07f6e1d9..0604f4f382 100644
--- a/src/backend/storage/ipc/Makefile
+++ b/src/backend/storage/ipc/Makefile
@@ -28,4 +28,8 @@ OBJS = \
standby.o \
waiteventset.o
+ifeq ($(PORTNAME), win32)
+OBJS += pg_job_object.o
+endif
+
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/ipc/meson.build b/src/backend/storage/ipc/meson.build
index b1b73dac3b..85ba13b70d 100644
--- a/src/backend/storage/ipc/meson.build
+++ b/src/backend/storage/ipc/meson.build
@@ -21,3 +21,10 @@ backend_sources += files(
'waiteventset.c',
)
+
+# Windows-specific files
+if host_system == 'windows'
+ backend_sources += files(
+ 'pg_job_object.c',
+ )
+endif
diff --git a/src/backend/storage/ipc/pg_job_object.c b/src/backend/storage/ipc/pg_job_object.c
new file mode 100644
index 0000000000..c84aeed9cd
--- /dev/null
+++ b/src/backend/storage/ipc/pg_job_object.c
@@ -0,0 +1,176 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_job_object.c
+ * Windows Job Object support for preventing orphaned backends
+ *
+ * On Unix, backends can detect when the postmaster dies via getppid().
+ * Windows has no equivalent mechanism. We solve this by using Job Objects,
+ * a Windows kernel feature that groups processes and can automatically
+ * terminate all members when the job handle closes.
+ *
+ * By configuring JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE, we ensure that if
+ * the postmaster exits (cleanly or via crash), Windows immediately kills
+ * all backends. This prevents orphaned processes that hold locks and
+ * prevent clean restart.
+ *
+ * The job object handle is stored in a static variable and never explicitly
+ * closed. This is intentional - we rely on Windows closing it automatically
+ * when the postmaster process exits, which triggers the child termination.
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/storage/ipc/pg_job_object.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#ifdef WIN32
+
+#include "postmaster/postmaster.h"
+#include "storage/ipc.h"
+#include "storage/pg_job_object.h"
+
+static HANDLE pg_job_object = NULL;
+
+
+/*
+ * pg_create_job_object
+ *
+ * Create job object for this PostgreSQL instance and configure it to
+ * kill all children when the postmaster exits.
+ *
+ * Failure is not fatal - we log a warning and continue. PostgreSQL will
+ * run without orphan protection, which is no worse than current behavior.
+ */
+void
+pg_create_job_object(void)
+{
+ JOBOBJECT_EXTENDED_LIMIT_INFORMATION limit_info;
+ char job_name[128];
+ DWORD error;
+
+ snprintf(job_name, sizeof(job_name), "PostgreSQL_Port_%d_PID_%lu",
+ PostPortNumber, GetCurrentProcessId());
+
+ pg_job_object = CreateJobObjectA(NULL, job_name);
+
+ if (pg_job_object == NULL)
+ {
+ error = GetLastError();
+ ereport(LOG,
+ (errmsg("could not create job object \"%s\": error code %lu",
+ job_name, error),
+ errdetail("Orphaned process cleanup will not be available.")));
+ return;
+ }
+
+ elog(DEBUG1, "created job object \"%s\"", job_name);
+
+ /*
+ * Set KILL_ON_JOB_CLOSE. When the job handle closes (either explicit
+ * close or process termination), all processes in the job are terminated.
+ *
+ * This is the critical flag that prevents orphaned backends.
+ */
+ memset(&limit_info, 0, sizeof(limit_info));
+ limit_info.BasicLimitInformation.LimitFlags = JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE;
+
+ if (!SetInformationJobObject(pg_job_object,
+ JobObjectExtendedLimitInformation,
+ &limit_info,
+ sizeof(limit_info)))
+ {
+ error = GetLastError();
+ ereport(WARNING,
+ (errmsg("could not configure job object: error code %lu", error),
+ errdetail("Job object created but KILL_ON_JOB_CLOSE not set."),
+ errhint("Orphaned processes may occur if postmaster crashes.")));
+ CloseHandle(pg_job_object);
+ pg_job_object = NULL;
+ return;
+ }
+
+ if (!AssignProcessToJobObject(pg_job_object, GetCurrentProcess()))
+ {
+ error = GetLastError();
+
+ /*
+ * ERROR_ACCESS_DENIED means we're already in a job. This can happen
+ * when PostgreSQL runs under a job-aware supervisor (Windows service
+ * on older Windows, or any process manager using nested jobs).
+ *
+ * On Windows 8+, we could use nested jobs, but for simplicity we
+ * just skip job creation. The parent job should handle cleanup.
+ */
+ if (error == ERROR_ACCESS_DENIED)
+ {
+ ereport(LOG,
+ (errmsg("postmaster is already in a job object"),
+ errdetail("This can occur when PostgreSQL is run under a job-aware supervisor."),
+ errhint("Automatic orphan cleanup will not be available.")));
+ }
+ else
+ {
+ ereport(WARNING,
+ (errmsg("could not assign postmaster to job object: error code %lu", error)));
+ }
+
+ CloseHandle(pg_job_object);
+ pg_job_object = NULL;
+ return;
+ }
+
+ elog(LOG, "PostgreSQL job object configured successfully - orphaned process prevention enabled");
+}
+
+
+/*
+ * pg_destroy_job_object
+ *
+ * Explicitly close the job object handle. This will trigger KILL_ON_JOB_CLOSE,
+ * terminating all backends.
+ *
+ * Note: In most cases we don't call this - we rely on Windows closing the
+ * handle automatically when the postmaster exits. Explicit close is only
+ * needed if we want to control the exact timing of backend termination.
+ */
+void
+pg_destroy_job_object(void)
+{
+ if (pg_job_object != NULL)
+ {
+ elog(DEBUG1, "closing job object - all child processes will terminate");
+ CloseHandle(pg_job_object);
+ pg_job_object = NULL;
+ }
+}
+
+
+/*
+ * pg_is_in_job_object
+ *
+ * Check if current process is in the PostgreSQL job object.
+ * Used primarily for testing and verification.
+ */
+bool
+pg_is_in_job_object(void)
+{
+ BOOL in_job = FALSE;
+
+ if (pg_job_object == NULL)
+ return false;
+
+ if (!IsProcessInJob(GetCurrentProcess(), pg_job_object, &in_job))
+ {
+ elog(DEBUG1, "IsProcessInJob failed: error code %lu", GetLastError());
+ return false;
+ }
+
+ return (bool) in_job;
+}
+
+#endif /* WIN32 */
diff --git a/src/include/storage/pg_job_object.h b/src/include/storage/pg_job_object.h
new file mode 100644
index 0000000000..67de54be5b
--- /dev/null
+++ b/src/include/storage/pg_job_object.h
@@ -0,0 +1,35 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_job_object.h
+ * Windows Job Object support for preventing orphaned backends
+ *
+ * When the postmaster crashes on Windows, child processes continue running
+ * because Windows has no equivalent to Unix's parent death detection. Job
+ * Objects solve this by allowing the kernel to terminate all children when
+ * the job handle closes (which happens automatically on process exit).
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/storage/pg_job_object.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_JOB_OBJECT_H
+#define PG_JOB_OBJECT_H
+
+#ifdef WIN32
+
+extern void pg_create_job_object(void);
+extern void pg_destroy_job_object(void);
+extern bool pg_is_in_job_object(void);
+
+#else /* !WIN32 */
+
+#define pg_create_job_object() ((void) 0)
+#define pg_destroy_job_object() ((void) 0)
+#define pg_is_in_job_object() (false)
+
+#endif /* WIN32 */
+
+#endif /* PG_JOB_OBJECT_H */
diff --git a/src/test/recovery/t/013_crash_restart.pl b/src/test/recovery/t/013_crash_restart.pl
index 4c5af018ee..db3af2c7a4 100644
--- a/src/test/recovery/t/013_crash_restart.pl
+++ b/src/test/recovery/t/013_crash_restart.pl
@@ -251,4 +251,115 @@ is( $node->safe_psql(
$node->stop();
+# Test Windows Job Objects orphan prevention
+if ($windows_os)
+{
+ note "testing Windows Job Object orphan prevention";
+
+ my $jobtest = PostgreSQL::Test::Cluster->new('jobtest');
+ $jobtest->init(allows_streaming => 1);
+ $jobtest->start();
+
+ my $log_contents = slurp_file($jobtest->logfile);
+ like($log_contents, qr/orphaned process prevention enabled/,
+ "Job Objects enabled on startup");
+
+ $jobtest->safe_psql('postgres',
+ q[ALTER SYSTEM SET restart_after_crash = 1;
+ SELECT pg_reload_conf();]);
+
+ my ($backend_stdin, $backend_stdout, $backend_stderr) = ('', '', '');
+ my $backend = IPC::Run::start(
+ [
+ 'psql', '--no-psqlrc', '--quiet', '--no-align', '--tuples-only',
+ '--file', '-', '--dbname' => $jobtest->connstr('postgres')
+ ],
+ '<' => \$backend_stdin,
+ '>' => \$backend_stdout,
+ '2>' => \$backend_stderr,
+ $psql_timeout);
+
+ $backend_stdin .= q[
+BEGIN;
+CREATE TABLE jobtest(id int);
+SELECT pg_backend_pid();
+];
+ ok( pump_until(
+ $backend, $psql_timeout,
+ \$backend_stdout, qr/[[:digit:]]+[\r\n]$/m),
+ 'acquired backend pid for job object test');
+ my $backend_pid = $backend_stdout;
+ chomp($backend_pid);
+ $backend_stdout = '';
+
+ note "backend PID: $backend_pid";
+
+ my $postmaster_pidfile = $jobtest->data_dir . '/postmaster.pid';
+ open(my $pidfh, '<', $postmaster_pidfile)
+ or die "cannot open postmaster.pid: $!";
+ my $postmaster_pid = <$pidfh>;
+ close($pidfh);
+ chomp($postmaster_pid);
+
+ note "postmaster PID: $postmaster_pid";
+ ok($postmaster_pid =~ /^\d+$/ && $postmaster_pid > 0,
+ "got valid postmaster PID: $postmaster_pid");
+
+ note "killing postmaster PID $postmaster_pid";
+ my $ret = PostgreSQL::Test::Utils::system_log('pg_ctl', 'kill', 'KILL',
+ $postmaster_pid);
+ is($ret, 0, "killed postmaster with pg_ctl kill KILL");
+
+ eval { $backend->finish; };
+
+ note "waiting for postmaster process to exit";
+ my $postmaster_gone = 0;
+ for (my $i = 0; $i < 30; $i++)
+ {
+ my $check = `tasklist /FI "PID eq $postmaster_pid" /NH 2>&1`;
+ if ($check !~ /postgres\.exe/)
+ {
+ $postmaster_gone = 1;
+ note "postmaster exited after " . (($i + 1) * 100) . "ms";
+ last;
+ }
+ select(undef, undef, undef, 0.1);
+ }
+
+ ok($postmaster_gone, "postmaster process exited");
+
+ $jobtest->{_pid} = undef;
+
+ unlink($postmaster_pidfile);
+
+ # THE REAL TEST: Can we restart without conflicts?
+ # If orphans exist, they'll block the port or shared memory
+ note "attempting restart (will fail if orphans present)";
+
+ my $restart_ok = 0;
+ eval {
+ $jobtest->start();
+ $restart_ok = 1;
+ };
+
+ if (!$restart_ok) {
+ fail("restart failed - orphans may be blocking resources: $@");
+ } else {
+ pass("restart succeeded - no orphans blocking port or shared memory");
+
+ eval {
+ my $result = $jobtest->safe_psql('postgres', 'SELECT 1');
+ is($result, '1', 'server functional after restart');
+ };
+ if ($@) {
+ note "query after restart failed: $@";
+ fail("query after restart failed");
+ }
+ }
+
+ note "stopping test cluster";
+ eval { $jobtest->stop('immediate'); };
+}
+
+
done_testing();
--
2.46.0.windows.1
On Tue, Nov 4, 2025 at 4:12 AM Bryan Green <dbryan.green@gmail.com> wrote:
Current approaches (inherited event handles, shared memory flags) depend
on the postmaster running code during exit. A segfault or kill bypasses
all of that.
Huh. I thought PostmasterHandle should be signalled by the kernel,
not by any code run by the postmaster itself, when taskkill /f
calls something like TerminateProcess(). Is that not the case? Are
you sure we haven't broken something on our side?
My proposed solution is to use Windows Job Objects with KILL_ON_JOB_CLOSE.
I'm not a Windows guy but I had been researching this independently,
and it does seem to be the standard approach, so +1 generally even
though I'm still very curious to know *why* the existing coding
doesn't work in such a simple case.
By coincidence, I'm getting close to posting a stack of patches for
better postmaster death handling on Unix too, along with better
subprocess and interrupt multiplexing and cleanup. That does more
stuff, with connections to multithreading, interrupts, critical
sections, a bunch of existing bugs we have with subprocess management
on Unix. I will gladly delete my own attempt at Windows job objects
from that effort and rebase on top of your patch, and see what review
feedback ideas come up in that process.
In nearby threads that triggered my work on that, I was a bit worried
about the change in behaviour on PM death in the syncrep and
syslogger, but I'm beginning to suspect that the vast majority of
Linux/systemd deployments probably just nuke the whole cgroup from
orbit in this case, so it seems like exceptional behaviour is really
just a recipe for fragility on rarer systems, not to mention that it
probably has to be like that in a potential multithreaded mode anyway.
We should probably just rip all such specialness out, which I'll show
in my patch set with some explanations soon.
On the other hand, I was thinking of this as a v19 feature, where one
can contemplate such changes, but you said:
Job creation can fail if postgres runs under an existing job (service
managers, debuggers). Windows 7 disallows nested jobs. We detect this
with IsProcessInJob(), and if AssignProcessToJobObject() returns
ERROR_ACCESS_DENIED, we log and continue without orphan protection.
We currently require Windows 10 (itself recently EOL'd), but
PostgreSQL 13-15 sort of claim to work on Windows all the way back to
7, so I'm guessing you're imagining back-patching this?
This patch does not include automated tests because the core
functionality (orphan prevention on crash) requires simulating process
termination, which is difficult to test reliably in CI.
Ah yes I ran into problems with that part too...
On 11/6/2025 6:46 AM, Thomas Munro wrote:
On Tue, Nov 4, 2025 at 4:12 AM Bryan Green <dbryan.green@gmail.com> wrote:
Current approaches (inherited event handles, shared memory flags) depend
on the postmaster running code during exit. A segfault or kill bypasses
all of that.Huh. I thought PostmasterHandle should be signalled by the kernel,
not by any code run by the postmaster itself, when taskkill /f
calls something like TerminateProcess(). Is that not the case? Are
you sure we haven't broken something on our side?
Yes, that is true. I may have been a bit sloppy with my wording here.
This patch is focused solely on quickly terminating the backend
processes when postmaster has crashed. Any children created in a
suspended state will not handle the signal. Additionally, we have
handles being inherited by the children that keep them from terminating.
See Fix Socket handle Inheritance on Windows
(https://commitfest.postgresql.org/patch/6207/) and O_CLOEXEC not
honored on Windows - handle inheritance chain
(https://commitfest.postgresql.org/patch/6197/). When the children are
in a suspended state or have handles inherited from the parent process
then we have to wait-- which we shouldn't do when postmaster crashes.
The reason to still do this patch and clean up the handle inheritance
mess is that there are states (suspended state, infinite loop, spinlock
hold, whatever) that a process can be in that keeps it from processing
the event. We don't need to wait on the children to voluntarily exit
when postmaster crashes.
My proposed solution is to use Windows Job Objects with KILL_ON_JOB_CLOSE.
I'm not a Windows guy but I had been researching this independently,
and it does seem to be the standard approach, so +1 generally even
though I'm still very curious to know *why* the existing coding
doesn't work in such a simple case.By coincidence, I'm getting close to posting a stack of patches for
better postmaster death handling on Unix too, along with better
subprocess and interrupt multiplexing and cleanup. That does more
stuff, with connections to multithreading, interrupts, critical
sections, a bunch of existing bugs we have with subprocess management
on Unix. I will gladly delete my own attempt at Windows job objects
from that effort and rebase on top of your patch, and see what review
feedback ideas come up in that process.
That sounds excellent. I'd be happy to coordinate - please feel free to
build on this work. The Windows Job Object approach is fairly
self-contained, so it should integrate cleanly with your broader
subprocess management improvements.
In nearby threads that triggered my work on that, I was a bit worried
about the change in behaviour on PM death in the syncrep and
syslogger, but I'm beginning to suspect that the vast majority of
Linux/systemd deployments probably just nuke the whole cgroup from
orbit in this case, so it seems like exceptional behaviour is really
just a recipe for fragility on rarer systems, not to mention that it
probably has to be like that in a potential multithreaded mode anyway.
We should probably just rip all such specialness out, which I'll show
in my patch set with some explanations soon.On the other hand, I was thinking of this as a v19 feature, where one
can contemplate such changes, but you said:Job creation can fail if postgres runs under an existing job (service
managers, debuggers). Windows 7 disallows nested jobs. We detect this
with IsProcessInJob(), and if AssignProcessToJobObject() returns
ERROR_ACCESS_DENIED, we log and continue without orphan protection.We currently require Windows 10 (itself recently EOL'd), but
PostgreSQL 13-15 sort of claim to work on Windows all the way back to
7, so I'm guessing you're imagining back-patching this?
No, I'm targeting v19 as a new feature. The Windows 7 comment was just
being defensive about the failure mode - since we allow job creation to
fail gracefully, there's no harm in having the code check for nested job
scenarios even though we don't officially support Win7 anymore. The code
degrades gracefully to "current behavior" if job creation fails for any
reason.
This patch does not include automated tests because the core
functionality (orphan prevention on crash) requires simulating process
termination, which is difficult to test reliably in CI.Ah yes I ran into problems with that part too...
Thanks for the review, I look forward to working on this problem area
with you. It's good to have more than one set of eyes looking at a problem.
--
Bryan Green
EDB: https://www.enterprisedb.com
On Fri, Nov 7, 2025 at 3:13 AM Bryan Green <dbryan.green@gmail.com> wrote:
The reason to still do this patch and clean up the handle inheritance
mess is that there are states (suspended state, infinite loop, spinlock
hold, whatever) that a process can be in that keeps it from processing
the event. We don't need to wait on the children to voluntarily exit
when postmaster crashes.
Agreed on all points. We'd recently come to the same conclusion on this thread:
/messages/by-id/B3C69B86-7F82-4111-B97F-0005497BB745@yandex-team.ru
I think there might arguably be a sort of weak forward progress
guarantee in the existing design and it's been a while since we've had
problem reports AFAIR*: locks were releases (which turns out to be
fundamentally unsafe at least while in a critical section as analysed
in that thread, but it does allow progress in blocked backends, so
that they can learn of the postmaster's demise), and no one should
enter WaitEventSet() while holding a spinlock, and infinite loops are
against the law, and it's previously been considered acceptable-ish
that a backend might continue to run a long query until completion
before exiting (without supporting auxiliary or worker backends, which
sounds potentially suspect, but at least you can't wait for another
backend without learning of the PostgreSQL's demise assuming the only
possible waits are LWLocks or latches). But clearly it's not good
enough.
The fact that Windows backends are born in suspended state until the
postmaster resumes them is indeed a new and significant hole in that
theory. Preemptive termination is the only thing that makes sense.
*We used to have places that waited but forgot to handle PM exit, and
I don't recall "manual orphan cleanup needed" reports since we
enforced a central handler. But see also my earlier note about
systemd potentially hiding problems these days, if using "mixed" mode
to SIGKILL the whole cgroup.
On 11/6/2025 8:39 PM, Thomas Munro wrote:
On Fri, Nov 7, 2025 at 3:13 AM Bryan Green <dbryan.green@gmail.com> wrote:
The reason to still do this patch and clean up the handle inheritance
mess is that there are states (suspended state, infinite loop, spinlock
hold, whatever) that a process can be in that keeps it from processing
the event. We don't need to wait on the children to voluntarily exit
when postmaster crashes.Agreed on all points. We'd recently come to the same conclusion on this thread:
/messages/by-id/B3C69B86-7F82-4111-B97F-0005497BB745@yandex-team.ru
Thanks for the link - I'll review that thread. It's reassuring to see
independent analysis reaching the same conclusions.
I think there might arguably be a sort of weak forward progress
guarantee in the existing design and it's been a while since we've had
problem reports AFAIR*: locks were releases (which turns out to be
fundamentally unsafe at least while in a critical section as analysed
in that thread, but it does allow progress in blocked backends, so
that they can learn of the postmaster's demise), and no one should
enter WaitEventSet() while holding a spinlock, and infinite loops are
against the law, and it's previously been considered acceptable-ish
that a backend might continue to run a long query until completion
before exiting (without supporting auxiliary or worker backends, which
sounds potentially suspect, but at least you can't wait for another
backend without learning of the PostgreSQL's demise assuming the only
possible waits are LWLocks or latches). But clearly it's not good
enough.The fact that Windows backends are born in suspended state until the
postmaster resumes them is indeed a new and significant hole in that
theory. Preemptive termination is the only thing that makes sense.
Exactly. The suspended state issue-- even with perfect handle
inheritance hygiene, a backend that hasn't been resumed yet cannot
receive or process any event notification. The postmaster could crash
after CreateProcess() but before ResumeThread(), leaving a zombie
process that will never wake up.
The handle inheritance problems I've been working on (socket handles,
O_CLOEXEC) are somewhat orthogonal - they cause *delayed* termination
rather than indefinite orphaning. But Job Objects solves both classes
of problems: it handles the suspended state case immediately, and
provides guaranteed cleanup regardless of handle state.
*We used to have places that waited but forgot to handle PM exit, and
I don't recall "manual orphan cleanup needed" reports since we
enforced a central handler. But see also my earlier note about
systemd potentially hiding problems these days, if using "mixed" mode
to SIGKILL the whole cgroup.
That's an interesting observation about systemd potentially masking
fragility. If Linux deployments are really just nuking the cgroup on
postmaster death, then the "voluntary exit" approach isn't actually
being tested in production at scale.
I'm looking forward to your broader subprocess management patchset.
--
Bryan Green
EDB: https://www.enterprisedb.com