Refactoring backend fork+exec code
Started by Heikki Linnakangasover 2 years ago63 messages
I started to look at the code in postmaster.c related to launching child
processes. I tried to reduce the difference between EXEC_BACKEND and
!EXEC_BACKEND code paths, and put the code that needs to differ behind a
better abstraction. I started doing this to help with implementing
multi-threading, but it doesn't introduce anything thread-related yet
and I think this improves readability anyway.
This is still work-inprogress, especially the last, big, patch in the
patch set. Mainly, I need to clean up the comments in the new
launch_backend.c file. But the other patches are in pretty good shape,
and if you ignore launch_backend.c, you can see the effect on the other
source files.
With these patches, there is a new function for launching a postmaster
child process:
pid_t postmaster_child_launch(PostmasterChildType child_type, char
*startup_data, size_t startup_data_len, ClientSocket *client_sock);
This function hides the differences between EXEC_BACKEND and
!EXEC_BACKEND cases.
In 'startup_data', the caller can pass a blob of data to the child
process, with different meaning for different kinds of child processes.
For a backend process, for example, it's used to pass the CAC_state,
which indicates whether the backend accepts the connection or just sends
"too many clients" error. And for background workers, it's used to pass
the BackgroundWorker struct. The startup data is passed to the child
process in the
ClientSocket is a new struct holds a socket FD, and the local and remote
address info. Before this patch set, postmaster initializes the Port
structs but only fills in those fields in it. With this patch set, we
have a new ClientSocket struct just for those fields, which makes it
more clear which fields are initialized where.
I haven't done much testing yet, and no testing at all on Windows, so
that's probably still broken.
--
Heikki Linnakangas
Neon (https://neon.tech)
Attachments:
0001-Allocate-Backend-structs-in-PostmasterContext.patchtext/x-patch; charset=UTF-8; name=0001-Allocate-Backend-structs-in-PostmasterContext.patchDownload
From 89cbf93c3490d38e7add149fd168f4f7a7eb670e Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Sun, 18 Jun 2023 10:41:29 +0300
Subject: [PATCH 1/9] Allocate Backend structs in PostmasterContext.
The child processes don't need them. By allocating them in
PostmasterContext, the memory gets free'd and is made available for
other stuff in the child processes.
---
src/backend/postmaster/bgworker.c | 11 ++++++++---
src/backend/postmaster/postmaster.c | 24 +++++++++++-------------
2 files changed, 19 insertions(+), 16 deletions(-)
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index 0dd22b23511..07ac68f50bc 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -33,6 +33,7 @@
#include "storage/shmem.h"
#include "tcop/tcopprot.h"
#include "utils/ascii.h"
+#include "utils/memutils.h"
#include "utils/ps_status.h"
#include "utils/timeout.h"
@@ -344,7 +345,9 @@ BackgroundWorkerStateChange(bool allow_new_workers)
/*
* Copy the registration data into the registered workers list.
*/
- rw = malloc(sizeof(RegisteredBgWorker));
+ rw = MemoryContextAllocExtended(PostmasterContext,
+ sizeof(RegisteredBgWorker),
+ MCXT_ALLOC_NO_OOM);
if (rw == NULL)
{
ereport(LOG,
@@ -452,7 +455,7 @@ ForgetBackgroundWorker(slist_mutable_iter *cur)
rw->rw_worker.bgw_name)));
slist_delete_current(cur);
- free(rw);
+ pfree(rw);
}
/*
@@ -926,7 +929,9 @@ RegisterBackgroundWorker(BackgroundWorker *worker)
/*
* Copy the registration data into the registered workers list.
*/
- rw = malloc(sizeof(RegisteredBgWorker));
+ rw = MemoryContextAllocExtended(PostmasterContext,
+ sizeof(RegisteredBgWorker),
+ MCXT_ALLOC_NO_OOM);
if (rw == NULL)
{
ereport(LOG,
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 4c49393fc5a..ee9e24e4e74 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -3357,7 +3357,7 @@ CleanupBackgroundWorker(int pid,
*/
if (rw->rw_backend->bgworker_notify)
BackgroundWorkerStopNotifications(rw->rw_pid);
- free(rw->rw_backend);
+ pfree(rw->rw_backend);
rw->rw_backend = NULL;
rw->rw_pid = 0;
rw->rw_child_slot = 0;
@@ -3450,7 +3450,7 @@ CleanupBackend(int pid,
BackgroundWorkerStopNotifications(bp->pid);
}
dlist_delete(iter.cur);
- free(bp);
+ pfree(bp);
break;
}
}
@@ -3506,7 +3506,7 @@ HandleChildCrash(int pid, int exitstatus, const char *procname)
#ifdef EXEC_BACKEND
ShmemBackendArrayRemove(rw->rw_backend);
#endif
- free(rw->rw_backend);
+ pfree(rw->rw_backend);
rw->rw_backend = NULL;
rw->rw_pid = 0;
rw->rw_child_slot = 0;
@@ -3543,7 +3543,7 @@ HandleChildCrash(int pid, int exitstatus, const char *procname)
#endif
}
dlist_delete(iter.cur);
- free(bp);
+ pfree(bp);
/* Keep looping so we can signal remaining backends */
}
else
@@ -4119,7 +4119,7 @@ BackendStartup(Port *port)
* Create backend data structure. Better before the fork() so we can
* handle failure cleanly.
*/
- bn = (Backend *) malloc(sizeof(Backend));
+ bn = (Backend *) palloc_extended(sizeof(Backend), MCXT_ALLOC_NO_OOM);
if (!bn)
{
ereport(LOG,
@@ -4135,7 +4135,7 @@ BackendStartup(Port *port)
*/
if (!RandomCancelKey(&MyCancelKey))
{
- free(bn);
+ pfree(bn);
ereport(LOG,
(errcode(ERRCODE_INTERNAL_ERROR),
errmsg("could not generate random cancel key")));
@@ -4165,8 +4165,6 @@ BackendStartup(Port *port)
pid = fork_process();
if (pid == 0) /* child */
{
- free(bn);
-
/* Detangle from postmaster */
InitPostmasterChild();
@@ -4197,7 +4195,7 @@ BackendStartup(Port *port)
if (!bn->dead_end)
(void) ReleasePostmasterChildSlot(bn->child_slot);
- free(bn);
+ pfree(bn);
errno = save_errno;
ereport(LOG,
(errmsg("could not fork new process for connection: %m")));
@@ -5460,7 +5458,7 @@ StartAutovacuumWorker(void)
return;
}
- bn = (Backend *) malloc(sizeof(Backend));
+ bn = (Backend *) palloc_extended(sizeof(Backend), MCXT_ALLOC_NO_OOM);
if (bn)
{
bn->cancel_key = MyCancelKey;
@@ -5487,7 +5485,7 @@ StartAutovacuumWorker(void)
* logged by StartAutoVacWorker
*/
(void) ReleasePostmasterChildSlot(bn->child_slot);
- free(bn);
+ pfree(bn);
}
else
ereport(LOG,
@@ -5732,7 +5730,7 @@ do_start_bgworker(RegisteredBgWorker *rw)
/* undo what assign_backendlist_entry did */
ReleasePostmasterChildSlot(rw->rw_child_slot);
rw->rw_child_slot = 0;
- free(rw->rw_backend);
+ pfree(rw->rw_backend);
rw->rw_backend = NULL;
/* mark entry as crashed, so we'll try again later */
rw->rw_crashed_at = GetCurrentTimestamp();
@@ -5858,7 +5856,7 @@ assign_backendlist_entry(RegisteredBgWorker *rw)
return false;
}
- bn = malloc(sizeof(Backend));
+ bn = palloc_extended(sizeof(Backend), MCXT_ALLOC_NO_OOM);
if (bn == NULL)
{
ereport(LOG,
--
2.30.2
0002-Pass-background-worker-entry-in-the-parameter-file.patchtext/x-patch; charset=UTF-8; name=0002-Pass-background-worker-entry-in-the-parameter-file.patchDownload
From 0a99a16741977ef2b10ebee8198692e818f8b700 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Sun, 18 Jun 2023 12:45:50 +0300
Subject: [PATCH 2/9] Pass background worker entry in the parameter file
This makes it possible to move InitProcess later in SubPostmasterMain
(in next commit), as we don't need an lwlock and hence a PGPROC entry
to get the background worker entry anymore.
---
src/backend/postmaster/bgworker.c | 21 ------------
src/backend/postmaster/postmaster.c | 37 ++++++++++++++-------
src/include/postmaster/bgworker_internals.h | 4 ---
3 files changed, 25 insertions(+), 37 deletions(-)
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index 07ac68f50bc..b7140306e43 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -628,27 +628,6 @@ ResetBackgroundWorkerCrashTimes(void)
}
}
-#ifdef EXEC_BACKEND
-/*
- * In EXEC_BACKEND mode, workers use this to retrieve their details from
- * shared memory.
- */
-BackgroundWorker *
-BackgroundWorkerEntry(int slotno)
-{
- static BackgroundWorker myEntry;
- BackgroundWorkerSlot *slot;
-
- Assert(slotno < BackgroundWorkerData->total_slots);
- slot = &BackgroundWorkerData->slot[slotno];
- Assert(slot->in_use);
-
- /* must copy this in case we don't intend to retain shmem access */
- memcpy(&myEntry, &slot->worker, sizeof myEntry);
- return &myEntry;
-}
-#endif
-
/*
* Complain about the BackgroundWorker definition using error level elevel.
* Return true if it looks ok, false if not (unless elevel >= ERROR, in
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index ee9e24e4e74..a7a9e061c5b 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -540,6 +540,8 @@ typedef struct
#endif
char my_exec_path[MAXPGPATH];
char pkglib_path[MAXPGPATH];
+
+ BackgroundWorker MyBgworkerEntry;
} BackendParameters;
static void read_backend_variables(char *id, Port *port);
@@ -4867,7 +4869,7 @@ SubPostmasterMain(int argc, char *argv[])
strcmp(argv[1], "--forkavlauncher") == 0 ||
strcmp(argv[1], "--forkavworker") == 0 ||
strcmp(argv[1], "--forkaux") == 0 ||
- strncmp(argv[1], "--forkbgworker=", 15) == 0)
+ strncmp(argv[1], "--forkbgworker", 14) == 0)
PGSharedMemoryReAttach();
else
PGSharedMemoryNoReAttach();
@@ -4998,10 +5000,8 @@ SubPostmasterMain(int argc, char *argv[])
AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
}
- if (strncmp(argv[1], "--forkbgworker=", 15) == 0)
+ if (strncmp(argv[1], "--forkbgworker", 14) == 0)
{
- int shmem_slot;
-
/* do this as early as possible; in particular, before InitProcess() */
IsBackgroundWorker = true;
@@ -5014,10 +5014,6 @@ SubPostmasterMain(int argc, char *argv[])
/* Attach process to shared data structures */
CreateSharedMemoryAndSemaphores();
- /* Fetch MyBgworkerEntry from shared memory */
- shmem_slot = atoi(argv[1] + 15);
- MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
-
StartBackgroundWorker();
}
if (strcmp(argv[1], "--forklog") == 0)
@@ -5662,13 +5658,14 @@ BackgroundWorkerUnblockSignals(void)
#ifdef EXEC_BACKEND
static pid_t
-bgworker_forkexec(int shmem_slot)
+bgworker_forkexec(BackgroundWorker *worker)
{
char *av[10];
int ac = 0;
char forkav[MAXPGPATH];
+ pid_t result;
- snprintf(forkav, MAXPGPATH, "--forkbgworker=%d", shmem_slot);
+ snprintf(forkav, MAXPGPATH, "--forkbgworker");
av[ac++] = "postgres";
av[ac++] = forkav;
@@ -5677,7 +5674,11 @@ bgworker_forkexec(int shmem_slot)
Assert(ac < lengthof(av));
- return postmaster_forkexec(ac, av);
+ MyBgworkerEntry = worker;
+ result = postmaster_forkexec(ac, av);
+ MyBgworkerEntry = NULL;
+
+ return result;
}
#endif
@@ -5718,7 +5719,7 @@ do_start_bgworker(RegisteredBgWorker *rw)
rw->rw_worker.bgw_name)));
#ifdef EXEC_BACKEND
- switch ((worker_pid = bgworker_forkexec(rw->rw_shmem_slot)))
+ switch ((worker_pid = bgworker_forkexec(&rw->rw_worker)))
#else
switch ((worker_pid = fork_process()))
#endif
@@ -6120,6 +6121,11 @@ save_backend_variables(BackendParameters *param, Port *port,
strlcpy(param->pkglib_path, pkglib_path, MAXPGPATH);
+ if (MyBgworkerEntry)
+ memcpy(¶m->MyBgworkerEntry, MyBgworkerEntry, sizeof(BackgroundWorker));
+ else
+ memset(¶m->MyBgworkerEntry, 0, sizeof(BackgroundWorker));
+
return true;
}
@@ -6350,6 +6356,13 @@ restore_backend_variables(BackendParameters *param, Port *port)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ if (param->MyBgworkerEntry.bgw_name[0] != '\0')
+ {
+ MyBgworkerEntry = (BackgroundWorker *)
+ MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
+ memcpy(MyBgworkerEntry, ¶m->MyBgworkerEntry, sizeof(BackgroundWorker));
+ }
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/postmaster/bgworker_internals.h b/src/include/postmaster/bgworker_internals.h
index 4ad63fd9bd7..7fa7ee2d878 100644
--- a/src/include/postmaster/bgworker_internals.h
+++ b/src/include/postmaster/bgworker_internals.h
@@ -57,8 +57,4 @@ extern void ResetBackgroundWorkerCrashTimes(void);
/* Function to start a background worker, called from postmaster.c */
extern void StartBackgroundWorker(void) pg_attribute_noreturn();
-#ifdef EXEC_BACKEND
-extern BackgroundWorker *BackgroundWorkerEntry(int slotno);
-#endif
-
#endif /* BGWORKER_INTERNALS_H */
--
2.30.2
0003-Refactor-CreateSharedMemoryAndSemaphores.patchtext/x-patch; charset=UTF-8; name=0003-Refactor-CreateSharedMemoryAndSemaphores.patchDownload
From 0cb6f8d665980d30a5d2a29013000744f16bf813 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Sun, 18 Jun 2023 11:00:21 +0300
Subject: [PATCH 3/9] Refactor CreateSharedMemoryAndSemaphores.
Moves InitProcess calls a little later in EXEC_BACKEND case.
For clarity, have separate functions for *creating* the shared memory
and semaphores, at postmaster or single-user backend startup, and for
*attaching* to existing shared memory structures in EXEC_BACKEND case.
I find it pretty confusing in all the *ShmemInit() functions too, that
they are called in two different contexts: in postmaster when creating
the shmem structs, and in the child process in EXEC_BACKEND when
attaching to the already existing structs. But this commit doesn't
change that.
---
src/backend/postmaster/autovacuum.c | 22 ++++++-------
src/backend/postmaster/auxprocess.c | 8 ++---
src/backend/postmaster/bgworker.c | 11 +++----
src/backend/postmaster/postmaster.c | 48 ++++++-----------------------
src/backend/storage/ipc/ipci.c | 11 +++++++
src/backend/storage/lmgr/proc.c | 2 +-
src/include/storage/ipc.h | 1 +
7 files changed, 41 insertions(+), 62 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index f929b62e8ad..7157c5466aa 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -476,14 +476,13 @@ AutoVacLauncherMain(int argc, char *argv[])
pqsignal(SIGCHLD, SIG_DFL);
/*
- * Create a per-backend PGPROC struct in shared memory, except in the
- * EXEC_BACKEND case where this was done in SubPostmasterMain. We must do
- * this before we can use LWLocks (and in the EXEC_BACKEND case we already
- * had to do some stuff with LWLocks).
+ * Create a per-backend PGPROC struct in shared memory. We must do
+ * this before we can use LWLocks.
*/
-#ifndef EXEC_BACKEND
InitProcess();
-#endif
+
+ /* Attach process to shared data structures */
+ AttachSharedMemoryAndSemaphores();
/* Early initialization */
BaseInit();
@@ -1548,14 +1547,13 @@ AutoVacWorkerMain(int argc, char *argv[])
pqsignal(SIGCHLD, SIG_DFL);
/*
- * Create a per-backend PGPROC struct in shared memory, except in the
- * EXEC_BACKEND case where this was done in SubPostmasterMain. We must do
- * this before we can use LWLocks (and in the EXEC_BACKEND case we already
- * had to do some stuff with LWLocks).
+ * Create a per-backend PGPROC struct in shared memory. We must do
+ * this before we can use LWLocks.
*/
-#ifndef EXEC_BACKEND
InitProcess();
-#endif
+
+ /* Attach process to shared data structures */
+ AttachSharedMemoryAndSemaphores();
/* Early initialization */
BaseInit();
diff --git a/src/backend/postmaster/auxprocess.c b/src/backend/postmaster/auxprocess.c
index cae6feb3562..536d9a2b3e4 100644
--- a/src/backend/postmaster/auxprocess.c
+++ b/src/backend/postmaster/auxprocess.c
@@ -97,12 +97,12 @@ AuxiliaryProcessMain(AuxProcType auxtype)
*/
/*
- * Create a PGPROC so we can use LWLocks. In the EXEC_BACKEND case, this
- * was already done by SubPostmasterMain().
+ * Create a PGPROC so we can use LWLocks.
*/
-#ifndef EXEC_BACKEND
InitAuxiliaryProcess();
-#endif
+
+ /* Attach process to shared data structures */
+ AttachSharedMemoryAndSemaphores();
BaseInit();
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index b7140306e43..13519ea0c45 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -810,14 +810,13 @@ StartBackgroundWorker(void)
PG_exception_stack = &local_sigjmp_buf;
/*
- * Create a per-backend PGPROC struct in shared memory, except in the
- * EXEC_BACKEND case where this was done in SubPostmasterMain. We must do
- * this before we can use LWLocks (and in the EXEC_BACKEND case we already
- * had to do some stuff with LWLocks).
+ * Create a per-backend PGPROC struct in shared memory. We must do
+ * this before we can use LWLocks.
*/
-#ifndef EXEC_BACKEND
InitProcess();
-#endif
+
+ /* Attach process to shared data structures */
+ AttachSharedMemoryAndSemaphores();
/*
* Early initialization.
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index a7a9e061c5b..bf9f0d27278 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -4176,15 +4176,6 @@ BackendStartup(Port *port)
/* Perform additional initialization and collect startup packet */
BackendInitialize(port);
- /*
- * Create a per-backend PGPROC struct in shared memory. We must do
- * this before we can use LWLocks. In the !EXEC_BACKEND case (here)
- * this could be delayed a bit further, but EXEC_BACKEND needs to do
- * stuff with LWLocks before PostgresMain(), so we do it here as well
- * for symmetry.
- */
- InitProcess();
-
/* And run the backend */
BackendRun(port);
}
@@ -4452,6 +4443,15 @@ BackendInitialize(Port *port)
static void
BackendRun(Port *port)
{
+ /*
+ * Create a per-backend PGPROC struct in shared memory. We must do
+ * this before we can use LWLocks (in AttachSharedMemoryAndSemaphores).
+ */
+ InitProcess();
+
+ /* Attach process to shared data structures */
+ AttachSharedMemoryAndSemaphores();
+
/*
* Make sure we aren't in PostmasterContext anymore. (We can't delete it
* just yet, though, because InitPostgres will need the HBA data.)
@@ -4947,12 +4947,6 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
/* And run the backend */
BackendRun(&port); /* does not return */
}
@@ -4965,12 +4959,6 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitAuxiliaryProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
auxtype = atoi(argv[3]);
AuxiliaryProcessMain(auxtype); /* does not return */
}
@@ -4979,12 +4967,6 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
AutoVacLauncherMain(argc - 2, argv + 2); /* does not return */
}
if (strcmp(argv[1], "--forkavworker") == 0)
@@ -4992,12 +4974,6 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
}
if (strncmp(argv[1], "--forkbgworker", 14) == 0)
@@ -5008,12 +4984,6 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
StartBackgroundWorker();
}
if (strcmp(argv[1], "--forklog") == 0)
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 8f1ded7338f..be6a774e351 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -170,6 +170,17 @@ CalculateShmemSize(int *num_semaphores)
* check IsUnderPostmaster, rather than EXEC_BACKEND, to detect this case.
* This is a bit code-wasteful and could be cleaned up.)
*/
+void
+AttachSharedMemoryAndSemaphores(void)
+{
+ /* InitProcess must've been called already */
+
+ /* Init !EXEC_BACKEND mode, we inherited everything through the fork */
+#ifdef EXEC_BACKEND
+ CreateSharedMemoryAndSemaphores();
+#endif
+}
+
void
CreateSharedMemoryAndSemaphores(void)
{
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index dac921219fa..91f415aa783 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -468,7 +468,7 @@ InitProcess(void)
*
* This is separate from InitProcess because we can't acquire LWLocks until
* we've created a PGPROC, but in the EXEC_BACKEND case ProcArrayAdd won't
- * work until after we've done CreateSharedMemoryAndSemaphores.
+ * work until after we've done AttachSharedMemoryAndSemaphores.
*/
void
InitProcessPhase2(void)
diff --git a/src/include/storage/ipc.h b/src/include/storage/ipc.h
index 888c08b3067..e75656f5242 100644
--- a/src/include/storage/ipc.h
+++ b/src/include/storage/ipc.h
@@ -79,6 +79,7 @@ extern PGDLLIMPORT shmem_startup_hook_type shmem_startup_hook;
extern Size CalculateShmemSize(int *num_semaphores);
extern void CreateSharedMemoryAndSemaphores(void);
+extern void AttachSharedMemoryAndSemaphores(void);
extern void InitializeShmemGUCs(void);
#endif /* IPC_H */
--
2.30.2
0004-Use-FD_CLOEXEC-on-ListenSockets.patchtext/x-patch; charset=UTF-8; name=0004-Use-FD_CLOEXEC-on-ListenSockets.patchDownload
From 1d89eec53c7fefa7a4a8c011c9f19e3df64dc436 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Mon, 12 Jun 2023 16:33:20 +0300
Subject: [PATCH 4/9] Use FD_CLOEXEC on ListenSockets
We went through some effort to close them in the child process. Better to
not hand them down to the child process in the first place.
---
src/backend/libpq/pqcomm.c | 6 +++++-
src/backend/postmaster/postmaster.c | 14 ++++++--------
2 files changed, 11 insertions(+), 9 deletions(-)
diff --git a/src/backend/libpq/pqcomm.c b/src/backend/libpq/pqcomm.c
index da5bb5fc5d3..9061daccc29 100644
--- a/src/backend/libpq/pqcomm.c
+++ b/src/backend/libpq/pqcomm.c
@@ -458,6 +458,9 @@ StreamServerPort(int family, const char *hostName, unsigned short portNumber,
}
#ifndef WIN32
+ /* Don't give the listen socket to any subprograms we execute. */
+ if (fcntl(fd, F_SETFD, FD_CLOEXEC) < 0)
+ elog(FATAL, "fcntl(F_SETFD) failed on socket: %m");
/*
* Without the SO_REUSEADDR flag, a new postmaster can't be started
@@ -831,7 +834,8 @@ StreamConnection(pgsocket server_fd, Port *port)
void
StreamClose(pgsocket sock)
{
- closesocket(sock);
+ if (closesocket(sock) != 0)
+ elog(LOG, "closesocket failed: %m");
}
/*
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index bf9f0d27278..228744ebee5 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -497,7 +497,6 @@ typedef struct
Port port;
InheritableSocket portsocket;
char DataDir[MAXPGPATH];
- pgsocket ListenSocket[MAXLISTEN];
int32 MyCancelKey;
int MyPMChildSlot;
#ifndef WIN32
@@ -2575,8 +2574,6 @@ ConnFree(Port *port)
void
ClosePostmasterPorts(bool am_syslogger)
{
- int i;
-
/* Release resources held by the postmaster's WaitEventSet. */
if (pm_wait_set)
{
@@ -2603,8 +2600,12 @@ ClosePostmasterPorts(bool am_syslogger)
/*
* Close the postmaster's listen sockets. These aren't tracked by fd.c,
* so we don't call ReleaseExternalFD() here.
+ *
+ * The listen sockets are marked as FD_CLOEXEC, so this isn't needed in
+ * EXEC_BACKEND mode.
*/
- for (i = 0; i < MAXLISTEN; i++)
+#ifndef EXEC_BACKEND
+ for (int i = 0; i < MAXLISTEN; i++)
{
if (ListenSocket[i] != PGINVALID_SOCKET)
{
@@ -2612,6 +2613,7 @@ ClosePostmasterPorts(bool am_syslogger)
ListenSocket[i] = PGINVALID_SOCKET;
}
}
+#endif
/*
* If using syslogger, close the read side of the pipe. We don't bother
@@ -6035,8 +6037,6 @@ save_backend_variables(BackendParameters *param, Port *port,
strlcpy(param->DataDir, DataDir, MAXPGPATH);
- memcpy(¶m->ListenSocket, &ListenSocket, sizeof(ListenSocket));
-
param->MyCancelKey = MyCancelKey;
param->MyPMChildSlot = MyPMChildSlot;
@@ -6273,8 +6273,6 @@ restore_backend_variables(BackendParameters *param, Port *port)
SetDataDir(param->DataDir);
- memcpy(&ListenSocket, ¶m->ListenSocket, sizeof(ListenSocket));
-
MyCancelKey = param->MyCancelKey;
MyPMChildSlot = param->MyPMChildSlot;
--
2.30.2
0005-Move-too-many-clients-already-et-al.-checks-from-Pro.patchtext/x-patch; charset=UTF-8; name=0005-Move-too-many-clients-already-et-al.-checks-from-Pro.patchDownload
From 2f518be9e96cfed1a1a49b4af8f7cb4a837aa784 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Mon, 12 Jun 2023 18:07:54 +0300
Subject: [PATCH 5/9] Move "too many clients already" et al. checks from
ProcessStartupPacket.
The check is not about processing the startup packet, so the calling
function seems like a more natural place.
---
src/backend/postmaster/postmaster.c | 86 ++++++++++++++---------------
1 file changed, 43 insertions(+), 43 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 228744ebee5..fa6f2c8e29c 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -2321,49 +2321,6 @@ retry1:
*/
MemoryContextSwitchTo(oldcontext);
- /*
- * If we're going to reject the connection due to database state, say so
- * now instead of wasting cycles on an authentication exchange. (This also
- * allows a pg_ping utility to be written.)
- */
- switch (port->canAcceptConnections)
- {
- case CAC_STARTUP:
- ereport(FATAL,
- (errcode(ERRCODE_CANNOT_CONNECT_NOW),
- errmsg("the database system is starting up")));
- break;
- case CAC_NOTCONSISTENT:
- if (EnableHotStandby)
- ereport(FATAL,
- (errcode(ERRCODE_CANNOT_CONNECT_NOW),
- errmsg("the database system is not yet accepting connections"),
- errdetail("Consistent recovery state has not been yet reached.")));
- else
- ereport(FATAL,
- (errcode(ERRCODE_CANNOT_CONNECT_NOW),
- errmsg("the database system is not accepting connections"),
- errdetail("Hot standby mode is disabled.")));
- break;
- case CAC_SHUTDOWN:
- ereport(FATAL,
- (errcode(ERRCODE_CANNOT_CONNECT_NOW),
- errmsg("the database system is shutting down")));
- break;
- case CAC_RECOVERY:
- ereport(FATAL,
- (errcode(ERRCODE_CANNOT_CONNECT_NOW),
- errmsg("the database system is in recovery mode")));
- break;
- case CAC_TOOMANY:
- ereport(FATAL,
- (errcode(ERRCODE_TOO_MANY_CONNECTIONS),
- errmsg("sorry, too many clients already")));
- break;
- case CAC_OK:
- break;
- }
-
return STATUS_OK;
}
@@ -4391,6 +4348,49 @@ BackendInitialize(Port *port)
*/
status = ProcessStartupPacket(port, false, false);
+ /*
+ * If we're going to reject the connection due to database state, say so
+ * now instead of wasting cycles on an authentication exchange. (This also
+ * allows a pg_ping utility to be written.)
+ */
+ switch (port->canAcceptConnections)
+ {
+ case CAC_STARTUP:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is starting up")));
+ break;
+ case CAC_NOTCONSISTENT:
+ if (EnableHotStandby)
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is not yet accepting connections"),
+ errdetail("Consistent recovery state has not been yet reached.")));
+ else
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is not accepting connections"),
+ errdetail("Hot standby mode is disabled.")));
+ break;
+ case CAC_SHUTDOWN:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is shutting down")));
+ break;
+ case CAC_RECOVERY:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is in recovery mode")));
+ break;
+ case CAC_TOOMANY:
+ ereport(FATAL,
+ (errcode(ERRCODE_TOO_MANY_CONNECTIONS),
+ errmsg("sorry, too many clients already")));
+ break;
+ case CAC_OK:
+ break;
+ }
+
/*
* Disable the timeout, and prevent SIGTERM again.
*/
--
2.30.2
0006-Pass-CAC-as-argument-to-backend-process.patchtext/x-patch; charset=UTF-8; name=0006-Pass-CAC-as-argument-to-backend-process.patchDownload
From c25b67c045018a2bf05e6ff53819d26e561fc83f Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Sun, 18 Jun 2023 14:11:16 +0300
Subject: [PATCH 6/9] Pass CAC as argument to backend process.
We used to smuggle it to the child process in the Port struct, but it seems
better to pass it down as a separate argument.
---
src/backend/postmaster/postmaster.c | 43 +++++++++++++++++++++--------
src/include/libpq/libpq-be.h | 12 --------
2 files changed, 31 insertions(+), 24 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index fa6f2c8e29c..3ce9d76a850 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -417,7 +417,18 @@ static void HandleChildCrash(int pid, int exitstatus, const char *procname);
static void LogChildExit(int lev, const char *procname,
int pid, int exitstatus);
static void PostmasterStateMachine(void);
-static void BackendInitialize(Port *port);
+
+typedef enum CAC_state
+{
+ CAC_OK,
+ CAC_STARTUP,
+ CAC_SHUTDOWN,
+ CAC_RECOVERY,
+ CAC_NOTCONSISTENT,
+ CAC_TOOMANY
+} CAC_state;
+
+static void BackendInitialize(Port *port, CAC_state cac);
static void BackendRun(Port *port) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
@@ -474,7 +485,7 @@ typedef struct
} win32_deadchild_waitinfo;
#endif /* WIN32 */
-static pid_t backend_forkexec(Port *port);
+static pid_t backend_forkexec(Port *port, CAC_state cac);
static pid_t internal_forkexec(int argc, char *argv[], Port *port);
/* Type for a socket that can be inherited to a client process */
@@ -4075,6 +4086,7 @@ BackendStartup(Port *port)
{
Backend *bn; /* for backend cleanup */
pid_t pid;
+ CAC_state cac;
/*
* Create backend data structure. Better before the fork() so we can
@@ -4106,8 +4118,8 @@ BackendStartup(Port *port)
bn->cancel_key = MyCancelKey;
/* Pass down canAcceptConnections state */
- port->canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
- bn->dead_end = (port->canAcceptConnections != CAC_OK);
+ cac = canAcceptConnections(BACKEND_TYPE_NORMAL);
+ bn->dead_end = (cac != CAC_OK);
/*
* Unless it's a dead_end child, assign it a child slot number
@@ -4121,7 +4133,7 @@ BackendStartup(Port *port)
bn->bgworker_notify = false;
#ifdef EXEC_BACKEND
- pid = backend_forkexec(port);
+ pid = backend_forkexec(port, cac);
#else /* !EXEC_BACKEND */
pid = fork_process();
if (pid == 0) /* child */
@@ -4133,7 +4145,7 @@ BackendStartup(Port *port)
ClosePostmasterPorts(false);
/* Perform additional initialization and collect startup packet */
- BackendInitialize(port);
+ BackendInitialize(port, cac);
/* And run the backend */
BackendRun(port);
@@ -4220,7 +4232,7 @@ report_fork_failure_to_client(Port *port, int errnum)
* but have not yet set up most of our local pointers to shmem structures.
*/
static void
-BackendInitialize(Port *port)
+BackendInitialize(Port *port, CAC_state cac)
{
int status;
int ret;
@@ -4353,7 +4365,7 @@ BackendInitialize(Port *port)
* now instead of wasting cycles on an authentication exchange. (This also
* allows a pg_ping utility to be written.)
*/
- switch (port->canAcceptConnections)
+ switch (cac)
{
case CAC_STARTUP:
ereport(FATAL,
@@ -4498,15 +4510,19 @@ postmaster_forkexec(int argc, char *argv[])
* returns the pid of the fork/exec'd process, or -1 on failure
*/
static pid_t
-backend_forkexec(Port *port)
+backend_forkexec(Port *port, CAC_state cac)
{
- char *av[4];
+ char *av[5];
int ac = 0;
+ char cacbuf[10];
av[ac++] = "postgres";
av[ac++] = "--forkbackend";
av[ac++] = NULL; /* filled in by internal_forkexec */
+ snprintf(cacbuf, sizeof(cacbuf), "%d", (int) cac);
+ av[ac++] = cacbuf;
+
av[ac] = NULL;
Assert(ac < lengthof(av));
@@ -4910,7 +4926,10 @@ SubPostmasterMain(int argc, char *argv[])
/* Run backend or appropriate child */
if (strcmp(argv[1], "--forkbackend") == 0)
{
- Assert(argc == 3); /* shouldn't be any more args */
+ CAC_state cac;
+
+ Assert(argc == 4);
+ cac = (CAC_state) atoi(argv[3]);
/*
* Need to reinitialize the SSL library in the backend, since the
@@ -4944,7 +4963,7 @@ SubPostmasterMain(int argc, char *argv[])
* PGPROC slots, we have already initialized libpq and are able to
* report the error to the client.
*/
- BackendInitialize(&port);
+ BackendInitialize(&port, cac);
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 3b2ce9908f8..625caa853a0 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -58,17 +58,6 @@ typedef struct
#include "libpq/pqcomm.h"
-typedef enum CAC_state
-{
- CAC_OK,
- CAC_STARTUP,
- CAC_SHUTDOWN,
- CAC_RECOVERY,
- CAC_NOTCONSISTENT,
- CAC_TOOMANY
-} CAC_state;
-
-
/*
* GSSAPI specific state information
*/
@@ -156,7 +145,6 @@ typedef struct Port
int remote_hostname_resolv; /* see above */
int remote_hostname_errcode; /* see above */
char *remote_port; /* text rep of remote port */
- CAC_state canAcceptConnections; /* postmaster connection status */
/*
* Information that needs to be saved from the startup packet and passed
--
2.30.2
0007-Remove-ConnCreate-and-ConnFree-and-allocate-Port-in-.patchtext/x-patch; charset=UTF-8; name=0007-Remove-ConnCreate-and-ConnFree-and-allocate-Port-in-.patchDownload
From 658cba5cdb2e5c45faff84566906d2fcaa8a3674 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Mon, 12 Jun 2023 18:03:03 +0300
Subject: [PATCH 7/9] Remove ConnCreate and ConnFree, and allocate Port in
stack.
By allocating Port in stack, we don't need to deal with out-of-memory
errors, which makes the code a little less verbose.
---
src/backend/postmaster/postmaster.c | 68 +++++------------------------
1 file changed, 10 insertions(+), 58 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 3ce9d76a850..8731b50e4a2 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -398,8 +398,6 @@ static void CloseServerPorts(int status, Datum arg);
static void unlink_external_pid_file(int status, Datum arg);
static void getInstallationPaths(const char *argv0);
static void checkControlFile(void);
-static Port *ConnCreate(int serverFd);
-static void ConnFree(Port *port);
static void handle_pm_pmsignal_signal(SIGNAL_ARGS);
static void handle_pm_child_exit_signal(SIGNAL_ARGS);
static void handle_pm_reload_request_signal(SIGNAL_ARGS);
@@ -1783,20 +1781,18 @@ ServerLoop(void)
if (events[i].events & WL_SOCKET_ACCEPT)
{
- Port *port;
+ Port port;
- port = ConnCreate(events[i].fd);
- if (port)
- {
- BackendStartup(port);
+ memset(&port, 0, sizeof(port));
+ if (StreamConnection(events[i].fd, &port) == STATUS_OK)
+ BackendStartup(&port);
- /*
- * We no longer need the open socket or port structure in
- * this process
- */
- StreamClose(port->sock);
- ConnFree(port);
- }
+ /*
+ * We no longer need the open socket or port structure in this
+ * process
+ */
+ if (port.sock != PGINVALID_SOCKET)
+ StreamClose(port.sock);
}
}
@@ -2485,50 +2481,6 @@ canAcceptConnections(int backend_type)
return result;
}
-
-/*
- * ConnCreate -- create a local connection data structure
- *
- * Returns NULL on failure, other than out-of-memory which is fatal.
- */
-static Port *
-ConnCreate(int serverFd)
-{
- Port *port;
-
- if (!(port = (Port *) calloc(1, sizeof(Port))))
- {
- ereport(LOG,
- (errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("out of memory")));
- ExitPostmaster(1);
- }
-
- if (StreamConnection(serverFd, port) != STATUS_OK)
- {
- if (port->sock != PGINVALID_SOCKET)
- StreamClose(port->sock);
- ConnFree(port);
- return NULL;
- }
-
- return port;
-}
-
-
-/*
- * ConnFree -- free a local connection data structure
- *
- * Caller has already closed the socket if any, so there's not much
- * to do here.
- */
-static void
-ConnFree(Port *port)
-{
- free(port);
-}
-
-
/*
* ClosePostmasterPorts -- close all the postmaster's open sockets
*
--
2.30.2
0008-Introduce-ClientSocket-rename-some-funcs.patchtext/x-patch; charset=UTF-8; name=0008-Introduce-ClientSocket-rename-some-funcs.patchDownload
From 65384b9a6cfb3b9b589041526216e0f64d64bea5 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Sun, 18 Jun 2023 13:56:44 +0300
Subject: [PATCH 8/9] Introduce ClientSocket, rename some funcs
- Move more of the work on a client socket to the child process.
- Reduce the amount of data that needs to be passed from postmaster to
child. (Used to pass a full Port struct, although most of the fields were
empty. Now we pass the much slimmer ClientSocket.)
---
src/backend/libpq/pqcomm.c | 93 ++++++++++---------
src/backend/postmaster/autovacuum.c | 8 +-
src/backend/postmaster/bgworker.c | 4 +-
src/backend/postmaster/postmaster.c | 139 +++++++++++++++-------------
src/include/libpq/libpq-be.h | 19 ++--
src/include/libpq/libpq.h | 6 +-
6 files changed, 147 insertions(+), 122 deletions(-)
diff --git a/src/backend/libpq/pqcomm.c b/src/backend/libpq/pqcomm.c
index 9061daccc29..e2789cb4a4f 100644
--- a/src/backend/libpq/pqcomm.c
+++ b/src/backend/libpq/pqcomm.c
@@ -29,9 +29,9 @@
* INTERFACE ROUTINES
*
* setup/teardown:
- * StreamServerPort - Open postmaster's server port
- * StreamConnection - Create new connection with client
- * StreamClose - Close a client/backend connection
+ * ListenServerPort - Open postmaster's server port
+ * AcceptClientConnection - Create new connection with client
+ * StreamConnection - Initialize a client connection
* TouchSocketFiles - Protect socket files against /tmp cleaners
* pq_init - initialize libpq at backend startup
* socket_comm_reset - reset libpq during error recovery
@@ -304,7 +304,7 @@ socket_close(int code, Datum arg)
/*
- * StreamServerPort -- open a "listening" port to accept connections.
+ * ListenServerPort -- open a "listening" port to accept connections.
*
* family should be AF_UNIX or AF_UNSPEC; portNumber is the port number.
* For AF_UNIX ports, hostName should be NULL and unixSocketDir must be
@@ -318,7 +318,7 @@ socket_close(int code, Datum arg)
*/
int
-StreamServerPort(int family, const char *hostName, unsigned short portNumber,
+ListenServerPort(int family, const char *hostName, unsigned short portNumber,
const char *unixSocketDir,
pgsocket ListenSocket[], int MaxListen)
{
@@ -689,8 +689,9 @@ Setup_AF_UNIX(const char *sock_path)
/*
- * StreamConnection -- create a new connection with client using
- * server port. Set port->sock to the FD of the new connection.
+ * AcceptClientConnection -- accept a new connection with client using
+ * server port. Fills *client_sock with the FD and endpoint info
+ * of the new connection.
*
* ASSUME: that this doesn't need to be non-blocking because
* the Postmaster waits for the socket to be ready to accept().
@@ -698,13 +699,13 @@ Setup_AF_UNIX(const char *sock_path)
* RETURNS: STATUS_OK or STATUS_ERROR
*/
int
-StreamConnection(pgsocket server_fd, Port *port)
+AcceptClientConnection(pgsocket server_fd, ClientSocket *client_sock)
{
/* accept connection and fill in the client (remote) address */
- port->raddr.salen = sizeof(port->raddr.addr);
- if ((port->sock = accept(server_fd,
- (struct sockaddr *) &port->raddr.addr,
- &port->raddr.salen)) == PGINVALID_SOCKET)
+ client_sock->raddr.salen = sizeof(client_sock->raddr.addr);
+ if ((client_sock->sock = accept(server_fd,
+ (struct sockaddr *) &client_sock->raddr.addr,
+ &client_sock->raddr.salen)) == PGINVALID_SOCKET)
{
ereport(LOG,
(errcode_for_socket_access(),
@@ -722,10 +723,10 @@ StreamConnection(pgsocket server_fd, Port *port)
}
/* fill in the server (local) address */
- port->laddr.salen = sizeof(port->laddr.addr);
- if (getsockname(port->sock,
- (struct sockaddr *) &port->laddr.addr,
- &port->laddr.salen) < 0)
+ client_sock->laddr.salen = sizeof(client_sock->laddr.addr);
+ if (getsockname(client_sock->sock,
+ (struct sockaddr *) &client_sock->laddr.addr,
+ &client_sock->laddr.salen) < 0)
{
ereport(LOG,
(errmsg("%s() failed: %m", "getsockname")));
@@ -733,7 +734,7 @@ StreamConnection(pgsocket server_fd, Port *port)
}
/* select NODELAY and KEEPALIVE options if it's a TCP connection */
- if (port->laddr.addr.ss_family != AF_UNIX)
+ if (client_sock->laddr.addr.ss_family != AF_UNIX)
{
int on;
#ifdef WIN32
@@ -744,7 +745,7 @@ StreamConnection(pgsocket server_fd, Port *port)
#ifdef TCP_NODELAY
on = 1;
- if (setsockopt(port->sock, IPPROTO_TCP, TCP_NODELAY,
+ if (setsockopt(client_sock->sock, IPPROTO_TCP, TCP_NODELAY,
(char *) &on, sizeof(on)) < 0)
{
ereport(LOG,
@@ -753,7 +754,7 @@ StreamConnection(pgsocket server_fd, Port *port)
}
#endif
on = 1;
- if (setsockopt(port->sock, SOL_SOCKET, SO_KEEPALIVE,
+ if (setsockopt(client_sock->sock, SOL_SOCKET, SO_KEEPALIVE,
(char *) &on, sizeof(on)) < 0)
{
ereport(LOG,
@@ -785,7 +786,7 @@ StreamConnection(pgsocket server_fd, Port *port)
* https://msdn.microsoft.com/en-us/library/bb736549%28v=vs.85%29.aspx
*/
optlen = sizeof(oldopt);
- if (getsockopt(port->sock, SOL_SOCKET, SO_SNDBUF, (char *) &oldopt,
+ if (getsockopt(client_sock->sock, SOL_SOCKET, SO_SNDBUF, (char *) &oldopt,
&optlen) < 0)
{
ereport(LOG,
@@ -795,7 +796,7 @@ StreamConnection(pgsocket server_fd, Port *port)
newopt = PQ_SEND_BUFFER_SIZE * 4;
if (oldopt < newopt)
{
- if (setsockopt(port->sock, SOL_SOCKET, SO_SNDBUF, (char *) &newopt,
+ if (setsockopt(client_sock->sock, SOL_SOCKET, SO_SNDBUF, (char *) &newopt,
sizeof(newopt)) < 0)
{
ereport(LOG,
@@ -804,13 +805,34 @@ StreamConnection(pgsocket server_fd, Port *port)
}
}
#endif
+ }
+ return STATUS_OK;
+}
+
+/*
+ * StreamConnection -- create a new connection from the given socket.
+ *
+ * This runs in the backend process.
+ */
+Port *
+StreamConnection(ClientSocket *client_sock)
+{
+ Port *port;
+
+ port = palloc0(sizeof(Port));
+ port->sock = client_sock->sock;
+ port->laddr = client_sock->laddr;
+ port->raddr = client_sock->raddr;
+ /* Apply the current keepalive parameters if it's a TCP connection */
+ if (port->laddr.addr.ss_family != AF_UNIX)
+ {
/*
- * Also apply the current keepalive parameters. If we fail to set a
- * parameter, don't error out, because these aren't universally
- * supported. (Note: you might think we need to reset the GUC
- * variables to 0 in such a case, but it's not necessary because the
- * show hooks for these variables report the truth anyway.)
+ * If we fail to set a parameter, don't error out, because these
+ * aren't universally supported. (Note: you might think we need to
+ * reset the GUC variables to 0 in such a case, but it's not necessary
+ * because the show hooks for these variables report the truth
+ * anyway.)
*/
(void) pq_setkeepalivesidle(tcp_keepalives_idle, port);
(void) pq_setkeepalivesinterval(tcp_keepalives_interval, port);
@@ -818,24 +840,7 @@ StreamConnection(pgsocket server_fd, Port *port)
(void) pq_settcpusertimeout(tcp_user_timeout, port);
}
- return STATUS_OK;
-}
-
-/*
- * StreamClose -- close a client/backend connection
- *
- * NOTE: this is NOT used to terminate a session; it is just used to release
- * the file descriptor in a process that should no longer have the socket
- * open. (For example, the postmaster calls this after passing ownership
- * of the connection to a child process.) It is expected that someone else
- * still has the socket open. So, we only want to close the descriptor,
- * we do NOT want to send anything to the far end.
- */
-void
-StreamClose(pgsocket sock)
-{
- if (closesocket(sock) != 0)
- elog(LOG, "closesocket failed: %m");
+ return port;
}
/*
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 7157c5466aa..1041376951e 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -476,8 +476,8 @@ AutoVacLauncherMain(int argc, char *argv[])
pqsignal(SIGCHLD, SIG_DFL);
/*
- * Create a per-backend PGPROC struct in shared memory. We must do
- * this before we can use LWLocks.
+ * Create a per-backend PGPROC struct in shared memory. We must do this
+ * before we can use LWLocks.
*/
InitProcess();
@@ -1547,8 +1547,8 @@ AutoVacWorkerMain(int argc, char *argv[])
pqsignal(SIGCHLD, SIG_DFL);
/*
- * Create a per-backend PGPROC struct in shared memory. We must do
- * this before we can use LWLocks.
+ * Create a per-backend PGPROC struct in shared memory. We must do this
+ * before we can use LWLocks.
*/
InitProcess();
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index 13519ea0c45..4c446ffd087 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -810,8 +810,8 @@ StartBackgroundWorker(void)
PG_exception_stack = &local_sigjmp_buf;
/*
- * Create a per-backend PGPROC struct in shared memory. We must do
- * this before we can use LWLocks.
+ * Create a per-backend PGPROC struct in shared memory. We must do this
+ * before we can use LWLocks.
*/
InitProcess();
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 8731b50e4a2..b73ba983f3e 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -426,15 +426,15 @@ typedef enum CAC_state
CAC_TOOMANY
} CAC_state;
-static void BackendInitialize(Port *port, CAC_state cac);
-static void BackendRun(Port *port) pg_attribute_noreturn();
+static void BackendInitialize(ClientSocket *client_sock, CAC_state cac);
+static void BackendRun(void) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
-static int BackendStartup(Port *port);
+static int BackendStartup(ClientSocket *port);
static int ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done);
static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
static void processCancelRequest(Port *port, void *pkt);
-static void report_fork_failure_to_client(Port *port, int errnum);
+static void report_fork_failure_to_client(ClientSocket *client_sock, int errnum);
static CAC_state canAcceptConnections(int backend_type);
static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
@@ -483,8 +483,8 @@ typedef struct
} win32_deadchild_waitinfo;
#endif /* WIN32 */
-static pid_t backend_forkexec(Port *port, CAC_state cac);
-static pid_t internal_forkexec(int argc, char *argv[], Port *port);
+static pid_t backend_forkexec(ClientSocket *client_sock, CAC_state cac);
+static pid_t internal_forkexec(int argc, char *argv[], ClientSocket *client_sock);
/* Type for a socket that can be inherited to a client process */
#ifdef WIN32
@@ -503,8 +503,8 @@ typedef int InheritableSocket;
*/
typedef struct
{
- Port port;
- InheritableSocket portsocket;
+ ClientSocket client_sock;
+ InheritableSocket serialized_sock;
char DataDir[MAXPGPATH];
int32 MyCancelKey;
int MyPMChildSlot;
@@ -552,13 +552,13 @@ typedef struct
BackgroundWorker MyBgworkerEntry;
} BackendParameters;
-static void read_backend_variables(char *id, Port *port);
-static void restore_backend_variables(BackendParameters *param, Port *port);
+static void read_backend_variables(char *id, ClientSocket *client_sock);
+static void restore_backend_variables(BackendParameters *param, ClientSocket *client_sock);
#ifndef WIN32
-static bool save_backend_variables(BackendParameters *param, Port *port);
+static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock);
#else
-static bool save_backend_variables(BackendParameters *param, Port *port,
+static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock,
HANDLE childProcess, pid_t childPid);
#endif
@@ -1221,12 +1221,12 @@ PostmasterMain(int argc, char *argv[])
char *curhost = (char *) lfirst(l);
if (strcmp(curhost, "*") == 0)
- status = StreamServerPort(AF_UNSPEC, NULL,
+ status = ListenServerPort(AF_UNSPEC, NULL,
(unsigned short) PostPortNumber,
NULL,
ListenSocket, MAXLISTEN);
else
- status = StreamServerPort(AF_UNSPEC, curhost,
+ status = ListenServerPort(AF_UNSPEC, curhost,
(unsigned short) PostPortNumber,
NULL,
ListenSocket, MAXLISTEN);
@@ -1318,7 +1318,7 @@ PostmasterMain(int argc, char *argv[])
{
char *socketdir = (char *) lfirst(l);
- status = StreamServerPort(AF_UNIX, NULL,
+ status = ListenServerPort(AF_UNIX, NULL,
(unsigned short) PostPortNumber,
socketdir,
ListenSocket, MAXLISTEN);
@@ -1499,7 +1499,7 @@ CloseServerPorts(int status, Datum arg)
{
if (ListenSocket[i] != PGINVALID_SOCKET)
{
- StreamClose(ListenSocket[i]);
+ closesocket(ListenSocket[i]);
ListenSocket[i] = PGINVALID_SOCKET;
}
}
@@ -1781,18 +1781,20 @@ ServerLoop(void)
if (events[i].events & WL_SOCKET_ACCEPT)
{
- Port port;
+ ClientSocket s;
- memset(&port, 0, sizeof(port));
- if (StreamConnection(events[i].fd, &port) == STATUS_OK)
- BackendStartup(&port);
+ if (AcceptClientConnection(events[i].fd, &s) == STATUS_OK)
+ BackendStartup(&s);
/*
* We no longer need the open socket or port structure in this
* process
*/
- if (port.sock != PGINVALID_SOCKET)
- StreamClose(port.sock);
+ if (s.sock != PGINVALID_SOCKET)
+ {
+ if (closesocket(s.sock) != 0)
+ elog(LOG, "could not close client socket: %m");
+ }
}
}
@@ -2529,7 +2531,8 @@ ClosePostmasterPorts(bool am_syslogger)
{
if (ListenSocket[i] != PGINVALID_SOCKET)
{
- StreamClose(ListenSocket[i]);
+ if (closesocket(ListenSocket[i]) != 0)
+ elog(LOG, "could not close listen socket: %m");
ListenSocket[i] = PGINVALID_SOCKET;
}
}
@@ -4034,7 +4037,7 @@ TerminateChildren(int signal)
* Note: if you change this code, also consider StartAutovacuumWorker.
*/
static int
-BackendStartup(Port *port)
+BackendStartup(ClientSocket *client_sock)
{
Backend *bn; /* for backend cleanup */
pid_t pid;
@@ -4085,7 +4088,7 @@ BackendStartup(Port *port)
bn->bgworker_notify = false;
#ifdef EXEC_BACKEND
- pid = backend_forkexec(port, cac);
+ pid = backend_forkexec(client_sock, cac);
#else /* !EXEC_BACKEND */
pid = fork_process();
if (pid == 0) /* child */
@@ -4097,10 +4100,10 @@ BackendStartup(Port *port)
ClosePostmasterPorts(false);
/* Perform additional initialization and collect startup packet */
- BackendInitialize(port, cac);
+ BackendInitialize(client_sock, cac);
/* And run the backend */
- BackendRun(port);
+ BackendRun();
}
#endif /* EXEC_BACKEND */
@@ -4115,14 +4118,14 @@ BackendStartup(Port *port)
errno = save_errno;
ereport(LOG,
(errmsg("could not fork new process for connection: %m")));
- report_fork_failure_to_client(port, save_errno);
+ report_fork_failure_to_client(client_sock, save_errno);
return STATUS_ERROR;
}
/* in parent, successful fork */
ereport(DEBUG2,
(errmsg_internal("forked new backend, pid=%d socket=%d",
- (int) pid, (int) port->sock)));
+ (int) pid, (int) client_sock->sock)));
/*
* Everything's been successful, it's safe to add this backend to our list
@@ -4149,7 +4152,7 @@ BackendStartup(Port *port)
* it's not up and running.
*/
static void
-report_fork_failure_to_client(Port *port, int errnum)
+report_fork_failure_to_client(ClientSocket *client_sock, int errnum)
{
char buffer[1000];
int rc;
@@ -4160,13 +4163,13 @@ report_fork_failure_to_client(Port *port, int errnum)
strerror(errnum));
/* Set port to non-blocking. Don't do send() if this fails */
- if (!pg_set_noblock(port->sock))
+ if (!pg_set_noblock(client_sock->sock))
return;
/* We'll retry after EINTR, but ignore all other failures */
do
{
- rc = send(port->sock, buffer, strlen(buffer) + 1, 0);
+ rc = send(client_sock->sock, buffer, strlen(buffer) + 1, 0);
} while (rc < 0 && errno == EINTR);
}
@@ -4184,16 +4187,24 @@ report_fork_failure_to_client(Port *port, int errnum)
* but have not yet set up most of our local pointers to shmem structures.
*/
static void
-BackendInitialize(Port *port, CAC_state cac)
+BackendInitialize(ClientSocket *client_sock, CAC_state cac)
{
int status;
int ret;
+ Port *port;
char remote_host[NI_MAXHOST];
char remote_port[NI_MAXSERV];
StringInfoData ps_data;
+ MemoryContext oldcontext;
- /* Save port etc. for ps status */
+ /*
+ * Create Port structure in TopMemoryContext, so that it survives into
+ * PostgresMain execution.
+ */
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
+ port = StreamConnection(client_sock);
MyProcPort = port;
+ MemoryContextSwitchTo(oldcontext);
/* Tell fd.c about the long-lived FD associated with the port */
ReserveExternalFD();
@@ -4254,8 +4265,9 @@ BackendInitialize(Port *port, CAC_state cac)
* Save remote_host and remote_port in port structure (after this, they
* will appear in log_line_prefix data for log messages).
*/
- port->remote_host = strdup(remote_host);
- port->remote_port = strdup(remote_port);
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
+ port->remote_host = pstrdup(remote_host);
+ port->remote_port = pstrdup(remote_port);
/* And now we can issue the Log_connections message, if wanted */
if (Log_connections)
@@ -4286,7 +4298,8 @@ BackendInitialize(Port *port, CAC_state cac)
ret == 0 &&
strspn(remote_host, "0123456789.") < strlen(remote_host) &&
strspn(remote_host, "0123456789ABCDEFabcdef:") < strlen(remote_host))
- port->remote_hostname = strdup(remote_host);
+ port->remote_hostname = pstrdup(remote_host);
+ MemoryContextSwitchTo(oldcontext);
/*
* Ready to begin client interaction. We will give up and _exit(1) after
@@ -4407,11 +4420,11 @@ BackendInitialize(Port *port, CAC_state cac)
* Doesn't return at all.
*/
static void
-BackendRun(Port *port)
+BackendRun(void)
{
/*
- * Create a per-backend PGPROC struct in shared memory. We must do
- * this before we can use LWLocks (in AttachSharedMemoryAndSemaphores).
+ * Create a per-backend PGPROC struct in shared memory. We must do this
+ * before we can use LWLocks (in AttachSharedMemoryAndSemaphores).
*/
InitProcess();
@@ -4424,7 +4437,7 @@ BackendRun(Port *port)
*/
MemoryContextSwitchTo(TopMemoryContext);
- PostgresMain(port->database_name, port->user_name);
+ PostgresMain(MyProcPort->database_name, MyProcPort->user_name);
}
@@ -4445,11 +4458,11 @@ BackendRun(Port *port)
pid_t
postmaster_forkexec(int argc, char *argv[])
{
- Port port;
+ ClientSocket client_sock;
- /* This entry point passes dummy values for the Port variables */
- memset(&port, 0, sizeof(port));
- return internal_forkexec(argc, argv, &port);
+ /* This entry point doesn't pass a client socket */
+ memset(&client_sock, 0, sizeof(ClientSocket));
+ return internal_forkexec(argc, argv, &client_sock);
}
/*
@@ -4462,7 +4475,7 @@ postmaster_forkexec(int argc, char *argv[])
* returns the pid of the fork/exec'd process, or -1 on failure
*/
static pid_t
-backend_forkexec(Port *port, CAC_state cac)
+backend_forkexec(ClientSocket *client_sock, CAC_state cac)
{
char *av[5];
int ac = 0;
@@ -4478,7 +4491,7 @@ backend_forkexec(Port *port, CAC_state cac)
av[ac] = NULL;
Assert(ac < lengthof(av));
- return internal_forkexec(ac, av, port);
+ return internal_forkexec(ac, av, client_sock);
}
#ifndef WIN32
@@ -4490,7 +4503,7 @@ backend_forkexec(Port *port, CAC_state cac)
* - fork():s, and then exec():s the child process
*/
static pid_t
-internal_forkexec(int argc, char *argv[], Port *port)
+internal_forkexec(int argc, char *argv[], ClientSocket *client_sock)
{
static unsigned long tmpBackendFileNum = 0;
pid_t pid;
@@ -4498,7 +4511,7 @@ internal_forkexec(int argc, char *argv[], Port *port)
BackendParameters param;
FILE *fp;
- if (!save_backend_variables(¶m, port))
+ if (!save_backend_variables(¶m, client_sock))
return -1; /* log made by save_backend_variables */
/* Calculate name for temp file */
@@ -4796,7 +4809,7 @@ retry:
void
SubPostmasterMain(int argc, char *argv[])
{
- Port port;
+ ClientSocket client_sock;
/* In EXEC_BACKEND case we will not have inherited these settings */
IsPostmasterEnvironment = true;
@@ -4810,8 +4823,8 @@ SubPostmasterMain(int argc, char *argv[])
elog(FATAL, "invalid subpostmaster invocation");
/* Read in the variables file */
- memset(&port, 0, sizeof(Port));
- read_backend_variables(argv[2], &port);
+ memset(&client_sock, 0, sizeof(ClientSocket));
+ read_backend_variables(argv[2], &client_sock);
/* Close the postmaster's sockets (as soon as we know them) */
ClosePostmasterPorts(strcmp(argv[1], "--forklog") == 0);
@@ -4915,13 +4928,13 @@ SubPostmasterMain(int argc, char *argv[])
* PGPROC slots, we have already initialized libpq and are able to
* report the error to the client.
*/
- BackendInitialize(&port, cac);
+ BackendInitialize(&client_sock, cac);
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
/* And run the backend */
- BackendRun(&port); /* does not return */
+ BackendRun(); /* does not return */
}
if (strcmp(argv[1], "--forkaux") == 0)
{
@@ -5995,15 +6008,15 @@ static void read_inheritable_socket(SOCKET *dest, InheritableSocket *src);
/* Save critical backend variables into the BackendParameters struct */
#ifndef WIN32
static bool
-save_backend_variables(BackendParameters *param, Port *port)
+save_backend_variables(BackendParameters *param, ClientSocket *client_sock)
#else
static bool
-save_backend_variables(BackendParameters *param, Port *port,
+save_backend_variables(BackendParameters *param, ClientSocket *client_sock,
HANDLE childProcess, pid_t childPid)
#endif
{
- memcpy(¶m->port, port, sizeof(Port));
- if (!write_inheritable_socket(¶m->portsocket, port->sock, childPid))
+ memcpy(¶m->client_sock, client_sock, sizeof(ClientSocket));
+ if (!write_inheritable_socket(¶m->serialized_sock, client_sock->sock, childPid))
return false;
strlcpy(param->DataDir, DataDir, MAXPGPATH);
@@ -6165,7 +6178,7 @@ read_inheritable_socket(SOCKET *dest, InheritableSocket *src)
#endif
static void
-read_backend_variables(char *id, Port *port)
+read_backend_variables(char *id, ClientSocket *client_sock)
{
BackendParameters param;
@@ -6232,15 +6245,15 @@ read_backend_variables(char *id, Port *port)
}
#endif
- restore_backend_variables(¶m, port);
+ restore_backend_variables(¶m, client_sock);
}
/* Restore critical backend variables from the BackendParameters struct */
static void
-restore_backend_variables(BackendParameters *param, Port *port)
+restore_backend_variables(BackendParameters *param, ClientSocket *client_sock)
{
- memcpy(port, ¶m->port, sizeof(Port));
- read_inheritable_socket(&port->sock, ¶m->portsocket);
+ memcpy(client_sock, ¶m->client_sock, sizeof(ClientSocket));
+ read_inheritable_socket(&client_sock->sock, ¶m->serialized_sock);
SetDataDir(param->DataDir);
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 625caa853a0..41867dc14ac 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -110,12 +110,9 @@ typedef struct ClientConnectionInfo
} ClientConnectionInfo;
/*
- * This is used by the postmaster in its communication with frontends. It
- * contains all state information needed during this communication before the
- * backend is run. The Port structure is kept in malloc'd memory and is
- * still available when a backend is running (see MyProcPort). The data
- * it points to must also be malloc'd, or else palloc'd in TopMemoryContext,
- * so that it survives into PostgresMain execution!
+ * The Port structure holds state information about a client connection in a
+ * backend process. It is available in the global variable MyProcPort. The
+ * struct and all the data it points are kept in TopMemoryContext.
*
* remote_hostname is set if we did a successful reverse lookup of the
* client's IP address during connection setup.
@@ -217,6 +214,16 @@ typedef struct Port
#endif
} Port;
+/*
+ * ClientSocket holds a socket for an accepted connection, along with the
+ * information about the endpoints.
+ */
+typedef struct ClientSocket {
+ pgsocket sock; /* File descriptor */
+ SockAddr laddr; /* local addr (postmaster) */
+ SockAddr raddr; /* remote addr (client) */
+} ClientSocket;
+
#ifdef USE_SSL
/*
* Hardcoded DH parameters, used in ephemeral DH keying. (See also
diff --git a/src/include/libpq/libpq.h b/src/include/libpq/libpq.h
index 50fc781f471..fcbcdbe2dbf 100644
--- a/src/include/libpq/libpq.h
+++ b/src/include/libpq/libpq.h
@@ -64,11 +64,11 @@ extern PGDLLIMPORT WaitEventSet *FeBeWaitSet;
#define FeBeWaitSetLatchPos 1
#define FeBeWaitSetNEvents 3
-extern int StreamServerPort(int family, const char *hostName,
+extern int ListenServerPort(int family, const char *hostName,
unsigned short portNumber, const char *unixSocketDir,
pgsocket ListenSocket[], int MaxListen);
-extern int StreamConnection(pgsocket server_fd, Port *port);
-extern void StreamClose(pgsocket sock);
+extern int AcceptClientConnection(pgsocket server_fd, ClientSocket *client_sock);
+extern Port *StreamConnection(ClientSocket *client_sock);
extern void TouchSocketFiles(void);
extern void RemoveSocketFiles(void);
extern void pq_init(void);
--
2.30.2
0009-Refactor-postmaster-child-process-launching.patchtext/x-patch; charset=UTF-8; name=0009-Refactor-postmaster-child-process-launching.patchDownload
From b33cfeb28a5419045acb659a01410b2b463bea3e Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Sun, 18 Jun 2023 13:59:48 +0300
Subject: [PATCH 9/9] Refactor postmaster child process launching
- Move code related to launching backend processes to new source file,
process_start.c
- Introduce new postmaster_child_launch() function that deals with the
differences between EXEC_BACKEND and fork mode.
- Refactor the mechanism of passing informaton from the parent to
child process. Instead of using different command-line arguments
when launching the child process in EXEC_BACKEND mode, pass a
variable-length blob of data along with all the global
variables. The contents of that blob depends on the kind of child
process being launched. In !EXEC_BACKEND mode, we use the same blob,
but it's simply inherited from the parent to child process.
---
src/backend/postmaster/Makefile | 2 +-
src/backend/postmaster/autovacuum.c | 173 +--
src/backend/postmaster/auxprocess.c | 67 +-
src/backend/postmaster/bgworker.c | 19 +-
src/backend/postmaster/bgwriter.c | 9 +-
src/backend/postmaster/checkpointer.c | 9 +-
src/backend/postmaster/fork_process.c | 126 --
src/backend/postmaster/launch_backend.c | 1314 ++++++++++++++++
src/backend/postmaster/meson.build | 2 +-
src/backend/postmaster/pgarch.c | 9 +-
src/backend/postmaster/postmaster.c | 1526 +++----------------
src/backend/postmaster/startup.c | 9 +-
src/backend/postmaster/syslogger.c | 275 ++--
src/backend/postmaster/walwriter.c | 9 +-
src/backend/replication/logical/launcher.c | 1 -
src/backend/replication/walreceiver.c | 10 +-
src/backend/storage/ipc/shmem.c | 2 +
src/backend/tcop/postgres.c | 1 +
src/backend/utils/init/globals.c | 1 +
src/backend/utils/init/miscinit.c | 133 --
src/include/libpq/libpq-be.h | 10 +-
src/include/miscadmin.h | 1 -
src/include/postmaster/autovacuum.h | 12 +-
src/include/postmaster/auxprocess.h | 4 +-
src/include/postmaster/bgworker_internals.h | 4 +-
src/include/postmaster/bgwriter.h | 4 +-
src/include/postmaster/fork_process.h | 17 -
src/include/postmaster/pgarch.h | 2 +-
src/include/postmaster/postmaster.h | 40 +-
src/include/postmaster/startup.h | 2 +-
src/include/postmaster/syslogger.h | 4 +-
src/include/postmaster/walwriter.h | 2 +-
src/include/replication/walreceiver.h | 2 +-
33 files changed, 1799 insertions(+), 2002 deletions(-)
delete mode 100644 src/backend/postmaster/fork_process.c
create mode 100644 src/backend/postmaster/launch_backend.c
delete mode 100644 src/include/postmaster/fork_process.h
diff --git a/src/backend/postmaster/Makefile b/src/backend/postmaster/Makefile
index 047448b34eb..fc88f5bae0b 100644
--- a/src/backend/postmaster/Makefile
+++ b/src/backend/postmaster/Makefile
@@ -18,8 +18,8 @@ OBJS = \
bgworker.o \
bgwriter.o \
checkpointer.o \
- fork_process.o \
interrupt.o \
+ launch_backend.o \
pgarch.o \
postmaster.o \
startup.o \
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 1041376951e..e13284b4b5c 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -84,7 +84,6 @@
#include "nodes/makefuncs.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "storage/bufmgr.h"
@@ -315,13 +314,6 @@ static WorkerInfo MyWorkerInfo = NULL;
/* PID of launcher, valid only in worker while shutting down */
int AutovacuumLauncherPid = 0;
-#ifdef EXEC_BACKEND
-static pid_t avlauncher_forkexec(void);
-static pid_t avworker_forkexec(void);
-#endif
-NON_EXEC_STATIC void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
-NON_EXEC_STATIC void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
-
static Oid do_start_worker(void);
static void HandleAutoVacLauncherInterrupts(void);
static void AutoVacLauncherShutdown(void) pg_attribute_noreturn();
@@ -365,85 +357,21 @@ static void avl_sigusr2_handler(SIGNAL_ARGS);
* AUTOVACUUM LAUNCHER CODE
********************************************************************/
-#ifdef EXEC_BACKEND
/*
- * forkexec routine for the autovacuum launcher process.
- *
- * Format up the arglist, then fork and exec.
- */
-static pid_t
-avlauncher_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkavlauncher";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-
-/*
- * We need this set from the outside, before InitProcess is called
+ * Main loop for the autovacuum launcher process.
*/
void
-AutovacuumLauncherIAm(void)
-{
- am_autovacuum_launcher = true;
-}
-#endif
-
-/*
- * Main entry point for autovacuum launcher process, to be called from the
- * postmaster.
- */
-int
-StartAutoVacLauncher(void)
+AutoVacLauncherMain(char *startup_data, size_t startup_data_len)
{
- pid_t AutoVacPID;
+ sigjmp_buf local_sigjmp_buf;
-#ifdef EXEC_BACKEND
- switch ((AutoVacPID = avlauncher_forkexec()))
-#else
- switch ((AutoVacPID = fork_process()))
-#endif
+ /* Release postmaster's working memory context */
+ if (PostmasterContext)
{
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum launcher process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacLauncherMain(0, NULL);
- break;
-#endif
- default:
- return (int) AutoVacPID;
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
}
- /* shouldn't get here */
- return 0;
-}
-
-/*
- * Main loop for the autovacuum launcher process.
- */
-NON_EXEC_STATIC void
-AutoVacLauncherMain(int argc, char *argv[])
-{
- sigjmp_buf local_sigjmp_buf;
-
am_autovacuum_launcher = true;
MyBackendType = B_AUTOVAC_LAUNCHER;
@@ -475,6 +403,9 @@ AutoVacLauncherMain(int argc, char *argv[])
pqsignal(SIGFPE, FloatExceptionHandler);
pqsignal(SIGCHLD, SIG_DFL);
+ /* autovacuum needs this set before calling InitProcess */
+ am_autovacuum_launcher = true;
+
/*
* Create a per-backend PGPROC struct in shared memory. We must do this
* before we can use LWLocks.
@@ -1435,87 +1366,22 @@ avl_sigusr2_handler(SIGNAL_ARGS)
* AUTOVACUUM WORKER CODE
********************************************************************/
-#ifdef EXEC_BACKEND
-/*
- * forkexec routines for the autovacuum worker.
- *
- * Format up the arglist, then fork and exec.
- */
-static pid_t
-avworker_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkavworker";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-
/*
- * We need this set from the outside, before InitProcess is called
+ * AutoVacWorkerMain
*/
void
-AutovacuumWorkerIAm(void)
+AutoVacWorkerMain(char *startup_data, size_t startup_data_len)
{
- am_autovacuum_worker = true;
-}
-#endif
-
-/*
- * Main entry point for autovacuum worker process.
- *
- * This code is heavily based on pgarch.c, q.v.
- */
-int
-StartAutoVacWorker(void)
-{
- pid_t worker_pid;
+ sigjmp_buf local_sigjmp_buf;
+ Oid dbid;
-#ifdef EXEC_BACKEND
- switch ((worker_pid = avworker_forkexec()))
-#else
- switch ((worker_pid = fork_process()))
-#endif
+ /* Release postmaster's working memory context */
+ if (PostmasterContext)
{
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum worker process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacWorkerMain(0, NULL);
- break;
-#endif
- default:
- return (int) worker_pid;
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
}
- /* shouldn't get here */
- return 0;
-}
-
-/*
- * AutoVacWorkerMain
- */
-NON_EXEC_STATIC void
-AutoVacWorkerMain(int argc, char *argv[])
-{
- sigjmp_buf local_sigjmp_buf;
- Oid dbid;
-
am_autovacuum_worker = true;
MyBackendType = B_AUTOVAC_WORKER;
@@ -1546,6 +1412,9 @@ AutoVacWorkerMain(int argc, char *argv[])
pqsignal(SIGFPE, FloatExceptionHandler);
pqsignal(SIGCHLD, SIG_DFL);
+ /* autovacuum needs this set before calling InitProcess */
+ am_autovacuum_worker = true;
+
/*
* Create a per-backend PGPROC struct in shared memory. We must do this
* before we can use LWLocks.
diff --git a/src/backend/postmaster/auxprocess.c b/src/backend/postmaster/auxprocess.c
index 536d9a2b3e4..76de39df003 100644
--- a/src/backend/postmaster/auxprocess.c
+++ b/src/backend/postmaster/auxprocess.c
@@ -46,7 +46,7 @@ AuxProcType MyAuxProcType = NotAnAuxProcess; /* declared in miscadmin.h */
/*
- * AuxiliaryProcessMain
+ * AuxiliaryProcessMain XXX
*
* The main entry point for auxiliary processes, such as the bgwriter,
* walwriter, walreceiver, bootstrapper and the shared memory checker code.
@@ -54,37 +54,17 @@ AuxProcType MyAuxProcType = NotAnAuxProcess; /* declared in miscadmin.h */
* This code is here just because of historical reasons.
*/
void
-AuxiliaryProcessMain(AuxProcType auxtype)
+AuxiliaryProcessInit(void)
{
- Assert(IsUnderPostmaster);
-
- MyAuxProcType = auxtype;
-
- switch (MyAuxProcType)
+ /* Release postmaster's working memory context */
+ if (PostmasterContext)
{
- case StartupProcess:
- MyBackendType = B_STARTUP;
- break;
- case ArchiverProcess:
- MyBackendType = B_ARCHIVER;
- break;
- case BgWriterProcess:
- MyBackendType = B_BG_WRITER;
- break;
- case CheckpointerProcess:
- MyBackendType = B_CHECKPOINTER;
- break;
- case WalWriterProcess:
- MyBackendType = B_WAL_WRITER;
- break;
- case WalReceiverProcess:
- MyBackendType = B_WAL_RECEIVER;
- break;
- default:
- elog(PANIC, "unrecognized process type: %d", (int) MyAuxProcType);
- MyBackendType = B_INVALID;
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
}
+ Assert(IsUnderPostmaster);
+
init_ps_display(NULL);
SetProcessingMode(BootstrapProcessing);
@@ -134,37 +114,6 @@ AuxiliaryProcessMain(AuxProcType auxtype)
before_shmem_exit(ShutdownAuxiliaryProcess, 0);
SetProcessingMode(NormalProcessing);
-
- switch (MyAuxProcType)
- {
- case StartupProcess:
- StartupProcessMain();
- proc_exit(1);
-
- case ArchiverProcess:
- PgArchiverMain();
- proc_exit(1);
-
- case BgWriterProcess:
- BackgroundWriterMain();
- proc_exit(1);
-
- case CheckpointerProcess:
- CheckpointerMain();
- proc_exit(1);
-
- case WalWriterProcess:
- WalWriterMain();
- proc_exit(1);
-
- case WalReceiverProcess:
- WalReceiverMain();
- proc_exit(1);
-
- default:
- elog(PANIC, "unrecognized process type: %d", (int) MyAuxProcType);
- proc_exit(1);
- }
}
/*
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index 4c446ffd087..97d62777c8a 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -717,23 +717,30 @@ bgworker_die(SIGNAL_ARGS)
}
/*
- * Start a new background worker
- *
- * This is the main entry point for background worker, to be called from
- * postmaster.
+ * This is the main entry point for background worker process.
*/
void
-StartBackgroundWorker(void)
+BackgroundWorkerMain(char *startup_data, size_t startup_data_len)
{
sigjmp_buf local_sigjmp_buf;
- BackgroundWorker *worker = MyBgworkerEntry;
+ BackgroundWorker *worker;
bgworker_main_type entrypt;
+ /* Release postmaster's working memory context */
+ if (PostmasterContext)
+ {
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
+ }
+
+ Assert(startup_data_len == sizeof(BackgroundWorker));
+ worker = (BackgroundWorker *) startup_data;
if (worker == NULL)
elog(FATAL, "unable to find bgworker entry");
IsBackgroundWorker = true;
+ MyBgworkerEntry = worker;
MyBackendType = B_BG_WORKER;
init_ps_display(worker->bgw_name);
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index caad642ec93..0b6fbec1dbe 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -36,6 +36,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/bgwriter.h"
#include "postmaster/interrupt.h"
#include "storage/buf_internals.h"
@@ -88,13 +89,19 @@ static XLogRecPtr last_snapshot_lsn = InvalidXLogRecPtr;
* basic execution environment, but not enabled signals yet.
*/
void
-BackgroundWriterMain(void)
+BackgroundWriterMain(char *startup_data, size_t startup_data_len)
{
sigjmp_buf local_sigjmp_buf;
MemoryContext bgwriter_context;
bool prev_hibernate;
WritebackContext wb_context;
+ Assert(startup_data_len == 0);
+
+ MyAuxProcType = BgWriterProcess;
+ MyBackendType = B_BG_WRITER;
+ AuxiliaryProcessInit();
+
/*
* Properly accept or ignore signals that might be sent to us.
*/
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index ace9893d957..e617066c385 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -42,6 +42,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/bgwriter.h"
#include "postmaster/interrupt.h"
#include "replication/syncrep.h"
@@ -178,11 +179,17 @@ static void ReqCheckpointHandler(SIGNAL_ARGS);
* basic execution environment, but not enabled signals yet.
*/
void
-CheckpointerMain(void)
+CheckpointerMain(char *startup_data, size_t startup_data_len)
{
sigjmp_buf local_sigjmp_buf;
MemoryContext checkpointer_context;
+ Assert(startup_data_len == 0);
+
+ MyAuxProcType = CheckpointerProcess;
+ MyBackendType = B_CHECKPOINTER;
+ AuxiliaryProcessInit();
+
CheckpointerShmem->checkpointer_pid = MyProcPid;
/*
diff --git a/src/backend/postmaster/fork_process.c b/src/backend/postmaster/fork_process.c
deleted file mode 100644
index 6f9c2765d68..00000000000
--- a/src/backend/postmaster/fork_process.c
+++ /dev/null
@@ -1,126 +0,0 @@
-/*
- * fork_process.c
- * A simple wrapper on top of fork(). This does not handle the
- * EXEC_BACKEND case; it might be extended to do so, but it would be
- * considerably more complex.
- *
- * Copyright (c) 1996-2023, PostgreSQL Global Development Group
- *
- * IDENTIFICATION
- * src/backend/postmaster/fork_process.c
- */
-#include "postgres.h"
-
-#include <fcntl.h>
-#include <signal.h>
-#include <time.h>
-#include <sys/stat.h>
-#include <sys/time.h>
-#include <unistd.h>
-
-#include "libpq/pqsignal.h"
-#include "postmaster/fork_process.h"
-
-#ifndef WIN32
-/*
- * Wrapper for fork(). Return values are the same as those for fork():
- * -1 if the fork failed, 0 in the child process, and the PID of the
- * child in the parent process. Signals are blocked while forking, so
- * the child must unblock.
- */
-pid_t
-fork_process(void)
-{
- pid_t result;
- const char *oomfilename;
- sigset_t save_mask;
-
-#ifdef LINUX_PROFILE
- struct itimerval prof_itimer;
-#endif
-
- /*
- * Flush stdio channels just before fork, to avoid double-output problems.
- */
- fflush(NULL);
-
-#ifdef LINUX_PROFILE
-
- /*
- * Linux's fork() resets the profiling timer in the child process. If we
- * want to profile child processes then we need to save and restore the
- * timer setting. This is a waste of time if not profiling, however, so
- * only do it if commanded by specific -DLINUX_PROFILE switch.
- */
- getitimer(ITIMER_PROF, &prof_itimer);
-#endif
-
- /*
- * We start postmaster children with signals blocked. This allows them to
- * install their own handlers before unblocking, to avoid races where they
- * might run the postmaster's handler and miss an important control
- * signal. With more analysis this could potentially be relaxed.
- */
- sigprocmask(SIG_SETMASK, &BlockSig, &save_mask);
- result = fork();
- if (result == 0)
- {
- /* fork succeeded, in child */
-#ifdef LINUX_PROFILE
- setitimer(ITIMER_PROF, &prof_itimer, NULL);
-#endif
-
- /*
- * By default, Linux tends to kill the postmaster in out-of-memory
- * situations, because it blames the postmaster for the sum of child
- * process sizes *including shared memory*. (This is unbelievably
- * stupid, but the kernel hackers seem uninterested in improving it.)
- * Therefore it's often a good idea to protect the postmaster by
- * setting its OOM score adjustment negative (which has to be done in
- * a root-owned startup script). Since the adjustment is inherited by
- * child processes, this would ordinarily mean that all the
- * postmaster's children are equally protected against OOM kill, which
- * is not such a good idea. So we provide this code to allow the
- * children to change their OOM score adjustments again. Both the
- * file name to write to and the value to write are controlled by
- * environment variables, which can be set by the same startup script
- * that did the original adjustment.
- */
- oomfilename = getenv("PG_OOM_ADJUST_FILE");
-
- if (oomfilename != NULL)
- {
- /*
- * Use open() not stdio, to ensure we control the open flags. Some
- * Linux security environments reject anything but O_WRONLY.
- */
- int fd = open(oomfilename, O_WRONLY, 0);
-
- /* We ignore all errors */
- if (fd >= 0)
- {
- const char *oomvalue = getenv("PG_OOM_ADJUST_VALUE");
- int rc;
-
- if (oomvalue == NULL) /* supply a useful default */
- oomvalue = "0";
-
- rc = write(fd, oomvalue, strlen(oomvalue));
- (void) rc;
- close(fd);
- }
- }
-
- /* do post-fork initialization for random number generation */
- pg_strong_random_init();
- }
- else
- {
- /* in parent, restore signal mask */
- sigprocmask(SIG_SETMASK, &save_mask, NULL);
- }
-
- return result;
-}
-
-#endif /* ! WIN32 */
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
new file mode 100644
index 00000000000..b1fc783a576
--- /dev/null
+++ b/src/backend/postmaster/launch_backend.c
@@ -0,0 +1,1314 @@
+/*-------------------------------------------------------------------------
+ *
+ * launch_backend.c
+ * Functions for launching backends and other postmaster child
+ * processes.
+ *
+ * - explain EXEC_BACKEND and Windows
+ * - postmaster calls postmaster_child_launch()
+ * - the child process will be restored to roughly the same state, whether
+ * EXEC_BACKEND is used or not: it will be attached to shared memory,
+ * and fds and other resources that we've inherited from postmaster that are not needed
+ * in a child process have been closed.
+ *
+ * When a request message is received, we now fork() immediately.
+ * The child process performs authentication of the request, and
+ * then becomes a backend if successful. This allows the auth code
+ * to be written in a simple single-threaded style (as opposed to the
+ * crufty "poor man's multitasking" code that used to be needed).
+ * More importantly, it ensures that blockages in non-multithreaded
+ * libraries like SSL or PAM cannot cause denial of service to other
+ * clients.
+ *
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/postmaster/launch_backend.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include <unistd.h>
+#include <signal.h>
+#include <time.h>
+#include <sys/wait.h>
+#include <sys/stat.h>
+#include <sys/socket.h>
+#include <fcntl.h>
+#include <sys/param.h>
+#include <netdb.h>
+#include <limits.h>
+
+#include "access/transam.h"
+#include "access/xlog.h"
+#include "libpq/libpq-be.h"
+#include "libpq/pqsignal.h"
+#include "miscadmin.h"
+#include "nodes/queryjumble.h"
+#include "port.h"
+#include "postmaster/autovacuum.h"
+#include "postmaster/auxprocess.h"
+#include "postmaster/bgworker_internals.h"
+#include "postmaster/bgwriter.h"
+#include "postmaster/interrupt.h"
+#include "postmaster/pgarch.h"
+#include "postmaster/postmaster.h"
+#include "postmaster/startup.h"
+#include "postmaster/syslogger.h"
+#include "postmaster/walwriter.h"
+#include "replication/walreceiver.h"
+#include "storage/fd.h"
+#include "storage/ipc.h"
+#include "storage/pg_shmem.h"
+#include "storage/pmsignal.h"
+#include "storage/proc.h"
+#include "tcop/tcopprot.h"
+#include "utils/builtins.h"
+#include "utils/datetime.h"
+#include "utils/guc.h"
+#include "utils/memutils.h"
+#include "utils/ps_status.h"
+#include "utils/timeout.h"
+#include "utils/timestamp.h"
+
+#ifdef EXEC_BACKEND
+#include "storage/spin.h"
+#endif
+
+
+/* Type for a socket that can be inherited to a client process */
+#ifdef WIN32
+typedef struct
+{
+ SOCKET origsocket; /* Original socket value, or PGINVALID_SOCKET
+ * if not a socket */
+ WSAPROTOCOL_INFO wsainfo;
+} InheritableSocket;
+#else
+typedef int InheritableSocket;
+#endif
+
+#ifdef EXEC_BACKEND
+
+/*
+ * Structure contains all global variables passed to exec:ed backends
+ */
+typedef struct
+{
+ char DataDir[MAXPGPATH];
+ int32 MyCancelKey;
+ int MyPMChildSlot;
+#ifndef WIN32
+ unsigned long UsedShmemSegID;
+#else
+ void *ShmemProtectiveRegion;
+ HANDLE UsedShmemSegID;
+#endif
+ void *UsedShmemSegAddr;
+ slock_t *ShmemLock;
+ VariableCache ShmemVariableCache;
+ struct bkend *ShmemBackendArray;
+#ifndef HAVE_SPINLOCKS
+ PGSemaphore *SpinlockSemaArray;
+#endif
+ int NamedLWLockTrancheRequests;
+ NamedLWLockTranche *NamedLWLockTrancheArray;
+ LWLockPadded *MainLWLockArray;
+ slock_t *ProcStructLock;
+ PROC_HDR *ProcGlobal;
+ PGPROC *AuxiliaryProcs;
+ PGPROC *PreparedXactProcs;
+ PMSignalData *PMSignalState;
+ pid_t PostmasterPid;
+ TimestampTz PgStartTime;
+ TimestampTz PgReloadTime;
+ pg_time_t first_syslogger_file_time;
+ bool redirection_done;
+ bool IsBinaryUpgrade;
+ bool query_id_enabled;
+ int max_safe_fds;
+ int MaxBackends;
+#ifdef WIN32
+ HANDLE PostmasterHandle;
+ HANDLE initial_signal_pipe;
+ HANDLE syslogPipe[2];
+#else
+ int postmaster_alive_fds[2];
+ int syslogPipe[2];
+#endif
+ char my_exec_path[MAXPGPATH];
+ char pkglib_path[MAXPGPATH];
+
+ /*
+ * These are only used by backend processes, but it's here because passing
+ * a socket needs some special handling on Windows. 'client_sock' is an
+ * explicit argument to postmaster_child_launch, but is stored in
+ * MyClientSocket in the child process.
+ */
+ ClientSocket client_sock;
+ InheritableSocket inh_sock;
+} BackendParameters;
+
+static void read_backend_variables(char *id, char **startup_data, size_t *startup_data_len);
+static void restore_backend_variables(BackendParameters *param);
+
+#ifndef WIN32
+static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock);
+#else
+static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock,
+ HANDLE childProcess, pid_t childPid);
+#endif
+#endif /* EXEC_BACKEND */
+
+static void InitPostmasterChild(bool am_syslogger);
+
+
+#ifdef EXEC_BACKEND
+static pid_t internal_forkexec(const char *entry_name, char *startup_data, size_t startup_data_len, ClientSocket *client_sock);
+#endif /* EXEC_BACKEND */
+
+#ifndef WIN32
+static pid_t fork_process(void);
+#endif
+
+typedef struct PMChildEntry
+{
+ const char *name;
+ ChildEntryPoint entry_fn;
+ bool shmem_attach;
+} PMChildEntry;
+
+const PMChildEntry entry_kinds[] = {
+ {"backend", BackendMain, true},
+
+ {"autovacuum launcher", AutoVacLauncherMain, true},
+ {"autovacuum worker", AutoVacWorkerMain, true},
+ {"bgworker", BackgroundWorkerMain, true},
+ {"syslogger", SysLoggerMain, false},
+
+ {"startup", StartupProcessMain, true},
+ {"bgwriter", BackgroundWriterMain, true},
+ {"archiver", PgArchiverMain, true},
+ {"checkpointer", CheckpointerMain, true},
+ {"wal_writer", WalWriterMain, true},
+ {"wal_receiver", WalReceiverMain, true},
+};
+
+const char *
+PostmasterChildName(PostmasterChildType child_type)
+{
+ Assert(child_type >= 0 && child_type < lengthof(entry_kinds));
+ return entry_kinds[child_type].name;
+}
+
+/*
+ * Start a new postmaster child process.
+ *
+ * The startup_data must be a contigous block that is passed to the child
+ * process.
+ *
+ * (In fork mode, it's inherited directly by the child process. In fork+exec mode,
+ * it is written to a file and read back in the child process)
+ */
+pid_t
+postmaster_child_launch(PostmasterChildType child_type, char *startup_data, size_t startup_data_len, ClientSocket *client_sock)
+{
+ pid_t pid;
+
+ Assert(child_type >= 0 && child_type < lengthof(entry_kinds));
+ Assert(IsPostmasterEnvironment && !IsUnderPostmaster);
+
+#ifdef EXEC_BACKEND
+ pid = internal_forkexec(entry_kinds[child_type].name, startup_data, startup_data_len, client_sock);
+ /* the child process will arrive in SubPostmasterMain */
+#else /* !EXEC_BACKEND */
+ pid = fork_process();
+ if (pid == 0) /* child */
+ {
+ /* Detangle from postmaster */
+ InitPostmasterChild(child_type == PMC_SYSLOGGER);
+
+ /*
+ * Before blowing away PostmasterContext, save the startup data
+ */
+ MemoryContextSwitchTo(TopMemoryContext);
+ if (startup_data != NULL)
+ {
+ char *cp = palloc(startup_data_len);
+
+ memcpy(cp, startup_data, startup_data_len);
+ startup_data = cp;
+ }
+
+ if (client_sock)
+ {
+ MyClientSocket = palloc(sizeof(ClientSocket));
+ memcpy(MyClientSocket, client_sock, sizeof(ClientSocket));
+ }
+
+ entry_kinds[child_type].entry_fn(startup_data, startup_data_len);
+ Assert(false);
+ }
+#endif /* EXEC_BACKEND */
+ return pid;
+}
+
+#ifndef WIN32
+/*
+ * Wrapper for fork(). Return values are the same as those for fork():
+ * -1 if the fork failed, 0 in the child process, and the PID of the
+ * child in the parent process. Signals are blocked while forking, so
+ * the child must unblock.
+ *
+ * XXX from fork_process.c file header:
+ * A simple wrapper on top of fork(). This does not handle the
+ * EXEC_BACKEND case; it might be extended to do so, but it would be
+ * considerably more complex.
+ */
+static pid_t
+fork_process(void)
+{
+ pid_t result;
+ const char *oomfilename;
+ sigset_t save_mask;
+
+#ifdef LINUX_PROFILE
+ struct itimerval prof_itimer;
+#endif
+
+ /*
+ * Flush stdio channels just before fork, to avoid double-output problems.
+ */
+ fflush(NULL);
+
+#ifdef LINUX_PROFILE
+
+ /*
+ * Linux's fork() resets the profiling timer in the child process. If we
+ * want to profile child processes then we need to save and restore the
+ * timer setting. This is a waste of time if not profiling, however, so
+ * only do it if commanded by specific -DLINUX_PROFILE switch.
+ */
+ getitimer(ITIMER_PROF, &prof_itimer);
+#endif
+
+ /*
+ * We start postmaster children with signals blocked. This allows them to
+ * install their own handlers before unblocking, to avoid races where they
+ * might run the postmaster's handler and miss an important control
+ * signal. With more analysis this could potentially be relaxed.
+ */
+ sigprocmask(SIG_SETMASK, &BlockSig, &save_mask);
+ result = fork();
+ if (result == 0)
+ {
+ /* fork succeeded, in child */
+#ifdef LINUX_PROFILE
+ setitimer(ITIMER_PROF, &prof_itimer, NULL);
+#endif
+
+ /*
+ * By default, Linux tends to kill the postmaster in out-of-memory
+ * situations, because it blames the postmaster for the sum of child
+ * process sizes *including shared memory*. (This is unbelievably
+ * stupid, but the kernel hackers seem uninterested in improving it.)
+ * Therefore it's often a good idea to protect the postmaster by
+ * setting its OOM score adjustment negative (which has to be done in
+ * a root-owned startup script). Since the adjustment is inherited by
+ * child processes, this would ordinarily mean that all the
+ * postmaster's children are equally protected against OOM kill, which
+ * is not such a good idea. So we provide this code to allow the
+ * children to change their OOM score adjustments again. Both the
+ * file name to write to and the value to write are controlled by
+ * environment variables, which can be set by the same startup script
+ * that did the original adjustment.
+ */
+ oomfilename = getenv("PG_OOM_ADJUST_FILE");
+
+ if (oomfilename != NULL)
+ {
+ /*
+ * Use open() not stdio, to ensure we control the open flags. Some
+ * Linux security environments reject anything but O_WRONLY.
+ */
+ int fd = open(oomfilename, O_WRONLY, 0);
+
+ /* We ignore all errors */
+ if (fd >= 0)
+ {
+ const char *oomvalue = getenv("PG_OOM_ADJUST_VALUE");
+ int rc;
+
+ if (oomvalue == NULL) /* supply a useful default */
+ oomvalue = "0";
+
+ rc = write(fd, oomvalue, strlen(oomvalue));
+ (void) rc;
+ close(fd);
+ }
+ }
+
+ /* do post-fork initialization for random number generation */
+ pg_strong_random_init();
+ }
+ else
+ {
+ /* in parent, restore signal mask */
+ sigprocmask(SIG_SETMASK, &save_mask, NULL);
+ }
+
+ return result;
+}
+
+#endif /* !WIN32 */
+
+#ifdef EXEC_BACKEND
+#ifndef WIN32
+
+/*
+ * internal_forkexec non-win32 implementation
+ *
+ * - writes out backend variables to the parameter file
+ * - fork():s, and then exec():s the child process
+ */
+static pid_t
+internal_forkexec(const char *entry_name, char *startup_data, size_t startup_data_len, ClientSocket *client_sock)
+{
+ static unsigned long tmpBackendFileNum = 0;
+ pid_t pid;
+ char tmpfilename[MAXPGPATH];
+ BackendParameters param;
+ FILE *fp;
+ char *argv[4];
+ char forkav[MAXPGPATH];
+
+ if (!save_backend_variables(¶m, client_sock))
+ return -1; /* log made by save_backend_variables */
+
+ /* Calculate name for temp file */
+ snprintf(tmpfilename, MAXPGPATH, "%s/%s.backend_var.%d.%lu",
+ PG_TEMP_FILES_DIR, PG_TEMP_FILE_PREFIX,
+ MyProcPid, ++tmpBackendFileNum);
+
+ /* Open file */
+ fp = AllocateFile(tmpfilename, PG_BINARY_W);
+ if (!fp)
+ {
+ /*
+ * As in OpenTemporaryFileInTablespace, try to make the temp-file
+ * directory, ignoring errors.
+ */
+ (void) MakePGDirectory(PG_TEMP_FILES_DIR);
+
+ fp = AllocateFile(tmpfilename, PG_BINARY_W);
+ if (!fp)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ tmpfilename)));
+ return -1;
+ }
+ }
+
+ if (fwrite(¶m, sizeof(param), 1, fp) != 1)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not write to file \"%s\": %m", tmpfilename)));
+ FreeFile(fp);
+ return -1;
+ }
+
+ /* write startup data */
+ if (fwrite((char *) &startup_data_len, sizeof(size_t), 1, fp) != 1)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not write to file \"%s\": %m", tmpfilename)));
+ FreeFile(fp);
+ return -1;
+ }
+ if (startup_data_len > 0)
+ {
+ if (fwrite(startup_data, startup_data_len, 1, fp) != 1)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not write to file \"%s\": %m", tmpfilename)));
+ FreeFile(fp);
+ return -1;
+ }
+ }
+
+ /* Release file */
+ if (FreeFile(fp))
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not write to file \"%s\": %m", tmpfilename)));
+ return -1;
+ }
+
+ /* set up argv properly */
+ argv[0] = "postgres";
+ snprintf(forkav, MAXPGPATH, "--forkchild=%s", entry_name);
+ argv[1] = forkav;
+ /* Insert temp file name after --fork argument */
+ argv[2] = tmpfilename;
+ argv[3] = NULL;
+
+ /* Fire off execv in child */
+ if ((pid = fork_process()) == 0)
+ {
+ if (execv(postgres_exec_path, argv) < 0)
+ {
+ ereport(LOG,
+ (errmsg("could not execute server process \"%s\": %m",
+ postgres_exec_path)));
+ /* We're already in the child process here, can't return */
+ exit(1);
+ }
+ }
+
+ return pid; /* Parent returns pid, or -1 on fork failure */
+}
+#else /* WIN32 */
+
+/*
+ * internal_forkexec win32 implementation
+ *
+ * - starts backend using CreateProcess(), in suspended state
+ * - writes out backend variables to the parameter file
+ * - during this, duplicates handles and sockets required for
+ * inheritance into the new process
+ * - resumes execution of the new process once the backend parameter
+ * file is complete.
+ */
+static pid_t
+internal_forkexec(const char *entry_name, char *startup_data, size_t startup_data_len, ClientSocket *client_sock)
+{
+ int retry_count = 0;
+ STARTUPINFO si;
+ PROCESS_INFORMATION pi;
+ int i;
+ int j;
+ char cmdLine[MAXPGPATH * 2];
+ HANDLE paramHandle;
+ BackendParameters *param;
+ SECURITY_ATTRIBUTES sa;
+ char paramHandleStr[32];
+ win32_deadchild_waitinfo *childinfo;
+ char *argv[4];
+
+ /* Resume here if we need to retry */
+retry:
+
+ /* Set up shared memory for parameter passing */
+ ZeroMemory(&sa, sizeof(sa));
+ sa.nLength = sizeof(sa);
+ sa.bInheritHandle = TRUE;
+ paramHandle = CreateFileMapping(INVALID_HANDLE_VALUE,
+ &sa,
+ PAGE_READWRITE,
+ 0,
+ sizeof(BackendParameters),
+ NULL);
+ if (paramHandle == INVALID_HANDLE_VALUE)
+ {
+ ereport(LOG,
+ (errmsg("could not create backend parameter file mapping: error code %lu",
+ GetLastError())));
+ return -1;
+ }
+
+ param = MapViewOfFile(paramHandle, FILE_MAP_WRITE, 0, 0, sizeof(BackendParameters));
+ if (!param)
+ {
+ ereport(LOG,
+ (errmsg("could not map backend parameter memory: error code %lu",
+ GetLastError())));
+ CloseHandle(paramHandle);
+ return -1;
+ }
+
+ /* Insert temp file name after --fork argument */
+#ifdef _WIN64
+ sprintf(paramHandleStr, "%llu", (LONG_PTR) paramHandle);
+#else
+ sprintf(paramHandleStr, "%lu", (DWORD) paramHandle);
+#endif
+
+ /* set up argv properly */
+ argv[0] = "postgres";
+ snprintf(forkav, MAXPGPATH, "--forkchild=%s", entry_name);
+ argv[1] = forkav;
+ /* Insert temp file name after --fork argument */
+ argv[2] = tmpfilename;
+ argv[3] = NULL;
+
+ /* Format the cmd line */
+ cmdLine[sizeof(cmdLine) - 1] = '\0';
+ cmdLine[sizeof(cmdLine) - 2] = '\0';
+ snprintf(cmdLine, sizeof(cmdLine) - 1, "\"%s\"", postgres_exec_path);
+ i = 0;
+ while (argv[++i] != NULL)
+ {
+ j = strlen(cmdLine);
+ snprintf(cmdLine + j, sizeof(cmdLine) - 1 - j, " \"%s\"", argv[i]);
+ }
+ if (cmdLine[sizeof(cmdLine) - 2] != '\0')
+ {
+ ereport(LOG,
+ (errmsg("subprocess command line too long")));
+ UnmapViewOfFile(param);
+ CloseHandle(paramHandle);
+ return -1;
+ }
+
+ memset(&pi, 0, sizeof(pi));
+ memset(&si, 0, sizeof(si));
+ si.cb = sizeof(si);
+
+ /*
+ * Create the subprocess in a suspended state. This will be resumed later,
+ * once we have written out the parameter file.
+ */
+ if (!CreateProcess(NULL, cmdLine, NULL, NULL, TRUE, CREATE_SUSPENDED,
+ NULL, NULL, &si, &pi))
+ {
+ ereport(LOG,
+ (errmsg("CreateProcess() call failed: %m (error code %lu)",
+ GetLastError())));
+ UnmapViewOfFile(param);
+ CloseHandle(paramHandle);
+ return -1;
+ }
+
+ if (!save_backend_variables(param, client_sock, pi.hProcess, pi.dwProcessId))
+ {
+ /*
+ * log made by save_backend_variables, but we have to clean up the
+ * mess with the half-started process
+ */
+ if (!TerminateProcess(pi.hProcess, 255))
+ ereport(LOG,
+ (errmsg_internal("could not terminate unstarted process: error code %lu",
+ GetLastError())));
+ CloseHandle(pi.hProcess);
+ CloseHandle(pi.hThread);
+ UnmapViewOfFile(param);
+ CloseHandle(paramHandle);
+ return -1; /* log made by save_backend_variables */
+ }
+
+ /* Drop the parameter shared memory that is now inherited to the backend */
+ if (!UnmapViewOfFile(param))
+ ereport(LOG,
+ (errmsg("could not unmap view of backend parameter file: error code %lu",
+ GetLastError())));
+ if (!CloseHandle(paramHandle))
+ ereport(LOG,
+ (errmsg("could not close handle to backend parameter file: error code %lu",
+ GetLastError())));
+
+ /*
+ * Reserve the memory region used by our main shared memory segment before
+ * we resume the child process. Normally this should succeed, but if ASLR
+ * is active then it might sometimes fail due to the stack or heap having
+ * gotten mapped into that range. In that case, just terminate the
+ * process and retry.
+ */
+ if (!pgwin32_ReserveSharedMemoryRegion(pi.hProcess))
+ {
+ /* pgwin32_ReserveSharedMemoryRegion already made a log entry */
+ if (!TerminateProcess(pi.hProcess, 255))
+ ereport(LOG,
+ (errmsg_internal("could not terminate process that failed to reserve memory: error code %lu",
+ GetLastError())));
+ CloseHandle(pi.hProcess);
+ CloseHandle(pi.hThread);
+ if (++retry_count < 100)
+ goto retry;
+ ereport(LOG,
+ (errmsg("giving up after too many tries to reserve shared memory"),
+ errhint("This might be caused by ASLR or antivirus software.")));
+ return -1;
+ }
+
+ /*
+ * Now that the backend variables are written out, we start the child
+ * thread so it can start initializing while we set up the rest of the
+ * parent state.
+ */
+ if (ResumeThread(pi.hThread) == -1)
+ {
+ if (!TerminateProcess(pi.hProcess, 255))
+ {
+ ereport(LOG,
+ (errmsg_internal("could not terminate unstartable process: error code %lu",
+ GetLastError())));
+ CloseHandle(pi.hProcess);
+ CloseHandle(pi.hThread);
+ return -1;
+ }
+ CloseHandle(pi.hProcess);
+ CloseHandle(pi.hThread);
+ ereport(LOG,
+ (errmsg_internal("could not resume thread of unstarted process: error code %lu",
+ GetLastError())));
+ return -1;
+ }
+
+ /*
+ * Queue a waiter to signal when this child dies. The wait will be handled
+ * automatically by an operating system thread pool. The memory will be
+ * freed by a later call to waitpid().
+ */
+ childinfo = palloc(sizeof(win32_deadchild_waitinfo));
+ childinfo->procHandle = pi.hProcess;
+ childinfo->procId = pi.dwProcessId;
+
+ if (!RegisterWaitForSingleObject(&childinfo->waitHandle,
+ pi.hProcess,
+ pgwin32_deadchild_callback,
+ childinfo,
+ INFINITE,
+ WT_EXECUTEONLYONCE | WT_EXECUTEINWAITTHREAD))
+ ereport(FATAL,
+ (errmsg_internal("could not register process for wait: error code %lu",
+ GetLastError())));
+
+ /* Don't close pi.hProcess here - waitpid() needs access to it */
+
+ CloseHandle(pi.hThread);
+
+ return pi.dwProcessId;
+}
+#endif /* WIN32 */
+
+/*
+ * SubPostmasterMain -- Get the fork/exec'd process into a state equivalent
+ * to what it would be if we'd simply forked on Unix, and then
+ * dispatch to the appropriate place.
+ *
+ * The first two command line arguments are expected to be "--forkFOO"
+ * (where FOO indicates which postmaster child we are to become), and
+ * the name of a variables file that we can read to load data that would
+ * have been inherited by fork() on Unix. Remaining arguments go to the
+ * subprocess FooMain() routine. XXX
+ */
+void
+SubPostmasterMain(int argc, char *argv[])
+{
+ PostmasterChildType child_type;
+ char *startup_data;
+ size_t startup_data_len;
+
+ /* In EXEC_BACKEND case we will not have inherited these settings */
+ IsPostmasterEnvironment = true;
+ whereToSendOutput = DestNone;
+
+ /* Setup essential subsystems (to ensure elog() behaves sanely) */
+ InitializeGUCOptions();
+
+ /* Check we got appropriate args */
+ if (argc < 3)
+ elog(FATAL, "invalid subpostmaster invocation");
+
+ if (strncmp(argv[1], "--forkchild=", 12) == 0)
+ {
+ char *entry_name = argv[1] + 12;
+ bool found = false;
+
+ for (int idx = 0; idx < lengthof(entry_kinds); idx++)
+ {
+ if (strcmp(entry_kinds[idx].name, entry_name) == 0)
+ {
+ child_type = idx;
+ found = true;
+ break;
+ }
+ }
+ if (!found)
+ elog(ERROR, "unknown child kind %s", entry_name);
+ }
+
+ /* Read in the variables file */
+ read_backend_variables(argv[2], &startup_data, &startup_data_len);
+
+ /* Setup as postmaster child */
+ InitPostmasterChild(child_type == PMC_SYSLOGGER);
+
+ /*
+ * If appropriate, physically re-attach to shared memory segment. We want
+ * to do this before going any further to ensure that we can attach at the
+ * same address the postmaster used. On the other hand, if we choose not
+ * to re-attach, we may have other cleanup to do.
+ *
+ * If testing EXEC_BACKEND on Linux, you should run this as root before
+ * starting the postmaster:
+ *
+ * sysctl -w kernel.randomize_va_space=0
+ *
+ * This prevents using randomized stack and code addresses that cause the
+ * child process's memory map to be different from the parent's, making it
+ * sometimes impossible to attach to shared memory at the desired address.
+ * Return the setting to its old value (usually '1' or '2') when finished.
+ */
+ if (entry_kinds[child_type].shmem_attach)
+ PGSharedMemoryReAttach();
+ else
+ PGSharedMemoryNoReAttach();
+
+ /* Read in remaining GUC variables */
+ read_nondefault_variables();
+
+ /*
+ * Check that the data directory looks valid, which will also check the
+ * privileges on the data directory and update our umask and file/group
+ * variables for creating files later. Note: this should really be done
+ * before we create any files or directories.
+ */
+ checkDataDir();
+
+ /*
+ * (re-)read control file, as it contains config. The postmaster will
+ * already have read this, but this process doesn't know about that.
+ */
+ LocalProcessControlFile(false);
+
+ /*
+ * Reload any libraries that were preloaded by the postmaster. Since we
+ * exec'd this process, those libraries didn't come along with us; but we
+ * should load them into all child processes to be consistent with the
+ * non-EXEC_BACKEND behavior.
+ */
+ process_shared_preload_libraries();
+
+ /* Restore basic shared memory pointers */
+ if (UsedShmemSegAddr != NULL)
+ InitShmemAccess(UsedShmemSegAddr);
+
+ /* Run backend or appropriate child */
+ entry_kinds[child_type].entry_fn(startup_data, startup_data_len);
+
+ abort(); /* shouldn't get here */
+}
+#endif /* EXEC_BACKEND */
+
+/* ----------------------------------------------------------------
+ * common process startup code
+ * ----------------------------------------------------------------
+ */
+
+/*
+ * Initialize the basic environment for a postmaster child
+ *
+ * Should be called as early as possible after the child's startup. However,
+ * on EXEC_BACKEND builds it does need to be after read_backend_variables().
+ */
+static void
+InitPostmasterChild(bool am_syslogger)
+{
+ /* Close the postmaster's sockets (as soon as we know them) */
+ ClosePostmasterPorts(am_syslogger);
+
+ IsUnderPostmaster = true; /* we are a postmaster subprocess now */
+
+ /*
+ * Start our win32 signal implementation. This has to be done after we
+ * read the backend variables, because we need to pick up the signal pipe
+ * from the parent process.
+ */
+#ifdef WIN32
+ pgwin32_signal_initialize();
+#endif
+
+ /*
+ * Set reference point for stack-depth checking. This might seem
+ * redundant in !EXEC_BACKEND builds; but it's not because the postmaster
+ * launches its children from signal handlers, so we might be running on
+ * an alternative stack. XXX still true?
+ */
+ (void) set_stack_base();
+
+ InitProcessGlobals();
+
+ /*
+ * make sure stderr is in binary mode before anything can possibly be
+ * written to it, in case it's actually the syslogger pipe, so the pipe
+ * chunking protocol isn't disturbed. Non-logpipe data gets translated on
+ * redirection (e.g. via pg_ctl -l) anyway.
+ */
+#ifdef WIN32
+ _setmode(fileno(stderr), _O_BINARY);
+#endif
+
+ /* We don't want the postmaster's proc_exit() handlers */
+ on_exit_reset();
+
+ /* In EXEC_BACKEND case we will not have inherited BlockSig etc values */
+#ifdef EXEC_BACKEND
+ pqinitmask();
+#endif
+
+ /* Initialize process-local latch support */
+ InitializeLatchSupport();
+ InitProcessLocalLatch();
+ InitializeLatchWaitSet();
+
+ /*
+ * If possible, make this process a group leader, so that the postmaster
+ * can signal any child processes too. Not all processes will have
+ * children, but for consistency we make all postmaster child processes do
+ * this.
+ */
+#ifdef HAVE_SETSID
+ if (setsid() < 0)
+ elog(FATAL, "setsid() failed: %m");
+#endif
+
+ /*
+ * Every postmaster child process is expected to respond promptly to
+ * SIGQUIT at all times. Therefore we centrally remove SIGQUIT from
+ * BlockSig and install a suitable signal handler. (Client-facing
+ * processes may choose to replace this default choice of handler with
+ * quickdie().) All other blockable signals remain blocked for now.
+ */
+ pqsignal(SIGQUIT, SignalHandlerForCrashExit);
+
+ sigdelset(&BlockSig, SIGQUIT);
+ sigprocmask(SIG_SETMASK, &BlockSig, NULL);
+
+ /* Request a signal if the postmaster dies, if possible. */
+ PostmasterDeathSignalInit();
+
+ /* Don't give the pipe to subprograms that we execute. */
+#ifndef WIN32
+ if (fcntl(postmaster_alive_fds[POSTMASTER_FD_WATCH], F_SETFD, FD_CLOEXEC) < 0)
+ ereport(FATAL,
+ (errcode_for_socket_access(),
+ errmsg_internal("could not set postmaster death monitoring pipe to FD_CLOEXEC mode: %m")));
+#endif
+}
+
+/*
+ * Initialize the basic environment for a standalone process.
+ *
+ * argv0 has to be suitable to find the program's executable.
+ */
+void
+InitStandaloneProcess(const char *argv0)
+{
+ Assert(!IsPostmasterEnvironment);
+
+ MyBackendType = B_STANDALONE_BACKEND;
+
+ /*
+ * Start our win32 signal implementation
+ */
+#ifdef WIN32
+ pgwin32_signal_initialize();
+#endif
+
+ InitProcessGlobals();
+
+ /* Initialize process-local latch support */
+ InitializeLatchSupport();
+ InitProcessLocalLatch();
+ InitializeLatchWaitSet();
+
+ /*
+ * For consistency with InitPostmasterChild, initialize signal mask here.
+ * But we don't unblock SIGQUIT or provide a default handler for it.
+ */
+ pqinitmask();
+ sigprocmask(SIG_SETMASK, &BlockSig, NULL);
+
+ /* Compute paths, no postmaster to inherit from */
+ if (my_exec_path[0] == '\0')
+ {
+ if (find_my_exec(argv0, my_exec_path) < 0)
+ elog(FATAL, "%s: could not locate my own executable path",
+ argv0);
+ }
+
+ if (pkglib_path[0] == '\0')
+ get_pkglib_path(my_exec_path, pkglib_path);
+}
+
+
+#ifdef EXEC_BACKEND
+
+/*
+ * The following need to be available to the save/restore_backend_variables
+ * functions. They are marked NON_EXEC_STATIC in their home modules.
+ */
+extern slock_t *ShmemLock;
+extern slock_t *ProcStructLock;
+extern PGPROC *AuxiliaryProcs;
+extern PMSignalData *PMSignalState;
+extern pg_time_t first_syslogger_file_time;
+extern struct bkend *ShmemBackendArray;
+extern bool redirection_done;
+
+#ifndef WIN32
+#define write_inheritable_socket(dest, src, childpid) ((*(dest) = (src)), true)
+#define read_inheritable_socket(dest, src) (*(dest) = *(src))
+#else
+static bool write_duplicated_handle(HANDLE *dest, HANDLE src, HANDLE child);
+static bool write_inheritable_socket(InheritableSocket *dest, SOCKET src,
+ pid_t childPid);
+static void read_inheritable_socket(SOCKET *dest, InheritableSocket *src);
+#endif
+
+
+/* Save critical backend variables into the BackendParameters struct */
+#ifndef WIN32
+static bool
+save_backend_variables(BackendParameters *param, ClientSocket *client_sock)
+#else
+static bool
+save_backend_variables(BackendParameters *param, ClientSocket *client_sock,
+ HANDLE childProcess, pid_t childPid)
+#endif
+{
+ if (client_sock)
+ memcpy(¶m->client_sock, client_sock, sizeof(ClientSocket));
+ else
+ memset(¶m->client_sock, 0, sizeof(ClientSocket));
+ if (!write_inheritable_socket(¶m->inh_sock,
+ client_sock ? client_sock->sock : PGINVALID_SOCKET,
+ childPid))
+ return false;
+
+ strlcpy(param->DataDir, DataDir, MAXPGPATH);
+
+ param->MyCancelKey = MyCancelKey;
+ param->MyPMChildSlot = MyPMChildSlot;
+
+#ifdef WIN32
+ param->ShmemProtectiveRegion = ShmemProtectiveRegion;
+#endif
+ param->UsedShmemSegID = UsedShmemSegID;
+ param->UsedShmemSegAddr = UsedShmemSegAddr;
+
+ param->ShmemLock = ShmemLock;
+ param->ShmemVariableCache = ShmemVariableCache;
+ param->ShmemBackendArray = ShmemBackendArray;
+
+#ifndef HAVE_SPINLOCKS
+ param->SpinlockSemaArray = SpinlockSemaArray;
+#endif
+ param->NamedLWLockTrancheRequests = NamedLWLockTrancheRequests;
+ param->NamedLWLockTrancheArray = NamedLWLockTrancheArray;
+ param->MainLWLockArray = MainLWLockArray;
+ param->ProcStructLock = ProcStructLock;
+ param->ProcGlobal = ProcGlobal;
+ param->AuxiliaryProcs = AuxiliaryProcs;
+ param->PreparedXactProcs = PreparedXactProcs;
+ param->PMSignalState = PMSignalState;
+
+ param->PostmasterPid = PostmasterPid;
+ param->PgStartTime = PgStartTime;
+ param->PgReloadTime = PgReloadTime;
+ param->first_syslogger_file_time = first_syslogger_file_time;
+
+ param->redirection_done = redirection_done;
+ param->IsBinaryUpgrade = IsBinaryUpgrade;
+ param->query_id_enabled = query_id_enabled;
+ param->max_safe_fds = max_safe_fds;
+
+ param->MaxBackends = MaxBackends;
+
+#ifdef WIN32
+ param->PostmasterHandle = PostmasterHandle;
+ if (!write_duplicated_handle(¶m->initial_signal_pipe,
+ pgwin32_create_signal_listener(childPid),
+ childProcess))
+ return false;
+#else
+ memcpy(¶m->postmaster_alive_fds, &postmaster_alive_fds,
+ sizeof(postmaster_alive_fds));
+#endif
+
+ memcpy(¶m->syslogPipe, &syslogPipe, sizeof(syslogPipe));
+
+ strlcpy(param->my_exec_path, my_exec_path, MAXPGPATH);
+
+ strlcpy(param->pkglib_path, pkglib_path, MAXPGPATH);
+
+ return true;
+}
+
+
+#ifdef WIN32
+/*
+ * Duplicate a handle for usage in a child process, and write the child
+ * process instance of the handle to the parameter file.
+ */
+static bool
+write_duplicated_handle(HANDLE *dest, HANDLE src, HANDLE childProcess)
+{
+ HANDLE hChild = INVALID_HANDLE_VALUE;
+
+ if (!DuplicateHandle(GetCurrentProcess(),
+ src,
+ childProcess,
+ &hChild,
+ 0,
+ TRUE,
+ DUPLICATE_CLOSE_SOURCE | DUPLICATE_SAME_ACCESS))
+ {
+ ereport(LOG,
+ (errmsg_internal("could not duplicate handle to be written to backend parameter file: error code %lu",
+ GetLastError())));
+ return false;
+ }
+
+ *dest = hChild;
+ return true;
+}
+
+/*
+ * Duplicate a socket for usage in a child process, and write the resulting
+ * structure to the parameter file.
+ * This is required because a number of LSPs (Layered Service Providers) very
+ * common on Windows (antivirus, firewalls, download managers etc) break
+ * straight socket inheritance.
+ */
+static bool
+write_inheritable_socket(InheritableSocket *dest, SOCKET src, pid_t childpid)
+{
+ dest->origsocket = src;
+ if (src != 0 && src != PGINVALID_SOCKET)
+ {
+ /* Actual socket */
+ if (WSADuplicateSocket(src, childpid, &dest->wsainfo) != 0)
+ {
+ ereport(LOG,
+ (errmsg("could not duplicate socket %d for use in backend: error code %d",
+ (int) src, WSAGetLastError())));
+ return false;
+ }
+ }
+ return true;
+}
+
+/*
+ * Read a duplicate socket structure back, and get the socket descriptor.
+ */
+static void
+read_inheritable_socket(SOCKET *dest, InheritableSocket *src)
+{
+ SOCKET s;
+
+ if (src->origsocket == PGINVALID_SOCKET || src->origsocket == 0)
+ {
+ /* Not a real socket! */
+ *dest = src->origsocket;
+ }
+ else
+ {
+ /* Actual socket, so create from structure */
+ s = WSASocket(FROM_PROTOCOL_INFO,
+ FROM_PROTOCOL_INFO,
+ FROM_PROTOCOL_INFO,
+ &src->wsainfo,
+ 0,
+ 0);
+ if (s == INVALID_SOCKET)
+ {
+ write_stderr("could not create inherited socket: error code %d\n",
+ WSAGetLastError());
+ exit(1);
+ }
+ *dest = s;
+
+ /*
+ * To make sure we don't get two references to the same socket, close
+ * the original one. (This would happen when inheritance actually
+ * works..
+ */
+ closesocket(src->origsocket);
+ }
+}
+#endif
+
+static void
+read_backend_variables(char *id, char **startup_data, size_t *startup_data_len)
+{
+ BackendParameters param;
+
+#ifndef WIN32
+ /* Non-win32 implementation reads from file */
+ FILE *fp;
+
+ /* Open file */
+ fp = AllocateFile(id, PG_BINARY_R);
+ if (!fp)
+ {
+ write_stderr("could not open backend variables file \"%s\": %s\n",
+ id, strerror(errno));
+ exit(1);
+ }
+
+ if (fread(¶m, sizeof(param), 1, fp) != 1)
+ {
+ write_stderr("could not read from backend variables file \"%s\": %s\n",
+ id, strerror(errno));
+ exit(1);
+ }
+
+ /* read startup data */
+ if (fread((char *) startup_data_len, sizeof(size_t), 1, fp) != 1)
+ {
+ write_stderr("could not read len from backend variables file \"%s\": %s\n",
+ id, strerror(errno));
+ exit(1);
+ }
+ if (*startup_data_len > 0)
+ {
+ *startup_data = palloc(*startup_data_len);
+ if (fread(*startup_data, *startup_data_len, 1, fp) != 1)
+ {
+ write_stderr("could not read startup data from backend variables file \"%s\": %s\n",
+ id, strerror(errno));
+ exit(1);
+ }
+ }
+ else
+ *startup_data = NULL;
+
+ /* Release file */
+ FreeFile(fp);
+ if (unlink(id) != 0)
+ {
+ write_stderr("could not remove file \"%s\": %s\n",
+ id, strerror(errno));
+ exit(1);
+ }
+#else
+ /* Win32 version uses mapped file */
+ HANDLE paramHandle;
+ BackendParameters *paramp;
+
+#ifdef _WIN64
+ paramHandle = (HANDLE) _atoi64(id);
+#else
+ paramHandle = (HANDLE) atol(id);
+#endif
+ paramp = MapViewOfFile(paramHandle, FILE_MAP_READ, 0, 0, 0);
+ if (!paramp)
+ {
+ write_stderr("could not map view of backend variables: error code %lu\n",
+ GetLastError());
+ exit(1);
+ }
+
+ memcpy(¶m, paramp, sizeof(BackendParameters));
+
+ /* XXX: read startup data */
+
+ if (!UnmapViewOfFile(paramp))
+ {
+ write_stderr("could not unmap view of backend variables: error code %lu\n",
+ GetLastError());
+ exit(1);
+ }
+
+ if (!CloseHandle(paramHandle))
+ {
+ write_stderr("could not close handle to backend parameter variables: error code %lu\n",
+ GetLastError());
+ exit(1);
+ }
+#endif
+
+ restore_backend_variables(¶m);
+}
+
+/* Restore critical backend variables from the BackendParameters struct */
+static void
+restore_backend_variables(BackendParameters *param)
+{
+ if (param->client_sock.sock != PGINVALID_SOCKET)
+ {
+ MyClientSocket = MemoryContextAlloc(TopMemoryContext, sizeof(ClientSocket));
+ memcpy(MyClientSocket, ¶m->client_sock, sizeof(ClientSocket));
+ read_inheritable_socket(&MyClientSocket->sock, ¶m->inh_sock);
+ }
+
+ SetDataDir(param->DataDir);
+
+ MyCancelKey = param->MyCancelKey;
+ MyPMChildSlot = param->MyPMChildSlot;
+
+#ifdef WIN32
+ ShmemProtectiveRegion = param->ShmemProtectiveRegion;
+#endif
+ UsedShmemSegID = param->UsedShmemSegID;
+ UsedShmemSegAddr = param->UsedShmemSegAddr;
+
+ ShmemLock = param->ShmemLock;
+ ShmemVariableCache = param->ShmemVariableCache;
+ ShmemBackendArray = param->ShmemBackendArray;
+
+#ifndef HAVE_SPINLOCKS
+ SpinlockSemaArray = param->SpinlockSemaArray;
+#endif
+ NamedLWLockTrancheRequests = param->NamedLWLockTrancheRequests;
+ NamedLWLockTrancheArray = param->NamedLWLockTrancheArray;
+ MainLWLockArray = param->MainLWLockArray;
+ ProcStructLock = param->ProcStructLock;
+ ProcGlobal = param->ProcGlobal;
+ AuxiliaryProcs = param->AuxiliaryProcs;
+ PreparedXactProcs = param->PreparedXactProcs;
+ PMSignalState = param->PMSignalState;
+
+ PostmasterPid = param->PostmasterPid;
+ PgStartTime = param->PgStartTime;
+ PgReloadTime = param->PgReloadTime;
+ first_syslogger_file_time = param->first_syslogger_file_time;
+
+ redirection_done = param->redirection_done;
+ IsBinaryUpgrade = param->IsBinaryUpgrade;
+ query_id_enabled = param->query_id_enabled;
+ max_safe_fds = param->max_safe_fds;
+
+ MaxBackends = param->MaxBackends;
+
+#ifdef WIN32
+ PostmasterHandle = param->PostmasterHandle;
+ pgwin32_initial_signal_pipe = param->initial_signal_pipe;
+#else
+ memcpy(&postmaster_alive_fds, ¶m->postmaster_alive_fds,
+ sizeof(postmaster_alive_fds));
+#endif
+
+ memcpy(&syslogPipe, ¶m->syslogPipe, sizeof(syslogPipe));
+
+ strlcpy(my_exec_path, param->my_exec_path, MAXPGPATH);
+
+ strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+
+ /*
+ * We need to restore fd.c's counts of externally-opened FDs; to avoid
+ * confusion, be sure to do this after restoring max_safe_fds. (Note:
+ * BackendInitialize will handle this for client_sock->sock.)
+ */
+#ifndef WIN32
+ if (postmaster_alive_fds[0] >= 0)
+ ReserveExternalFD();
+ if (postmaster_alive_fds[1] >= 0)
+ ReserveExternalFD();
+#endif
+}
+
+#endif /* EXEC_BACKEND */
diff --git a/src/backend/postmaster/meson.build b/src/backend/postmaster/meson.build
index cda921fd10b..89ff11beb0a 100644
--- a/src/backend/postmaster/meson.build
+++ b/src/backend/postmaster/meson.build
@@ -6,8 +6,8 @@ backend_sources += files(
'bgworker.c',
'bgwriter.c',
'checkpointer.c',
- 'fork_process.c',
'interrupt.c',
+ 'launch_backend.c',
'pgarch.c',
'postmaster.c',
'startup.c',
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 46af3495644..b82d66fab52 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -36,6 +36,7 @@
#include "lib/binaryheap.h"
#include "libpq/pqsignal.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/interrupt.h"
#include "postmaster/pgarch.h"
#include "storage/fd.h"
@@ -211,8 +212,14 @@ PgArchCanRestart(void)
/* Main entry point for archiver process */
void
-PgArchiverMain(void)
+PgArchiverMain(char *startup_data, size_t startup_data_len)
{
+ Assert(startup_data_len == 0);
+
+ MyAuxProcType = ArchiverProcess;
+ MyBackendType = B_ARCHIVER;
+ AuxiliaryProcessInit();
+
/*
* Ignore all signals usually bound to some action in the postmaster,
* except for SIGHUP, SIGTERM, SIGUSR1, SIGUSR2, and SIGQUIT.
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index b73ba983f3e..dc5cac19878 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -109,7 +109,6 @@
#include "postmaster/autovacuum.h"
#include "postmaster/auxprocess.h"
#include "postmaster/bgworker_internals.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
@@ -131,10 +130,6 @@
#include "utils/timestamp.h"
#include "utils/varlena.h"
-#ifdef EXEC_BACKEND
-#include "storage/spin.h"
-#endif
-
/*
* Possible types of a backend. Beyond being the possible bkend_type values in
@@ -187,7 +182,7 @@ typedef struct bkend
static dlist_head BackendList = DLIST_STATIC_INIT(BackendList);
#ifdef EXEC_BACKEND
-static Backend *ShmemBackendArray;
+Backend *ShmemBackendArray;
#endif
BackgroundWorker *MyBgworkerEntry = NULL;
@@ -412,7 +407,7 @@ static void StartupPacketTimeoutHandler(void);
static void CleanupBackend(int pid, int exitstatus);
static bool CleanupBackgroundWorker(int pid, int exitstatus);
static void HandleChildCrash(int pid, int exitstatus, const char *procname);
-static void LogChildExit(int lev, const char *procname,
+static void LogChildExit(int lev, const char *procnamBackendInitializee,
int pid, int exitstatus);
static void PostmasterStateMachine(void);
@@ -427,7 +422,6 @@ typedef enum CAC_state
} CAC_state;
static void BackendInitialize(ClientSocket *client_sock, CAC_state cac);
-static void BackendRun(void) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
static int BackendStartup(ClientSocket *port);
@@ -448,7 +442,7 @@ static int CountChildren(int target);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
-static pid_t StartChildProcess(AuxProcType type);
+static pid_t StartChildProcess(PostmasterChildType type);
static void StartAutovacuumWorker(void);
static void MaybeStartWalReceiver(void);
static void InitPostmasterDeathWatchHandle(void);
@@ -483,95 +477,19 @@ typedef struct
} win32_deadchild_waitinfo;
#endif /* WIN32 */
-static pid_t backend_forkexec(ClientSocket *client_sock, CAC_state cac);
-static pid_t internal_forkexec(int argc, char *argv[], ClientSocket *client_sock);
-
-/* Type for a socket that can be inherited to a client process */
-#ifdef WIN32
-typedef struct
-{
- SOCKET origsocket; /* Original socket value, or PGINVALID_SOCKET
- * if not a socket */
- WSAPROTOCOL_INFO wsainfo;
-} InheritableSocket;
-#else
-typedef int InheritableSocket;
-#endif
-
-/*
- * Structure contains all variables passed to exec:ed backends
- */
-typedef struct
-{
- ClientSocket client_sock;
- InheritableSocket serialized_sock;
- char DataDir[MAXPGPATH];
- int32 MyCancelKey;
- int MyPMChildSlot;
-#ifndef WIN32
- unsigned long UsedShmemSegID;
-#else
- void *ShmemProtectiveRegion;
- HANDLE UsedShmemSegID;
-#endif
- void *UsedShmemSegAddr;
- slock_t *ShmemLock;
- VariableCache ShmemVariableCache;
- Backend *ShmemBackendArray;
-#ifndef HAVE_SPINLOCKS
- PGSemaphore *SpinlockSemaArray;
-#endif
- int NamedLWLockTrancheRequests;
- NamedLWLockTranche *NamedLWLockTrancheArray;
- LWLockPadded *MainLWLockArray;
- slock_t *ProcStructLock;
- PROC_HDR *ProcGlobal;
- PGPROC *AuxiliaryProcs;
- PGPROC *PreparedXactProcs;
- PMSignalData *PMSignalState;
- pid_t PostmasterPid;
- TimestampTz PgStartTime;
- TimestampTz PgReloadTime;
- pg_time_t first_syslogger_file_time;
- bool redirection_done;
- bool IsBinaryUpgrade;
- bool query_id_enabled;
- int max_safe_fds;
- int MaxBackends;
-#ifdef WIN32
- HANDLE PostmasterHandle;
- HANDLE initial_signal_pipe;
- HANDLE syslogPipe[2];
-#else
- int postmaster_alive_fds[2];
- int syslogPipe[2];
-#endif
- char my_exec_path[MAXPGPATH];
- char pkglib_path[MAXPGPATH];
-
- BackgroundWorker MyBgworkerEntry;
-} BackendParameters;
-
-static void read_backend_variables(char *id, ClientSocket *client_sock);
-static void restore_backend_variables(BackendParameters *param, ClientSocket *client_sock);
-
-#ifndef WIN32
-static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock);
-#else
-static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock,
- HANDLE childProcess, pid_t childPid);
-#endif
-
static void ShmemBackendArrayAdd(Backend *bn);
static void ShmemBackendArrayRemove(Backend *bn);
#endif /* EXEC_BACKEND */
-#define StartupDataBase() StartChildProcess(StartupProcess)
-#define StartArchiver() StartChildProcess(ArchiverProcess)
-#define StartBackgroundWriter() StartChildProcess(BgWriterProcess)
-#define StartCheckpointer() StartChildProcess(CheckpointerProcess)
-#define StartWalWriter() StartChildProcess(WalWriterProcess)
-#define StartWalReceiver() StartChildProcess(WalReceiverProcess)
+#define StartupDataBase() StartChildProcess(PMC_STARTUP)
+#define StartArchiver() StartChildProcess(PMC_ARCHIVER)
+#define StartBackgroundWriter() StartChildProcess(PMC_BGWRITER)
+#define StartCheckpointer() StartChildProcess(PMC_CHECKPOINTER)
+#define StartWalWriter() StartChildProcess(PMC_WAL_WRITER)
+#define StartWalReceiver() StartChildProcess(PMC_WAL_RECEIVER)
+
+#define StartAutoVacLauncher() StartChildProcess(PMC_AV_LAUNCHER);
+#define StartAutoVacWorker() StartChildProcess(PMC_AV_WORKER);
/* Macros to check exit status of a child process */
#define EXIT_STATUS_0(st) ((st) == 0)
@@ -1115,11 +1033,11 @@ PostmasterMain(int argc, char *argv[])
/*
* Clean out the temp directory used to transmit parameters to child
- * processes (see internal_forkexec, below). We must do this before
- * launching any child processes, else we have a race condition: we could
- * remove a parameter file before the child can read it. It should be
- * safe to do so now, because we verified earlier that there are no
- * conflicting Postgres processes in this data directory.
+ * processes (see internal_forkexec). We must do this before launching
+ * any child processes, else we have a race condition: we could remove a
+ * parameter file before the child can read it. It should be safe to do
+ * so now, because we verified earlier that there are no conflicting
+ * Postgres processes in this data directory.
*/
RemovePgTempFilesInDir(PG_TEMP_FILES_DIR, true, false);
#endif
@@ -4029,6 +3947,11 @@ TerminateChildren(int signal)
signal_child(PgArchPID, signal);
}
+typedef struct BackendStartupInfo
+{
+ bool canAcceptConnections;
+} BackendStartupInfo;
+
/*
* BackendStartup -- start backend process
*
@@ -4041,7 +3964,7 @@ BackendStartup(ClientSocket *client_sock)
{
Backend *bn; /* for backend cleanup */
pid_t pid;
- CAC_state cac;
+ BackendStartupInfo info;
/*
* Create backend data structure. Better before the fork() so we can
@@ -4070,11 +3993,9 @@ BackendStartup(ClientSocket *client_sock)
return STATUS_ERROR;
}
- bn->cancel_key = MyCancelKey;
-
/* Pass down canAcceptConnections state */
- cac = canAcceptConnections(BACKEND_TYPE_NORMAL);
- bn->dead_end = (cac != CAC_OK);
+ info.canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
+ bn->cancel_key = MyCancelKey;
/*
* Unless it's a dead_end child, assign it a child slot number
@@ -4087,26 +4008,7 @@ BackendStartup(ClientSocket *client_sock)
/* Hasn't asked to be notified about any bgworkers yet */
bn->bgworker_notify = false;
-#ifdef EXEC_BACKEND
- pid = backend_forkexec(client_sock, cac);
-#else /* !EXEC_BACKEND */
- pid = fork_process();
- if (pid == 0) /* child */
- {
- /* Detangle from postmaster */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Perform additional initialization and collect startup packet */
- BackendInitialize(client_sock, cac);
-
- /* And run the backend */
- BackendRun();
- }
-#endif /* EXEC_BACKEND */
-
+ pid = postmaster_child_launch(PMC_BACKEND, (char *) &info, sizeof(info), client_sock);
if (pid < 0)
{
/* in parent, fork failed */
@@ -4143,6 +4045,63 @@ BackendStartup(ClientSocket *client_sock)
return STATUS_OK;
}
+void
+BackendMain(char *startup_data, size_t startup_data_len)
+{
+ BackendStartupInfo *info = (BackendStartupInfo *) startup_data;
+
+ Assert(startup_data_len == sizeof(BackendStartupInfo));
+ Assert(MyClientSocket != NULL);
+
+#ifdef EXEC_BACKEND
+
+ /*
+ * Need to reinitialize the SSL library in the backend, since the context
+ * structures contain function pointers and cannot be passed through the
+ * parameter file.
+ *
+ * If for some reason reload fails (maybe the user installed broken key
+ * files), soldier on without SSL; that's better than all connections
+ * becoming impossible.
+ *
+ * XXX should we do this in all child processes? For the moment it's
+ * enough to do it in backend children.
+ */
+#ifdef USE_SSL
+ if (EnableSSL)
+ {
+ if (secure_initialize(false) == 0)
+ LoadedSSL = true;
+ else
+ ereport(LOG,
+ (errmsg("SSL configuration could not be loaded in child process")));
+ }
+#endif
+#endif
+
+ /* Perform additional initialization and collect startup packet */
+ BackendInitialize(MyClientSocket, info->canAcceptConnections);
+
+ /*
+ * Create a per-backend PGPROC struct in shared memory. We must do this
+ * before we can use LWLocks. In the !EXEC_BACKEND case (here) this could
+ * be delayed a bit further, but EXEC_BACKEND needs to do stuff with
+ * LWLocks before PostgresMain(), so we do it here as well for symmetry.
+ */
+ InitProcess();
+
+ /* Attach process to shared data structures */
+ AttachSharedMemoryAndSemaphores();
+
+ /*
+ * Make sure we aren't in PostmasterContext anymore. (We can't delete it
+ * just yet, though, because InitPostgres will need the HBA data.)
+ */
+ MemoryContextSwitchTo(TopMemoryContext);
+
+ PostgresMain(MyProcPort->database_name, MyProcPort->user_name);
+}
+
/*
* Try to report backend fork() failure to client before we close the
* connection. Since we do not care to risk blocking the postmaster on
@@ -4414,732 +4373,161 @@ BackendInitialize(ClientSocket *client_sock, CAC_state cac)
/*
- * BackendRun -- set up the backend's argument list and invoke PostgresMain()
+ * ExitPostmaster -- cleanup
*
- * returns:
- * Doesn't return at all.
+ * Do NOT call exit() directly --- always go through here!
*/
static void
-BackendRun(void)
+ExitPostmaster(int status)
{
+#ifdef HAVE_PTHREAD_IS_THREADED_NP
+
/*
- * Create a per-backend PGPROC struct in shared memory. We must do this
- * before we can use LWLocks (in AttachSharedMemoryAndSemaphores).
+ * There is no known cause for a postmaster to become multithreaded after
+ * startup. Recheck to account for the possibility of unknown causes.
+ * This message uses LOG level, because an unclean shutdown at this point
+ * would usually not look much different from a clean shutdown.
*/
- InitProcess();
+ if (pthread_is_threaded_np() != 0)
+ ereport(LOG,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg_internal("postmaster became multithreaded"),
+ errdetail("Please report this to <%s>.", PACKAGE_BUGREPORT)));
+#endif
- /* Attach process to shared data structures */
- AttachSharedMemoryAndSemaphores();
+ /* should cleanup shared memory and kill all backends */
/*
- * Make sure we aren't in PostmasterContext anymore. (We can't delete it
- * just yet, though, because InitPostgres will need the HBA data.)
+ * Not sure of the semantics here. When the Postmaster dies, should the
+ * backends all be killed? probably not.
+ *
+ * MUST -- vadim 05-10-1999
*/
- MemoryContextSwitchTo(TopMemoryContext);
-
- PostgresMain(MyProcPort->database_name, MyProcPort->user_name);
-}
-
-
-#ifdef EXEC_BACKEND
-
-/*
- * postmaster_forkexec -- fork and exec a postmaster subprocess
- *
- * The caller must have set up the argv array already, except for argv[2]
- * which will be filled with the name of the temp variable file.
- *
- * Returns the child process PID, or -1 on fork failure (a suitable error
- * message has been logged on failure).
- *
- * All uses of this routine will dispatch to SubPostmasterMain in the
- * child process.
- */
-pid_t
-postmaster_forkexec(int argc, char *argv[])
-{
- ClientSocket client_sock;
-
- /* This entry point doesn't pass a client socket */
- memset(&client_sock, 0, sizeof(ClientSocket));
- return internal_forkexec(argc, argv, &client_sock);
-}
-
-/*
- * backend_forkexec -- fork/exec off a backend process
- *
- * Some operating systems (WIN32) don't have fork() so we have to simulate
- * it by storing parameters that need to be passed to the child and
- * then create a new child process.
- *
- * returns the pid of the fork/exec'd process, or -1 on failure
- */
-static pid_t
-backend_forkexec(ClientSocket *client_sock, CAC_state cac)
-{
- char *av[5];
- int ac = 0;
- char cacbuf[10];
- av[ac++] = "postgres";
- av[ac++] = "--forkbackend";
- av[ac++] = NULL; /* filled in by internal_forkexec */
-
- snprintf(cacbuf, sizeof(cacbuf), "%d", (int) cac);
- av[ac++] = cacbuf;
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return internal_forkexec(ac, av, client_sock);
+ proc_exit(status);
}
-#ifndef WIN32
-
/*
- * internal_forkexec non-win32 implementation
- *
- * - writes out backend variables to the parameter file
- * - fork():s, and then exec():s the child process
+ * Handle pmsignal conditions representing requests from backends,
+ * and check for promote and logrotate requests from pg_ctl.
*/
-static pid_t
-internal_forkexec(int argc, char *argv[], ClientSocket *client_sock)
+static void
+process_pm_pmsignal(void)
{
- static unsigned long tmpBackendFileNum = 0;
- pid_t pid;
- char tmpfilename[MAXPGPATH];
- BackendParameters param;
- FILE *fp;
-
- if (!save_backend_variables(¶m, client_sock))
- return -1; /* log made by save_backend_variables */
+ pending_pm_pmsignal = false;
- /* Calculate name for temp file */
- snprintf(tmpfilename, MAXPGPATH, "%s/%s.backend_var.%d.%lu",
- PG_TEMP_FILES_DIR, PG_TEMP_FILE_PREFIX,
- MyProcPid, ++tmpBackendFileNum);
+ ereport(DEBUG2,
+ (errmsg_internal("postmaster received pmsignal signal")));
- /* Open file */
- fp = AllocateFile(tmpfilename, PG_BINARY_W);
- if (!fp)
+ /*
+ * RECOVERY_STARTED and BEGIN_HOT_STANDBY signals are ignored in
+ * unexpected states. If the startup process quickly starts up, completes
+ * recovery, exits, we might process the death of the startup process
+ * first. We don't want to go back to recovery in that case.
+ */
+ if (CheckPostmasterSignal(PMSIGNAL_RECOVERY_STARTED) &&
+ pmState == PM_STARTUP && Shutdown == NoShutdown)
{
+ /* WAL redo has started. We're out of reinitialization. */
+ FatalError = false;
+ AbortStartTime = 0;
+
/*
- * As in OpenTemporaryFileInTablespace, try to make the temp-file
- * directory, ignoring errors.
+ * Start the archiver if we're responsible for (re-)archiving received
+ * files.
*/
- (void) MakePGDirectory(PG_TEMP_FILES_DIR);
+ Assert(PgArchPID == 0);
+ if (XLogArchivingAlways())
+ PgArchPID = StartArchiver();
- fp = AllocateFile(tmpfilename, PG_BINARY_W);
- if (!fp)
+ /*
+ * If we aren't planning to enter hot standby mode later, treat
+ * RECOVERY_STARTED as meaning we're out of startup, and report status
+ * accordingly.
+ */
+ if (!EnableHotStandby)
{
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not create file \"%s\": %m",
- tmpfilename)));
- return -1;
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+#ifdef USE_SYSTEMD
+ sd_notify(0, "READY=1");
+#endif
}
+
+ pmState = PM_RECOVERY;
}
- if (fwrite(¶m, sizeof(param), 1, fp) != 1)
+ if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
+ pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not write to file \"%s\": %m", tmpfilename)));
- FreeFile(fp);
- return -1;
+ (errmsg("database system is ready to accept read-only connections")));
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_READY);
+#ifdef USE_SYSTEMD
+ sd_notify(0, "READY=1");
+#endif
+
+ pmState = PM_HOT_STANDBY;
+ connsAllowed = true;
+
+ /* Some workers may be scheduled to start now */
+ StartWorkerNeeded = true;
}
- /* Release file */
- if (FreeFile(fp))
+ /* Process background worker state changes. */
+ if (CheckPostmasterSignal(PMSIGNAL_BACKGROUND_WORKER_CHANGE))
{
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not write to file \"%s\": %m", tmpfilename)));
- return -1;
+ /* Accept new worker requests only if not stopping. */
+ BackgroundWorkerStateChange(pmState < PM_STOP_BACKENDS);
+ StartWorkerNeeded = true;
}
- /* Make sure caller set up argv properly */
- Assert(argc >= 3);
- Assert(argv[argc] == NULL);
- Assert(strncmp(argv[1], "--fork", 6) == 0);
- Assert(argv[2] == NULL);
-
- /* Insert temp file name after --fork argument */
- argv[2] = tmpfilename;
+ if (StartWorkerNeeded || HaveCrashedWorker)
+ maybe_start_bgworkers();
- /* Fire off execv in child */
- if ((pid = fork_process()) == 0)
+ /* Tell syslogger to rotate logfile if requested */
+ if (SysLoggerPID != 0)
{
- if (execv(postgres_exec_path, argv) < 0)
+ if (CheckLogrotateSignal())
{
- ereport(LOG,
- (errmsg("could not execute server process \"%s\": %m",
- postgres_exec_path)));
- /* We're already in the child process here, can't return */
- exit(1);
+ signal_child(SysLoggerPID, SIGUSR1);
+ RemoveLogrotateSignalFiles();
+ }
+ else if (CheckPostmasterSignal(PMSIGNAL_ROTATE_LOGFILE))
+ {
+ signal_child(SysLoggerPID, SIGUSR1);
}
}
- return pid; /* Parent returns pid, or -1 on fork failure */
-}
-#else /* WIN32 */
-
-/*
- * internal_forkexec win32 implementation
- *
- * - starts backend using CreateProcess(), in suspended state
- * - writes out backend variables to the parameter file
- * - during this, duplicates handles and sockets required for
- * inheritance into the new process
- * - resumes execution of the new process once the backend parameter
- * file is complete.
- */
-static pid_t
-internal_forkexec(int argc, char *argv[], Port *port)
-{
- int retry_count = 0;
- STARTUPINFO si;
- PROCESS_INFORMATION pi;
- int i;
- int j;
- char cmdLine[MAXPGPATH * 2];
- HANDLE paramHandle;
- BackendParameters *param;
- SECURITY_ATTRIBUTES sa;
- char paramHandleStr[32];
- win32_deadchild_waitinfo *childinfo;
-
- /* Make sure caller set up argv properly */
- Assert(argc >= 3);
- Assert(argv[argc] == NULL);
- Assert(strncmp(argv[1], "--fork", 6) == 0);
- Assert(argv[2] == NULL);
-
- /* Resume here if we need to retry */
-retry:
-
- /* Set up shared memory for parameter passing */
- ZeroMemory(&sa, sizeof(sa));
- sa.nLength = sizeof(sa);
- sa.bInheritHandle = TRUE;
- paramHandle = CreateFileMapping(INVALID_HANDLE_VALUE,
- &sa,
- PAGE_READWRITE,
- 0,
- sizeof(BackendParameters),
- NULL);
- if (paramHandle == INVALID_HANDLE_VALUE)
+ if (CheckPostmasterSignal(PMSIGNAL_START_AUTOVAC_LAUNCHER) &&
+ Shutdown <= SmartShutdown && pmState < PM_STOP_BACKENDS)
{
- ereport(LOG,
- (errmsg("could not create backend parameter file mapping: error code %lu",
- GetLastError())));
- return -1;
+ /*
+ * Start one iteration of the autovacuum daemon, even if autovacuuming
+ * is nominally not enabled. This is so we can have an active defense
+ * against transaction ID wraparound. We set a flag for the main loop
+ * to do it rather than trying to do it here --- this is because the
+ * autovac process itself may send the signal, and we want to handle
+ * that by launching another iteration as soon as the current one
+ * completes.
+ */
+ start_autovac_launcher = true;
}
- param = MapViewOfFile(paramHandle, FILE_MAP_WRITE, 0, 0, sizeof(BackendParameters));
- if (!param)
+ if (CheckPostmasterSignal(PMSIGNAL_START_AUTOVAC_WORKER) &&
+ Shutdown <= SmartShutdown && pmState < PM_STOP_BACKENDS)
{
- ereport(LOG,
- (errmsg("could not map backend parameter memory: error code %lu",
- GetLastError())));
- CloseHandle(paramHandle);
- return -1;
+ /* The autovacuum launcher wants us to start a worker process. */
+ StartAutovacuumWorker();
}
- /* Insert temp file name after --fork argument */
-#ifdef _WIN64
- sprintf(paramHandleStr, "%llu", (LONG_PTR) paramHandle);
-#else
- sprintf(paramHandleStr, "%lu", (DWORD) paramHandle);
-#endif
- argv[2] = paramHandleStr;
-
- /* Format the cmd line */
- cmdLine[sizeof(cmdLine) - 1] = '\0';
- cmdLine[sizeof(cmdLine) - 2] = '\0';
- snprintf(cmdLine, sizeof(cmdLine) - 1, "\"%s\"", postgres_exec_path);
- i = 0;
- while (argv[++i] != NULL)
- {
- j = strlen(cmdLine);
- snprintf(cmdLine + j, sizeof(cmdLine) - 1 - j, " \"%s\"", argv[i]);
- }
- if (cmdLine[sizeof(cmdLine) - 2] != '\0')
+ if (CheckPostmasterSignal(PMSIGNAL_START_WALRECEIVER))
{
- ereport(LOG,
- (errmsg("subprocess command line too long")));
- UnmapViewOfFile(param);
- CloseHandle(paramHandle);
- return -1;
- }
-
- memset(&pi, 0, sizeof(pi));
- memset(&si, 0, sizeof(si));
- si.cb = sizeof(si);
-
- /*
- * Create the subprocess in a suspended state. This will be resumed later,
- * once we have written out the parameter file.
- */
- if (!CreateProcess(NULL, cmdLine, NULL, NULL, TRUE, CREATE_SUSPENDED,
- NULL, NULL, &si, &pi))
- {
- ereport(LOG,
- (errmsg("CreateProcess() call failed: %m (error code %lu)",
- GetLastError())));
- UnmapViewOfFile(param);
- CloseHandle(paramHandle);
- return -1;
- }
-
- if (!save_backend_variables(param, port, pi.hProcess, pi.dwProcessId))
- {
- /*
- * log made by save_backend_variables, but we have to clean up the
- * mess with the half-started process
- */
- if (!TerminateProcess(pi.hProcess, 255))
- ereport(LOG,
- (errmsg_internal("could not terminate unstarted process: error code %lu",
- GetLastError())));
- CloseHandle(pi.hProcess);
- CloseHandle(pi.hThread);
- UnmapViewOfFile(param);
- CloseHandle(paramHandle);
- return -1; /* log made by save_backend_variables */
- }
-
- /* Drop the parameter shared memory that is now inherited to the backend */
- if (!UnmapViewOfFile(param))
- ereport(LOG,
- (errmsg("could not unmap view of backend parameter file: error code %lu",
- GetLastError())));
- if (!CloseHandle(paramHandle))
- ereport(LOG,
- (errmsg("could not close handle to backend parameter file: error code %lu",
- GetLastError())));
-
- /*
- * Reserve the memory region used by our main shared memory segment before
- * we resume the child process. Normally this should succeed, but if ASLR
- * is active then it might sometimes fail due to the stack or heap having
- * gotten mapped into that range. In that case, just terminate the
- * process and retry.
- */
- if (!pgwin32_ReserveSharedMemoryRegion(pi.hProcess))
- {
- /* pgwin32_ReserveSharedMemoryRegion already made a log entry */
- if (!TerminateProcess(pi.hProcess, 255))
- ereport(LOG,
- (errmsg_internal("could not terminate process that failed to reserve memory: error code %lu",
- GetLastError())));
- CloseHandle(pi.hProcess);
- CloseHandle(pi.hThread);
- if (++retry_count < 100)
- goto retry;
- ereport(LOG,
- (errmsg("giving up after too many tries to reserve shared memory"),
- errhint("This might be caused by ASLR or antivirus software.")));
- return -1;
- }
-
- /*
- * Now that the backend variables are written out, we start the child
- * thread so it can start initializing while we set up the rest of the
- * parent state.
- */
- if (ResumeThread(pi.hThread) == -1)
- {
- if (!TerminateProcess(pi.hProcess, 255))
- {
- ereport(LOG,
- (errmsg_internal("could not terminate unstartable process: error code %lu",
- GetLastError())));
- CloseHandle(pi.hProcess);
- CloseHandle(pi.hThread);
- return -1;
- }
- CloseHandle(pi.hProcess);
- CloseHandle(pi.hThread);
- ereport(LOG,
- (errmsg_internal("could not resume thread of unstarted process: error code %lu",
- GetLastError())));
- return -1;
- }
-
- /*
- * Queue a waiter to signal when this child dies. The wait will be handled
- * automatically by an operating system thread pool. The memory will be
- * freed by a later call to waitpid().
- */
- childinfo = palloc(sizeof(win32_deadchild_waitinfo));
- childinfo->procHandle = pi.hProcess;
- childinfo->procId = pi.dwProcessId;
-
- if (!RegisterWaitForSingleObject(&childinfo->waitHandle,
- pi.hProcess,
- pgwin32_deadchild_callback,
- childinfo,
- INFINITE,
- WT_EXECUTEONLYONCE | WT_EXECUTEINWAITTHREAD))
- ereport(FATAL,
- (errmsg_internal("could not register process for wait: error code %lu",
- GetLastError())));
-
- /* Don't close pi.hProcess here - waitpid() needs access to it */
-
- CloseHandle(pi.hThread);
-
- return pi.dwProcessId;
-}
-#endif /* WIN32 */
-
-
-/*
- * SubPostmasterMain -- Get the fork/exec'd process into a state equivalent
- * to what it would be if we'd simply forked on Unix, and then
- * dispatch to the appropriate place.
- *
- * The first two command line arguments are expected to be "--forkFOO"
- * (where FOO indicates which postmaster child we are to become), and
- * the name of a variables file that we can read to load data that would
- * have been inherited by fork() on Unix. Remaining arguments go to the
- * subprocess FooMain() routine.
- */
-void
-SubPostmasterMain(int argc, char *argv[])
-{
- ClientSocket client_sock;
-
- /* In EXEC_BACKEND case we will not have inherited these settings */
- IsPostmasterEnvironment = true;
- whereToSendOutput = DestNone;
-
- /* Setup essential subsystems (to ensure elog() behaves sanely) */
- InitializeGUCOptions();
-
- /* Check we got appropriate args */
- if (argc < 3)
- elog(FATAL, "invalid subpostmaster invocation");
-
- /* Read in the variables file */
- memset(&client_sock, 0, sizeof(ClientSocket));
- read_backend_variables(argv[2], &client_sock);
-
- /* Close the postmaster's sockets (as soon as we know them) */
- ClosePostmasterPorts(strcmp(argv[1], "--forklog") == 0);
-
- /* Setup as postmaster child */
- InitPostmasterChild();
-
- /*
- * If appropriate, physically re-attach to shared memory segment. We want
- * to do this before going any further to ensure that we can attach at the
- * same address the postmaster used. On the other hand, if we choose not
- * to re-attach, we may have other cleanup to do.
- *
- * If testing EXEC_BACKEND on Linux, you should run this as root before
- * starting the postmaster:
- *
- * sysctl -w kernel.randomize_va_space=0
- *
- * This prevents using randomized stack and code addresses that cause the
- * child process's memory map to be different from the parent's, making it
- * sometimes impossible to attach to shared memory at the desired address.
- * Return the setting to its old value (usually '1' or '2') when finished.
- */
- if (strcmp(argv[1], "--forkbackend") == 0 ||
- strcmp(argv[1], "--forkavlauncher") == 0 ||
- strcmp(argv[1], "--forkavworker") == 0 ||
- strcmp(argv[1], "--forkaux") == 0 ||
- strncmp(argv[1], "--forkbgworker", 14) == 0)
- PGSharedMemoryReAttach();
- else
- PGSharedMemoryNoReAttach();
-
- /* autovacuum needs this set before calling InitProcess */
- if (strcmp(argv[1], "--forkavlauncher") == 0)
- AutovacuumLauncherIAm();
- if (strcmp(argv[1], "--forkavworker") == 0)
- AutovacuumWorkerIAm();
-
- /* Read in remaining GUC variables */
- read_nondefault_variables();
-
- /*
- * Check that the data directory looks valid, which will also check the
- * privileges on the data directory and update our umask and file/group
- * variables for creating files later. Note: this should really be done
- * before we create any files or directories.
- */
- checkDataDir();
-
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
- /*
- * Reload any libraries that were preloaded by the postmaster. Since we
- * exec'd this process, those libraries didn't come along with us; but we
- * should load them into all child processes to be consistent with the
- * non-EXEC_BACKEND behavior.
- */
- process_shared_preload_libraries();
-
- /* Run backend or appropriate child */
- if (strcmp(argv[1], "--forkbackend") == 0)
- {
- CAC_state cac;
-
- Assert(argc == 4);
- cac = (CAC_state) atoi(argv[3]);
-
- /*
- * Need to reinitialize the SSL library in the backend, since the
- * context structures contain function pointers and cannot be passed
- * through the parameter file.
- *
- * If for some reason reload fails (maybe the user installed broken
- * key files), soldier on without SSL; that's better than all
- * connections becoming impossible.
- *
- * XXX should we do this in all child processes? For the moment it's
- * enough to do it in backend children.
- */
-#ifdef USE_SSL
- if (EnableSSL)
- {
- if (secure_initialize(false) == 0)
- LoadedSSL = true;
- else
- ereport(LOG,
- (errmsg("SSL configuration could not be loaded in child process")));
- }
-#endif
-
- /*
- * Perform additional initialization and collect startup packet.
- *
- * We want to do this before InitProcess() for a couple of reasons: 1.
- * so that we aren't eating up a PGPROC slot while waiting on the
- * client. 2. so that if InitProcess() fails due to being out of
- * PGPROC slots, we have already initialized libpq and are able to
- * report the error to the client.
- */
- BackendInitialize(&client_sock, cac);
-
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* And run the backend */
- BackendRun(); /* does not return */
- }
- if (strcmp(argv[1], "--forkaux") == 0)
- {
- AuxProcType auxtype;
-
- Assert(argc == 4);
-
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- auxtype = atoi(argv[3]);
- AuxiliaryProcessMain(auxtype); /* does not return */
- }
- if (strcmp(argv[1], "--forkavlauncher") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- AutoVacLauncherMain(argc - 2, argv + 2); /* does not return */
- }
- if (strcmp(argv[1], "--forkavworker") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
- }
- if (strncmp(argv[1], "--forkbgworker", 14) == 0)
- {
- /* do this as early as possible; in particular, before InitProcess() */
- IsBackgroundWorker = true;
-
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- StartBackgroundWorker();
- }
- if (strcmp(argv[1], "--forklog") == 0)
- {
- /* Do not want to attach to shared memory */
-
- SysLoggerMain(argc, argv); /* does not return */
- }
-
- abort(); /* shouldn't get here */
-}
-#endif /* EXEC_BACKEND */
-
-
-/*
- * ExitPostmaster -- cleanup
- *
- * Do NOT call exit() directly --- always go through here!
- */
-static void
-ExitPostmaster(int status)
-{
-#ifdef HAVE_PTHREAD_IS_THREADED_NP
-
- /*
- * There is no known cause for a postmaster to become multithreaded after
- * startup. Recheck to account for the possibility of unknown causes.
- * This message uses LOG level, because an unclean shutdown at this point
- * would usually not look much different from a clean shutdown.
- */
- if (pthread_is_threaded_np() != 0)
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg_internal("postmaster became multithreaded"),
- errdetail("Please report this to <%s>.", PACKAGE_BUGREPORT)));
-#endif
-
- /* should cleanup shared memory and kill all backends */
-
- /*
- * Not sure of the semantics here. When the Postmaster dies, should the
- * backends all be killed? probably not.
- *
- * MUST -- vadim 05-10-1999
- */
-
- proc_exit(status);
-}
-
-/*
- * Handle pmsignal conditions representing requests from backends,
- * and check for promote and logrotate requests from pg_ctl.
- */
-static void
-process_pm_pmsignal(void)
-{
- pending_pm_pmsignal = false;
-
- ereport(DEBUG2,
- (errmsg_internal("postmaster received pmsignal signal")));
-
- /*
- * RECOVERY_STARTED and BEGIN_HOT_STANDBY signals are ignored in
- * unexpected states. If the startup process quickly starts up, completes
- * recovery, exits, we might process the death of the startup process
- * first. We don't want to go back to recovery in that case.
- */
- if (CheckPostmasterSignal(PMSIGNAL_RECOVERY_STARTED) &&
- pmState == PM_STARTUP && Shutdown == NoShutdown)
- {
- /* WAL redo has started. We're out of reinitialization. */
- FatalError = false;
- AbortStartTime = 0;
-
- /*
- * Start the archiver if we're responsible for (re-)archiving received
- * files.
- */
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
- PgArchPID = StartArchiver();
-
- /*
- * If we aren't planning to enter hot standby mode later, treat
- * RECOVERY_STARTED as meaning we're out of startup, and report status
- * accordingly.
- */
- if (!EnableHotStandby)
- {
- AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
-#ifdef USE_SYSTEMD
- sd_notify(0, "READY=1");
-#endif
- }
-
- pmState = PM_RECOVERY;
- }
-
- if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
- pmState == PM_RECOVERY && Shutdown == NoShutdown)
- {
- ereport(LOG,
- (errmsg("database system is ready to accept read-only connections")));
-
- /* Report status */
- AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_READY);
-#ifdef USE_SYSTEMD
- sd_notify(0, "READY=1");
-#endif
-
- pmState = PM_HOT_STANDBY;
- connsAllowed = true;
-
- /* Some workers may be scheduled to start now */
- StartWorkerNeeded = true;
- }
-
- /* Process background worker state changes. */
- if (CheckPostmasterSignal(PMSIGNAL_BACKGROUND_WORKER_CHANGE))
- {
- /* Accept new worker requests only if not stopping. */
- BackgroundWorkerStateChange(pmState < PM_STOP_BACKENDS);
- StartWorkerNeeded = true;
- }
-
- if (StartWorkerNeeded || HaveCrashedWorker)
- maybe_start_bgworkers();
-
- /* Tell syslogger to rotate logfile if requested */
- if (SysLoggerPID != 0)
- {
- if (CheckLogrotateSignal())
- {
- signal_child(SysLoggerPID, SIGUSR1);
- RemoveLogrotateSignalFiles();
- }
- else if (CheckPostmasterSignal(PMSIGNAL_ROTATE_LOGFILE))
- {
- signal_child(SysLoggerPID, SIGUSR1);
- }
- }
-
- if (CheckPostmasterSignal(PMSIGNAL_START_AUTOVAC_LAUNCHER) &&
- Shutdown <= SmartShutdown && pmState < PM_STOP_BACKENDS)
- {
- /*
- * Start one iteration of the autovacuum daemon, even if autovacuuming
- * is nominally not enabled. This is so we can have an active defense
- * against transaction ID wraparound. We set a flag for the main loop
- * to do it rather than trying to do it here --- this is because the
- * autovac process itself may send the signal, and we want to handle
- * that by launching another iteration as soon as the current one
- * completes.
- */
- start_autovac_launcher = true;
- }
-
- if (CheckPostmasterSignal(PMSIGNAL_START_AUTOVAC_WORKER) &&
- Shutdown <= SmartShutdown && pmState < PM_STOP_BACKENDS)
- {
- /* The autovacuum launcher wants us to start a worker process. */
- StartAutovacuumWorker();
- }
-
- if (CheckPostmasterSignal(PMSIGNAL_START_WALRECEIVER))
- {
- /* Startup Process wants us to start the walreceiver process. */
- /* Start immediately if possible, else remember request for later. */
- WalReceiverRequested = true;
- MaybeStartWalReceiver();
+ /* Startup Process wants us to start the walreceiver process. */
+ /* Start immediately if possible, else remember request for later. */
+ WalReceiverRequested = true;
+ MaybeStartWalReceiver();
}
/*
@@ -5276,93 +4664,23 @@ CountChildren(int target)
* to start subprocess.
*/
static pid_t
-StartChildProcess(AuxProcType type)
+StartChildProcess(PostmasterChildType type)
{
pid_t pid;
-#ifdef EXEC_BACKEND
- {
- char *av[10];
- int ac = 0;
- char typebuf[32];
-
- /*
- * Set up command-line arguments for subprocess
- */
- av[ac++] = "postgres";
- av[ac++] = "--forkaux";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- snprintf(typebuf, sizeof(typebuf), "%d", type);
- av[ac++] = typebuf;
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- pid = postmaster_forkexec(ac, av);
- }
-#else /* !EXEC_BACKEND */
- pid = fork_process();
-
- if (pid == 0) /* child */
- {
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Release postmaster's working memory context */
- MemoryContextSwitchTo(TopMemoryContext);
- MemoryContextDelete(PostmasterContext);
- PostmasterContext = NULL;
-
- AuxiliaryProcessMain(type); /* does not return */
- }
-#endif /* EXEC_BACKEND */
-
+ pid = postmaster_child_launch(type, NULL, 0, NULL);
if (pid < 0)
{
/* in parent, fork failed */
- int save_errno = errno;
-
- errno = save_errno;
- switch (type)
- {
- case StartupProcess:
- ereport(LOG,
- (errmsg("could not fork startup process: %m")));
- break;
- case ArchiverProcess:
- ereport(LOG,
- (errmsg("could not fork archiver process: %m")));
- break;
- case BgWriterProcess:
- ereport(LOG,
- (errmsg("could not fork background writer process: %m")));
- break;
- case CheckpointerProcess:
- ereport(LOG,
- (errmsg("could not fork checkpointer process: %m")));
- break;
- case WalWriterProcess:
- ereport(LOG,
- (errmsg("could not fork WAL writer process: %m")));
- break;
- case WalReceiverProcess:
- ereport(LOG,
- (errmsg("could not fork WAL receiver process: %m")));
- break;
- default:
- ereport(LOG,
- (errmsg("could not fork process: %m")));
- break;
- }
+ /* XXX: translation? */
+ ereport(LOG,
+ (errmsg("could not fork %s process: %m", PostmasterChildName(type))));
/*
* fork failure is fatal during startup, but there's no need to choke
* immediately if starting other child types fails.
*/
- if (type == StartupProcess)
+ if (type == PMC_STARTUP)
ExitPostmaster(1);
return 0;
}
@@ -5612,32 +4930,6 @@ BackgroundWorkerUnblockSignals(void)
sigprocmask(SIG_SETMASK, &UnBlockSig, NULL);
}
-#ifdef EXEC_BACKEND
-static pid_t
-bgworker_forkexec(BackgroundWorker *worker)
-{
- char *av[10];
- int ac = 0;
- char forkav[MAXPGPATH];
- pid_t result;
-
- snprintf(forkav, MAXPGPATH, "--forkbgworker");
-
- av[ac++] = "postgres";
- av[ac++] = forkav;
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- MyBgworkerEntry = worker;
- result = postmaster_forkexec(ac, av);
- MyBgworkerEntry = NULL;
-
- return result;
-}
-#endif
-
/*
* Start a new bgworker.
* Starting time conditions must have been checked already.
@@ -5674,65 +4966,32 @@ do_start_bgworker(RegisteredBgWorker *rw)
(errmsg_internal("starting background worker process \"%s\"",
rw->rw_worker.bgw_name)));
-#ifdef EXEC_BACKEND
- switch ((worker_pid = bgworker_forkexec(&rw->rw_worker)))
-#else
- switch ((worker_pid = fork_process()))
-#endif
+ worker_pid = postmaster_child_launch(PMC_BGWORKER, (char *) &rw->rw_worker, sizeof(BackgroundWorker), NULL);
+ if (worker_pid == -1)
{
- case -1:
- /* in postmaster, fork failed ... */
- ereport(LOG,
- (errmsg("could not fork worker process: %m")));
- /* undo what assign_backendlist_entry did */
- ReleasePostmasterChildSlot(rw->rw_child_slot);
- rw->rw_child_slot = 0;
- pfree(rw->rw_backend);
- rw->rw_backend = NULL;
- /* mark entry as crashed, so we'll try again later */
- rw->rw_crashed_at = GetCurrentTimestamp();
- break;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /*
- * Before blowing away PostmasterContext, save this bgworker's
- * data where it can find it.
- */
- MyBgworkerEntry = (BackgroundWorker *)
- MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
- memcpy(MyBgworkerEntry, &rw->rw_worker, sizeof(BackgroundWorker));
-
- /* Release postmaster's working memory context */
- MemoryContextSwitchTo(TopMemoryContext);
- MemoryContextDelete(PostmasterContext);
- PostmasterContext = NULL;
-
- StartBackgroundWorker();
+ /* in postmaster, fork failed ... */
+ ereport(LOG,
+ (errmsg("could not fork worker process: %m")));
+ /* undo what assign_backendlist_entry did */
+ ReleasePostmasterChildSlot(rw->rw_child_slot);
+ rw->rw_child_slot = 0;
+ pfree(rw->rw_backend);
+ rw->rw_backend = NULL;
+ /* mark entry as crashed, so we'll try again later */
+ rw->rw_crashed_at = GetCurrentTimestamp();
+ return false;
+ }
- exit(1); /* should not get here */
- break;
-#endif
- default:
- /* in postmaster, fork successful ... */
- rw->rw_pid = worker_pid;
- rw->rw_backend->pid = rw->rw_pid;
- ReportBackgroundWorkerPID(rw);
- /* add new worker to lists of backends */
- dlist_push_head(&BackendList, &rw->rw_backend->elem);
+ /* in postmaster, fork successful ... */
+ rw->rw_pid = worker_pid;
+ rw->rw_backend->pid = rw->rw_pid;
+ ReportBackgroundWorkerPID(rw);
+ /* add new worker to lists of backends */
+ dlist_push_head(&BackendList, &rw->rw_backend->elem);
#ifdef EXEC_BACKEND
- ShmemBackendArrayAdd(rw->rw_backend);
+ ShmemBackendArrayAdd(rw->rw_backend);
#endif
- return true;
- }
-
- return false;
+ return true;
}
/*
@@ -5984,351 +5243,6 @@ PostmasterMarkPIDForWorkerNotify(int pid)
#ifdef EXEC_BACKEND
-/*
- * The following need to be available to the save/restore_backend_variables
- * functions. They are marked NON_EXEC_STATIC in their home modules.
- */
-extern slock_t *ShmemLock;
-extern slock_t *ProcStructLock;
-extern PGPROC *AuxiliaryProcs;
-extern PMSignalData *PMSignalState;
-extern pg_time_t first_syslogger_file_time;
-
-#ifndef WIN32
-#define write_inheritable_socket(dest, src, childpid) ((*(dest) = (src)), true)
-#define read_inheritable_socket(dest, src) (*(dest) = *(src))
-#else
-static bool write_duplicated_handle(HANDLE *dest, HANDLE src, HANDLE child);
-static bool write_inheritable_socket(InheritableSocket *dest, SOCKET src,
- pid_t childPid);
-static void read_inheritable_socket(SOCKET *dest, InheritableSocket *src);
-#endif
-
-
-/* Save critical backend variables into the BackendParameters struct */
-#ifndef WIN32
-static bool
-save_backend_variables(BackendParameters *param, ClientSocket *client_sock)
-#else
-static bool
-save_backend_variables(BackendParameters *param, ClientSocket *client_sock,
- HANDLE childProcess, pid_t childPid)
-#endif
-{
- memcpy(¶m->client_sock, client_sock, sizeof(ClientSocket));
- if (!write_inheritable_socket(¶m->serialized_sock, client_sock->sock, childPid))
- return false;
-
- strlcpy(param->DataDir, DataDir, MAXPGPATH);
-
- param->MyCancelKey = MyCancelKey;
- param->MyPMChildSlot = MyPMChildSlot;
-
-#ifdef WIN32
- param->ShmemProtectiveRegion = ShmemProtectiveRegion;
-#endif
- param->UsedShmemSegID = UsedShmemSegID;
- param->UsedShmemSegAddr = UsedShmemSegAddr;
-
- param->ShmemLock = ShmemLock;
- param->ShmemVariableCache = ShmemVariableCache;
- param->ShmemBackendArray = ShmemBackendArray;
-
-#ifndef HAVE_SPINLOCKS
- param->SpinlockSemaArray = SpinlockSemaArray;
-#endif
- param->NamedLWLockTrancheRequests = NamedLWLockTrancheRequests;
- param->NamedLWLockTrancheArray = NamedLWLockTrancheArray;
- param->MainLWLockArray = MainLWLockArray;
- param->ProcStructLock = ProcStructLock;
- param->ProcGlobal = ProcGlobal;
- param->AuxiliaryProcs = AuxiliaryProcs;
- param->PreparedXactProcs = PreparedXactProcs;
- param->PMSignalState = PMSignalState;
-
- param->PostmasterPid = PostmasterPid;
- param->PgStartTime = PgStartTime;
- param->PgReloadTime = PgReloadTime;
- param->first_syslogger_file_time = first_syslogger_file_time;
-
- param->redirection_done = redirection_done;
- param->IsBinaryUpgrade = IsBinaryUpgrade;
- param->query_id_enabled = query_id_enabled;
- param->max_safe_fds = max_safe_fds;
-
- param->MaxBackends = MaxBackends;
-
-#ifdef WIN32
- param->PostmasterHandle = PostmasterHandle;
- if (!write_duplicated_handle(¶m->initial_signal_pipe,
- pgwin32_create_signal_listener(childPid),
- childProcess))
- return false;
-#else
- memcpy(¶m->postmaster_alive_fds, &postmaster_alive_fds,
- sizeof(postmaster_alive_fds));
-#endif
-
- memcpy(¶m->syslogPipe, &syslogPipe, sizeof(syslogPipe));
-
- strlcpy(param->my_exec_path, my_exec_path, MAXPGPATH);
-
- strlcpy(param->pkglib_path, pkglib_path, MAXPGPATH);
-
- if (MyBgworkerEntry)
- memcpy(¶m->MyBgworkerEntry, MyBgworkerEntry, sizeof(BackgroundWorker));
- else
- memset(¶m->MyBgworkerEntry, 0, sizeof(BackgroundWorker));
-
- return true;
-}
-
-
-#ifdef WIN32
-/*
- * Duplicate a handle for usage in a child process, and write the child
- * process instance of the handle to the parameter file.
- */
-static bool
-write_duplicated_handle(HANDLE *dest, HANDLE src, HANDLE childProcess)
-{
- HANDLE hChild = INVALID_HANDLE_VALUE;
-
- if (!DuplicateHandle(GetCurrentProcess(),
- src,
- childProcess,
- &hChild,
- 0,
- TRUE,
- DUPLICATE_CLOSE_SOURCE | DUPLICATE_SAME_ACCESS))
- {
- ereport(LOG,
- (errmsg_internal("could not duplicate handle to be written to backend parameter file: error code %lu",
- GetLastError())));
- return false;
- }
-
- *dest = hChild;
- return true;
-}
-
-/*
- * Duplicate a socket for usage in a child process, and write the resulting
- * structure to the parameter file.
- * This is required because a number of LSPs (Layered Service Providers) very
- * common on Windows (antivirus, firewalls, download managers etc) break
- * straight socket inheritance.
- */
-static bool
-write_inheritable_socket(InheritableSocket *dest, SOCKET src, pid_t childpid)
-{
- dest->origsocket = src;
- if (src != 0 && src != PGINVALID_SOCKET)
- {
- /* Actual socket */
- if (WSADuplicateSocket(src, childpid, &dest->wsainfo) != 0)
- {
- ereport(LOG,
- (errmsg("could not duplicate socket %d for use in backend: error code %d",
- (int) src, WSAGetLastError())));
- return false;
- }
- }
- return true;
-}
-
-/*
- * Read a duplicate socket structure back, and get the socket descriptor.
- */
-static void
-read_inheritable_socket(SOCKET *dest, InheritableSocket *src)
-{
- SOCKET s;
-
- if (src->origsocket == PGINVALID_SOCKET || src->origsocket == 0)
- {
- /* Not a real socket! */
- *dest = src->origsocket;
- }
- else
- {
- /* Actual socket, so create from structure */
- s = WSASocket(FROM_PROTOCOL_INFO,
- FROM_PROTOCOL_INFO,
- FROM_PROTOCOL_INFO,
- &src->wsainfo,
- 0,
- 0);
- if (s == INVALID_SOCKET)
- {
- write_stderr("could not create inherited socket: error code %d\n",
- WSAGetLastError());
- exit(1);
- }
- *dest = s;
-
- /*
- * To make sure we don't get two references to the same socket, close
- * the original one. (This would happen when inheritance actually
- * works..
- */
- closesocket(src->origsocket);
- }
-}
-#endif
-
-static void
-read_backend_variables(char *id, ClientSocket *client_sock)
-{
- BackendParameters param;
-
-#ifndef WIN32
- /* Non-win32 implementation reads from file */
- FILE *fp;
-
- /* Open file */
- fp = AllocateFile(id, PG_BINARY_R);
- if (!fp)
- {
- write_stderr("could not open backend variables file \"%s\": %s\n",
- id, strerror(errno));
- exit(1);
- }
-
- if (fread(¶m, sizeof(param), 1, fp) != 1)
- {
- write_stderr("could not read from backend variables file \"%s\": %s\n",
- id, strerror(errno));
- exit(1);
- }
-
- /* Release file */
- FreeFile(fp);
- if (unlink(id) != 0)
- {
- write_stderr("could not remove file \"%s\": %s\n",
- id, strerror(errno));
- exit(1);
- }
-#else
- /* Win32 version uses mapped file */
- HANDLE paramHandle;
- BackendParameters *paramp;
-
-#ifdef _WIN64
- paramHandle = (HANDLE) _atoi64(id);
-#else
- paramHandle = (HANDLE) atol(id);
-#endif
- paramp = MapViewOfFile(paramHandle, FILE_MAP_READ, 0, 0, 0);
- if (!paramp)
- {
- write_stderr("could not map view of backend variables: error code %lu\n",
- GetLastError());
- exit(1);
- }
-
- memcpy(¶m, paramp, sizeof(BackendParameters));
-
- if (!UnmapViewOfFile(paramp))
- {
- write_stderr("could not unmap view of backend variables: error code %lu\n",
- GetLastError());
- exit(1);
- }
-
- if (!CloseHandle(paramHandle))
- {
- write_stderr("could not close handle to backend parameter variables: error code %lu\n",
- GetLastError());
- exit(1);
- }
-#endif
-
- restore_backend_variables(¶m, client_sock);
-}
-
-/* Restore critical backend variables from the BackendParameters struct */
-static void
-restore_backend_variables(BackendParameters *param, ClientSocket *client_sock)
-{
- memcpy(client_sock, ¶m->client_sock, sizeof(ClientSocket));
- read_inheritable_socket(&client_sock->sock, ¶m->serialized_sock);
-
- SetDataDir(param->DataDir);
-
- MyCancelKey = param->MyCancelKey;
- MyPMChildSlot = param->MyPMChildSlot;
-
-#ifdef WIN32
- ShmemProtectiveRegion = param->ShmemProtectiveRegion;
-#endif
- UsedShmemSegID = param->UsedShmemSegID;
- UsedShmemSegAddr = param->UsedShmemSegAddr;
-
- ShmemLock = param->ShmemLock;
- ShmemVariableCache = param->ShmemVariableCache;
- ShmemBackendArray = param->ShmemBackendArray;
-
-#ifndef HAVE_SPINLOCKS
- SpinlockSemaArray = param->SpinlockSemaArray;
-#endif
- NamedLWLockTrancheRequests = param->NamedLWLockTrancheRequests;
- NamedLWLockTrancheArray = param->NamedLWLockTrancheArray;
- MainLWLockArray = param->MainLWLockArray;
- ProcStructLock = param->ProcStructLock;
- ProcGlobal = param->ProcGlobal;
- AuxiliaryProcs = param->AuxiliaryProcs;
- PreparedXactProcs = param->PreparedXactProcs;
- PMSignalState = param->PMSignalState;
-
- PostmasterPid = param->PostmasterPid;
- PgStartTime = param->PgStartTime;
- PgReloadTime = param->PgReloadTime;
- first_syslogger_file_time = param->first_syslogger_file_time;
-
- redirection_done = param->redirection_done;
- IsBinaryUpgrade = param->IsBinaryUpgrade;
- query_id_enabled = param->query_id_enabled;
- max_safe_fds = param->max_safe_fds;
-
- MaxBackends = param->MaxBackends;
-
-#ifdef WIN32
- PostmasterHandle = param->PostmasterHandle;
- pgwin32_initial_signal_pipe = param->initial_signal_pipe;
-#else
- memcpy(&postmaster_alive_fds, ¶m->postmaster_alive_fds,
- sizeof(postmaster_alive_fds));
-#endif
-
- memcpy(&syslogPipe, ¶m->syslogPipe, sizeof(syslogPipe));
-
- strlcpy(my_exec_path, param->my_exec_path, MAXPGPATH);
-
- strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
-
- if (param->MyBgworkerEntry.bgw_name[0] != '\0')
- {
- MyBgworkerEntry = (BackgroundWorker *)
- MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
- memcpy(MyBgworkerEntry, ¶m->MyBgworkerEntry, sizeof(BackgroundWorker));
- }
-
- /*
- * We need to restore fd.c's counts of externally-opened FDs; to avoid
- * confusion, be sure to do this after restoring max_safe_fds. (Note:
- * BackendInitialize will handle this for port->sock.)
- */
-#ifndef WIN32
- if (postmaster_alive_fds[0] >= 0)
- ReserveExternalFD();
- if (postmaster_alive_fds[1] >= 0)
- ReserveExternalFD();
-#endif
-}
-
-
Size
ShmemBackendArraySize(void)
{
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index efc2580536a..0db0843ab8e 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -25,6 +25,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/interrupt.h"
#include "postmaster/startup.h"
#include "storage/ipc.h"
@@ -227,8 +228,14 @@ StartupProcExit(int code, Datum arg)
* ----------------------------------
*/
void
-StartupProcessMain(void)
+StartupProcessMain(char *startup_data, size_t startup_data_len)
{
+ Assert(startup_data_len == 0);
+
+ MyAuxProcType = StartupProcess;
+ MyBackendType = B_STARTUP;
+ AuxiliaryProcessInit();
+
/* Arrange to clean up at startup process exit */
on_shmem_exit(StartupProcExit, 0);
diff --git a/src/backend/postmaster/syslogger.c b/src/backend/postmaster/syslogger.c
index 858a2f6b2b9..6b06311c9d5 100644
--- a/src/backend/postmaster/syslogger.c
+++ b/src/backend/postmaster/syslogger.c
@@ -39,7 +39,6 @@
#include "pgstat.h"
#include "pgtime.h"
#include "port/pg_bitutils.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "postmaster/syslogger.h"
@@ -50,6 +49,7 @@
#include "storage/pg_shmem.h"
#include "tcop/tcopprot.h"
#include "utils/guc.h"
+#include "utils/memutils.h"
#include "utils/ps_status.h"
#include "utils/timestamp.h"
@@ -134,10 +134,7 @@ static volatile sig_atomic_t rotation_requested = false;
#ifdef EXEC_BACKEND
static int syslogger_fdget(FILE *file);
static FILE *syslogger_fdopen(int fd);
-static pid_t syslogger_forkexec(void);
-static void syslogger_parseArgs(int argc, char *argv[]);
#endif
-NON_EXEC_STATIC void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
static void process_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static void flush_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static FILE *logfile_open(const char *filename, const char *mode,
@@ -156,13 +153,18 @@ static void set_next_rotation_time(void);
static void sigUsr1Handler(SIGNAL_ARGS);
static void update_metainfo_datafile(void);
+typedef struct {
+ int syslogFile;
+ int csvlogFile;
+ int jsonlogFile;
+} syslogger_startup_data;
/*
* Main entry point for syslogger process
* argc/argv parameters are valid only in EXEC_BACKEND case.
*/
-NON_EXEC_STATIC void
-SysLoggerMain(int argc, char *argv[])
+void
+SysLoggerMain(char *startup_data, size_t startup_data_len)
{
#ifndef WIN32
char logbuffer[READ_BUF_SIZE];
@@ -174,11 +176,35 @@ SysLoggerMain(int argc, char *argv[])
pg_time_t now;
WaitEventSet *wes;
- now = MyStartTime;
+ /* Release postmaster's working memory context */
+ if (PostmasterContext)
+ {
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
+ }
+ /*
+ * Re-open the error output files that were opened by SysLogger_Start().
+ *
+ * We expect this will always succeed, which is too optimistic, but if it
+ * fails there's not a lot we can do to report the problem anyway. As
+ * coded, we'll just crash on a null pointer dereference after failure...
+ */
#ifdef EXEC_BACKEND
- syslogger_parseArgs(argc, argv);
-#endif /* EXEC_BACKEND */
+ {
+ syslogger_startup_data *info;
+
+ Assert(startup_data_len == sizeof(info));
+ info = (syslogger_startup_data *) startup_data;
+ syslogFile = syslogger_fdopen(info->syslogFile);
+ csvlogFile = syslogger_fdopen(info->csvlogFile);
+ jsonlogFile = syslogger_fdopen(info->jsonlogFile);
+ }
+#else
+ Assert(startup_data_len == 0);
+#endif
+
+ now = MyStartTime;
MyBackendType = B_LOGGER;
init_ps_display(NULL);
@@ -568,6 +594,9 @@ SysLogger_Start(void)
{
pid_t sysloggerPid;
char *filename;
+#ifdef EXEC_BACKEND
+ syslogger_startup_data startup_data;
+#endif /* EXEC_BACKEND */
if (!Logging_collector)
return 0;
@@ -667,112 +696,95 @@ SysLogger_Start(void)
}
#ifdef EXEC_BACKEND
- switch ((sysloggerPid = syslogger_forkexec()))
+ startup_data.syslogFile = syslogger_fdget(syslogFile);
+ startup_data.csvlogFile = syslogger_fdget(csvlogFile);
+ startup_data.jsonlogFile = syslogger_fdget(jsonlogFile);
+ sysloggerPid = postmaster_child_launch(PMC_SYSLOGGER, (char *) &startup_data, sizeof(startup_data), NULL);
#else
- switch ((sysloggerPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork system logger: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(true);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
+ sysloggerPid = postmaster_child_launch(PMC_SYSLOGGER, NULL, 0, NULL);
+#endif /* EXEC_BACKEND */
- /* do the work */
- SysLoggerMain(0, NULL);
- break;
-#endif
+ if (sysloggerPid == -1)
+ {
+ ereport(LOG,
+ (errmsg("could not fork system logger: %m")));
+ return 0;
+ }
- default:
- /* success, in postmaster */
+ /* success, in postmaster */
- /* now we redirect stderr, if not done already */
- if (!redirection_done)
- {
+ /* now we redirect stderr, if not done already */
+ if (!redirection_done)
+ {
#ifdef WIN32
- int fd;
+ int fd;
#endif
- /*
- * Leave a breadcrumb trail when redirecting, in case the user
- * forgets that redirection is active and looks only at the
- * original stderr target file.
- */
- ereport(LOG,
- (errmsg("redirecting log output to logging collector process"),
- errhint("Future log output will appear in directory \"%s\".",
- Log_directory)));
+ /*
+ * Leave a breadcrumb trail when redirecting, in case the user
+ * forgets that redirection is active and looks only at the
+ * original stderr target file.
+ */
+ ereport(LOG,
+ (errmsg("redirecting log output to logging collector process"),
+ errhint("Future log output will appear in directory \"%s\".",
+ Log_directory)));
#ifndef WIN32
- fflush(stdout);
- if (dup2(syslogPipe[1], STDOUT_FILENO) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stdout: %m")));
- fflush(stderr);
- if (dup2(syslogPipe[1], STDERR_FILENO) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stderr: %m")));
- /* Now we are done with the write end of the pipe. */
- close(syslogPipe[1]);
- syslogPipe[1] = -1;
+ fflush(stdout);
+ if (dup2(syslogPipe[1], STDOUT_FILENO) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stdout: %m")));
+ fflush(stderr);
+ if (dup2(syslogPipe[1], STDERR_FILENO) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stderr: %m")));
+ /* Now we are done with the write end of the pipe. */
+ close(syslogPipe[1]);
+ syslogPipe[1] = -1;
#else
- /*
- * open the pipe in binary mode and make sure stderr is binary
- * after it's been dup'ed into, to avoid disturbing the pipe
- * chunking protocol.
- */
- fflush(stderr);
- fd = _open_osfhandle((intptr_t) syslogPipe[1],
- _O_APPEND | _O_BINARY);
- if (dup2(fd, STDERR_FILENO) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stderr: %m")));
- close(fd);
- _setmode(STDERR_FILENO, _O_BINARY);
+ /*
+ * open the pipe in binary mode and make sure stderr is binary
+ * after it's been dup'ed into, to avoid disturbing the pipe
+ * chunking protocol.
+ */
+ fflush(stderr);
+ fd = _open_osfhandle((intptr_t) syslogPipe[1],
+ _O_APPEND | _O_BINARY);
+ if (dup2(fd, STDERR_FILENO) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stderr: %m")));
+ close(fd);
+ _setmode(STDERR_FILENO, _O_BINARY);
- /*
- * Now we are done with the write end of the pipe.
- * CloseHandle() must not be called because the preceding
- * close() closes the underlying handle.
- */
- syslogPipe[1] = 0;
+ /*
+ * Now we are done with the write end of the pipe.
+ * CloseHandle() must not be called because the preceding
+ * close() closes the underlying handle.
+ */
+ syslogPipe[1] = 0;
#endif
- redirection_done = true;
- }
-
- /* postmaster will never write the file(s); close 'em */
- fclose(syslogFile);
- syslogFile = NULL;
- if (csvlogFile != NULL)
- {
- fclose(csvlogFile);
- csvlogFile = NULL;
- }
- if (jsonlogFile != NULL)
- {
- fclose(jsonlogFile);
- jsonlogFile = NULL;
- }
- return (int) sysloggerPid;
+ redirection_done = true;
}
- /* we should never reach here */
- return 0;
+ /* postmaster will never write the file(s); close 'em */
+ fclose(syslogFile);
+ syslogFile = NULL;
+ if (csvlogFile != NULL)
+ {
+ fclose(csvlogFile);
+ csvlogFile = NULL;
+ }
+ if (jsonlogFile != NULL)
+ {
+ fclose(jsonlogFile);
+ jsonlogFile = NULL;
+ }
+ return (int) sysloggerPid;
}
@@ -831,69 +843,6 @@ syslogger_fdopen(int fd)
return file;
}
-
-/*
- * syslogger_forkexec() -
- *
- * Format up the arglist for, then fork and exec, a syslogger process
- */
-static pid_t
-syslogger_forkexec(void)
-{
- char *av[10];
- int ac = 0;
- char filenobuf[32];
- char csvfilenobuf[32];
- char jsonfilenobuf[32];
-
- av[ac++] = "postgres";
- av[ac++] = "--forklog";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- /* static variables (those not passed by write_backend_variables) */
- snprintf(filenobuf, sizeof(filenobuf), "%d",
- syslogger_fdget(syslogFile));
- av[ac++] = filenobuf;
- snprintf(csvfilenobuf, sizeof(csvfilenobuf), "%d",
- syslogger_fdget(csvlogFile));
- av[ac++] = csvfilenobuf;
- snprintf(jsonfilenobuf, sizeof(jsonfilenobuf), "%d",
- syslogger_fdget(jsonlogFile));
- av[ac++] = jsonfilenobuf;
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-
-/*
- * syslogger_parseArgs() -
- *
- * Extract data from the arglist for exec'ed syslogger process
- */
-static void
-syslogger_parseArgs(int argc, char *argv[])
-{
- int fd;
-
- Assert(argc == 6);
- argv += 3;
-
- /*
- * Re-open the error output files that were opened by SysLogger_Start().
- *
- * We expect this will always succeed, which is too optimistic, but if it
- * fails there's not a lot we can do to report the problem anyway. As
- * coded, we'll just crash on a null pointer dereference after failure...
- */
- fd = atoi(*argv++);
- syslogFile = syslogger_fdopen(fd);
- fd = atoi(*argv++);
- csvlogFile = syslogger_fdopen(fd);
- fd = atoi(*argv++);
- jsonlogFile = syslogger_fdopen(fd);
-}
#endif /* EXEC_BACKEND */
diff --git a/src/backend/postmaster/walwriter.c b/src/backend/postmaster/walwriter.c
index 266fbc23399..886c8bd8f6a 100644
--- a/src/backend/postmaster/walwriter.c
+++ b/src/backend/postmaster/walwriter.c
@@ -48,6 +48,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/interrupt.h"
#include "postmaster/walwriter.h"
#include "storage/bufmgr.h"
@@ -88,13 +89,19 @@ static void HandleWalWriterInterrupts(void);
* basic execution environment, but not enabled signals yet.
*/
void
-WalWriterMain(void)
+WalWriterMain(char *startup_data, size_t startup_data_len)
{
sigjmp_buf local_sigjmp_buf;
MemoryContext walwriter_context;
int left_till_hibernate;
bool hibernating;
+ Assert(startup_data_len == 0);
+
+ MyAuxProcType = WalWriterProcess;
+ MyBackendType = B_WAL_WRITER;
+ AuxiliaryProcessInit();
+
/*
* Properly accept or ignore signals the postmaster might send us
*
diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c
index 87b5593d2db..91f037f070b 100644
--- a/src/backend/replication/logical/launcher.c
+++ b/src/backend/replication/logical/launcher.c
@@ -30,7 +30,6 @@
#include "miscadmin.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "replication/logicallauncher.h"
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index feff7094351..01e8c3fd937 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -65,6 +65,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/interrupt.h"
#include "replication/walreceiver.h"
#include "replication/walsender.h"
@@ -184,7 +185,7 @@ ProcessWalRcvInterrupts(void)
/* Main entry point for walreceiver process */
void
-WalReceiverMain(void)
+WalReceiverMain(char *startup_data, size_t startup_data_len)
{
char conninfo[MAXCONNINFO];
char *tmp_conninfo;
@@ -200,10 +201,17 @@ WalReceiverMain(void)
char *sender_host = NULL;
int sender_port = 0;
+ Assert(startup_data_len == 0);
+
+ MyAuxProcType = WalReceiverProcess;
+ MyBackendType = B_WAL_RECEIVER;
+ AuxiliaryProcessInit();
+
/*
* WalRcv should be set up already (if we are a backend, we inherit this
* by fork() or EXEC_BACKEND mechanism from the postmaster).
*/
+ // FIXME: could this be passed in startup_data instead? Would it be better?
Assert(walrcv != NULL);
/*
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 5465fa19646..d65d461340c 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -144,6 +144,8 @@ InitShmemAllocation(void)
/*
* Initialize ShmemVariableCache for transaction manager. (This doesn't
* really belong here, but not worth moving.)
+ *
+ * XXX: we really should move this
*/
ShmemVariableCache = (VariableCache)
ShmemAlloc(sizeof(*ShmemVariableCache));
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 01b6cc1f7d3..6bc5a47864e 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -4204,6 +4204,7 @@ PostgresMain(const char *dbname, const char *username)
* *MyProcPort, because ConnCreate() allocated that space with malloc()
* ... else we'd need to copy the Port data first. Also, subsidiary data
* such as the username isn't lost either; see ProcessStartupPacket().
+ * XXX
*/
if (PostmasterContext)
{
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 011ec18015a..e7dde1b4e1f 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -44,6 +44,7 @@ volatile uint32 CritSectionCount = 0;
int MyProcPid;
pg_time_t MyStartTime;
TimestampTz MyStartTimestamp;
+struct ClientSocket *MyClientSocket;
struct Port *MyProcPort;
int32 MyCancelKey;
int MyPMChildSlot;
diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c
index a604432126c..fb0f26d1814 100644
--- a/src/backend/utils/init/miscinit.c
+++ b/src/backend/utils/init/miscinit.c
@@ -85,139 +85,6 @@ bool IgnoreSystemIndexes = false;
* ----------------------------------------------------------------
*/
-/*
- * Initialize the basic environment for a postmaster child
- *
- * Should be called as early as possible after the child's startup. However,
- * on EXEC_BACKEND builds it does need to be after read_backend_variables().
- */
-void
-InitPostmasterChild(void)
-{
- IsUnderPostmaster = true; /* we are a postmaster subprocess now */
-
- /*
- * Start our win32 signal implementation. This has to be done after we
- * read the backend variables, because we need to pick up the signal pipe
- * from the parent process.
- */
-#ifdef WIN32
- pgwin32_signal_initialize();
-#endif
-
- /*
- * Set reference point for stack-depth checking. This might seem
- * redundant in !EXEC_BACKEND builds; but it's not because the postmaster
- * launches its children from signal handlers, so we might be running on
- * an alternative stack.
- */
- (void) set_stack_base();
-
- InitProcessGlobals();
-
- /*
- * make sure stderr is in binary mode before anything can possibly be
- * written to it, in case it's actually the syslogger pipe, so the pipe
- * chunking protocol isn't disturbed. Non-logpipe data gets translated on
- * redirection (e.g. via pg_ctl -l) anyway.
- */
-#ifdef WIN32
- _setmode(fileno(stderr), _O_BINARY);
-#endif
-
- /* We don't want the postmaster's proc_exit() handlers */
- on_exit_reset();
-
- /* In EXEC_BACKEND case we will not have inherited BlockSig etc values */
-#ifdef EXEC_BACKEND
- pqinitmask();
-#endif
-
- /* Initialize process-local latch support */
- InitializeLatchSupport();
- InitProcessLocalLatch();
- InitializeLatchWaitSet();
-
- /*
- * If possible, make this process a group leader, so that the postmaster
- * can signal any child processes too. Not all processes will have
- * children, but for consistency we make all postmaster child processes do
- * this.
- */
-#ifdef HAVE_SETSID
- if (setsid() < 0)
- elog(FATAL, "setsid() failed: %m");
-#endif
-
- /*
- * Every postmaster child process is expected to respond promptly to
- * SIGQUIT at all times. Therefore we centrally remove SIGQUIT from
- * BlockSig and install a suitable signal handler. (Client-facing
- * processes may choose to replace this default choice of handler with
- * quickdie().) All other blockable signals remain blocked for now.
- */
- pqsignal(SIGQUIT, SignalHandlerForCrashExit);
-
- sigdelset(&BlockSig, SIGQUIT);
- sigprocmask(SIG_SETMASK, &BlockSig, NULL);
-
- /* Request a signal if the postmaster dies, if possible. */
- PostmasterDeathSignalInit();
-
- /* Don't give the pipe to subprograms that we execute. */
-#ifndef WIN32
- if (fcntl(postmaster_alive_fds[POSTMASTER_FD_WATCH], F_SETFD, FD_CLOEXEC) < 0)
- ereport(FATAL,
- (errcode_for_socket_access(),
- errmsg_internal("could not set postmaster death monitoring pipe to FD_CLOEXEC mode: %m")));
-#endif
-}
-
-/*
- * Initialize the basic environment for a standalone process.
- *
- * argv0 has to be suitable to find the program's executable.
- */
-void
-InitStandaloneProcess(const char *argv0)
-{
- Assert(!IsPostmasterEnvironment);
-
- MyBackendType = B_STANDALONE_BACKEND;
-
- /*
- * Start our win32 signal implementation
- */
-#ifdef WIN32
- pgwin32_signal_initialize();
-#endif
-
- InitProcessGlobals();
-
- /* Initialize process-local latch support */
- InitializeLatchSupport();
- InitProcessLocalLatch();
- InitializeLatchWaitSet();
-
- /*
- * For consistency with InitPostmasterChild, initialize signal mask here.
- * But we don't unblock SIGQUIT or provide a default handler for it.
- */
- pqinitmask();
- sigprocmask(SIG_SETMASK, &BlockSig, NULL);
-
- /* Compute paths, no postmaster to inherit from */
- if (my_exec_path[0] == '\0')
- {
- if (find_my_exec(argv0, my_exec_path) < 0)
- elog(FATAL, "%s: could not locate my own executable path",
- argv0);
- }
-
- if (pkglib_path[0] == '\0')
- get_pkglib_path(my_exec_path, pkglib_path);
-}
-
void
SwitchToSharedLatch(void)
{
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 41867dc14ac..53f6178a8a9 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -132,10 +132,11 @@ typedef struct ClientConnectionInfo
typedef struct Port
{
pgsocket sock; /* File descriptor */
- bool noblock; /* is the socket in non-blocking mode? */
- ProtocolVersion proto; /* FE/BE protocol version */
SockAddr laddr; /* local addr (postmaster) */
SockAddr raddr; /* remote addr (client) */
+
+ bool noblock; /* is the socket in non-blocking mode? */
+ ProtocolVersion proto; /* FE/BE protocol version */
char *remote_host; /* name (or ip addr) of remote host */
char *remote_hostname; /* name (not ip addr) of remote host, if
* available */
@@ -218,11 +219,12 @@ typedef struct Port
* ClientSocket holds a socket for an accepted connection, along with the
* information about the endpoints.
*/
-typedef struct ClientSocket {
+struct ClientSocket {
pgsocket sock; /* File descriptor */
SockAddr laddr; /* local addr (postmaster) */
SockAddr raddr; /* remote addr (client) */
-} ClientSocket;
+};
+typedef struct ClientSocket ClientSocket;
#ifdef USE_SSL
/*
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 14bd574fc24..0b41254920c 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -317,7 +317,6 @@ extern int trace_recovery(int trace_level);
extern PGDLLIMPORT char *DatabasePath;
/* now in utils/init/miscinit.c */
-extern void InitPostmasterChild(void);
extern void InitStandaloneProcess(const char *argv0);
extern void InitProcessLocalLatch(void);
extern void SwitchToSharedLatch(void);
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 65afd1ea1e8..b6b338a4971 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -55,20 +55,14 @@ extern bool IsAutoVacuumWorkerProcess(void);
#define IsAnyAutoVacuumProcess() \
(IsAutoVacuumLauncherProcess() || IsAutoVacuumWorkerProcess())
-/* Functions to start autovacuum process, called from postmaster */
+/* called from postmaster at server startup */
extern void autovac_init(void);
-extern int StartAutoVacLauncher(void);
-extern int StartAutoVacWorker(void);
/* called from postmaster when a worker could not be forked */
extern void AutoVacWorkerFailed(void);
-#ifdef EXEC_BACKEND
-extern void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
-extern void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
-extern void AutovacuumWorkerIAm(void);
-extern void AutovacuumLauncherIAm(void);
-#endif
+extern void AutoVacLauncherMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
+extern void AutoVacWorkerMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern bool AutoVacuumRequestWork(AutoVacuumWorkItemType type,
Oid relationId, BlockNumber blkno);
diff --git a/src/include/postmaster/auxprocess.h b/src/include/postmaster/auxprocess.h
index 5c2d6527ff6..75394ca0155 100644
--- a/src/include/postmaster/auxprocess.h
+++ b/src/include/postmaster/auxprocess.h
@@ -13,8 +13,6 @@
#ifndef AUXPROCESS_H
#define AUXPROCESS_H
-#include "miscadmin.h"
-
-extern void AuxiliaryProcessMain(AuxProcType auxtype) pg_attribute_noreturn();
+extern void AuxiliaryProcessInit(void);
#endif /* AUXPROCESS_H */
diff --git a/src/include/postmaster/bgworker_internals.h b/src/include/postmaster/bgworker_internals.h
index 7fa7ee2d878..c29c37df9cc 100644
--- a/src/include/postmaster/bgworker_internals.h
+++ b/src/include/postmaster/bgworker_internals.h
@@ -54,7 +54,7 @@ extern void BackgroundWorkerStopNotifications(pid_t pid);
extern void ForgetUnstartedBackgroundWorkers(void);
extern void ResetBackgroundWorkerCrashTimes(void);
-/* Function to start a background worker, called from postmaster.c */
-extern void StartBackgroundWorker(void) pg_attribute_noreturn();
+/* Entry point of a background worker process, called from postmaster.c */
+extern void BackgroundWorkerMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
#endif /* BGWORKER_INTERNALS_H */
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index a66722873f4..ee54fc401ef 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -27,8 +27,8 @@ extern PGDLLIMPORT int CheckPointTimeout;
extern PGDLLIMPORT int CheckPointWarning;
extern PGDLLIMPORT double CheckPointCompletionTarget;
-extern void BackgroundWriterMain(void) pg_attribute_noreturn();
-extern void CheckpointerMain(void) pg_attribute_noreturn();
+extern void BackgroundWriterMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
+extern void CheckpointerMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern void RequestCheckpoint(int flags);
extern void CheckpointWriteDelay(int flags, double progress);
diff --git a/src/include/postmaster/fork_process.h b/src/include/postmaster/fork_process.h
deleted file mode 100644
index 12decc8133b..00000000000
--- a/src/include/postmaster/fork_process.h
+++ /dev/null
@@ -1,17 +0,0 @@
-/*-------------------------------------------------------------------------
- *
- * fork_process.h
- * Exports from postmaster/fork_process.c.
- *
- * Copyright (c) 1996-2023, PostgreSQL Global Development Group
- *
- * src/include/postmaster/fork_process.h
- *
- *-------------------------------------------------------------------------
- */
-#ifndef FORK_PROCESS_H
-#define FORK_PROCESS_H
-
-extern pid_t fork_process(void);
-
-#endif /* FORK_PROCESS_H */
diff --git a/src/include/postmaster/pgarch.h b/src/include/postmaster/pgarch.h
index 3bd4fac71e5..577fc14e1d0 100644
--- a/src/include/postmaster/pgarch.h
+++ b/src/include/postmaster/pgarch.h
@@ -29,7 +29,7 @@
extern Size PgArchShmemSize(void);
extern void PgArchShmemInit(void);
extern bool PgArchCanRestart(void);
-extern void PgArchiverMain(void) pg_attribute_noreturn();
+extern void PgArchiverMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern void PgArchWakeup(void);
extern void PgArchForceDirScan(void);
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index 3b3889c58c0..bb0b15fcf32 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -58,14 +58,48 @@ extern int MaxLivePostmasterChildren(void);
extern bool PostmasterMarkPIDForWorkerNotify(int);
-#ifdef EXEC_BACKEND
-extern pid_t postmaster_forkexec(int argc, char *argv[]);
-extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
+extern void BackendMain(char *startup_data, size_t startup_data_len);
+#ifdef EXEC_BACKEND
extern Size ShmemBackendArraySize(void);
extern void ShmemBackendArrayAllocation(void);
#endif
+/* in process_start.c */
+
+/* this better match the list in process_start.c */
+typedef enum PostmasterChildType {
+ PMC_BACKEND = 0,
+ PMC_AV_LAUNCHER,
+ PMC_AV_WORKER,
+ PMC_BGWORKER,
+ PMC_SYSLOGGER,
+
+ /*
+ * so-called "aux processes". These access shared memory, but are not attached to
+ * any particular database. Only one of each of these can be running at a time.
+ */
+ PMC_STARTUP,
+ PMC_BGWRITER,
+ PMC_ARCHIVER,
+ PMC_CHECKPOINTER,
+ PMC_WAL_WRITER,
+ PMC_WAL_RECEIVER,
+} PostmasterChildType;
+
+typedef void (*ChildEntryPoint) (char *startup_data, size_t startup_data_len);
+
+/* defined in libpq-be.h */
+extern struct ClientSocket *MyClientSocket;
+
+extern pid_t postmaster_child_launch(PostmasterChildType child_type, char *startup_data, size_t startup_data_len, struct ClientSocket *sock);
+
+#ifdef EXEC_BACKEND
+extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
+#endif
+
+const char *PostmasterChildName(PostmasterChildType child_type);
+
/*
* Note: MAX_BACKENDS is limited to 2^18-1 because that's the width reserved
* for buffer references in buf_internals.h. This limitation could be lifted
diff --git a/src/include/postmaster/startup.h b/src/include/postmaster/startup.h
index 6a2e4c4526b..ec885063aab 100644
--- a/src/include/postmaster/startup.h
+++ b/src/include/postmaster/startup.h
@@ -26,7 +26,7 @@
extern PGDLLIMPORT int log_startup_progress_interval;
extern void HandleStartupProcInterrupts(void);
-extern void StartupProcessMain(void) pg_attribute_noreturn();
+extern void StartupProcessMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern void PreRestoreCommand(void);
extern void PostRestoreCommand(void);
extern bool IsPromoteSignaled(void);
diff --git a/src/include/postmaster/syslogger.h b/src/include/postmaster/syslogger.h
index 34da778f1ef..7dc41b30e7c 100644
--- a/src/include/postmaster/syslogger.h
+++ b/src/include/postmaster/syslogger.h
@@ -86,9 +86,7 @@ extern int SysLogger_Start(void);
extern void write_syslogger_file(const char *buffer, int count, int destination);
-#ifdef EXEC_BACKEND
-extern void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
+extern void SysLoggerMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern bool CheckLogrotateSignal(void);
extern void RemoveLogrotateSignalFiles(void);
diff --git a/src/include/postmaster/walwriter.h b/src/include/postmaster/walwriter.h
index 6eba7ad79cf..99b7cc07fb2 100644
--- a/src/include/postmaster/walwriter.h
+++ b/src/include/postmaster/walwriter.h
@@ -18,6 +18,6 @@
extern PGDLLIMPORT int WalWriterDelay;
extern PGDLLIMPORT int WalWriterFlushAfter;
-extern void WalWriterMain(void) pg_attribute_noreturn();
+extern void WalWriterMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
#endif /* _WALWRITER_H */
diff --git a/src/include/replication/walreceiver.h b/src/include/replication/walreceiver.h
index 281626fa6f5..92dead4db7b 100644
--- a/src/include/replication/walreceiver.h
+++ b/src/include/replication/walreceiver.h
@@ -457,7 +457,7 @@ walrcv_clear_result(WalRcvExecResult *walres)
}
/* prototypes for functions in walreceiver.c */
-extern void WalReceiverMain(void) pg_attribute_noreturn();
+extern void WalReceiverMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern void ProcessWalRcvInterrupts(void);
extern void WalRcvForceReply(void);
--
2.30.2
Re: Refactoring backend fork+exec code
From 1d89eec53c7fefa7a4a8c011c9f19e3df64dc436 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Mon, 12 Jun 2023 16:33:20 +0300
Subject: [PATCH 4/9] Use FD_CLOEXEC on ListenSockets
@@ -831,7 +834,8 @@ StreamConnection(pgsocket server_fd, Port *port) void StreamClose(pgsocket sock) { - closesocket(sock); + if (closesocket(sock) != 0) + elog(LOG, "closesocket failed: %m"); }/*
Do you think WARNING would be a more appropriate log level?
From 2f518be9e96cfed1a1a49b4af8f7cb4a837aa784 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Mon, 12 Jun 2023 18:07:54 +0300
Subject: [PATCH 5/9] Move "too many clients already" et al. checks from
ProcessStartupPacket.
This seems like a change you could push already (assuming another
maintainer agrees with you), which makes reviews for this patchset even
easier.
From c25b67c045018a2bf05e6ff53819d26e561fc83f Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Sun, 18 Jun 2023 14:11:16 +0300
Subject: [PATCH 6/9] Pass CAC as argument to backend process.
Could you expand a bit more on why it is better to pass it as a separate
argument? Does it not fit well conceptually in struct Port?
@@ -4498,15 +4510,19 @@ postmaster_forkexec(int argc, char *argv[]) * returns the pid of the fork/exec'd process, or -1 on failure */ static pid_t -backend_forkexec(Port *port) +backend_forkexec(Port *port, CAC_state cac) { - char *av[4]; + char *av[5]; int ac = 0; + char cacbuf[10];av[ac++] = "postgres";
av[ac++] = "--forkbackend";
av[ac++] = NULL; /* filled in by internal_forkexec */+ snprintf(cacbuf, sizeof(cacbuf), "%d", (int) cac); + av[ac++] = cacbuf;
Might be worth a sanity check that there wasn't any truncation into
cacbuf, which is an impossibility as the code is written, but still
useful for catching a future developer error.
Is it worth adding a command line option at all instead of having the
naked positional argument? It would help anybody who might read the
command line what the seemingly random integer stands for.
@@ -4910,7 +4926,10 @@ SubPostmasterMain(int argc, char *argv[]) /* Run backend or appropriate child */ if (strcmp(argv[1], "--forkbackend") == 0) { - Assert(argc == 3); /* shouldn't be any more args */ + CAC_state cac; + + Assert(argc == 4); + cac = (CAC_state) atoi(argv[3]);
Perhaps an assert or full error checking that atoi succeeds would be
useful for similar reasons to my previous comment.
From 658cba5cdb2e5c45faff84566906d2fcaa8a3674 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Mon, 12 Jun 2023 18:03:03 +0300
Subject: [PATCH 7/9] Remove ConnCreate and ConnFree, and allocate Port in
stack.
Again, seems like another patch that could be pushed separately assuming
others don't have any comments.
From 65384b9a6cfb3b9b589041526216e0f64d64bea5 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Sun, 18 Jun 2023 13:56:44 +0300
Subject: [PATCH 8/9] Introduce ClientSocket, rename some funcs
@@ -1499,7 +1499,7 @@ CloseServerPorts(int status, Datum arg) { if (ListenSocket[i] != PGINVALID_SOCKET) { - StreamClose(ListenSocket[i]); + closesocket(ListenSocket[i]); ListenSocket[i] = PGINVALID_SOCKET; } }
I see you have been adding log messages in the case of closesocket()
failing. Do you think it is worth doing here as well?
One strange part about this patch is that in patch 4, you edit
StreamClose() to emit a log message in the case of closesocket()
failure, but then this patch just completely removes it.
@@ -4407,11 +4420,11 @@ BackendInitialize(Port *port, CAC_state cac) * Doesn't return at all. */ static void -BackendRun(Port *port) +BackendRun(void) { /* - * Create a per-backend PGPROC struct in shared memory. We must do - * this before we can use LWLocks (in AttachSharedMemoryAndSemaphores). + * Create a per-backend PGPROC struct in shared memory. We must do this + * before we can use LWLocks (in AttachSharedMemoryAndSemaphores). */ InitProcess();
This comment reflow probably fits better in the patch that added the
AttachSharedMemoryAndSemaphores function.
From b33cfeb28a5419045acb659a01410b2b463bea3e Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Sun, 18 Jun 2023 13:59:48 +0300
Subject: [PATCH 9/9] Refactor postmaster child process launching
- Move code related to launching backend processes to new source file,
process_start.c
Since this seems pretty self-contained, my be easier to review if this
was its own commit.
- Refactor the mechanism of passing informaton from the parent to
child process. Instead of using different command-line arguments
when launching the child process in EXEC_BACKEND mode, pass a
variable-length blob of data along with all the global
variables. The contents of that blob depends on the kind of child
process being launched. In !EXEC_BACKEND mode, we use the same blob,
but it's simply inherited from the parent to child process.
Same with this. Perhaps others would disagree.
+const PMChildEntry entry_kinds[] = { + {"backend", BackendMain, true}, + + {"autovacuum launcher", AutoVacLauncherMain, true}, + {"autovacuum worker", AutoVacWorkerMain, true}, + {"bgworker", BackgroundWorkerMain, true}, + {"syslogger", SysLoggerMain, false}, + + {"startup", StartupProcessMain, true}, + {"bgwriter", BackgroundWriterMain, true}, + {"archiver", PgArchiverMain, true}, + {"checkpointer", CheckpointerMain, true}, + {"wal_writer", WalWriterMain, true}, + {"wal_receiver", WalReceiverMain, true}, +};
It seems like this could be made static. I didn't see it getting exposed
in a header file anywhere, but I also admit that I can be blind at
times.
I need to spend more time looking at this last patch.
Nice work so far!
--
Tristan Partin
Neon (https://neon.tech)
Re: Refactoring backend fork+exec code
Hi,
On 2023-06-18 14:22:33 +0300, Heikki Linnakangas wrote:
I started to look at the code in postmaster.c related to launching child
processes. I tried to reduce the difference between EXEC_BACKEND and
!EXEC_BACKEND code paths, and put the code that needs to differ behind a
better abstraction. I started doing this to help with implementing
multi-threading, but it doesn't introduce anything thread-related yet and I
think this improves readability anyway.
Yes please! This code is absolutely awful.
From 0cb6f8d665980d30a5d2a29013000744f16bf813 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Sun, 18 Jun 2023 11:00:21 +0300
Subject: [PATCH 3/9] Refactor CreateSharedMemoryAndSemaphores.Moves InitProcess calls a little later in EXEC_BACKEND case.
What's the reason for this part? ISTM that we'd really want to get away from
plastering duplicated InitProcess() etc everywhere.
I think this might be easier to understand if you just changed did the
CreateSharedMemoryAndSemaphores() -> AttachSharedMemoryAndSemaphores() piece
in this commit, and the rest later.
+void +AttachSharedMemoryAndSemaphores(void) +{ + /* InitProcess must've been called already */
Perhaps worth an assertion to make it easier to see that the order is wrong?
From 1d89eec53c7fefa7a4a8c011c9f19e3df64dc436 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Mon, 12 Jun 2023 16:33:20 +0300
Subject: [PATCH 4/9] Use FD_CLOEXEC on ListenSocketsWe went through some effort to close them in the child process. Better to
not hand them down to the child process in the first place.
I think Thomas has a larger version of this patch:
/messages/by-id/CA+hUKGKPNFcfBQduqof4-7C=avjcSfdkKBGvQoRuAvfocnvY0A@mail.gmail.com
From 65384b9a6cfb3b9b589041526216e0f64d64bea5 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Sun, 18 Jun 2023 13:56:44 +0300
Subject: [PATCH 8/9] Introduce ClientSocket, rename some funcs- Move more of the work on a client socket to the child process.
- Reduce the amount of data that needs to be passed from postmaster to
child. (Used to pass a full Port struct, although most of the fields were
empty. Now we pass the much slimmer ClientSocket.)
I think there might be extensions accessing Port. Not sure if it's worth
worrying about, but ...
--- a/src/backend/postmaster/autovacuum.c +++ b/src/backend/postmaster/autovacuum.c @@ -476,8 +476,8 @@ AutoVacLauncherMain(int argc, char *argv[]) pqsignal(SIGCHLD, SIG_DFL);/* - * Create a per-backend PGPROC struct in shared memory. We must do - * this before we can use LWLocks. + * Create a per-backend PGPROC struct in shared memory. We must do this + * before we can use LWLocks. */ InitProcess();
Don't think this was intentional?
From b33cfeb28a5419045acb659a01410b2b463bea3e Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Sun, 18 Jun 2023 13:59:48 +0300
Subject: [PATCH 9/9] Refactor postmaster child process launching- Move code related to launching backend processes to new source file,
process_start.c
I think you might have renamed this to launch_backend.c?
- Introduce new postmaster_child_launch() function that deals with the
differences between EXEC_BACKEND and fork mode.- Refactor the mechanism of passing informaton from the parent to
child process. Instead of using different command-line arguments
when launching the child process in EXEC_BACKEND mode, pass a
variable-length blob of data along with all the global
variables. The contents of that blob depends on the kind of child
process being launched. In !EXEC_BACKEND mode, we use the same blob,
but it's simply inherited from the parent to child process.
+const PMChildEntry entry_kinds[] = { + {"backend", BackendMain, true}, + + {"autovacuum launcher", AutoVacLauncherMain, true}, + {"autovacuum worker", AutoVacWorkerMain, true}, + {"bgworker", BackgroundWorkerMain, true}, + {"syslogger", SysLoggerMain, false}, + + {"startup", StartupProcessMain, true}, + {"bgwriter", BackgroundWriterMain, true}, + {"archiver", PgArchiverMain, true}, + {"checkpointer", CheckpointerMain, true}, + {"wal_writer", WalWriterMain, true}, + {"wal_receiver", WalReceiverMain, true}, +};
I'd assign them with the PostmasterChildType as index, so there's no danger of
getting out of order.
const PMChildEntry entry_kinds = {
[PMC_AV_LAUNCHER] = {"autovacuum launcher", AutoVacLauncherMain, true},
...
}
or such should work.
I'd also use designated initializers for the fields, it's otherwise hard to
know what true means etc.
I think it might be good to put more into array. If we e.g. knew whether a
particular child type is a backend-like, and aux process or syslogger, we
could avoid the duplicated InitAuxiliaryProcess(),
MemoryContextDelete(PostmasterContext) etc calls everywhere.
+/* + * SubPostmasterMain -- Get the fork/exec'd process into a state equivalent + * to what it would be if we'd simply forked on Unix, and then + * dispatch to the appropriate place. + * + * The first two command line arguments are expected to be "--forkFOO" + * (where FOO indicates which postmaster child we are to become), and + * the name of a variables file that we can read to load data that would + * have been inherited by fork() on Unix. Remaining arguments go to the + * subprocess FooMain() routine. XXX + */ +void +SubPostmasterMain(int argc, char *argv[]) +{ + PostmasterChildType child_type; + char *startup_data; + size_t startup_data_len; + + /* In EXEC_BACKEND case we will not have inherited these settings */ + IsPostmasterEnvironment = true; + whereToSendOutput = DestNone; + + /* Setup essential subsystems (to ensure elog() behaves sanely) */ + InitializeGUCOptions(); + + /* Check we got appropriate args */ + if (argc < 3) + elog(FATAL, "invalid subpostmaster invocation"); + + if (strncmp(argv[1], "--forkchild=", 12) == 0) + { + char *entry_name = argv[1] + 12; + bool found = false; + + for (int idx = 0; idx < lengthof(entry_kinds); idx++) + { + if (strcmp(entry_kinds[idx].name, entry_name) == 0) + { + child_type = idx; + found = true; + break; + } + } + if (!found) + elog(ERROR, "unknown child kind %s", entry_name); + }
Hm, shouldn't we error out when called without --forkchild?
+/* Save critical backend variables into the BackendParameters struct */ +#ifndef WIN32 +static bool +save_backend_variables(BackendParameters *param, ClientSocket *client_sock) +#else
There's so much of this kind of thing. Could we hide it in a struct or such
instead of needing ifdefs everywhere?
--- a/src/backend/storage/ipc/shmem.c +++ b/src/backend/storage/ipc/shmem.c @@ -144,6 +144,8 @@ InitShmemAllocation(void) /* * Initialize ShmemVariableCache for transaction manager. (This doesn't * really belong here, but not worth moving.) + * + * XXX: we really should move this */ ShmemVariableCache = (VariableCache) ShmemAlloc(sizeof(*ShmemVariableCache));
Heh. Indeed. And probably just rename it to something less insane.
Greetings,
Andres Freund
Use FD_CLOEXEC on ListenSockets (was Re: Refactoring backend fork+exec code)
Focusing on this one patch in this series:
On 11/07/2023 01:50, Andres Freund wrote:
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Mon, 12 Jun 2023 16:33:20 +0300
Subject: [PATCH 4/9] Use FD_CLOEXEC on ListenSocketsWe went through some effort to close them in the child process. Better to
not hand them down to the child process in the first place.I think Thomas has a larger version of this patch:
/messages/by-id/CA+hUKGKPNFcfBQduqof4-7C=avjcSfdkKBGvQoRuAvfocnvY0A@mail.gmail.com
Hmm, no, that's a little different. Thomas added the FD_CLOEXEC option
to the *accepted* socket in commit 1da569ca1f. That was part of that
thread. This patch adds the option to the *listen* sockets. That was not
discussed in that thread, but it's certainly in the same vein.
Thomas: What do you think of the attached?
On 11/07/2023 00:07, Tristan Partin wrote:
@@ -831,7 +834,8 @@ StreamConnection(pgsocket server_fd, Port *port) void StreamClose(pgsocket sock) { - closesocket(sock); + if (closesocket(sock) != 0) + elog(LOG, "closesocket failed: %m"); }/*
Do you think WARNING would be a more appropriate log level?
No, WARNING is for messages that you expect the client to receive. This
failure is unexpected at the system level, the message is for the
administrator. The distinction isn't always very clear, but LOG seems
more appropriate in this case.
--
Heikki Linnakangas
Neon (https://neon.tech)
Attachments:
v2-0001-Use-FD_CLOEXEC-on-ListenSockets.patchtext/x-patch; charset=UTF-8; name=v2-0001-Use-FD_CLOEXEC-on-ListenSockets.patchDownload
From 95bd11720c8e88a1f65b9d600f50d71662241ba9 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Thu, 24 Aug 2023 13:53:33 +0300
Subject: [PATCH v2 1/1] Use FD_CLOEXEC on ListenSockets
It's good hygiene if e.g. an extension launches a subprogram when
being loaded. And we went through some effort to close them in the
child process in EXEC_BACKEND mode; better to not hand them down to
the child process in the first place. We still need to close them
after fork when !EXEC_BACKEND, but it's a little simpler.
In the passing, LOG a message if closing the client connection or
listen socket fails. Shouldn't happen, but if it does, would be nice
to know.
Reviewed-by: Tristan Partin, Andres Freund
Discussion: https://www.postgresql.org/message-id/7a59b073-5b5b-151e-7ed3-8b01ff7ce9ef@iki.fi
---
src/backend/libpq/pqcomm.c | 6 +++++-
src/backend/postmaster/postmaster.c | 14 ++++++--------
2 files changed, 11 insertions(+), 9 deletions(-)
diff --git a/src/backend/libpq/pqcomm.c b/src/backend/libpq/pqcomm.c
index 1bdcbe4f636..8d217b06455 100644
--- a/src/backend/libpq/pqcomm.c
+++ b/src/backend/libpq/pqcomm.c
@@ -458,6 +458,9 @@ StreamServerPort(int family, const char *hostName, unsigned short portNumber,
}
#ifndef WIN32
+ /* Don't give the listen socket to any subprograms we execute. */
+ if (fcntl(fd, F_SETFD, FD_CLOEXEC) < 0)
+ elog(FATAL, "fcntl(F_SETFD) failed on socket: %m");
/*
* Without the SO_REUSEADDR flag, a new postmaster can't be started
@@ -831,7 +834,8 @@ StreamConnection(pgsocket server_fd, Port *port)
void
StreamClose(pgsocket sock)
{
- closesocket(sock);
+ if (closesocket(sock) != 0)
+ elog(LOG, "could not close client or listen socket: %m");
}
/*
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 07d376d77ec..41bccb46a87 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -496,7 +496,6 @@ typedef struct
Port port;
InheritableSocket portsocket;
char DataDir[MAXPGPATH];
- pgsocket ListenSocket[MAXLISTEN];
int32 MyCancelKey;
int MyPMChildSlot;
#ifndef WIN32
@@ -2545,8 +2544,6 @@ ConnFree(Port *port)
void
ClosePostmasterPorts(bool am_syslogger)
{
- int i;
-
/* Release resources held by the postmaster's WaitEventSet. */
if (pm_wait_set)
{
@@ -2573,8 +2570,12 @@ ClosePostmasterPorts(bool am_syslogger)
/*
* Close the postmaster's listen sockets. These aren't tracked by fd.c,
* so we don't call ReleaseExternalFD() here.
+ *
+ * The listen sockets are marked as FD_CLOEXEC, so this isn't needed in
+ * EXEC_BACKEND mode.
*/
- for (i = 0; i < MAXLISTEN; i++)
+#ifndef EXEC_BACKEND
+ for (int i = 0; i < MAXLISTEN; i++)
{
if (ListenSocket[i] != PGINVALID_SOCKET)
{
@@ -2582,6 +2583,7 @@ ClosePostmasterPorts(bool am_syslogger)
ListenSocket[i] = PGINVALID_SOCKET;
}
}
+#endif
/*
* If using syslogger, close the read side of the pipe. We don't bother
@@ -6038,8 +6040,6 @@ save_backend_variables(BackendParameters *param, Port *port,
strlcpy(param->DataDir, DataDir, MAXPGPATH);
- memcpy(¶m->ListenSocket, &ListenSocket, sizeof(ListenSocket));
-
param->MyCancelKey = MyCancelKey;
param->MyPMChildSlot = MyPMChildSlot;
@@ -6271,8 +6271,6 @@ restore_backend_variables(BackendParameters *param, Port *port)
SetDataDir(param->DataDir);
- memcpy(&ListenSocket, ¶m->ListenSocket, sizeof(ListenSocket));
-
MyCancelKey = param->MyCancelKey;
MyPMChildSlot = param->MyPMChildSlot;
--
2.39.2
Re: Use FD_CLOEXEC on ListenSockets (was Re: Refactoring backend fork+exec code)
On Thu, Aug 24, 2023 at 11:41 PM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
On 11/07/2023 01:50, Andres Freund wrote:
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Mon, 12 Jun 2023 16:33:20 +0300
Subject: [PATCH 4/9] Use FD_CLOEXEC on ListenSocketsWe went through some effort to close them in the child process. Better to
not hand them down to the child process in the first place.I think Thomas has a larger version of this patch:
/messages/by-id/CA+hUKGKPNFcfBQduqof4-7C=avjcSfdkKBGvQoRuAvfocnvY0A@mail.gmail.comHmm, no, that's a little different. Thomas added the FD_CLOEXEC option
to the *accepted* socket in commit 1da569ca1f. That was part of that
thread. This patch adds the option to the *listen* sockets. That was not
discussed in that thread, but it's certainly in the same vein.Thomas: What do you think of the attached?
LGTM. I vaguely recall thinking that it might be better to keep
EXEC_BACKEND and !EXEC_BACKEND working the same which might be why I
didn't try this one, but it looks fine with the comment to explain, as
you have it. (It's a shame we can't use O_CLOFORK.)
There was some question in the other thread about whether doing that
to the server socket might affect accepted sockets too on some OS, but
I can at least confirm that your patch works fine on FreeBSD in an
EXEC_BACKEND build. I think there were some historical disagreements
about which socket properties were inherited, but not that.
Re: Use FD_CLOEXEC on ListenSockets (was Re: Refactoring backend fork+exec code)
On 24/08/2023 15:48, Thomas Munro wrote:
LGTM. I vaguely recall thinking that it might be better to keep
EXEC_BACKEND and !EXEC_BACKEND working the same which might be why I
didn't try this one, but it looks fine with the comment to explain, as
you have it. (It's a shame we can't use O_CLOFORK.)
Yeah, O_CLOFORK would be nice..
Committed, thanks!
--
Heikki Linnakangas
Neon (https://neon.tech)
Re: Use FD_CLOEXEC on ListenSockets (was Re: Refactoring backend fork+exec code)
On Thu, Aug 24, 2023 at 10:05 AM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
On 24/08/2023 15:48, Thomas Munro wrote:
LGTM. I vaguely recall thinking that it might be better to keep
EXEC_BACKEND and !EXEC_BACKEND working the same which might be why I
didn't try this one, but it looks fine with the comment to explain, as
you have it. (It's a shame we can't use O_CLOFORK.)Yeah, O_CLOFORK would be nice..
Committed, thanks!
Since this commit, I'm getting a lot (63 per restart) of messages:
LOG: could not close client or listen socket: Bad file descriptor
All I have to do to get the message is turn logging_collector = on and
restart.
The close failure condition existed before the commit, it just wasn't
logged before. So, did the extra logging added here just uncover a
pre-existing bug?
The LOG message is sent to the terminal, not to the log file.
Cheers,
Jeff
Re: Use FD_CLOEXEC on ListenSockets (was Re: Refactoring backend fork+exec code)
On 28/08/2023 18:55, Jeff Janes wrote:
Since this commit, I'm getting a lot (63 per restart) of messages:
LOG: could not close client or listen socket: Bad file descriptor
All I have to do to get the message is turn logging_collector = on and
restart.The close failure condition existed before the commit, it just wasn't
logged before. So, did the extra logging added here just uncover a
pre-existing bug?
Yes, so it seems. Syslogger is started before the ListenSockets array is
initialized, so its still all zeros. When syslogger is forked, the child
process tries to close all the listen sockets, which are all zeros. So
syslogger calls close(0) MAXLISTEN (64) times. Attached patch moves the
array initialization earlier.
The first close(0) actually does have an effect: it closes stdin, which
is fd 0. That is surely accidental, but I wonder if we should indeed
close stdin in child processes? The postmaster process doesn't do
anything with stdin either, although I guess a library might try to read
a passphrase from stdin before starting up, for example.
--
Heikki Linnakangas
Neon (https://neon.tech)
Attachments:
fix-syslogger-closesocket-errors.patchtext/x-patch; charset=UTF-8; name=fix-syslogger-closesocket-errors.patchDownload
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 41bccb46a87..6b9e10bffa0 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -596,6 +596,9 @@ PostmasterMain(int argc, char *argv[])
IsPostmasterEnvironment = true;
+ for (i = 0; i < MAXLISTEN; i++)
+ ListenSocket[i] = PGINVALID_SOCKET;
+
/*
* Start our win32 signal implementation
*/
@@ -1176,12 +1179,9 @@ PostmasterMain(int argc, char *argv[])
/*
* Establish input sockets.
*
- * First, mark them all closed, and set up an on_proc_exit function that's
- * charged with closing the sockets again at postmaster shutdown.
+ * Set up an on_proc_exit function that's charged with closing the sockets
+ * again at postmaster shutdown.
*/
- for (i = 0; i < MAXLISTEN; i++)
- ListenSocket[i] = PGINVALID_SOCKET;
-
on_proc_exit(CloseServerPorts, 0);
if (ListenAddresses)
Re: Use FD_CLOEXEC on ListenSockets (was Re: Refactoring backend fork+exec code)
On Mon, Aug 28, 2023 at 11:52:15PM +0300, Heikki Linnakangas wrote:
On 28/08/2023 18:55, Jeff Janes wrote:
Since this commit, I'm getting a lot (63 per restart) of messages:
LOG: could not close client or listen socket: Bad file descriptor
All I have to do to get the message is turn logging_collector = on and
restart.The close failure condition existed before the commit, it just wasn't
logged before. So, did the extra logging added here just uncover a
pre-existing bug?
In case you've not noticed:
/messages/by-id/ZOvvuQe0rdj2slA9@paquier.xyz
But it does not really matter now ;)
Yes, so it seems. Syslogger is started before the ListenSockets array is
initialized, so its still all zeros. When syslogger is forked, the child
process tries to close all the listen sockets, which are all zeros. So
syslogger calls close(0) MAXLISTEN (64) times. Attached patch moves the
array initialization earlier.
Yep, I've reached the same conclusion. Wouldn't it be cleaner to move
the callback registration of CloseServerPorts() closer to the array
initialization, though?
The first close(0) actually does have an effect: it closes stdin, which is
fd 0. That is surely accidental, but I wonder if we should indeed close
stdin in child processes? The postmaster process doesn't do anything with
stdin either, although I guess a library might try to read a passphrase from
stdin before starting up, for example.
We would have heard about that, wouldn't we? I may be missing
something of course, but on HEAD, the array initialization is done
before starting any child processes, and the syslogger is the first
one forked.
--
Michael
Re: Use FD_CLOEXEC on ListenSockets (was Re: Refactoring backend fork+exec code)
On 29/08/2023 01:28, Michael Paquier wrote:
In case you've not noticed:
/messages/by-id/ZOvvuQe0rdj2slA9@paquier.xyz
But it does not really matter now ;)
Ah sorry, missed that thread.
Yes, so it seems. Syslogger is started before the ListenSockets array is
initialized, so its still all zeros. When syslogger is forked, the child
process tries to close all the listen sockets, which are all zeros. So
syslogger calls close(0) MAXLISTEN (64) times. Attached patch moves the
array initialization earlier.Yep, I've reached the same conclusion. Wouldn't it be cleaner to move
the callback registration of CloseServerPorts() closer to the array
initialization, though?
Ok, pushed that way.
I checked the history of this: it goes back to commit 9a86f03b4e in
version 13. The SysLogger_Start() call used to be later, after setting p
ListenSockets, but that commit moved it. So I backpatched this to v13.
The first close(0) actually does have an effect: it closes stdin, which is
fd 0. That is surely accidental, but I wonder if we should indeed close
stdin in child processes? The postmaster process doesn't do anything with
stdin either, although I guess a library might try to read a passphrase from
stdin before starting up, for example.We would have heard about that, wouldn't we? I may be missing
something of course, but on HEAD, the array initialization is done
before starting any child processes, and the syslogger is the first
one forked.
Yes, syslogger is the only process that closes stdin. After moving the
initialization, it doesn't close it either.
Thinking about this some more, the ListenSockets array is a bit silly
anyway. We fill the array starting from index 0, always append to the
end, and never remove entries from it. It would seem more
straightforward to keep track of the used size of the array. Currently
we always loop through the unused parts too, and e.g.
ConfigurePostmasterWaitSet() needs to walk the array to count how many
elements are in use.
--
Heikki Linnakangas
Neon (https://neon.tech)
Re: Use FD_CLOEXEC on ListenSockets (was Re: Refactoring backend fork+exec code)
On 29/08/2023 09:21, Heikki Linnakangas wrote:
Thinking about this some more, the ListenSockets array is a bit silly
anyway. We fill the array starting from index 0, always append to the
end, and never remove entries from it. It would seem more
straightforward to keep track of the used size of the array. Currently
we always loop through the unused parts too, and e.g.
ConfigurePostmasterWaitSet() needs to walk the array to count how many
elements are in use.
Like this.
--
Heikki Linnakangas
Neon (https://neon.tech)
Attachments:
0001-Refactor-ListenSocket-array.patchtext/x-patch; charset=UTF-8; name=0001-Refactor-ListenSocket-array.patchDownload
From 796280f07dd5dbf50897c9895715ab5e2dbf187b Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Tue, 29 Aug 2023 09:53:08 +0300
Subject: [PATCH 1/1] Refactor ListenSocket array.
Keep track of the used size of the array. That avoids looping through
the whole array in a few places. It doesn't matter from a performance
point of view since the array is small anyway, but this feels less
surprising and is a little less code. Now that we have an explicit
NumListenSockets variable that is statically initialized to 0, we
don't need the loop to initialize the array.
Allocate the array in PostmasterContext. The array isn't needed in
child processes, so this allows reusing that memory. We could easily
make the array resizable now, but we haven't heard any complaints
about the current 64 sockets limit.
---
src/backend/libpq/pqcomm.c | 18 +++----
src/backend/postmaster/postmaster.c | 76 +++++++++++------------------
src/include/libpq/libpq.h | 2 +-
3 files changed, 36 insertions(+), 60 deletions(-)
diff --git a/src/backend/libpq/pqcomm.c b/src/backend/libpq/pqcomm.c
index 8d217b0645..48ae7704fb 100644
--- a/src/backend/libpq/pqcomm.c
+++ b/src/backend/libpq/pqcomm.c
@@ -311,8 +311,9 @@ socket_close(int code, Datum arg)
* specified. For TCP ports, hostName is either NULL for all interfaces or
* the interface to listen on, and unixSocketDir is ignored (can be NULL).
*
- * Successfully opened sockets are added to the ListenSocket[] array (of
- * length MaxListen), at the first position that isn't PGINVALID_SOCKET.
+ * Successfully opened sockets are appended to the ListenSockets[] array. The
+ * current size of the array is *NumListenSockets, it is updated to reflect
+ * the opened sockets. MaxListen is the allocated size of the array.
*
* RETURNS: STATUS_OK or STATUS_ERROR
*/
@@ -320,7 +321,7 @@ socket_close(int code, Datum arg)
int
StreamServerPort(int family, const char *hostName, unsigned short portNumber,
const char *unixSocketDir,
- pgsocket ListenSocket[], int MaxListen)
+ pgsocket ListenSockets[], int *NumListenSockets, int MaxListen)
{
pgsocket fd;
int err;
@@ -335,7 +336,6 @@ StreamServerPort(int family, const char *hostName, unsigned short portNumber,
struct addrinfo *addrs = NULL,
*addr;
struct addrinfo hint;
- int listen_index = 0;
int added = 0;
char unixSocketPath[MAXPGPATH];
#if !defined(WIN32) || defined(IPV6_V6ONLY)
@@ -401,12 +401,7 @@ StreamServerPort(int family, const char *hostName, unsigned short portNumber,
}
/* See if there is still room to add 1 more socket. */
- for (; listen_index < MaxListen; listen_index++)
- {
- if (ListenSocket[listen_index] == PGINVALID_SOCKET)
- break;
- }
- if (listen_index >= MaxListen)
+ if (*NumListenSockets == MaxListen)
{
ereport(LOG,
(errmsg("could not bind to all requested addresses: MAXLISTEN (%d) exceeded",
@@ -573,7 +568,8 @@ StreamServerPort(int family, const char *hostName, unsigned short portNumber,
(errmsg("listening on %s address \"%s\", port %d",
familyDesc, addrDesc, (int) portNumber)));
- ListenSocket[listen_index] = fd;
+ ListenSockets[*NumListenSockets] = fd;
+ (*NumListenSockets)++;
added++;
}
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index d7bfb28ff3..2659329b82 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -226,7 +226,8 @@ int ReservedConnections;
/* The socket(s) we're listening to. */
#define MAXLISTEN 64
-static pgsocket ListenSocket[MAXLISTEN];
+static int NumListenSockets = 0;
+static pgsocket *ListenSockets = NULL;
/* still more option variables */
bool EnableSSL = false;
@@ -587,7 +588,6 @@ PostmasterMain(int argc, char *argv[])
int status;
char *userDoption = NULL;
bool listen_addr_saved = false;
- int i;
char *output_config_variable = NULL;
InitProcessGlobals();
@@ -1141,17 +1141,6 @@ PostmasterMain(int argc, char *argv[])
errmsg("could not remove file \"%s\": %m",
LOG_METAINFO_DATAFILE)));
- /*
- * Initialize input sockets.
- *
- * Mark them all closed, and set up an on_proc_exit function that's
- * charged with closing the sockets again at postmaster shutdown.
- */
- for (i = 0; i < MAXLISTEN; i++)
- ListenSocket[i] = PGINVALID_SOCKET;
-
- on_proc_exit(CloseServerPorts, 0);
-
/*
* If enabled, start up syslogger collection subprocess
*/
@@ -1186,7 +1175,13 @@ PostmasterMain(int argc, char *argv[])
/*
* Establish input sockets.
+ *
+ * First set up an on_proc_exit function that's charged with closing the
+ * sockets again at postmaster shutdown.
*/
+ ListenSockets = palloc(MAXLISTEN * sizeof(pgsocket));
+ on_proc_exit(CloseServerPorts, 0);
+
if (ListenAddresses)
{
char *rawstring;
@@ -1215,12 +1210,16 @@ PostmasterMain(int argc, char *argv[])
status = StreamServerPort(AF_UNSPEC, NULL,
(unsigned short) PostPortNumber,
NULL,
- ListenSocket, MAXLISTEN);
+ ListenSockets,
+ &NumListenSockets,
+ MAXLISTEN);
else
status = StreamServerPort(AF_UNSPEC, curhost,
(unsigned short) PostPortNumber,
NULL,
- ListenSocket, MAXLISTEN);
+ ListenSockets,
+ &NumListenSockets,
+ MAXLISTEN);
if (status == STATUS_OK)
{
@@ -1248,7 +1247,7 @@ PostmasterMain(int argc, char *argv[])
#ifdef USE_BONJOUR
/* Register for Bonjour only if we opened TCP socket(s) */
- if (enable_bonjour && ListenSocket[0] != PGINVALID_SOCKET)
+ if (enable_bonjour && NumListenSockets > 0)
{
DNSServiceErrorType err;
@@ -1312,7 +1311,9 @@ PostmasterMain(int argc, char *argv[])
status = StreamServerPort(AF_UNIX, NULL,
(unsigned short) PostPortNumber,
socketdir,
- ListenSocket, MAXLISTEN);
+ ListenSockets,
+ &NumListenSockets,
+ MAXLISTEN);
if (status == STATUS_OK)
{
@@ -1338,7 +1339,7 @@ PostmasterMain(int argc, char *argv[])
/*
* check that we have some socket to listen on
*/
- if (ListenSocket[0] == PGINVALID_SOCKET)
+ if (NumListenSockets == 0)
ereport(FATAL,
(errmsg("no socket created for listening")));
@@ -1486,14 +1487,8 @@ CloseServerPorts(int status, Datum arg)
* before we remove the postmaster.pid lockfile; otherwise there's a race
* condition if a new postmaster wants to re-use the TCP port number.
*/
- for (i = 0; i < MAXLISTEN; i++)
- {
- if (ListenSocket[i] != PGINVALID_SOCKET)
- {
- StreamClose(ListenSocket[i]);
- ListenSocket[i] = PGINVALID_SOCKET;
- }
- }
+ for (i = 0; i < NumListenSockets; i++)
+ StreamClose(ListenSockets[i]);
/*
* Next, remove any filesystem entries for Unix sockets. To avoid race
@@ -1694,29 +1689,19 @@ DetermineSleepTime(void)
static void
ConfigurePostmasterWaitSet(bool accept_connections)
{
- int nsockets;
-
if (pm_wait_set)
FreeWaitEventSet(pm_wait_set);
pm_wait_set = NULL;
- /* How many server sockets do we need to wait for? */
- nsockets = 0;
- if (accept_connections)
- {
- while (nsockets < MAXLISTEN &&
- ListenSocket[nsockets] != PGINVALID_SOCKET)
- ++nsockets;
- }
-
- pm_wait_set = CreateWaitEventSet(CurrentMemoryContext, 1 + nsockets);
+ pm_wait_set = CreateWaitEventSet(CurrentMemoryContext,
+ 1 + (accept_connections ? NumListenSockets : 0));
AddWaitEventToSet(pm_wait_set, WL_LATCH_SET, PGINVALID_SOCKET, MyLatch,
NULL);
if (accept_connections)
{
- for (int i = 0; i < nsockets; i++)
- AddWaitEventToSet(pm_wait_set, WL_SOCKET_ACCEPT, ListenSocket[i],
+ for (int i = 0; i < NumListenSockets; i++)
+ AddWaitEventToSet(pm_wait_set, WL_SOCKET_ACCEPT, ListenSockets[i],
NULL, NULL);
}
}
@@ -2578,14 +2563,9 @@ ClosePostmasterPorts(bool am_syslogger)
* EXEC_BACKEND mode.
*/
#ifndef EXEC_BACKEND
- for (int i = 0; i < MAXLISTEN; i++)
- {
- if (ListenSocket[i] != PGINVALID_SOCKET)
- {
- StreamClose(ListenSocket[i]);
- ListenSocket[i] = PGINVALID_SOCKET;
- }
- }
+ for (int i = 0; i < NumListenSockets; i++)
+ StreamClose(ListenSockets[i]);
+ NumListenSockets = 0;
#endif
/*
diff --git a/src/include/libpq/libpq.h b/src/include/libpq/libpq.h
index 50fc781f47..a6104d8cd0 100644
--- a/src/include/libpq/libpq.h
+++ b/src/include/libpq/libpq.h
@@ -66,7 +66,7 @@ extern PGDLLIMPORT WaitEventSet *FeBeWaitSet;
extern int StreamServerPort(int family, const char *hostName,
unsigned short portNumber, const char *unixSocketDir,
- pgsocket ListenSocket[], int MaxListen);
+ pgsocket ListenSocket[], int *NumListenSockets, int MaxListen);
extern int StreamConnection(pgsocket server_fd, Port *port);
extern void StreamClose(pgsocket sock);
extern void TouchSocketFiles(void);
--
2.39.2
Re: Use FD_CLOEXEC on ListenSockets (was Re: Refactoring backend fork+exec code)
On 29/08/2023 09:58, Heikki Linnakangas wrote:
On 29/08/2023 09:21, Heikki Linnakangas wrote:
Thinking about this some more, the ListenSockets array is a bit silly
anyway. We fill the array starting from index 0, always append to the
end, and never remove entries from it. It would seem more
straightforward to keep track of the used size of the array. Currently
we always loop through the unused parts too, and e.g.
ConfigurePostmasterWaitSet() needs to walk the array to count how many
elements are in use.Like this.
This seems pretty uncontroversial, and I heard no objections, so I went
ahead and committed that.
--
Heikki Linnakangas
Neon (https://neon.tech)
Re: Use FD_CLOEXEC on ListenSockets (was Re: Refactoring backend fork+exec code)
On Thu, Oct 05, 2023 at 03:08:37PM +0300, Heikki Linnakangas wrote:
This seems pretty uncontroversial, and I heard no objections, so I went
ahead and committed that.
It looks like e29c4643951 is causing issues here. While doing
benchmarking on a cluster compiled with -O2, I got a crash:
LOG: system logger process (PID 27924) was terminated by signal 11: Segmentation fault
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x000055ef3b9aed20 in pfree ()
(gdb) bt
#0 0x000055ef3b9aed20 in pfree ()
#1 0x000055ef3b7e0e41 in ClosePostmasterPorts ()
#2 0x000055ef3b7e6649 in SysLogger_Start ()
#3 0x000055ef3b7e4413 in PostmasterMain ()
Okay, the backtrace is not that useful. I'll see if I can get
something better, still it seems like this has broken the way the
syslogger closes these ports.
--
Michael
Re: Use FD_CLOEXEC on ListenSockets (was Re: Refactoring backend fork+exec code)
On Fri, Oct 06, 2023 at 02:30:16PM +0900, Michael Paquier wrote:
Okay, the backtrace is not that useful. I'll see if I can get
something better, still it seems like this has broken the way the
syslogger closes these ports.
And here you go:
Program terminated with signal SIGSEGV, Segmentation fault.
#0 GetMemoryChunkMethodID (pointer=0x0) at mcxt.c:196 196 header =
*((const uint64 *) ((const char *) pointer - sizeof(uint64)));
(gdb) bt
#0 GetMemoryChunkMethodID (pointer=0x0) at mcxt.c:196
#1 0x0000557d04176d59 in pfree (pointer=0x0) at mcxt.c:1463
#2 0x0000557d03e8eab3 in ClosePostmasterPorts (am_syslogger=true) at postmaster.c:2571
#3 0x0000557d03e93ac2 in SysLogger_Start () at syslogger.c:686
#4 0x0000557d03e8c5b7 in PostmasterMain (argc=3, argv=0x557d0471ed00)
at postmaster.c:1148
#5 0x0000557d03d48e34 in main (argc=3, argv=0x557d0471ed00) at main.c:198
(gdb) up 2
#2 0x0000557d03e8eab3 in ClosePostmasterPorts (am_syslogger=true) at
postmaster.c:2571
2571 pfree(ListenSockets);
(gdb) p ListenSockets $1 = (pgsocket *) 0x0
--
Michael
Re: Use FD_CLOEXEC on ListenSockets (was Re: Refactoring backend fork+exec code)
On 06/10/2023 09:50, Michael Paquier wrote:
On Fri, Oct 06, 2023 at 02:30:16PM +0900, Michael Paquier wrote:
Okay, the backtrace is not that useful. I'll see if I can get
something better, still it seems like this has broken the way the
syslogger closes these ports.And here you go:
Program terminated with signal SIGSEGV, Segmentation fault.
#0 GetMemoryChunkMethodID (pointer=0x0) at mcxt.c:196 196 header =
*((const uint64 *) ((const char *) pointer - sizeof(uint64)));
(gdb) bt
#0 GetMemoryChunkMethodID (pointer=0x0) at mcxt.c:196
#1 0x0000557d04176d59 in pfree (pointer=0x0) at mcxt.c:1463
#2 0x0000557d03e8eab3 in ClosePostmasterPorts (am_syslogger=true) at postmaster.c:2571
#3 0x0000557d03e93ac2 in SysLogger_Start () at syslogger.c:686
#4 0x0000557d03e8c5b7 in PostmasterMain (argc=3, argv=0x557d0471ed00)
at postmaster.c:1148
#5 0x0000557d03d48e34 in main (argc=3, argv=0x557d0471ed00) at main.c:198
(gdb) up 2
#2 0x0000557d03e8eab3 in ClosePostmasterPorts (am_syslogger=true) at
postmaster.c:2571
2571 pfree(ListenSockets);
(gdb) p ListenSockets $1 = (pgsocket *) 0x0
Fixed, thanks!
I did a quick test with syslogger enabled before committing, but didn't
notice the segfault. I missed it because syslogger gets restarted and
then it worked.
--
Heikki Linnakangas
Neon (https://neon.tech)
Re: Use FD_CLOEXEC on ListenSockets (was Re: Refactoring backend fork+exec code)
On Fri, Oct 06, 2023 at 10:27:22AM +0300, Heikki Linnakangas wrote:
I did a quick test with syslogger enabled before committing, but didn't
notice the segfault. I missed it because syslogger gets restarted and then
it worked.
Thanks, Heikki.
--
Michael
Re: Refactoring backend fork+exec code
I updated this patch set, addressing some of the straightforward
comments from Tristan and Andres, and did some more cleanups, commenting
etc. Works on Windows now.
Replies to some of the individual comments below:
On 11/07/2023 00:07, Tristan Partin wrote:
@@ -4498,15 +4510,19 @@ postmaster_forkexec(int argc, char *argv[]) * returns the pid of the fork/exec'd process, or -1 on failure */ static pid_t -backend_forkexec(Port *port) +backend_forkexec(Port *port, CAC_state cac) { - char *av[4]; + char *av[5]; int ac = 0; + char cacbuf[10];av[ac++] = "postgres";
av[ac++] = "--forkbackend";
av[ac++] = NULL; /* filled in by internal_forkexec */+ snprintf(cacbuf, sizeof(cacbuf), "%d", (int) cac); + av[ac++] = cacbuf;Might be worth a sanity check that there wasn't any truncation into
cacbuf, which is an impossibility as the code is written, but still
useful for catching a future developer error.Is it worth adding a command line option at all instead of having the
naked positional argument? It would help anybody who might read the
command line what the seemingly random integer stands for.
+1. This gets refactored away in the last patch though. In the last
patch, I used a child process name instead of an integer precisely
because it looks nicer in "ps".
I wonder if we should add more command line arguments, just for
informational purposes. Autovacuum worker process could display the
database name it's connected to, for example. I don't know how important
the command line is on Windows, is it displayed by tools that people
care about?
On 11/07/2023 01:50, Andres Freund wrote:
On 2023-06-18 14:22:33 +0300, Heikki Linnakangas wrote:
From 0cb6f8d665980d30a5d2a29013000744f16bf813 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Sun, 18 Jun 2023 11:00:21 +0300
Subject: [PATCH 3/9] Refactor CreateSharedMemoryAndSemaphores.Moves InitProcess calls a little later in EXEC_BACKEND case.
What's the reason for this part?
The point is that with this commit, InitProcess() is called at same
place in EXEC_BACKEND mode and !EXEC_BACKEND. It feels more consistent
that way.
ISTM that we'd really want to get away from plastering duplicated
InitProcess() etc everywhere.
Sure, we could do more to reduce the duplication. I think this is a step
in the right direction, though.
From 65384b9a6cfb3b9b589041526216e0f64d64bea5 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Sun, 18 Jun 2023 13:56:44 +0300
Subject: [PATCH 8/9] Introduce ClientSocket, rename some funcs- Move more of the work on a client socket to the child process.
- Reduce the amount of data that needs to be passed from postmaster to
child. (Used to pass a full Port struct, although most of the fields were
empty. Now we pass the much slimmer ClientSocket.)I think there might be extensions accessing Port. Not sure if it's worth
worrying about, but ...
That's OK. Port still exists, it's just created a little later. It will
be initialized by the time extensions might look at it.
+const PMChildEntry entry_kinds[] = { + {"backend", BackendMain, true}, + + {"autovacuum launcher", AutoVacLauncherMain, true}, + {"autovacuum worker", AutoVacWorkerMain, true}, + {"bgworker", BackgroundWorkerMain, true}, + {"syslogger", SysLoggerMain, false}, + + {"startup", StartupProcessMain, true}, + {"bgwriter", BackgroundWriterMain, true}, + {"archiver", PgArchiverMain, true}, + {"checkpointer", CheckpointerMain, true}, + {"wal_writer", WalWriterMain, true}, + {"wal_receiver", WalReceiverMain, true}, +};I'd assign them with the PostmasterChildType as index, so there's no danger of
getting out of order.const PMChildEntry entry_kinds = {
[PMC_AV_LAUNCHER] = {"autovacuum launcher", AutoVacLauncherMain, true},
...
}or such should work.
Nice, I didn't know about that syntax! Changed it that way.
I'd also use designated initializers for the fields, it's otherwise hard to
know what true means etc.
I think with one boolean and the struct declaration nearby, it's fine.
If this becomes more complex in the future, with more fields, I agree.
I think it might be good to put more into array. If we e.g. knew whether a
particular child type is a backend-like, and aux process or syslogger, we
could avoid the duplicated InitAuxiliaryProcess(),
MemoryContextDelete(PostmasterContext) etc calls everywhere.
I agree we could do more refactoring here. I don't agree with adding
more to this struct though. I'm trying to limit the code in
launch_backend.c to hiding the differences between EXEC_BACKEND and
!EXEC_BACKEND. In EXEC_BACKEND mode, it restores the child process to
the same state as it is after fork() in !EXEC_BACKEND mode. Any other
initialization steps belong elsewhere.
Some of the steps between InitPostmasterChild() and the *Main()
functions could probably be moved around and refactored. I didn't think
hard about that. I think that can be done separately as follow-up patch.
+/* Save critical backend variables into the BackendParameters struct */ +#ifndef WIN32 +static bool +save_backend_variables(BackendParameters *param, ClientSocket *client_sock) +#elseThere's so much of this kind of thing. Could we hide it in a struct or such
instead of needing ifdefs everywhere?
A lot of #ifdefs you mean? I agree launch_backend.c has a lot of those.
I haven't come up with any good ideas on reducing them, unfortunately.
--
Heikki Linnakangas
Neon (https://neon.tech)
Attachments:
v2-0001-Pass-background-worker-entry-in-the-parameter-fil.patchtext/x-patch; charset=UTF-8; name=v2-0001-Pass-background-worker-entry-in-the-parameter-fil.patchDownload
From f48172bb716dede354ab6cd68f84779a6e63678a Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 11 Oct 2023 13:35:20 +0300
Subject: [PATCH v2 1/6] Pass background worker entry in the parameter file
This makes it possible to move InitProcess later in SubPostmasterMain
(in next commit), as we don't need an lwlock and hence a PGPROC entry
to get the background worker entry anymore.
Reviewed-by: Tristan Partin, Andres Freund
Discussion: https://www.postgresql.org/message-id/7a59b073-5b5b-151e-7ed3-8b01ff7ce9ef@iki.fi
---
src/backend/postmaster/bgworker.c | 21 -----------
src/backend/postmaster/postmaster.c | 39 ++++++++++++++-------
src/include/postmaster/bgworker_internals.h | 4 ---
3 files changed, 26 insertions(+), 38 deletions(-)
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index 48a9924527e..b3c2e65ba9f 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -631,27 +631,6 @@ ResetBackgroundWorkerCrashTimes(void)
}
}
-#ifdef EXEC_BACKEND
-/*
- * In EXEC_BACKEND mode, workers use this to retrieve their details from
- * shared memory.
- */
-BackgroundWorker *
-BackgroundWorkerEntry(int slotno)
-{
- static BackgroundWorker myEntry;
- BackgroundWorkerSlot *slot;
-
- Assert(slotno < BackgroundWorkerData->total_slots);
- slot = &BackgroundWorkerData->slot[slotno];
- Assert(slot->in_use);
-
- /* must copy this in case we don't intend to retain shmem access */
- memcpy(&myEntry, &slot->worker, sizeof myEntry);
- return &myEntry;
-}
-#endif
-
/*
* Complain about the BackgroundWorker definition using error level elevel.
* Return true if it looks ok, false if not (unless elevel >= ERROR, in
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 282e6486948..e55f3a152aa 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -540,6 +540,8 @@ typedef struct
#endif
char my_exec_path[MAXPGPATH];
char pkglib_path[MAXPGPATH];
+
+ BackgroundWorker MyBgworkerEntry;
} BackendParameters;
static void read_backend_variables(char *id, Port *port);
@@ -4831,7 +4833,7 @@ SubPostmasterMain(int argc, char *argv[])
strcmp(argv[1], "--forkavlauncher") == 0 ||
strcmp(argv[1], "--forkavworker") == 0 ||
strcmp(argv[1], "--forkaux") == 0 ||
- strncmp(argv[1], "--forkbgworker=", 15) == 0)
+ strncmp(argv[1], "--forkbgworker", 14) == 0)
PGSharedMemoryReAttach();
else
PGSharedMemoryNoReAttach();
@@ -4962,10 +4964,8 @@ SubPostmasterMain(int argc, char *argv[])
AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
}
- if (strncmp(argv[1], "--forkbgworker=", 15) == 0)
+ if (strncmp(argv[1], "--forkbgworker", 14) == 0)
{
- int shmem_slot;
-
/* do this as early as possible; in particular, before InitProcess() */
IsBackgroundWorker = true;
@@ -4978,11 +4978,7 @@ SubPostmasterMain(int argc, char *argv[])
/* Attach process to shared data structures */
CreateSharedMemoryAndSemaphores();
- /* Fetch MyBgworkerEntry from shared memory */
- shmem_slot = atoi(argv[1] + 15);
- MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
-
- BackgroundWorkerMain();
+ StartBackgroundWorker();
}
if (strcmp(argv[1], "--forklog") == 0)
{
@@ -5634,13 +5630,14 @@ BackgroundWorkerUnblockSignals(void)
#ifdef EXEC_BACKEND
static pid_t
-bgworker_forkexec(int shmem_slot)
+bgworker_forkexec(BackgroundWorker *worker)
{
char *av[10];
int ac = 0;
char forkav[MAXPGPATH];
+ pid_t result;
- snprintf(forkav, MAXPGPATH, "--forkbgworker=%d", shmem_slot);
+ snprintf(forkav, MAXPGPATH, "--forkbgworker");
av[ac++] = "postgres";
av[ac++] = forkav;
@@ -5649,7 +5646,11 @@ bgworker_forkexec(int shmem_slot)
Assert(ac < lengthof(av));
- return postmaster_forkexec(ac, av);
+ MyBgworkerEntry = worker;
+ result = postmaster_forkexec(ac, av);
+ MyBgworkerEntry = NULL;
+
+ return result;
}
#endif
@@ -5690,7 +5691,7 @@ do_start_bgworker(RegisteredBgWorker *rw)
rw->rw_worker.bgw_name)));
#ifdef EXEC_BACKEND
- switch ((worker_pid = bgworker_forkexec(rw->rw_shmem_slot)))
+ switch ((worker_pid = bgworker_forkexec(&rw->rw_worker)))
#else
switch ((worker_pid = fork_process()))
#endif
@@ -6090,6 +6091,11 @@ save_backend_variables(BackendParameters *param, Port *port,
strlcpy(param->pkglib_path, pkglib_path, MAXPGPATH);
+ if (MyBgworkerEntry)
+ memcpy(¶m->MyBgworkerEntry, MyBgworkerEntry, sizeof(BackgroundWorker));
+ else
+ memset(¶m->MyBgworkerEntry, 0, sizeof(BackgroundWorker));
+
return true;
}
@@ -6318,6 +6324,13 @@ restore_backend_variables(BackendParameters *param, Port *port)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ if (param->MyBgworkerEntry.bgw_name[0] != '\0')
+ {
+ MyBgworkerEntry = (BackgroundWorker *)
+ MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
+ memcpy(MyBgworkerEntry, ¶m->MyBgworkerEntry, sizeof(BackgroundWorker));
+ }
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/postmaster/bgworker_internals.h b/src/include/postmaster/bgworker_internals.h
index 09df054fcce..323f1e07291 100644
--- a/src/include/postmaster/bgworker_internals.h
+++ b/src/include/postmaster/bgworker_internals.h
@@ -57,8 +57,4 @@ extern void ResetBackgroundWorkerCrashTimes(void);
/* Entry point for background worker processes */
extern void BackgroundWorkerMain(void) pg_attribute_noreturn();
-#ifdef EXEC_BACKEND
-extern BackgroundWorker *BackgroundWorkerEntry(int slotno);
-#endif
-
#endif /* BGWORKER_INTERNALS_H */
--
2.39.2
v2-0002-Refactor-CreateSharedMemoryAndSemaphores.patchtext/x-patch; charset=UTF-8; name=v2-0002-Refactor-CreateSharedMemoryAndSemaphores.patchDownload
From 03f53ab9208240f45b7ec90bc435c81fe1140671 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 11 Oct 2023 13:36:16 +0300
Subject: [PATCH v2 2/6] Refactor CreateSharedMemoryAndSemaphores.
The order of process initialization steps is now more consistent
between !EXEC_BACKEND and EXEC_BACKEND modes. InitProcess() is called
at the same place in either mode. In EXEC_BACKEND mode, there's an
extra AttachSharedMemoryAndSemaphores() call, which is a no-op in
!EXEC_BACKEND mode. This reduces the number of "#ifdef EXEC_BACKEND"
blocks.
For clarity, have separate functions for *creating* the shared memory
and semaphores, at postmaster or single-user backend startup, and
for *attaching* to existing shared memory structures in EXEC_BACKEND
case.
I find it pretty confusing in all the *ShmemInit() functions too that
they are called in two different contexts: in postmaster when creating
the shmem structs, and in the child process in EXEC_BACKEND when
attaching to the already existing structs. But this commit doesn't
change that.
Reviewed-by: Tristan Partin, Andres Freund
Discussion: https://www.postgresql.org/message-id/7a59b073-5b5b-151e-7ed3-8b01ff7ce9ef@iki.fi
---
src/backend/postmaster/autovacuum.c | 22 ++++++------
src/backend/postmaster/auxprocess.c | 8 ++---
src/backend/postmaster/bgworker.c | 11 +++---
src/backend/postmaster/postmaster.c | 48 +++++----------------------
src/backend/replication/walreceiver.c | 3 +-
src/backend/storage/ipc/ipci.c | 12 +++++++
src/backend/storage/lmgr/proc.c | 2 +-
src/include/storage/ipc.h | 1 +
8 files changed, 44 insertions(+), 63 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 327ea0d45ad..17f2b36b28a 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -476,14 +476,13 @@ AutoVacLauncherMain(int argc, char *argv[])
pqsignal(SIGCHLD, SIG_DFL);
/*
- * Create a per-backend PGPROC struct in shared memory, except in the
- * EXEC_BACKEND case where this was done in SubPostmasterMain. We must do
- * this before we can use LWLocks (and in the EXEC_BACKEND case we already
- * had to do some stuff with LWLocks).
+ * Create a per-backend PGPROC struct in shared memory. We must do this
+ * before we can use LWLocks.
*/
-#ifndef EXEC_BACKEND
InitProcess();
-#endif
+
+ /* Attach process to shared data structures */
+ AttachSharedMemoryAndSemaphores();
/* Early initialization */
BaseInit();
@@ -1548,14 +1547,13 @@ AutoVacWorkerMain(int argc, char *argv[])
pqsignal(SIGCHLD, SIG_DFL);
/*
- * Create a per-backend PGPROC struct in shared memory, except in the
- * EXEC_BACKEND case where this was done in SubPostmasterMain. We must do
- * this before we can use LWLocks (and in the EXEC_BACKEND case we already
- * had to do some stuff with LWLocks).
+ * Create a per-backend PGPROC struct in shared memory. We must do this
+ * before we can use LWLocks.
*/
-#ifndef EXEC_BACKEND
InitProcess();
-#endif
+
+ /* Attach process to shared data structures */
+ AttachSharedMemoryAndSemaphores();
/* Early initialization */
BaseInit();
diff --git a/src/backend/postmaster/auxprocess.c b/src/backend/postmaster/auxprocess.c
index cae6feb3562..536d9a2b3e4 100644
--- a/src/backend/postmaster/auxprocess.c
+++ b/src/backend/postmaster/auxprocess.c
@@ -97,12 +97,12 @@ AuxiliaryProcessMain(AuxProcType auxtype)
*/
/*
- * Create a PGPROC so we can use LWLocks. In the EXEC_BACKEND case, this
- * was already done by SubPostmasterMain().
+ * Create a PGPROC so we can use LWLocks.
*/
-#ifndef EXEC_BACKEND
InitAuxiliaryProcess();
-#endif
+
+ /* Attach process to shared data structures */
+ AttachSharedMemoryAndSemaphores();
BaseInit();
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index b3c2e65ba9f..15435965bf8 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -810,14 +810,13 @@ BackgroundWorkerMain(void)
PG_exception_stack = &local_sigjmp_buf;
/*
- * Create a per-backend PGPROC struct in shared memory, except in the
- * EXEC_BACKEND case where this was done in SubPostmasterMain. We must do
- * this before we can use LWLocks (and in the EXEC_BACKEND case we already
- * had to do some stuff with LWLocks).
+ * Create a per-backend PGPROC struct in shared memory. We must do this
+ * before we can use LWLocks.
*/
-#ifndef EXEC_BACKEND
InitProcess();
-#endif
+
+ /* Attach process to shared data structures */
+ AttachSharedMemoryAndSemaphores();
/*
* Early initialization.
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index e55f3a152aa..13ebf7ac4de 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -4097,15 +4097,6 @@ BackendStartup(Port *port)
/* Perform additional initialization and collect startup packet */
BackendInitialize(port);
- /*
- * Create a per-backend PGPROC struct in shared memory. We must do
- * this before we can use LWLocks. In the !EXEC_BACKEND case (here)
- * this could be delayed a bit further, but EXEC_BACKEND needs to do
- * stuff with LWLocks before PostgresMain(), so we do it here as well
- * for symmetry.
- */
- InitProcess();
-
/* And run the backend */
BackendRun(port);
}
@@ -4416,6 +4407,15 @@ BackendInitialize(Port *port)
static void
BackendRun(Port *port)
{
+ /*
+ * Create a per-backend PGPROC struct in shared memory. We must do this
+ * before we can use LWLocks (in AttachSharedMemoryAndSemaphores).
+ */
+ InitProcess();
+
+ /* Attach process to shared data structures */
+ AttachSharedMemoryAndSemaphores();
+
/*
* Make sure we aren't in PostmasterContext anymore. (We can't delete it
* just yet, though, because InitPostgres will need the HBA data.)
@@ -4911,12 +4911,6 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
/* And run the backend */
BackendRun(&port); /* does not return */
}
@@ -4929,12 +4923,6 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitAuxiliaryProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
auxtype = atoi(argv[3]);
AuxiliaryProcessMain(auxtype); /* does not return */
}
@@ -4943,12 +4931,6 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
AutoVacLauncherMain(argc - 2, argv + 2); /* does not return */
}
if (strcmp(argv[1], "--forkavworker") == 0)
@@ -4956,12 +4938,6 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
}
if (strncmp(argv[1], "--forkbgworker", 14) == 0)
@@ -4972,12 +4948,6 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
StartBackgroundWorker();
}
if (strcmp(argv[1], "--forklog") == 0)
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index feff7094351..073668d4f2e 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -194,7 +194,7 @@ WalReceiverMain(void)
TimeLineID startpointTLI;
TimeLineID primaryTLI;
bool first_stream;
- WalRcvData *walrcv = WalRcv;
+ WalRcvData *walrcv;
TimestampTz now;
char *err;
char *sender_host = NULL;
@@ -204,6 +204,7 @@ WalReceiverMain(void)
* WalRcv should be set up already (if we are a backend, we inherit this
* by fork() or EXEC_BACKEND mechanism from the postmaster).
*/
+ walrcv = WalRcv;
Assert(walrcv != NULL);
/*
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index a3d8eacb8dc..4f31ceca073 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -171,6 +171,18 @@ CalculateShmemSize(int *num_semaphores)
* check IsUnderPostmaster, rather than EXEC_BACKEND, to detect this case.
* This is a bit code-wasteful and could be cleaned up.)
*/
+void
+AttachSharedMemoryAndSemaphores(void)
+{
+ /* InitProcess must've been called already */
+ Assert(MyProc != NULL);
+
+ /* Init !EXEC_BACKEND mode, we inherited everything through the fork */
+#ifdef EXEC_BACKEND
+ CreateSharedMemoryAndSemaphores();
+#endif
+}
+
void
CreateSharedMemoryAndSemaphores(void)
{
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index 5b663a2997c..c1d259b1405 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -468,7 +468,7 @@ InitProcess(void)
*
* This is separate from InitProcess because we can't acquire LWLocks until
* we've created a PGPROC, but in the EXEC_BACKEND case ProcArrayAdd won't
- * work until after we've done CreateSharedMemoryAndSemaphores.
+ * work until after we've done AttachSharedMemoryAndSemaphores.
*/
void
InitProcessPhase2(void)
diff --git a/src/include/storage/ipc.h b/src/include/storage/ipc.h
index 888c08b3067..e75656f5242 100644
--- a/src/include/storage/ipc.h
+++ b/src/include/storage/ipc.h
@@ -79,6 +79,7 @@ extern PGDLLIMPORT shmem_startup_hook_type shmem_startup_hook;
extern Size CalculateShmemSize(int *num_semaphores);
extern void CreateSharedMemoryAndSemaphores(void);
+extern void AttachSharedMemoryAndSemaphores(void);
extern void InitializeShmemGUCs(void);
#endif /* IPC_H */
--
2.39.2
v2-0003-Pass-CAC-as-argument-to-backend-process.patchtext/x-patch; charset=UTF-8; name=v2-0003-Pass-CAC-as-argument-to-backend-process.patchDownload
From 0473f732fc3ca894dfdf6e98297582ef10fea7d8 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 11 Oct 2023 13:38:00 +0300
Subject: [PATCH v2 3/6] Pass CAC as argument to backend process.
We used to smuggle it to the child process in the Port struct, but it seems
better to pass it down as a separate argument.
Reviewed-by: Tristan Partin, Andres Freund
Discussion: https://www.postgresql.org/message-id/7a59b073-5b5b-151e-7ed3-8b01ff7ce9ef@iki.fi
---
src/backend/postmaster/postmaster.c | 43 +++++++++++++++++++++--------
src/include/libpq/libpq-be.h | 12 --------
2 files changed, 31 insertions(+), 24 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 13ebf7ac4de..936828df010 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -418,7 +418,18 @@ static void HandleChildCrash(int pid, int exitstatus, const char *procname);
static void LogChildExit(int lev, const char *procname,
int pid, int exitstatus);
static void PostmasterStateMachine(void);
-static void BackendInitialize(Port *port);
+
+typedef enum CAC_state
+{
+ CAC_OK,
+ CAC_STARTUP,
+ CAC_SHUTDOWN,
+ CAC_RECOVERY,
+ CAC_NOTCONSISTENT,
+ CAC_TOOMANY
+} CAC_state;
+
+static void BackendInitialize(Port *port, CAC_state cac);
static void BackendRun(Port *port) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
@@ -475,7 +486,7 @@ typedef struct
} win32_deadchild_waitinfo;
#endif /* WIN32 */
-static pid_t backend_forkexec(Port *port);
+static pid_t backend_forkexec(Port *port, CAC_state cac);
static pid_t internal_forkexec(int argc, char *argv[], Port *port);
/* Type for a socket that can be inherited to a client process */
@@ -4037,6 +4048,7 @@ BackendStartup(Port *port)
{
Backend *bn; /* for backend cleanup */
pid_t pid;
+ CAC_state cac;
/*
* Create backend data structure. Better before the fork() so we can
@@ -4068,8 +4080,8 @@ BackendStartup(Port *port)
bn->cancel_key = MyCancelKey;
/* Pass down canAcceptConnections state */
- port->canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
- bn->dead_end = (port->canAcceptConnections != CAC_OK);
+ cac = canAcceptConnections(BACKEND_TYPE_NORMAL);
+ bn->dead_end = (cac != CAC_OK);
/*
* Unless it's a dead_end child, assign it a child slot number
@@ -4083,7 +4095,7 @@ BackendStartup(Port *port)
bn->bgworker_notify = false;
#ifdef EXEC_BACKEND
- pid = backend_forkexec(port);
+ pid = backend_forkexec(port, cac);
#else /* !EXEC_BACKEND */
pid = fork_process();
if (pid == 0) /* child */
@@ -4095,7 +4107,7 @@ BackendStartup(Port *port)
ClosePostmasterPorts(false);
/* Perform additional initialization and collect startup packet */
- BackendInitialize(port);
+ BackendInitialize(port, cac);
/* And run the backend */
BackendRun(port);
@@ -4182,7 +4194,7 @@ report_fork_failure_to_client(Port *port, int errnum)
* but have not yet set up most of our local pointers to shmem structures.
*/
static void
-BackendInitialize(Port *port)
+BackendInitialize(Port *port, CAC_state cac)
{
int status;
int ret;
@@ -4315,7 +4327,7 @@ BackendInitialize(Port *port)
* now instead of wasting cycles on an authentication exchange. (This also
* allows a pg_ping utility to be written.)
*/
- switch (port->canAcceptConnections)
+ switch (cac)
{
case CAC_STARTUP:
ereport(FATAL,
@@ -4460,15 +4472,19 @@ postmaster_forkexec(int argc, char *argv[])
* returns the pid of the fork/exec'd process, or -1 on failure
*/
static pid_t
-backend_forkexec(Port *port)
+backend_forkexec(Port *port, CAC_state cac)
{
- char *av[4];
+ char *av[5];
int ac = 0;
+ char cacbuf[10];
av[ac++] = "postgres";
av[ac++] = "--forkbackend";
av[ac++] = NULL; /* filled in by internal_forkexec */
+ snprintf(cacbuf, sizeof(cacbuf), "%d", (int) cac);
+ av[ac++] = cacbuf;
+
av[ac] = NULL;
Assert(ac < lengthof(av));
@@ -4872,7 +4888,10 @@ SubPostmasterMain(int argc, char *argv[])
/* Run backend or appropriate child */
if (strcmp(argv[1], "--forkbackend") == 0)
{
- Assert(argc == 3); /* shouldn't be any more args */
+ CAC_state cac;
+
+ Assert(argc == 4);
+ cac = (CAC_state) atoi(argv[3]);
/*
* Need to reinitialize the SSL library in the backend, since the
@@ -4906,7 +4925,7 @@ SubPostmasterMain(int argc, char *argv[])
* PGPROC slots, we have already initialized libpq and are able to
* report the error to the client.
*/
- BackendInitialize(&port);
+ BackendInitialize(&port, cac);
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index a0b74c8095f..335cb2de44a 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -58,17 +58,6 @@ typedef struct
#include "libpq/pqcomm.h"
-typedef enum CAC_state
-{
- CAC_OK,
- CAC_STARTUP,
- CAC_SHUTDOWN,
- CAC_RECOVERY,
- CAC_NOTCONSISTENT,
- CAC_TOOMANY
-} CAC_state;
-
-
/*
* GSSAPI specific state information
*/
@@ -156,7 +145,6 @@ typedef struct Port
int remote_hostname_resolv; /* see above */
int remote_hostname_errcode; /* see above */
char *remote_port; /* text rep of remote port */
- CAC_state canAcceptConnections; /* postmaster connection status */
/*
* Information that needs to be saved from the startup packet and passed
--
2.39.2
v2-0004-Remove-ConnCreate-and-ConnFree-and-allocate-Port-.patchtext/x-patch; charset=UTF-8; name=v2-0004-Remove-ConnCreate-and-ConnFree-and-allocate-Port-.patchDownload
From 145fe47c788b0e02d4b0e827cfaa32fe9b04dfe3 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 11 Oct 2023 13:38:06 +0300
Subject: [PATCH v2 4/6] Remove ConnCreate and ConnFree, and allocate Port in
stack.
By allocating Port in stack, we don't need to deal with out-of-memory
errors, which makes the code a little less verbose.
XXX: This should perhaps be squashed with the next commit. It changes
how the Port structure is allocated again. But maybe it's easier to
review separately.
Reviewed-by: Tristan Partin, Andres Freund
Discussion: https://www.postgresql.org/message-id/7a59b073-5b5b-151e-7ed3-8b01ff7ce9ef@iki.fi
---
src/backend/postmaster/postmaster.c | 68 +++++------------------------
src/backend/tcop/postgres.c | 2 +-
2 files changed, 11 insertions(+), 59 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 936828df010..ff52b9acf35 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -399,8 +399,6 @@ static void CloseServerPorts(int status, Datum arg);
static void unlink_external_pid_file(int status, Datum arg);
static void getInstallationPaths(const char *argv0);
static void checkControlFile(void);
-static Port *ConnCreate(int serverFd);
-static void ConnFree(Port *port);
static void handle_pm_pmsignal_signal(SIGNAL_ARGS);
static void handle_pm_child_exit_signal(SIGNAL_ARGS);
static void handle_pm_reload_request_signal(SIGNAL_ARGS);
@@ -1772,20 +1770,18 @@ ServerLoop(void)
if (events[i].events & WL_SOCKET_ACCEPT)
{
- Port *port;
+ Port port;
- port = ConnCreate(events[i].fd);
- if (port)
- {
- BackendStartup(port);
+ memset(&port, 0, sizeof(port));
+ if (StreamConnection(events[i].fd, &port) == STATUS_OK)
+ BackendStartup(&port);
- /*
- * We no longer need the open socket or port structure in
- * this process
- */
- StreamClose(port->sock);
- ConnFree(port);
- }
+ /*
+ * We no longer need the open socket or port structure in this
+ * process
+ */
+ if (port.sock != PGINVALID_SOCKET)
+ StreamClose(port.sock);
}
}
@@ -2447,50 +2443,6 @@ canAcceptConnections(int backend_type)
return result;
}
-
-/*
- * ConnCreate -- create a local connection data structure
- *
- * Returns NULL on failure, other than out-of-memory which is fatal.
- */
-static Port *
-ConnCreate(int serverFd)
-{
- Port *port;
-
- if (!(port = (Port *) calloc(1, sizeof(Port))))
- {
- ereport(LOG,
- (errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("out of memory")));
- ExitPostmaster(1);
- }
-
- if (StreamConnection(serverFd, port) != STATUS_OK)
- {
- if (port->sock != PGINVALID_SOCKET)
- StreamClose(port->sock);
- ConnFree(port);
- return NULL;
- }
-
- return port;
-}
-
-
-/*
- * ConnFree -- free a local connection data structure
- *
- * Caller has already closed the socket if any, so there's not much
- * to do here.
- */
-static void
-ConnFree(Port *port)
-{
- free(port);
-}
-
-
/*
* ClosePostmasterPorts -- close all the postmaster's open sockets
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index f3c9f1f9bab..dca5eb78297 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -4217,7 +4217,7 @@ PostgresMain(const char *dbname, const char *username)
/*
* If the PostmasterContext is still around, recycle the space; we don't
* need it anymore after InitPostgres completes. Note this does not trash
- * *MyProcPort, because ConnCreate() allocated that space with malloc()
+ * *MyProcPort, because that space is allocated in stack
* ... else we'd need to copy the Port data first. Also, subsidiary data
* such as the username isn't lost either; see ProcessStartupPacket().
*/
--
2.39.2
v2-0005-Introduce-ClientSocket-rename-some-funcs.patchtext/x-patch; charset=UTF-8; name=v2-0005-Introduce-ClientSocket-rename-some-funcs.patchDownload
From abfefd9ee0112c095d49d3979c04d9cb16af94eb Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 11 Oct 2023 13:38:10 +0300
Subject: [PATCH v2 5/6] Introduce ClientSocket, rename some funcs
- Move more of the work on a client socket to the child process.
- Reduce the amount of data that needs to be passed from postmaster to
child. (Used to pass a full Port struct, although most of the fields were
empty. Now we pass the much slimmer ClientSocket.)
Reviewed-by: Tristan Partin, Andres Freund
Discussion: https://www.postgresql.org/message-id/7a59b073-5b5b-151e-7ed3-8b01ff7ce9ef@iki.fi
---
src/backend/libpq/pqcomm.c | 93 +++++++++--------
src/backend/postmaster/postmaster.c | 149 +++++++++++++++-------------
src/backend/tcop/postgres.c | 5 +-
src/include/libpq/libpq-be.h | 19 ++--
src/include/libpq/libpq.h | 6 +-
src/tools/pgindent/typedefs.list | 1 +
6 files changed, 149 insertions(+), 124 deletions(-)
diff --git a/src/backend/libpq/pqcomm.c b/src/backend/libpq/pqcomm.c
index 522584e5978..93ca3b1b8c1 100644
--- a/src/backend/libpq/pqcomm.c
+++ b/src/backend/libpq/pqcomm.c
@@ -29,9 +29,9 @@
* INTERFACE ROUTINES
*
* setup/teardown:
- * StreamServerPort - Open postmaster's server port
- * StreamConnection - Create new connection with client
- * StreamClose - Close a client/backend connection
+ * ListenServerPort - Open postmaster's server port
+ * AcceptClientConnection - Create new connection with client
+ * StreamConnection - Initialize a client connection
* TouchSocketFiles - Protect socket files against /tmp cleaners
* pq_init - initialize libpq at backend startup
* socket_comm_reset - reset libpq during error recovery
@@ -304,7 +304,7 @@ socket_close(int code, Datum arg)
/*
- * StreamServerPort -- open a "listening" port to accept connections.
+ * ListenServerPort -- open a "listening" port to accept connections.
*
* family should be AF_UNIX or AF_UNSPEC; portNumber is the port number.
* For AF_UNIX ports, hostName should be NULL and unixSocketDir must be
@@ -319,7 +319,7 @@ socket_close(int code, Datum arg)
* RETURNS: STATUS_OK or STATUS_ERROR
*/
int
-StreamServerPort(int family, const char *hostName, unsigned short portNumber,
+ListenServerPort(int family, const char *hostName, unsigned short portNumber,
const char *unixSocketDir,
pgsocket ListenSockets[], int *NumListenSockets, int MaxListen)
{
@@ -685,8 +685,9 @@ Setup_AF_UNIX(const char *sock_path)
/*
- * StreamConnection -- create a new connection with client using
- * server port. Set port->sock to the FD of the new connection.
+ * AcceptClientConnection -- accept a new connection with client using
+ * server port. Fills *client_sock with the FD and endpoint info
+ * of the new connection.
*
* ASSUME: that this doesn't need to be non-blocking because
* the Postmaster waits for the socket to be ready to accept().
@@ -694,13 +695,13 @@ Setup_AF_UNIX(const char *sock_path)
* RETURNS: STATUS_OK or STATUS_ERROR
*/
int
-StreamConnection(pgsocket server_fd, Port *port)
+AcceptClientConnection(pgsocket server_fd, ClientSocket *client_sock)
{
/* accept connection and fill in the client (remote) address */
- port->raddr.salen = sizeof(port->raddr.addr);
- if ((port->sock = accept(server_fd,
- (struct sockaddr *) &port->raddr.addr,
- &port->raddr.salen)) == PGINVALID_SOCKET)
+ client_sock->raddr.salen = sizeof(client_sock->raddr.addr);
+ if ((client_sock->sock = accept(server_fd,
+ (struct sockaddr *) &client_sock->raddr.addr,
+ &client_sock->raddr.salen)) == PGINVALID_SOCKET)
{
ereport(LOG,
(errcode_for_socket_access(),
@@ -718,10 +719,10 @@ StreamConnection(pgsocket server_fd, Port *port)
}
/* fill in the server (local) address */
- port->laddr.salen = sizeof(port->laddr.addr);
- if (getsockname(port->sock,
- (struct sockaddr *) &port->laddr.addr,
- &port->laddr.salen) < 0)
+ client_sock->laddr.salen = sizeof(client_sock->laddr.addr);
+ if (getsockname(client_sock->sock,
+ (struct sockaddr *) &client_sock->laddr.addr,
+ &client_sock->laddr.salen) < 0)
{
ereport(LOG,
(errmsg("%s() failed: %m", "getsockname")));
@@ -729,7 +730,7 @@ StreamConnection(pgsocket server_fd, Port *port)
}
/* select NODELAY and KEEPALIVE options if it's a TCP connection */
- if (port->laddr.addr.ss_family != AF_UNIX)
+ if (client_sock->laddr.addr.ss_family != AF_UNIX)
{
int on;
#ifdef WIN32
@@ -740,7 +741,7 @@ StreamConnection(pgsocket server_fd, Port *port)
#ifdef TCP_NODELAY
on = 1;
- if (setsockopt(port->sock, IPPROTO_TCP, TCP_NODELAY,
+ if (setsockopt(client_sock->sock, IPPROTO_TCP, TCP_NODELAY,
(char *) &on, sizeof(on)) < 0)
{
ereport(LOG,
@@ -749,7 +750,7 @@ StreamConnection(pgsocket server_fd, Port *port)
}
#endif
on = 1;
- if (setsockopt(port->sock, SOL_SOCKET, SO_KEEPALIVE,
+ if (setsockopt(client_sock->sock, SOL_SOCKET, SO_KEEPALIVE,
(char *) &on, sizeof(on)) < 0)
{
ereport(LOG,
@@ -781,7 +782,7 @@ StreamConnection(pgsocket server_fd, Port *port)
* https://msdn.microsoft.com/en-us/library/bb736549%28v=vs.85%29.aspx
*/
optlen = sizeof(oldopt);
- if (getsockopt(port->sock, SOL_SOCKET, SO_SNDBUF, (char *) &oldopt,
+ if (getsockopt(client_sock->sock, SOL_SOCKET, SO_SNDBUF, (char *) &oldopt,
&optlen) < 0)
{
ereport(LOG,
@@ -791,7 +792,7 @@ StreamConnection(pgsocket server_fd, Port *port)
newopt = PQ_SEND_BUFFER_SIZE * 4;
if (oldopt < newopt)
{
- if (setsockopt(port->sock, SOL_SOCKET, SO_SNDBUF, (char *) &newopt,
+ if (setsockopt(client_sock->sock, SOL_SOCKET, SO_SNDBUF, (char *) &newopt,
sizeof(newopt)) < 0)
{
ereport(LOG,
@@ -800,13 +801,34 @@ StreamConnection(pgsocket server_fd, Port *port)
}
}
#endif
+ }
+ return STATUS_OK;
+}
+
+/*
+ * StreamConnection -- create a new connection from the given socket.
+ *
+ * This runs in the backend process.
+ */
+Port *
+StreamConnection(ClientSocket *client_sock)
+{
+ Port *port;
+
+ port = palloc0(sizeof(Port));
+ port->sock = client_sock->sock;
+ port->laddr = client_sock->laddr;
+ port->raddr = client_sock->raddr;
+ /* Apply the current keepalive parameters if it's a TCP connection */
+ if (port->laddr.addr.ss_family != AF_UNIX)
+ {
/*
- * Also apply the current keepalive parameters. If we fail to set a
- * parameter, don't error out, because these aren't universally
- * supported. (Note: you might think we need to reset the GUC
- * variables to 0 in such a case, but it's not necessary because the
- * show hooks for these variables report the truth anyway.)
+ * If we fail to set a parameter, don't error out, because these
+ * aren't universally supported. (Note: you might think we need to
+ * reset the GUC variables to 0 in such a case, but it's not necessary
+ * because the show hooks for these variables report the truth
+ * anyway.)
*/
(void) pq_setkeepalivesidle(tcp_keepalives_idle, port);
(void) pq_setkeepalivesinterval(tcp_keepalives_interval, port);
@@ -814,24 +836,7 @@ StreamConnection(pgsocket server_fd, Port *port)
(void) pq_settcpusertimeout(tcp_user_timeout, port);
}
- return STATUS_OK;
-}
-
-/*
- * StreamClose -- close a client/backend connection
- *
- * NOTE: this is NOT used to terminate a session; it is just used to release
- * the file descriptor in a process that should no longer have the socket
- * open. (For example, the postmaster calls this after passing ownership
- * of the connection to a child process.) It is expected that someone else
- * still has the socket open. So, we only want to close the descriptor,
- * we do NOT want to send anything to the far end.
- */
-void
-StreamClose(pgsocket sock)
-{
- if (closesocket(sock) != 0)
- elog(LOG, "could not close client or listen socket: %m");
+ return port;
}
/*
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index ff52b9acf35..1b1a44e3ea8 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -427,15 +427,15 @@ typedef enum CAC_state
CAC_TOOMANY
} CAC_state;
-static void BackendInitialize(Port *port, CAC_state cac);
-static void BackendRun(Port *port) pg_attribute_noreturn();
+static void BackendInitialize(ClientSocket *client_sock, CAC_state cac);
+static void BackendRun(void) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
-static int BackendStartup(Port *port);
+static int BackendStartup(ClientSocket *port);
static int ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done);
static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
static void processCancelRequest(Port *port, void *pkt);
-static void report_fork_failure_to_client(Port *port, int errnum);
+static void report_fork_failure_to_client(ClientSocket *client_sock, int errnum);
static CAC_state canAcceptConnections(int backend_type);
static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
@@ -484,8 +484,8 @@ typedef struct
} win32_deadchild_waitinfo;
#endif /* WIN32 */
-static pid_t backend_forkexec(Port *port, CAC_state cac);
-static pid_t internal_forkexec(int argc, char *argv[], Port *port);
+static pid_t backend_forkexec(ClientSocket *client_sock, CAC_state cac);
+static pid_t internal_forkexec(int argc, char *argv[], ClientSocket *client_sock);
/* Type for a socket that can be inherited to a client process */
#ifdef WIN32
@@ -504,8 +504,8 @@ typedef int InheritableSocket;
*/
typedef struct
{
- Port port;
- InheritableSocket portsocket;
+ ClientSocket client_sock;
+ InheritableSocket serialized_sock;
char DataDir[MAXPGPATH];
int32 MyCancelKey;
int MyPMChildSlot;
@@ -553,13 +553,13 @@ typedef struct
BackgroundWorker MyBgworkerEntry;
} BackendParameters;
-static void read_backend_variables(char *id, Port *port);
-static void restore_backend_variables(BackendParameters *param, Port *port);
+static void read_backend_variables(char *id, ClientSocket *client_sock);
+static void restore_backend_variables(BackendParameters *param, ClientSocket *client_sock);
#ifndef WIN32
-static bool save_backend_variables(BackendParameters *param, Port *port);
+static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock);
#else
-static bool save_backend_variables(BackendParameters *param, Port *port,
+static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock,
HANDLE childProcess, pid_t childPid);
#endif
@@ -1219,14 +1219,14 @@ PostmasterMain(int argc, char *argv[])
char *curhost = (char *) lfirst(l);
if (strcmp(curhost, "*") == 0)
- status = StreamServerPort(AF_UNSPEC, NULL,
+ status = ListenServerPort(AF_UNSPEC, NULL,
(unsigned short) PostPortNumber,
NULL,
ListenSockets,
&NumListenSockets,
MAXLISTEN);
else
- status = StreamServerPort(AF_UNSPEC, curhost,
+ status = ListenServerPort(AF_UNSPEC, curhost,
(unsigned short) PostPortNumber,
NULL,
ListenSockets,
@@ -1320,7 +1320,7 @@ PostmasterMain(int argc, char *argv[])
{
char *socketdir = (char *) lfirst(l);
- status = StreamServerPort(AF_UNIX, NULL,
+ status = ListenServerPort(AF_UNIX, NULL,
(unsigned short) PostPortNumber,
socketdir,
ListenSockets,
@@ -1500,7 +1500,7 @@ CloseServerPorts(int status, Datum arg)
* condition if a new postmaster wants to re-use the TCP port number.
*/
for (i = 0; i < NumListenSockets; i++)
- StreamClose(ListenSockets[i]);
+ closesocket(ListenSockets[i]);
NumListenSockets = 0;
/*
@@ -1770,18 +1770,20 @@ ServerLoop(void)
if (events[i].events & WL_SOCKET_ACCEPT)
{
- Port port;
+ ClientSocket s;
- memset(&port, 0, sizeof(port));
- if (StreamConnection(events[i].fd, &port) == STATUS_OK)
- BackendStartup(&port);
+ if (AcceptClientConnection(events[i].fd, &s) == STATUS_OK)
+ BackendStartup(&s);
/*
* We no longer need the open socket or port structure in this
* process
*/
- if (port.sock != PGINVALID_SOCKET)
- StreamClose(port.sock);
+ if (s.sock != PGINVALID_SOCKET)
+ {
+ if (closesocket(s.sock) != 0)
+ elog(LOG, "could not close client socket: %m");
+ }
}
}
@@ -2146,11 +2148,7 @@ retry1:
/*
* Now fetch parameters out of startup packet and save them into the Port
- * structure. All data structures attached to the Port struct must be
- * allocated in TopMemoryContext so that they will remain available in a
- * running backend (even after PostmasterContext is destroyed). We need
- * not worry about leaking this storage on failure, since we aren't in the
- * postmaster process anymore.
+ * structure.
*/
oldcontext = MemoryContextSwitchTo(TopMemoryContext);
@@ -2286,7 +2284,7 @@ retry1:
port->database_name[0] = '\0';
/*
- * Done putting stuff in TopMemoryContext.
+ * Done filling the Port structure
*/
MemoryContextSwitchTo(oldcontext);
@@ -2490,7 +2488,10 @@ ClosePostmasterPorts(bool am_syslogger)
if (ListenSockets)
{
for (int i = 0; i < NumListenSockets; i++)
- StreamClose(ListenSockets[i]);
+ {
+ if (closesocket(ListenSockets[i]) != 0)
+ elog(LOG, "could not close listen socket: %m");
+ }
pfree(ListenSockets);
}
NumListenSockets = 0;
@@ -3996,7 +3997,7 @@ TerminateChildren(int signal)
* Note: if you change this code, also consider StartAutovacuumWorker.
*/
static int
-BackendStartup(Port *port)
+BackendStartup(ClientSocket *client_sock)
{
Backend *bn; /* for backend cleanup */
pid_t pid;
@@ -4047,7 +4048,7 @@ BackendStartup(Port *port)
bn->bgworker_notify = false;
#ifdef EXEC_BACKEND
- pid = backend_forkexec(port, cac);
+ pid = backend_forkexec(client_sock, cac);
#else /* !EXEC_BACKEND */
pid = fork_process();
if (pid == 0) /* child */
@@ -4059,10 +4060,10 @@ BackendStartup(Port *port)
ClosePostmasterPorts(false);
/* Perform additional initialization and collect startup packet */
- BackendInitialize(port, cac);
+ BackendInitialize(client_sock, cac);
/* And run the backend */
- BackendRun(port);
+ BackendRun();
}
#endif /* EXEC_BACKEND */
@@ -4077,14 +4078,14 @@ BackendStartup(Port *port)
errno = save_errno;
ereport(LOG,
(errmsg("could not fork new process for connection: %m")));
- report_fork_failure_to_client(port, save_errno);
+ report_fork_failure_to_client(client_sock, save_errno);
return STATUS_ERROR;
}
/* in parent, successful fork */
ereport(DEBUG2,
(errmsg_internal("forked new backend, pid=%d socket=%d",
- (int) pid, (int) port->sock)));
+ (int) pid, (int) client_sock->sock)));
/*
* Everything's been successful, it's safe to add this backend to our list
@@ -4111,7 +4112,7 @@ BackendStartup(Port *port)
* it's not up and running.
*/
static void
-report_fork_failure_to_client(Port *port, int errnum)
+report_fork_failure_to_client(ClientSocket *client_sock, int errnum)
{
char buffer[1000];
int rc;
@@ -4122,13 +4123,13 @@ report_fork_failure_to_client(Port *port, int errnum)
strerror(errnum));
/* Set port to non-blocking. Don't do send() if this fails */
- if (!pg_set_noblock(port->sock))
+ if (!pg_set_noblock(client_sock->sock))
return;
/* We'll retry after EINTR, but ignore all other failures */
do
{
- rc = send(port->sock, buffer, strlen(buffer) + 1, 0);
+ rc = send(client_sock->sock, buffer, strlen(buffer) + 1, 0);
} while (rc < 0 && errno == EINTR);
}
@@ -4146,16 +4147,28 @@ report_fork_failure_to_client(Port *port, int errnum)
* but have not yet set up most of our local pointers to shmem structures.
*/
static void
-BackendInitialize(Port *port, CAC_state cac)
+BackendInitialize(ClientSocket *client_sock, CAC_state cac)
{
int status;
int ret;
+ Port *port;
char remote_host[NI_MAXHOST];
char remote_port[NI_MAXSERV];
StringInfoData ps_data;
+ MemoryContext oldcontext;
- /* Save port etc. for ps status */
+ /*
+ * Create the Port structure.
+ *
+ * The Port structure and all data structures attached to it are allocated
+ * in TopMemoryContext, so that they survive into PostgresMain execution.
+ * We need not worry about leaking this storage on failure, since we
+ * aren't in the postmaster process anymore.
+ */
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
+ port = StreamConnection(client_sock);
MyProcPort = port;
+ MemoryContextSwitchTo(oldcontext);
/* Tell fd.c about the long-lived FD associated with the port */
ReserveExternalFD();
@@ -4216,8 +4229,9 @@ BackendInitialize(Port *port, CAC_state cac)
* Save remote_host and remote_port in port structure (after this, they
* will appear in log_line_prefix data for log messages).
*/
- port->remote_host = strdup(remote_host);
- port->remote_port = strdup(remote_port);
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
+ port->remote_host = pstrdup(remote_host);
+ port->remote_port = pstrdup(remote_port);
/* And now we can issue the Log_connections message, if wanted */
if (Log_connections)
@@ -4248,7 +4262,8 @@ BackendInitialize(Port *port, CAC_state cac)
ret == 0 &&
strspn(remote_host, "0123456789.") < strlen(remote_host) &&
strspn(remote_host, "0123456789ABCDEFabcdef:") < strlen(remote_host))
- port->remote_hostname = strdup(remote_host);
+ port->remote_hostname = pstrdup(remote_host);
+ MemoryContextSwitchTo(oldcontext);
/*
* Ready to begin client interaction. We will give up and _exit(1) after
@@ -4369,7 +4384,7 @@ BackendInitialize(Port *port, CAC_state cac)
* Doesn't return at all.
*/
static void
-BackendRun(Port *port)
+BackendRun(void)
{
/*
* Create a per-backend PGPROC struct in shared memory. We must do this
@@ -4386,7 +4401,7 @@ BackendRun(Port *port)
*/
MemoryContextSwitchTo(TopMemoryContext);
- PostgresMain(port->database_name, port->user_name);
+ PostgresMain(MyProcPort->database_name, MyProcPort->user_name);
}
@@ -4407,11 +4422,11 @@ BackendRun(Port *port)
pid_t
postmaster_forkexec(int argc, char *argv[])
{
- Port port;
+ ClientSocket client_sock;
- /* This entry point passes dummy values for the Port variables */
- memset(&port, 0, sizeof(port));
- return internal_forkexec(argc, argv, &port);
+ /* This entry point doesn't pass a client socket */
+ memset(&client_sock, 0, sizeof(ClientSocket));
+ return internal_forkexec(argc, argv, &client_sock);
}
/*
@@ -4424,7 +4439,7 @@ postmaster_forkexec(int argc, char *argv[])
* returns the pid of the fork/exec'd process, or -1 on failure
*/
static pid_t
-backend_forkexec(Port *port, CAC_state cac)
+backend_forkexec(ClientSocket *client_sock, CAC_state cac)
{
char *av[5];
int ac = 0;
@@ -4440,7 +4455,7 @@ backend_forkexec(Port *port, CAC_state cac)
av[ac] = NULL;
Assert(ac < lengthof(av));
- return internal_forkexec(ac, av, port);
+ return internal_forkexec(ac, av, client_sock);
}
#ifndef WIN32
@@ -4452,7 +4467,7 @@ backend_forkexec(Port *port, CAC_state cac)
* - fork():s, and then exec():s the child process
*/
static pid_t
-internal_forkexec(int argc, char *argv[], Port *port)
+internal_forkexec(int argc, char *argv[], ClientSocket *client_sock)
{
static unsigned long tmpBackendFileNum = 0;
pid_t pid;
@@ -4460,7 +4475,7 @@ internal_forkexec(int argc, char *argv[], Port *port)
BackendParameters param;
FILE *fp;
- if (!save_backend_variables(¶m, port))
+ if (!save_backend_variables(¶m, client_sock))
return -1; /* log made by save_backend_variables */
/* Calculate name for temp file */
@@ -4758,7 +4773,7 @@ retry:
void
SubPostmasterMain(int argc, char *argv[])
{
- Port port;
+ ClientSocket client_sock;
/* In EXEC_BACKEND case we will not have inherited these settings */
IsPostmasterEnvironment = true;
@@ -4772,8 +4787,8 @@ SubPostmasterMain(int argc, char *argv[])
elog(FATAL, "invalid subpostmaster invocation");
/* Read in the variables file */
- memset(&port, 0, sizeof(Port));
- read_backend_variables(argv[2], &port);
+ memset(&client_sock, 0, sizeof(ClientSocket));
+ read_backend_variables(argv[2], &client_sock);
/* Close the postmaster's sockets (as soon as we know them) */
ClosePostmasterPorts(strcmp(argv[1], "--forklog") == 0);
@@ -4877,13 +4892,13 @@ SubPostmasterMain(int argc, char *argv[])
* PGPROC slots, we have already initialized libpq and are able to
* report the error to the client.
*/
- BackendInitialize(&port, cac);
+ BackendInitialize(&client_sock, cac);
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
/* And run the backend */
- BackendRun(&port); /* does not return */
+ BackendRun(); /* does not return */
}
if (strcmp(argv[1], "--forkaux") == 0)
{
@@ -5965,15 +5980,15 @@ static void read_inheritable_socket(SOCKET *dest, InheritableSocket *src);
/* Save critical backend variables into the BackendParameters struct */
#ifndef WIN32
static bool
-save_backend_variables(BackendParameters *param, Port *port)
+save_backend_variables(BackendParameters *param, ClientSocket *client_sock)
#else
static bool
-save_backend_variables(BackendParameters *param, Port *port,
+save_backend_variables(BackendParameters *param, ClientSocket *client_sock,
HANDLE childProcess, pid_t childPid)
#endif
{
- memcpy(¶m->port, port, sizeof(Port));
- if (!write_inheritable_socket(¶m->portsocket, port->sock, childPid))
+ memcpy(¶m->client_sock, client_sock, sizeof(ClientSocket));
+ if (!write_inheritable_socket(¶m->serialized_sock, client_sock->sock, childPid))
return false;
strlcpy(param->DataDir, DataDir, MAXPGPATH);
@@ -6135,7 +6150,7 @@ read_inheritable_socket(SOCKET *dest, InheritableSocket *src)
#endif
static void
-read_backend_variables(char *id, Port *port)
+read_backend_variables(char *id, ClientSocket *client_sock)
{
BackendParameters param;
@@ -6202,15 +6217,15 @@ read_backend_variables(char *id, Port *port)
}
#endif
- restore_backend_variables(¶m, port);
+ restore_backend_variables(¶m, client_sock);
}
/* Restore critical backend variables from the BackendParameters struct */
static void
-restore_backend_variables(BackendParameters *param, Port *port)
+restore_backend_variables(BackendParameters *param, ClientSocket *client_sock)
{
- memcpy(port, ¶m->port, sizeof(Port));
- read_inheritable_socket(&port->sock, ¶m->portsocket);
+ memcpy(client_sock, ¶m->client_sock, sizeof(ClientSocket));
+ read_inheritable_socket(&client_sock->sock, ¶m->serialized_sock);
SetDataDir(param->DataDir);
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index dca5eb78297..087a1b43d7a 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -4216,10 +4216,7 @@ PostgresMain(const char *dbname, const char *username)
/*
* If the PostmasterContext is still around, recycle the space; we don't
- * need it anymore after InitPostgres completes. Note this does not trash
- * *MyProcPort, because that space is allocated in stack
- * ... else we'd need to copy the Port data first. Also, subsidiary data
- * such as the username isn't lost either; see ProcessStartupPacket().
+ * need it anymore after InitPostgres completes.
*/
if (PostmasterContext)
{
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 335cb2de44a..9b6d8fc5571 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -110,12 +110,9 @@ typedef struct ClientConnectionInfo
} ClientConnectionInfo;
/*
- * This is used by the postmaster in its communication with frontends. It
- * contains all state information needed during this communication before the
- * backend is run. The Port structure is kept in malloc'd memory and is
- * still available when a backend is running (see MyProcPort). The data
- * it points to must also be malloc'd, or else palloc'd in TopMemoryContext,
- * so that it survives into PostgresMain execution!
+ * The Port structure holds state information about a client connection in a
+ * backend process. It is available in the global variable MyProcPort. The
+ * struct and all the data it points are kept in TopMemoryContext.
*
* remote_hostname is set if we did a successful reverse lookup of the
* client's IP address during connection setup.
@@ -217,6 +214,16 @@ typedef struct Port
#endif
} Port;
+/*
+ * ClientSocket holds a socket for an accepted connection, along with the
+ * information about the endpoints.
+ */
+typedef struct ClientSocket {
+ pgsocket sock; /* File descriptor */
+ SockAddr laddr; /* local addr (postmaster) */
+ SockAddr raddr; /* remote addr (client) */
+} ClientSocket;
+
#ifdef USE_SSL
/*
* Hardcoded DH parameters, used in ephemeral DH keying. (See also
diff --git a/src/include/libpq/libpq.h b/src/include/libpq/libpq.h
index a6104d8cd02..889e86a7f61 100644
--- a/src/include/libpq/libpq.h
+++ b/src/include/libpq/libpq.h
@@ -64,11 +64,11 @@ extern PGDLLIMPORT WaitEventSet *FeBeWaitSet;
#define FeBeWaitSetLatchPos 1
#define FeBeWaitSetNEvents 3
-extern int StreamServerPort(int family, const char *hostName,
+extern int ListenServerPort(int family, const char *hostName,
unsigned short portNumber, const char *unixSocketDir,
pgsocket ListenSocket[], int *NumListenSockets, int MaxListen);
-extern int StreamConnection(pgsocket server_fd, Port *port);
-extern void StreamClose(pgsocket sock);
+extern int AcceptClientConnection(pgsocket server_fd, ClientSocket *client_sock);
+extern Port *StreamConnection(ClientSocket *client_sock);
extern void TouchSocketFiles(void);
extern void RemoveSocketFiles(void);
extern void pq_init(void);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 8de90c49585..78a9e0ebbc8 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -380,6 +380,7 @@ ClientCertMode
ClientCertName
ClientConnectionInfo
ClientData
+ClientSocket
ClonePtrType
ClosePortalStmt
ClosePtrType
--
2.39.2
v2-0006-Refactor-postmaster-child-process-launching.patchtext/x-patch; charset=UTF-8; name=v2-0006-Refactor-postmaster-child-process-launching.patchDownload
From 2d1a5a1b20e94911228143ae68da7ee0210de424 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 11 Oct 2023 13:38:42 +0300
Subject: [PATCH v2 6/6] Refactor postmaster child process launching
- Move code related to launching backend processes to new source file,
launch_backend.c
- Introduce new postmaster_child_launch() function that deals with the
differences between EXEC_BACKEND and fork mode.
- Refactor the mechanism of passing information from the parent to
child process. Instead of using different command-line arguments when
launching the child process in EXEC_BACKEND mode, pass a
variable-length blob of data along with all the global variables. The
contents of that blob depends on the kind of child process being
launched. In !EXEC_BACKEND mode, we use the same blob, but it's simply
inherited from the parent to child process.
Reviewed-by: Tristan Partin, Andres Freund
Discussion: https://www.postgresql.org/message-id/7a59b073-5b5b-151e-7ed3-8b01ff7ce9ef@iki.fi
---
src/backend/postmaster/Makefile | 2 +-
src/backend/postmaster/autovacuum.c | 173 +--
src/backend/postmaster/auxprocess.c | 72 +-
src/backend/postmaster/bgworker.c | 14 +-
src/backend/postmaster/bgwriter.c | 9 +-
src/backend/postmaster/checkpointer.c | 9 +-
src/backend/postmaster/fork_process.c | 126 --
src/backend/postmaster/launch_backend.c | 1272 +++++++++++++++
src/backend/postmaster/meson.build | 2 +-
src/backend/postmaster/pgarch.c | 9 +-
src/backend/postmaster/postmaster.c | 1557 +++----------------
src/backend/postmaster/startup.c | 9 +-
src/backend/postmaster/syslogger.c | 275 ++--
src/backend/postmaster/walwriter.c | 9 +-
src/backend/replication/logical/launcher.c | 1 -
src/backend/replication/walreceiver.c | 9 +-
src/backend/storage/ipc/shmem.c | 2 +
src/backend/utils/init/globals.c | 1 +
src/backend/utils/init/miscinit.c | 133 --
src/include/libpq/libpq-be.h | 11 +-
src/include/miscadmin.h | 1 -
src/include/postmaster/autovacuum.h | 12 +-
src/include/postmaster/auxprocess.h | 4 +-
src/include/postmaster/bgworker_internals.h | 2 +-
src/include/postmaster/bgwriter.h | 4 +-
src/include/postmaster/fork_process.h | 17 -
src/include/postmaster/pgarch.h | 2 +-
src/include/postmaster/postmaster.h | 44 +-
src/include/postmaster/startup.h | 2 +-
src/include/postmaster/syslogger.h | 4 +-
src/include/postmaster/walwriter.h | 2 +-
src/include/replication/walreceiver.h | 2 +-
src/tools/pgindent/typedefs.list | 2 +
33 files changed, 1789 insertions(+), 2004 deletions(-)
delete mode 100644 src/backend/postmaster/fork_process.c
create mode 100644 src/backend/postmaster/launch_backend.c
delete mode 100644 src/include/postmaster/fork_process.h
diff --git a/src/backend/postmaster/Makefile b/src/backend/postmaster/Makefile
index 047448b34eb..fc88f5bae0b 100644
--- a/src/backend/postmaster/Makefile
+++ b/src/backend/postmaster/Makefile
@@ -18,8 +18,8 @@ OBJS = \
bgworker.o \
bgwriter.o \
checkpointer.o \
- fork_process.o \
interrupt.o \
+ launch_backend.o \
pgarch.o \
postmaster.o \
startup.o \
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 17f2b36b28a..a6ffe7749be 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -84,7 +84,6 @@
#include "nodes/makefuncs.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "storage/bufmgr.h"
@@ -315,13 +314,6 @@ static WorkerInfo MyWorkerInfo = NULL;
/* PID of launcher, valid only in worker while shutting down */
int AutovacuumLauncherPid = 0;
-#ifdef EXEC_BACKEND
-static pid_t avlauncher_forkexec(void);
-static pid_t avworker_forkexec(void);
-#endif
-NON_EXEC_STATIC void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
-NON_EXEC_STATIC void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
-
static Oid do_start_worker(void);
static void HandleAutoVacLauncherInterrupts(void);
static void AutoVacLauncherShutdown(void) pg_attribute_noreturn();
@@ -365,85 +357,21 @@ static void avl_sigusr2_handler(SIGNAL_ARGS);
* AUTOVACUUM LAUNCHER CODE
********************************************************************/
-#ifdef EXEC_BACKEND
/*
- * forkexec routine for the autovacuum launcher process.
- *
- * Format up the arglist, then fork and exec.
- */
-static pid_t
-avlauncher_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkavlauncher";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-
-/*
- * We need this set from the outside, before InitProcess is called
+ * Main loop for the autovacuum launcher process.
*/
void
-AutovacuumLauncherIAm(void)
-{
- am_autovacuum_launcher = true;
-}
-#endif
-
-/*
- * Main entry point for autovacuum launcher process, to be called from the
- * postmaster.
- */
-int
-StartAutoVacLauncher(void)
+AutoVacLauncherMain(char *startup_data, size_t startup_data_len)
{
- pid_t AutoVacPID;
+ sigjmp_buf local_sigjmp_buf;
-#ifdef EXEC_BACKEND
- switch ((AutoVacPID = avlauncher_forkexec()))
-#else
- switch ((AutoVacPID = fork_process()))
-#endif
+ /* Release postmaster's working memory context */
+ if (PostmasterContext)
{
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum launcher process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacLauncherMain(0, NULL);
- break;
-#endif
- default:
- return (int) AutoVacPID;
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
}
- /* shouldn't get here */
- return 0;
-}
-
-/*
- * Main loop for the autovacuum launcher process.
- */
-NON_EXEC_STATIC void
-AutoVacLauncherMain(int argc, char *argv[])
-{
- sigjmp_buf local_sigjmp_buf;
-
am_autovacuum_launcher = true;
MyBackendType = B_AUTOVAC_LAUNCHER;
@@ -475,6 +403,9 @@ AutoVacLauncherMain(int argc, char *argv[])
pqsignal(SIGFPE, FloatExceptionHandler);
pqsignal(SIGCHLD, SIG_DFL);
+ /* autovacuum needs this set before calling InitProcess */
+ am_autovacuum_launcher = true;
+
/*
* Create a per-backend PGPROC struct in shared memory. We must do this
* before we can use LWLocks.
@@ -1435,87 +1366,22 @@ avl_sigusr2_handler(SIGNAL_ARGS)
* AUTOVACUUM WORKER CODE
********************************************************************/
-#ifdef EXEC_BACKEND
-/*
- * forkexec routines for the autovacuum worker.
- *
- * Format up the arglist, then fork and exec.
- */
-static pid_t
-avworker_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkavworker";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-
/*
- * We need this set from the outside, before InitProcess is called
+ * AutoVacWorkerMain
*/
void
-AutovacuumWorkerIAm(void)
+AutoVacWorkerMain(char *startup_data, size_t startup_data_len)
{
- am_autovacuum_worker = true;
-}
-#endif
-
-/*
- * Main entry point for autovacuum worker process.
- *
- * This code is heavily based on pgarch.c, q.v.
- */
-int
-StartAutoVacWorker(void)
-{
- pid_t worker_pid;
+ sigjmp_buf local_sigjmp_buf;
+ Oid dbid;
-#ifdef EXEC_BACKEND
- switch ((worker_pid = avworker_forkexec()))
-#else
- switch ((worker_pid = fork_process()))
-#endif
+ /* Release postmaster's working memory context */
+ if (PostmasterContext)
{
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum worker process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacWorkerMain(0, NULL);
- break;
-#endif
- default:
- return (int) worker_pid;
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
}
- /* shouldn't get here */
- return 0;
-}
-
-/*
- * AutoVacWorkerMain
- */
-NON_EXEC_STATIC void
-AutoVacWorkerMain(int argc, char *argv[])
-{
- sigjmp_buf local_sigjmp_buf;
- Oid dbid;
-
am_autovacuum_worker = true;
MyBackendType = B_AUTOVAC_WORKER;
@@ -1546,6 +1412,9 @@ AutoVacWorkerMain(int argc, char *argv[])
pqsignal(SIGFPE, FloatExceptionHandler);
pqsignal(SIGCHLD, SIG_DFL);
+ /* autovacuum needs this set before calling InitProcess */
+ am_autovacuum_worker = true;
+
/*
* Create a per-backend PGPROC struct in shared memory. We must do this
* before we can use LWLocks.
diff --git a/src/backend/postmaster/auxprocess.c b/src/backend/postmaster/auxprocess.c
index 536d9a2b3e4..5c2b628bf9d 100644
--- a/src/backend/postmaster/auxprocess.c
+++ b/src/backend/postmaster/auxprocess.c
@@ -46,45 +46,23 @@ AuxProcType MyAuxProcType = NotAnAuxProcess; /* declared in miscadmin.h */
/*
- * AuxiliaryProcessMain
+ * AuxiliaryProcessInit
*
- * The main entry point for auxiliary processes, such as the bgwriter,
+ * Common initialization code for auxiliary processes, such as the bgwriter,
* walwriter, walreceiver, bootstrapper and the shared memory checker code.
- *
- * This code is here just because of historical reasons.
*/
void
-AuxiliaryProcessMain(AuxProcType auxtype)
+AuxiliaryProcessInit(void)
{
- Assert(IsUnderPostmaster);
-
- MyAuxProcType = auxtype;
-
- switch (MyAuxProcType)
+ /* Release postmaster's working memory context */
+ if (PostmasterContext)
{
- case StartupProcess:
- MyBackendType = B_STARTUP;
- break;
- case ArchiverProcess:
- MyBackendType = B_ARCHIVER;
- break;
- case BgWriterProcess:
- MyBackendType = B_BG_WRITER;
- break;
- case CheckpointerProcess:
- MyBackendType = B_CHECKPOINTER;
- break;
- case WalWriterProcess:
- MyBackendType = B_WAL_WRITER;
- break;
- case WalReceiverProcess:
- MyBackendType = B_WAL_RECEIVER;
- break;
- default:
- elog(PANIC, "unrecognized process type: %d", (int) MyAuxProcType);
- MyBackendType = B_INVALID;
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
}
+ Assert(IsUnderPostmaster);
+
init_ps_display(NULL);
SetProcessingMode(BootstrapProcessing);
@@ -125,7 +103,6 @@ AuxiliaryProcessMain(AuxProcType auxtype)
*/
CreateAuxProcessResourceOwner();
-
/* Initialize backend status information */
pgstat_beinit();
pgstat_bestart();
@@ -134,37 +111,6 @@ AuxiliaryProcessMain(AuxProcType auxtype)
before_shmem_exit(ShutdownAuxiliaryProcess, 0);
SetProcessingMode(NormalProcessing);
-
- switch (MyAuxProcType)
- {
- case StartupProcess:
- StartupProcessMain();
- proc_exit(1);
-
- case ArchiverProcess:
- PgArchiverMain();
- proc_exit(1);
-
- case BgWriterProcess:
- BackgroundWriterMain();
- proc_exit(1);
-
- case CheckpointerProcess:
- CheckpointerMain();
- proc_exit(1);
-
- case WalWriterProcess:
- WalWriterMain();
- proc_exit(1);
-
- case WalReceiverProcess:
- WalReceiverMain();
- proc_exit(1);
-
- default:
- elog(PANIC, "unrecognized process type: %d", (int) MyAuxProcType);
- proc_exit(1);
- }
}
/*
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index 15435965bf8..9910691255e 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -723,17 +723,27 @@ bgworker_die(SIGNAL_ARGS)
* Main entry point for background worker processes.
*/
void
-BackgroundWorkerMain(void)
+BackgroundWorkerMain(char *startup_data, size_t startup_data_len)
{
sigjmp_buf local_sigjmp_buf;
- BackgroundWorker *worker = MyBgworkerEntry;
+ BackgroundWorker *worker;
bgworker_main_type entrypt;
+ /* Release postmaster's working memory context */
+ if (PostmasterContext)
+ {
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
+ }
+
+ Assert(startup_data_len == sizeof(BackgroundWorker));
+ worker = (BackgroundWorker *) startup_data;
if (worker == NULL)
elog(FATAL, "unable to find bgworker entry");
IsBackgroundWorker = true;
+ MyBgworkerEntry = worker;
MyBackendType = B_BG_WORKER;
init_ps_display(worker->bgw_name);
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index f2e4f23d9fc..b8b505b25fd 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -36,6 +36,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/bgwriter.h"
#include "postmaster/interrupt.h"
#include "storage/buf_internals.h"
@@ -88,13 +89,19 @@ static XLogRecPtr last_snapshot_lsn = InvalidXLogRecPtr;
* basic execution environment, but not enabled signals yet.
*/
void
-BackgroundWriterMain(void)
+BackgroundWriterMain(char *startup_data, size_t startup_data_len)
{
sigjmp_buf local_sigjmp_buf;
MemoryContext bgwriter_context;
bool prev_hibernate;
WritebackContext wb_context;
+ Assert(startup_data_len == 0);
+
+ MyAuxProcType = BgWriterProcess;
+ MyBackendType = B_BG_WRITER;
+ AuxiliaryProcessInit();
+
/*
* Properly accept or ignore signals that might be sent to us.
*/
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index ace9893d957..e617066c385 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -42,6 +42,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/bgwriter.h"
#include "postmaster/interrupt.h"
#include "replication/syncrep.h"
@@ -178,11 +179,17 @@ static void ReqCheckpointHandler(SIGNAL_ARGS);
* basic execution environment, but not enabled signals yet.
*/
void
-CheckpointerMain(void)
+CheckpointerMain(char *startup_data, size_t startup_data_len)
{
sigjmp_buf local_sigjmp_buf;
MemoryContext checkpointer_context;
+ Assert(startup_data_len == 0);
+
+ MyAuxProcType = CheckpointerProcess;
+ MyBackendType = B_CHECKPOINTER;
+ AuxiliaryProcessInit();
+
CheckpointerShmem->checkpointer_pid = MyProcPid;
/*
diff --git a/src/backend/postmaster/fork_process.c b/src/backend/postmaster/fork_process.c
deleted file mode 100644
index 6f9c2765d68..00000000000
--- a/src/backend/postmaster/fork_process.c
+++ /dev/null
@@ -1,126 +0,0 @@
-/*
- * fork_process.c
- * A simple wrapper on top of fork(). This does not handle the
- * EXEC_BACKEND case; it might be extended to do so, but it would be
- * considerably more complex.
- *
- * Copyright (c) 1996-2023, PostgreSQL Global Development Group
- *
- * IDENTIFICATION
- * src/backend/postmaster/fork_process.c
- */
-#include "postgres.h"
-
-#include <fcntl.h>
-#include <signal.h>
-#include <time.h>
-#include <sys/stat.h>
-#include <sys/time.h>
-#include <unistd.h>
-
-#include "libpq/pqsignal.h"
-#include "postmaster/fork_process.h"
-
-#ifndef WIN32
-/*
- * Wrapper for fork(). Return values are the same as those for fork():
- * -1 if the fork failed, 0 in the child process, and the PID of the
- * child in the parent process. Signals are blocked while forking, so
- * the child must unblock.
- */
-pid_t
-fork_process(void)
-{
- pid_t result;
- const char *oomfilename;
- sigset_t save_mask;
-
-#ifdef LINUX_PROFILE
- struct itimerval prof_itimer;
-#endif
-
- /*
- * Flush stdio channels just before fork, to avoid double-output problems.
- */
- fflush(NULL);
-
-#ifdef LINUX_PROFILE
-
- /*
- * Linux's fork() resets the profiling timer in the child process. If we
- * want to profile child processes then we need to save and restore the
- * timer setting. This is a waste of time if not profiling, however, so
- * only do it if commanded by specific -DLINUX_PROFILE switch.
- */
- getitimer(ITIMER_PROF, &prof_itimer);
-#endif
-
- /*
- * We start postmaster children with signals blocked. This allows them to
- * install their own handlers before unblocking, to avoid races where they
- * might run the postmaster's handler and miss an important control
- * signal. With more analysis this could potentially be relaxed.
- */
- sigprocmask(SIG_SETMASK, &BlockSig, &save_mask);
- result = fork();
- if (result == 0)
- {
- /* fork succeeded, in child */
-#ifdef LINUX_PROFILE
- setitimer(ITIMER_PROF, &prof_itimer, NULL);
-#endif
-
- /*
- * By default, Linux tends to kill the postmaster in out-of-memory
- * situations, because it blames the postmaster for the sum of child
- * process sizes *including shared memory*. (This is unbelievably
- * stupid, but the kernel hackers seem uninterested in improving it.)
- * Therefore it's often a good idea to protect the postmaster by
- * setting its OOM score adjustment negative (which has to be done in
- * a root-owned startup script). Since the adjustment is inherited by
- * child processes, this would ordinarily mean that all the
- * postmaster's children are equally protected against OOM kill, which
- * is not such a good idea. So we provide this code to allow the
- * children to change their OOM score adjustments again. Both the
- * file name to write to and the value to write are controlled by
- * environment variables, which can be set by the same startup script
- * that did the original adjustment.
- */
- oomfilename = getenv("PG_OOM_ADJUST_FILE");
-
- if (oomfilename != NULL)
- {
- /*
- * Use open() not stdio, to ensure we control the open flags. Some
- * Linux security environments reject anything but O_WRONLY.
- */
- int fd = open(oomfilename, O_WRONLY, 0);
-
- /* We ignore all errors */
- if (fd >= 0)
- {
- const char *oomvalue = getenv("PG_OOM_ADJUST_VALUE");
- int rc;
-
- if (oomvalue == NULL) /* supply a useful default */
- oomvalue = "0";
-
- rc = write(fd, oomvalue, strlen(oomvalue));
- (void) rc;
- close(fd);
- }
- }
-
- /* do post-fork initialization for random number generation */
- pg_strong_random_init();
- }
- else
- {
- /* in parent, restore signal mask */
- sigprocmask(SIG_SETMASK, &save_mask, NULL);
- }
-
- return result;
-}
-
-#endif /* ! WIN32 */
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
new file mode 100644
index 00000000000..fac658ba687
--- /dev/null
+++ b/src/backend/postmaster/launch_backend.c
@@ -0,0 +1,1272 @@
+/*-------------------------------------------------------------------------
+ *
+ * launch_backend.c
+ * Functions for launching backends and other postmaster child
+ * processes.
+ *
+ * On Unix systems, a new child process is launched with fork(). It inherits
+ * all the global variables and data structures that had been initialized in
+ * the postmaster. After forking, the child process closes the file
+ * descriptors that are not needed in the child process, and sets up the
+ * mechanism to detect death of the parent postmaster process, etc. After
+ * that, it calls the right Main function depending on the kind of child
+ * process.
+ *
+ * In EXEC_BACKEND mode, which is used on Windows but can be enabled on other
+ * platforms for testing, the child process is launched by fork() + exec() (or
+ * CreateProcess() on Windows). It does not inherit the state from the
+ * postmaster, so it needs to re-attach to the shared memory, re-initialize
+ * global variables, re-load the config file etc.
+ *
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/postmaster/launch_backend.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include <unistd.h>
+#include <signal.h>
+#include <time.h>
+#include <sys/wait.h>
+#include <sys/stat.h>
+#include <sys/socket.h>
+#include <fcntl.h>
+#include <sys/param.h>
+#include <netdb.h>
+#include <limits.h>
+
+#include "access/transam.h"
+#include "access/xlog.h"
+#include "common/file_utils.h"
+#include "libpq/libpq-be.h"
+#include "libpq/pqsignal.h"
+#include "miscadmin.h"
+#include "nodes/queryjumble.h"
+#include "port.h"
+#include "postmaster/autovacuum.h"
+#include "postmaster/auxprocess.h"
+#include "postmaster/bgworker_internals.h"
+#include "postmaster/bgwriter.h"
+#include "postmaster/interrupt.h"
+#include "postmaster/pgarch.h"
+#include "postmaster/postmaster.h"
+#include "postmaster/startup.h"
+#include "postmaster/syslogger.h"
+#include "postmaster/walwriter.h"
+#include "replication/walreceiver.h"
+#include "storage/fd.h"
+#include "storage/ipc.h"
+#include "storage/pg_shmem.h"
+#include "storage/pmsignal.h"
+#include "storage/proc.h"
+#include "tcop/tcopprot.h"
+#include "utils/builtins.h"
+#include "utils/datetime.h"
+#include "utils/guc.h"
+#include "utils/memutils.h"
+#include "utils/ps_status.h"
+#include "utils/timeout.h"
+#include "utils/timestamp.h"
+
+#ifdef EXEC_BACKEND
+#include "storage/spin.h"
+#endif
+
+
+/* Type for a socket that can be inherited to a client process */
+#ifdef WIN32
+typedef struct
+{
+ SOCKET origsocket; /* Original socket value, or PGINVALID_SOCKET
+ * if not a socket */
+ WSAPROTOCOL_INFO wsainfo;
+} InheritableSocket;
+#else
+typedef int InheritableSocket;
+#endif
+
+#ifdef EXEC_BACKEND
+
+/*
+ * Structure contains all global variables passed to exec:ed backends
+ */
+typedef struct
+{
+ char DataDir[MAXPGPATH];
+ int32 MyCancelKey;
+ int MyPMChildSlot;
+#ifndef WIN32
+ unsigned long UsedShmemSegID;
+#else
+ void *ShmemProtectiveRegion;
+ HANDLE UsedShmemSegID;
+#endif
+ void *UsedShmemSegAddr;
+ slock_t *ShmemLock;
+ VariableCache ShmemVariableCache;
+ struct bkend *ShmemBackendArray;
+#ifndef HAVE_SPINLOCKS
+ PGSemaphore *SpinlockSemaArray;
+#endif
+ int NamedLWLockTrancheRequests;
+ NamedLWLockTranche *NamedLWLockTrancheArray;
+ LWLockPadded *MainLWLockArray;
+ slock_t *ProcStructLock;
+ PROC_HDR *ProcGlobal;
+ PGPROC *AuxiliaryProcs;
+ PGPROC *PreparedXactProcs;
+ PMSignalData *PMSignalState;
+ pid_t PostmasterPid;
+ TimestampTz PgStartTime;
+ TimestampTz PgReloadTime;
+ pg_time_t first_syslogger_file_time;
+ bool redirection_done;
+ bool IsBinaryUpgrade;
+ bool query_id_enabled;
+ int max_safe_fds;
+ int MaxBackends;
+#ifdef WIN32
+ HANDLE PostmasterHandle;
+ HANDLE initial_signal_pipe;
+ HANDLE syslogPipe[2];
+#else
+ int postmaster_alive_fds[2];
+ int syslogPipe[2];
+#endif
+ char my_exec_path[MAXPGPATH];
+ char pkglib_path[MAXPGPATH];
+
+ /*
+ * These are only used by backend processes, but it's here because passing
+ * a socket needs some special handling on Windows. 'client_sock' is an
+ * explicit argument to postmaster_child_launch, but is stored in
+ * MyClientSocket in the child process.
+ */
+ ClientSocket client_sock;
+ InheritableSocket inh_sock;
+
+ size_t startup_data_len;
+ /* startup data follows */
+ char startup_data[FLEXIBLE_ARRAY_MEMBER];
+} BackendParameters;
+
+#define SizeOfBackendParameters(startup_data_len) (offsetof(BackendParameters, startup_data) + startup_data_len)
+
+static void read_backend_variables(char *id, char **startup_data, size_t *startup_data_len);
+static void restore_backend_variables(BackendParameters *param);
+
+static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock,
+#ifdef WIN32
+ HANDLE childProcess, pid_t childPid,
+#endif
+ char *startup_data, size_t startup_data_len);
+
+#endif /* EXEC_BACKEND */
+
+static void InitPostmasterChild(bool am_syslogger);
+
+
+#ifdef EXEC_BACKEND
+static pid_t internal_forkexec(const char *main_fn_name, char *startup_data, size_t startup_data_len, ClientSocket *client_sock);
+#endif /* EXEC_BACKEND */
+
+#ifndef WIN32
+static pid_t fork_process(void);
+#endif
+
+/*
+ * Information needed to launch different kinds of child processes.
+ */
+static const struct
+{
+ const char *name;
+ void (*main_fn) (char *startup_data, size_t startup_data_len);
+ bool shmem_attach;
+} entry_kinds[] = {
+ [PMC_BACKEND] = {"backend", BackendMain, true},
+
+ [PMC_AV_LAUNCHER] = {"autovacuum launcher", AutoVacLauncherMain, true},
+ [PMC_AV_WORKER] = {"autovacuum worker", AutoVacWorkerMain, true},
+ [PMC_BGWORKER] = {"bgworker", BackgroundWorkerMain, true},
+ [PMC_SYSLOGGER] = {"syslogger", SysLoggerMain, false},
+
+ [PMC_STARTUP] = {"startup", StartupProcessMain, true},
+ [PMC_BGWRITER] = {"bgwriter", BackgroundWriterMain, true},
+ [PMC_ARCHIVER] = {"archiver", PgArchiverMain, true},
+ [PMC_CHECKPOINTER] = {"checkpointer", CheckpointerMain, true},
+ [PMC_WAL_WRITER] = {"wal_writer", WalWriterMain, true},
+ [PMC_WAL_RECEIVER] = {"wal_receiver", WalReceiverMain, true},
+};
+
+const char *
+PostmasterChildName(PostmasterChildType child_type)
+{
+ Assert(child_type >= 0 && child_type < lengthof(entry_kinds));
+ return entry_kinds[child_type].name;
+}
+
+/*
+ * Start a new postmaster child process.
+ *
+ * The child process will be restored to roughly the same state, whether
+ * EXEC_BACKEND is used or not: it will be attached to shared memory, and fds
+ * and other resources that we've inherited from postmaster that are not
+ * needed in a child process have been closed.
+ *
+ * 'startup_data' is an optional contiguous chunk of data that is passed to
+ * the child process.
+ */
+pid_t
+postmaster_child_launch(PostmasterChildType child_type, char *startup_data, size_t startup_data_len, ClientSocket *client_sock)
+{
+ pid_t pid;
+
+ Assert(child_type >= 0 && child_type < lengthof(entry_kinds));
+ Assert(IsPostmasterEnvironment && !IsUnderPostmaster);
+
+#ifdef EXEC_BACKEND
+ pid = internal_forkexec(entry_kinds[child_type].name, startup_data, startup_data_len, client_sock);
+ /* the child process will arrive in SubPostmasterMain */
+#else /* !EXEC_BACKEND */
+ pid = fork_process();
+ if (pid == 0) /* child */
+ {
+ /* Detangle from postmaster */
+ InitPostmasterChild(child_type == PMC_SYSLOGGER);
+
+ /*
+ * Before blowing away PostmasterContext (in the Main function), save
+ * the startup data.
+ */
+ MemoryContextSwitchTo(TopMemoryContext);
+ if (startup_data != NULL)
+ {
+ char *cp = palloc(startup_data_len);
+
+ memcpy(cp, startup_data, startup_data_len);
+ startup_data = cp;
+ }
+
+ if (client_sock)
+ {
+ MyClientSocket = palloc(sizeof(ClientSocket));
+ memcpy(MyClientSocket, client_sock, sizeof(ClientSocket));
+ }
+
+ entry_kinds[child_type].main_fn(startup_data, startup_data_len);
+ Assert(false);
+ }
+#endif /* EXEC_BACKEND */
+ return pid;
+}
+
+#ifndef WIN32
+/*
+ * Wrapper for fork(). Performs initialization steps that are the same in
+ * !EXEC_BACKEND and EXEC_BACKEND modes.
+ *
+ * Return values are the same as those for fork(): -1 if the fork failed, 0 in
+ * the child process, and the PID of the child in the parent process. Signals
+ * are blocked while forking, so the child must unblock.
+ */
+static pid_t
+fork_process(void)
+{
+ pid_t result;
+ const char *oomfilename;
+ sigset_t save_mask;
+
+#ifdef LINUX_PROFILE
+ struct itimerval prof_itimer;
+#endif
+
+ /*
+ * Flush stdio channels just before fork, to avoid double-output problems.
+ */
+ fflush(NULL);
+
+#ifdef LINUX_PROFILE
+
+ /*
+ * Linux's fork() resets the profiling timer in the child process. If we
+ * want to profile child processes then we need to save and restore the
+ * timer setting. This is a waste of time if not profiling, however, so
+ * only do it if commanded by specific -DLINUX_PROFILE switch.
+ */
+ getitimer(ITIMER_PROF, &prof_itimer);
+#endif
+
+ /*
+ * We start postmaster children with signals blocked. This allows them to
+ * install their own handlers before unblocking, to avoid races where they
+ * might run the postmaster's handler and miss an important control
+ * signal. With more analysis this could potentially be relaxed.
+ */
+ sigprocmask(SIG_SETMASK, &BlockSig, &save_mask);
+ result = fork();
+ if (result == 0)
+ {
+ /* fork succeeded, in child */
+#ifdef LINUX_PROFILE
+ setitimer(ITIMER_PROF, &prof_itimer, NULL);
+#endif
+
+ /*
+ * By default, Linux tends to kill the postmaster in out-of-memory
+ * situations, because it blames the postmaster for the sum of child
+ * process sizes *including shared memory*. (This is unbelievably
+ * stupid, but the kernel hackers seem uninterested in improving it.)
+ * Therefore it's often a good idea to protect the postmaster by
+ * setting its OOM score adjustment negative (which has to be done in
+ * a root-owned startup script). Since the adjustment is inherited by
+ * child processes, this would ordinarily mean that all the
+ * postmaster's children are equally protected against OOM kill, which
+ * is not such a good idea. So we provide this code to allow the
+ * children to change their OOM score adjustments again. Both the
+ * file name to write to and the value to write are controlled by
+ * environment variables, which can be set by the same startup script
+ * that did the original adjustment.
+ */
+ oomfilename = getenv("PG_OOM_ADJUST_FILE");
+
+ if (oomfilename != NULL)
+ {
+ /*
+ * Use open() not stdio, to ensure we control the open flags. Some
+ * Linux security environments reject anything but O_WRONLY.
+ */
+ int fd = open(oomfilename, O_WRONLY, 0);
+
+ /* We ignore all errors */
+ if (fd >= 0)
+ {
+ const char *oomvalue = getenv("PG_OOM_ADJUST_VALUE");
+ int rc;
+
+ if (oomvalue == NULL) /* supply a useful default */
+ oomvalue = "0";
+
+ rc = write(fd, oomvalue, strlen(oomvalue));
+ (void) rc;
+ close(fd);
+ }
+ }
+
+ /* do post-fork initialization for random number generation */
+ pg_strong_random_init();
+ }
+ else
+ {
+ /* in parent, restore signal mask */
+ sigprocmask(SIG_SETMASK, &save_mask, NULL);
+ }
+
+ return result;
+}
+
+#endif /* !WIN32 */
+
+#ifdef EXEC_BACKEND
+#ifndef WIN32
+
+/*
+ * internal_forkexec non-win32 implementation
+ *
+ * - writes out backend variables to the parameter file
+ * - fork():s, and then exec():s the child process
+ */
+static pid_t
+internal_forkexec(const char *entry_name, char *startup_data, size_t startup_data_len, ClientSocket *client_sock)
+{
+ static unsigned long tmpBackendFileNum = 0;
+ pid_t pid;
+ char tmpfilename[MAXPGPATH];
+ size_t paramsz;
+ BackendParameters *param;
+ FILE *fp;
+ char *argv[4];
+ char forkav[MAXPGPATH];
+
+ paramsz = SizeOfBackendParameters(startup_data_len);
+ param = palloc(paramsz);
+ if (!save_backend_variables(param, client_sock, startup_data, startup_data_len))
+ {
+ pfree(param);
+ return -1; /* log made by save_backend_variables */
+ }
+
+ /* Calculate name for temp file */
+ snprintf(tmpfilename, MAXPGPATH, "%s/%s.backend_var.%d.%lu",
+ PG_TEMP_FILES_DIR, PG_TEMP_FILE_PREFIX,
+ MyProcPid, ++tmpBackendFileNum);
+
+ /* Open file */
+ fp = AllocateFile(tmpfilename, PG_BINARY_W);
+ if (!fp)
+ {
+ /*
+ * As in OpenTemporaryFileInTablespace, try to make the temp-file
+ * directory, ignoring errors.
+ */
+ (void) MakePGDirectory(PG_TEMP_FILES_DIR);
+
+ fp = AllocateFile(tmpfilename, PG_BINARY_W);
+ if (!fp)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ tmpfilename)));
+ return -1;
+ }
+ }
+
+ if (fwrite(param, paramsz, 1, fp) != 1)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not write to file \"%s\": %m", tmpfilename)));
+ FreeFile(fp);
+ return -1;
+ }
+
+ /* Release file */
+ if (FreeFile(fp))
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not write to file \"%s\": %m", tmpfilename)));
+ return -1;
+ }
+
+ /* set up argv properly */
+ argv[0] = "postgres";
+ snprintf(forkav, MAXPGPATH, "--forkchild=%s", entry_name);
+ argv[1] = forkav;
+ /* Insert temp file name after --forkchild argument */
+ argv[2] = tmpfilename;
+ argv[3] = NULL;
+
+ /* Fire off execv in child */
+ if ((pid = fork_process()) == 0)
+ {
+ if (execv(postgres_exec_path, argv) < 0)
+ {
+ ereport(LOG,
+ (errmsg("could not execute server process \"%s\": %m",
+ postgres_exec_path)));
+ /* We're already in the child process here, can't return */
+ exit(1);
+ }
+ }
+
+ return pid; /* Parent returns pid, or -1 on fork failure */
+}
+#else /* WIN32 */
+
+/*
+ * internal_forkexec win32 implementation
+ *
+ * - starts backend using CreateProcess(), in suspended state
+ * - writes out backend variables to the parameter file
+ * - during this, duplicates handles and sockets required for
+ * inheritance into the new process
+ * - resumes execution of the new process once the backend parameter
+ * file is complete.
+ */
+static pid_t
+internal_forkexec(const char *entry_name, char *startup_data, size_t startup_data_len, ClientSocket *client_sock)
+{
+ int retry_count = 0;
+ STARTUPINFO si;
+ PROCESS_INFORMATION pi;
+ char cmdLine[MAXPGPATH * 2];
+ HANDLE paramHandle;
+ BackendParameters *param;
+ SECURITY_ATTRIBUTES sa;
+ size_t paramsz;
+ char paramHandleStr[32];
+ int l;
+
+ paramsz = SizeOfBackendParameters(startup_data_len);
+
+ /* Resume here if we need to retry */
+retry:
+
+ /* Set up shared memory for parameter passing */
+ ZeroMemory(&sa, sizeof(sa));
+ sa.nLength = sizeof(sa);
+ sa.bInheritHandle = TRUE;
+ paramHandle = CreateFileMapping(INVALID_HANDLE_VALUE,
+ &sa,
+ PAGE_READWRITE,
+ 0,
+ paramsz,
+ NULL);
+ if (paramHandle == INVALID_HANDLE_VALUE)
+ {
+ ereport(LOG,
+ (errmsg("could not create backend parameter file mapping: error code %lu",
+ GetLastError())));
+ return -1;
+ }
+ param = MapViewOfFile(paramHandle, FILE_MAP_WRITE, 0, 0, paramsz);
+ if (!param)
+ {
+ ereport(LOG,
+ (errmsg("could not map backend parameter memory: error code %lu",
+ GetLastError())));
+ CloseHandle(paramHandle);
+ return -1;
+ }
+
+ /* Format the cmd line */
+#ifdef _WIN64
+ sprintf(paramHandleStr, "%llu", (LONG_PTR) paramHandle);
+#else
+ sprintf(paramHandleStr, "%lu", (DWORD) paramHandle);
+#endif
+ l = snprintf(cmdLine, sizeof(cmdLine) - 1, "\"%s\" --forkchild=\"%s\" %s",
+ postgres_exec_path, entry_name, paramHandleStr);
+ if (l >= sizeof(cmdLine))
+ {
+ ereport(LOG,
+ (errmsg("subprocess command line too long")));
+ UnmapViewOfFile(param);
+ CloseHandle(paramHandle);
+ return -1;
+ }
+
+ memset(&pi, 0, sizeof(pi));
+ memset(&si, 0, sizeof(si));
+ si.cb = sizeof(si);
+
+ /*
+ * Create the subprocess in a suspended state. This will be resumed later,
+ * once we have written out the parameter file.
+ */
+ if (!CreateProcess(NULL, cmdLine, NULL, NULL, TRUE, CREATE_SUSPENDED,
+ NULL, NULL, &si, &pi))
+ {
+ ereport(LOG,
+ (errmsg("CreateProcess() call failed: %m (error code %lu)",
+ GetLastError())));
+ UnmapViewOfFile(param);
+ CloseHandle(paramHandle);
+ return -1;
+ }
+
+ if (!save_backend_variables(param, client_sock, pi.hProcess, pi.dwProcessId, startup_data, startup_data_len))
+ {
+ /*
+ * log made by save_backend_variables, but we have to clean up the
+ * mess with the half-started process
+ */
+ if (!TerminateProcess(pi.hProcess, 255))
+ ereport(LOG,
+ (errmsg_internal("could not terminate unstarted process: error code %lu",
+ GetLastError())));
+ CloseHandle(pi.hProcess);
+ CloseHandle(pi.hThread);
+ UnmapViewOfFile(param);
+ CloseHandle(paramHandle);
+ return -1; /* log made by save_backend_variables */
+ }
+
+ /* Drop the parameter shared memory that is now inherited to the backend */
+ if (!UnmapViewOfFile(param))
+ ereport(LOG,
+ (errmsg("could not unmap view of backend parameter file: error code %lu",
+ GetLastError())));
+ if (!CloseHandle(paramHandle))
+ ereport(LOG,
+ (errmsg("could not close handle to backend parameter file: error code %lu",
+ GetLastError())));
+
+ /*
+ * Reserve the memory region used by our main shared memory segment before
+ * we resume the child process. Normally this should succeed, but if ASLR
+ * is active then it might sometimes fail due to the stack or heap having
+ * gotten mapped into that range. In that case, just terminate the
+ * process and retry.
+ */
+ if (!pgwin32_ReserveSharedMemoryRegion(pi.hProcess))
+ {
+ /* pgwin32_ReserveSharedMemoryRegion already made a log entry */
+ if (!TerminateProcess(pi.hProcess, 255))
+ ereport(LOG,
+ (errmsg_internal("could not terminate process that failed to reserve memory: error code %lu",
+ GetLastError())));
+ CloseHandle(pi.hProcess);
+ CloseHandle(pi.hThread);
+ if (++retry_count < 100)
+ goto retry;
+ ereport(LOG,
+ (errmsg("giving up after too many tries to reserve shared memory"),
+ errhint("This might be caused by ASLR or antivirus software.")));
+ return -1;
+ }
+
+ /*
+ * Now that the backend variables are written out, we start the child
+ * thread so it can start initializing while we set up the rest of the
+ * parent state.
+ */
+ if (ResumeThread(pi.hThread) == -1)
+ {
+ if (!TerminateProcess(pi.hProcess, 255))
+ {
+ ereport(LOG,
+ (errmsg_internal("could not terminate unstartable process: error code %lu",
+ GetLastError())));
+ CloseHandle(pi.hProcess);
+ CloseHandle(pi.hThread);
+ return -1;
+ }
+ CloseHandle(pi.hProcess);
+ CloseHandle(pi.hThread);
+ ereport(LOG,
+ (errmsg_internal("could not resume thread of unstarted process: error code %lu",
+ GetLastError())));
+ return -1;
+ }
+
+ pgwin32_register_deadchild_callback(pi.hProcess, pi.dwProcessId);
+
+ /* Don't close pi.hProcess here - waitpid() needs access to it */
+ CloseHandle(pi.hThread);
+
+ return pi.dwProcessId;
+}
+#endif /* WIN32 */
+
+/*
+ * SubPostmasterMain -- Get the fork/exec'd process into a state equivalent
+ * to what it would be if we'd simply forked on Unix, and then
+ * dispatch to the appropriate place.
+ *
+ * The first two command line arguments are expected to be "--forkchild=<name>",
+ * where <name> indicates which postmaster child we are to become, and
+ * the name of a variables file that we can read to load data that would
+ * have been inherited by fork() on Unix.
+ */
+void
+SubPostmasterMain(int argc, char *argv[])
+{
+ PostmasterChildType child_type;
+ char *startup_data;
+ size_t startup_data_len;
+ char *entry_name;
+ bool found = false;
+
+ /* In EXEC_BACKEND case we will not have inherited these settings */
+ IsPostmasterEnvironment = true;
+ whereToSendOutput = DestNone;
+
+ /* Setup essential subsystems (to ensure elog() behaves sanely) */
+ InitializeGUCOptions();
+
+ /* Check we got appropriate args */
+ if (argc != 3)
+ elog(FATAL, "invalid subpostmaster invocation");
+
+ if (strncmp(argv[1], "--forkchild=", 12) != 0)
+ elog(FATAL, "invalid subpostmaster invocation (--forkchild argument missing)");
+ entry_name = argv[1] + 12;
+ found = false;
+ for (int idx = 0; idx < lengthof(entry_kinds); idx++)
+ {
+ if (strcmp(entry_kinds[idx].name, entry_name) == 0)
+ {
+ child_type = idx;
+ found = true;
+ break;
+ }
+ }
+ if (!found)
+ elog(ERROR, "unknown child kind %s", entry_name);
+
+ /* Read in the variables file */
+ read_backend_variables(argv[2], &startup_data, &startup_data_len);
+
+ /* Setup as postmaster child */
+ InitPostmasterChild(child_type == PMC_SYSLOGGER);
+
+ /*
+ * If appropriate, physically re-attach to shared memory segment. We want
+ * to do this before going any further to ensure that we can attach at the
+ * same address the postmaster used. On the other hand, if we choose not
+ * to re-attach, we may have other cleanup to do.
+ *
+ * If testing EXEC_BACKEND on Linux, you should run this as root before
+ * starting the postmaster:
+ *
+ * sysctl -w kernel.randomize_va_space=0
+ *
+ * This prevents using randomized stack and code addresses that cause the
+ * child process's memory map to be different from the parent's, making it
+ * sometimes impossible to attach to shared memory at the desired address.
+ * Return the setting to its old value (usually '1' or '2') when finished.
+ */
+ if (entry_kinds[child_type].shmem_attach)
+ PGSharedMemoryReAttach();
+ else
+ PGSharedMemoryNoReAttach();
+
+ /* Read in remaining GUC variables */
+ read_nondefault_variables();
+
+ /*
+ * Check that the data directory looks valid, which will also check the
+ * privileges on the data directory and update our umask and file/group
+ * variables for creating files later. Note: this should really be done
+ * before we create any files or directories.
+ */
+ checkDataDir();
+
+ /*
+ * (re-)read control file, as it contains config. The postmaster will
+ * already have read this, but this process doesn't know about that.
+ */
+ LocalProcessControlFile(false);
+
+ /*
+ * Reload any libraries that were preloaded by the postmaster. Since we
+ * exec'd this process, those libraries didn't come along with us; but we
+ * should load them into all child processes to be consistent with the
+ * non-EXEC_BACKEND behavior.
+ */
+ process_shared_preload_libraries();
+
+ /* Restore basic shared memory pointers */
+ if (UsedShmemSegAddr != NULL)
+ InitShmemAccess(UsedShmemSegAddr);
+
+ /* Run backend or appropriate child */
+ entry_kinds[child_type].main_fn(startup_data, startup_data_len);
+
+ abort(); /* shouldn't get here */
+}
+#endif /* EXEC_BACKEND */
+
+/* ----------------------------------------------------------------
+ * common process startup code
+ * ----------------------------------------------------------------
+ */
+
+/*
+ * Initialize the basic environment for a postmaster child
+ *
+ * Should be called as early as possible after the child's startup. However,
+ * on EXEC_BACKEND builds it does need to be after read_backend_variables().
+ */
+static void
+InitPostmasterChild(bool am_syslogger)
+{
+ /* Close the postmaster's sockets (as soon as we know them) */
+ ClosePostmasterPorts(am_syslogger);
+
+ IsUnderPostmaster = true; /* we are a postmaster subprocess now */
+
+ /*
+ * Start our win32 signal implementation. This has to be done after we
+ * read the backend variables, because we need to pick up the signal pipe
+ * from the parent process.
+ */
+#ifdef WIN32
+ pgwin32_signal_initialize();
+#endif
+
+ /*
+ * Set reference point for stack-depth checking. This might seem
+ * redundant in !EXEC_BACKEND builds; but it's not because the postmaster
+ * launches its children from signal handlers, so we might be running on
+ * an alternative stack. XXX still true?
+ */
+ (void) set_stack_base();
+
+ InitProcessGlobals();
+
+ /*
+ * make sure stderr is in binary mode before anything can possibly be
+ * written to it, in case it's actually the syslogger pipe, so the pipe
+ * chunking protocol isn't disturbed. Non-logpipe data gets translated on
+ * redirection (e.g. via pg_ctl -l) anyway.
+ */
+#ifdef WIN32
+ _setmode(fileno(stderr), _O_BINARY);
+#endif
+
+ /* We don't want the postmaster's proc_exit() handlers */
+ on_exit_reset();
+
+ /* In EXEC_BACKEND case we will not have inherited BlockSig etc values */
+#ifdef EXEC_BACKEND
+ pqinitmask();
+#endif
+
+ /* Initialize process-local latch support */
+ InitializeLatchSupport();
+ InitProcessLocalLatch();
+ InitializeLatchWaitSet();
+
+ /*
+ * If possible, make this process a group leader, so that the postmaster
+ * can signal any child processes too. Not all processes will have
+ * children, but for consistency we make all postmaster child processes do
+ * this.
+ */
+#ifdef HAVE_SETSID
+ if (setsid() < 0)
+ elog(FATAL, "setsid() failed: %m");
+#endif
+
+ /*
+ * Every postmaster child process is expected to respond promptly to
+ * SIGQUIT at all times. Therefore we centrally remove SIGQUIT from
+ * BlockSig and install a suitable signal handler. (Client-facing
+ * processes may choose to replace this default choice of handler with
+ * quickdie().) All other blockable signals remain blocked for now.
+ */
+ pqsignal(SIGQUIT, SignalHandlerForCrashExit);
+
+ sigdelset(&BlockSig, SIGQUIT);
+ sigprocmask(SIG_SETMASK, &BlockSig, NULL);
+
+ /* Request a signal if the postmaster dies, if possible. */
+ PostmasterDeathSignalInit();
+
+ /* Don't give the pipe to subprograms that we execute. */
+#ifndef WIN32
+ if (fcntl(postmaster_alive_fds[POSTMASTER_FD_WATCH], F_SETFD, FD_CLOEXEC) < 0)
+ ereport(FATAL,
+ (errcode_for_socket_access(),
+ errmsg_internal("could not set postmaster death monitoring pipe to FD_CLOEXEC mode: %m")));
+#endif
+}
+
+/*
+ * Initialize the basic environment for a standalone process.
+ *
+ * argv0 has to be suitable to find the program's executable.
+ */
+void
+InitStandaloneProcess(const char *argv0)
+{
+ Assert(!IsPostmasterEnvironment);
+
+ MyBackendType = B_STANDALONE_BACKEND;
+
+ /*
+ * Start our win32 signal implementation
+ */
+#ifdef WIN32
+ pgwin32_signal_initialize();
+#endif
+
+ InitProcessGlobals();
+
+ /* Initialize process-local latch support */
+ InitializeLatchSupport();
+ InitProcessLocalLatch();
+ InitializeLatchWaitSet();
+
+ /*
+ * For consistency with InitPostmasterChild, initialize signal mask here.
+ * But we don't unblock SIGQUIT or provide a default handler for it.
+ */
+ pqinitmask();
+ sigprocmask(SIG_SETMASK, &BlockSig, NULL);
+
+ /* Compute paths, no postmaster to inherit from */
+ if (my_exec_path[0] == '\0')
+ {
+ if (find_my_exec(argv0, my_exec_path) < 0)
+ elog(FATAL, "%s: could not locate my own executable path",
+ argv0);
+ }
+
+ if (pkglib_path[0] == '\0')
+ get_pkglib_path(my_exec_path, pkglib_path);
+}
+
+
+#ifdef EXEC_BACKEND
+
+/*
+ * The following need to be available to the save/restore_backend_variables
+ * functions. They are marked NON_EXEC_STATIC in their home modules.
+ */
+extern slock_t *ShmemLock;
+extern slock_t *ProcStructLock;
+extern PGPROC *AuxiliaryProcs;
+extern PMSignalData *PMSignalState;
+extern pg_time_t first_syslogger_file_time;
+extern struct bkend *ShmemBackendArray;
+extern bool redirection_done;
+
+#ifndef WIN32
+#define write_inheritable_socket(dest, src, childpid) ((*(dest) = (src)), true)
+#define read_inheritable_socket(dest, src) (*(dest) = *(src))
+#else
+static bool write_duplicated_handle(HANDLE *dest, HANDLE src, HANDLE child);
+static bool write_inheritable_socket(InheritableSocket *dest, SOCKET src,
+ pid_t childPid);
+static void read_inheritable_socket(SOCKET *dest, InheritableSocket *src);
+#endif
+
+
+/* Save critical backend variables into the BackendParameters struct */
+static bool
+save_backend_variables(BackendParameters *param, ClientSocket *client_sock,
+#ifdef WIN32
+ HANDLE childProcess, pid_t childPid,
+#endif
+ char *startup_data, size_t startup_data_len)
+{
+ if (client_sock)
+ memcpy(¶m->client_sock, client_sock, sizeof(ClientSocket));
+ else
+ memset(¶m->client_sock, 0, sizeof(ClientSocket));
+ if (!write_inheritable_socket(¶m->inh_sock,
+ client_sock ? client_sock->sock : PGINVALID_SOCKET,
+ childPid))
+ return false;
+
+ strlcpy(param->DataDir, DataDir, MAXPGPATH);
+
+ param->MyCancelKey = MyCancelKey;
+ param->MyPMChildSlot = MyPMChildSlot;
+
+#ifdef WIN32
+ param->ShmemProtectiveRegion = ShmemProtectiveRegion;
+#endif
+ param->UsedShmemSegID = UsedShmemSegID;
+ param->UsedShmemSegAddr = UsedShmemSegAddr;
+
+ param->ShmemLock = ShmemLock;
+ param->ShmemVariableCache = ShmemVariableCache;
+ param->ShmemBackendArray = ShmemBackendArray;
+
+#ifndef HAVE_SPINLOCKS
+ param->SpinlockSemaArray = SpinlockSemaArray;
+#endif
+ param->NamedLWLockTrancheRequests = NamedLWLockTrancheRequests;
+ param->NamedLWLockTrancheArray = NamedLWLockTrancheArray;
+ param->MainLWLockArray = MainLWLockArray;
+ param->ProcStructLock = ProcStructLock;
+ param->ProcGlobal = ProcGlobal;
+ param->AuxiliaryProcs = AuxiliaryProcs;
+ param->PreparedXactProcs = PreparedXactProcs;
+ param->PMSignalState = PMSignalState;
+
+ param->PostmasterPid = PostmasterPid;
+ param->PgStartTime = PgStartTime;
+ param->PgReloadTime = PgReloadTime;
+ param->first_syslogger_file_time = first_syslogger_file_time;
+
+ param->redirection_done = redirection_done;
+ param->IsBinaryUpgrade = IsBinaryUpgrade;
+ param->query_id_enabled = query_id_enabled;
+ param->max_safe_fds = max_safe_fds;
+
+ param->MaxBackends = MaxBackends;
+
+#ifdef WIN32
+ param->PostmasterHandle = PostmasterHandle;
+ if (!write_duplicated_handle(¶m->initial_signal_pipe,
+ pgwin32_create_signal_listener(childPid),
+ childProcess))
+ return false;
+#else
+ memcpy(¶m->postmaster_alive_fds, &postmaster_alive_fds,
+ sizeof(postmaster_alive_fds));
+#endif
+
+ memcpy(¶m->syslogPipe, &syslogPipe, sizeof(syslogPipe));
+
+ strlcpy(param->my_exec_path, my_exec_path, MAXPGPATH);
+
+ strlcpy(param->pkglib_path, pkglib_path, MAXPGPATH);
+
+ param->startup_data_len = startup_data_len;
+ memcpy(param->startup_data, startup_data, startup_data_len);
+
+ return true;
+}
+
+
+#ifdef WIN32
+/*
+ * Duplicate a handle for usage in a child process, and write the child
+ * process instance of the handle to the parameter file.
+ */
+static bool
+write_duplicated_handle(HANDLE *dest, HANDLE src, HANDLE childProcess)
+{
+ HANDLE hChild = INVALID_HANDLE_VALUE;
+
+ if (!DuplicateHandle(GetCurrentProcess(),
+ src,
+ childProcess,
+ &hChild,
+ 0,
+ TRUE,
+ DUPLICATE_CLOSE_SOURCE | DUPLICATE_SAME_ACCESS))
+ {
+ ereport(LOG,
+ (errmsg_internal("could not duplicate handle to be written to backend parameter file: error code %lu",
+ GetLastError())));
+ return false;
+ }
+
+ *dest = hChild;
+ return true;
+}
+
+/*
+ * Duplicate a socket for usage in a child process, and write the resulting
+ * structure to the parameter file.
+ * This is required because a number of LSPs (Layered Service Providers) very
+ * common on Windows (antivirus, firewalls, download managers etc) break
+ * straight socket inheritance.
+ */
+static bool
+write_inheritable_socket(InheritableSocket *dest, SOCKET src, pid_t childpid)
+{
+ dest->origsocket = src;
+ if (src != 0 && src != PGINVALID_SOCKET)
+ {
+ /* Actual socket */
+ if (WSADuplicateSocket(src, childpid, &dest->wsainfo) != 0)
+ {
+ ereport(LOG,
+ (errmsg("could not duplicate socket %d for use in backend: error code %d",
+ (int) src, WSAGetLastError())));
+ return false;
+ }
+ }
+ return true;
+}
+
+/*
+ * Read a duplicate socket structure back, and get the socket descriptor.
+ */
+static void
+read_inheritable_socket(SOCKET *dest, InheritableSocket *src)
+{
+ SOCKET s;
+
+ if (src->origsocket == PGINVALID_SOCKET || src->origsocket == 0)
+ {
+ /* Not a real socket! */
+ *dest = src->origsocket;
+ }
+ else
+ {
+ /* Actual socket, so create from structure */
+ s = WSASocket(FROM_PROTOCOL_INFO,
+ FROM_PROTOCOL_INFO,
+ FROM_PROTOCOL_INFO,
+ &src->wsainfo,
+ 0,
+ 0);
+ if (s == INVALID_SOCKET)
+ {
+ write_stderr("could not create inherited socket: error code %d\n",
+ WSAGetLastError());
+ exit(1);
+ }
+ *dest = s;
+
+ /*
+ * To make sure we don't get two references to the same socket, close
+ * the original one. (This would happen when inheritance actually
+ * works..
+ */
+ closesocket(src->origsocket);
+ }
+}
+#endif
+
+static void
+read_backend_variables(char *id, char **startup_data, size_t *startup_data_len)
+{
+ BackendParameters param;
+
+#ifndef WIN32
+ /* Non-win32 implementation reads from file */
+ FILE *fp;
+
+ /* Open file */
+ fp = AllocateFile(id, PG_BINARY_R);
+ if (!fp)
+ {
+ write_stderr("could not open backend variables file \"%s\": %s\n",
+ id, strerror(errno));
+ exit(1);
+ }
+
+ if (fread(¶m, sizeof(param), 1, fp) != 1)
+ {
+ write_stderr("could not read from backend variables file \"%s\": %s\n",
+ id, strerror(errno));
+ exit(1);
+ }
+
+ /* read startup data */
+ *startup_data_len = param.startup_data_len;
+ if (param.startup_data_len > 0)
+ {
+ *startup_data = palloc(*startup_data_len);
+ if (fread(*startup_data, *startup_data_len, 1, fp) != 1)
+ {
+ write_stderr("could not read startup data from backend variables file \"%s\": %s\n",
+ id, strerror(errno));
+ exit(1);
+ }
+ }
+ else
+ *startup_data = NULL;
+
+ /* Release file */
+ FreeFile(fp);
+ if (unlink(id) != 0)
+ {
+ write_stderr("could not remove file \"%s\": %s\n",
+ id, strerror(errno));
+ exit(1);
+ }
+#else
+ /* Win32 version uses mapped file */
+ HANDLE paramHandle;
+ BackendParameters *filep;
+
+#ifdef _WIN64
+ paramHandle = (HANDLE) _atoi64(id);
+#else
+ paramHandle = (HANDLE) atol(id);
+#endif
+ filep = MapViewOfFile(paramHandle, FILE_MAP_READ, 0, 0, 0);
+ if (!filep)
+ {
+ write_stderr("could not map view of backend variables: error code %lu\n",
+ GetLastError());
+ exit(1);
+ }
+
+ memcpy(¶m, filep, sizeof(BackendParameters));
+
+ /* read startup data */
+ *startup_data_len = param.startup_data_len;
+ if (param.startup_data_len > 0)
+ {
+ *startup_data = palloc(param.startup_data_len);
+ memcpy(*startup_data, filep->startup_data, param.startup_data_len);
+ }
+ else
+ *startup_data = NULL;
+
+ if (!UnmapViewOfFile(filep))
+ {
+ write_stderr("could not unmap view of backend variables: error code %lu\n",
+ GetLastError());
+ exit(1);
+ }
+
+ if (!CloseHandle(paramHandle))
+ {
+ write_stderr("could not close handle to backend parameter variables: error code %lu\n",
+ GetLastError());
+ exit(1);
+ }
+#endif
+
+ restore_backend_variables(¶m);
+}
+
+/* Restore critical backend variables from the BackendParameters struct */
+static void
+restore_backend_variables(BackendParameters *param)
+{
+ if (param->client_sock.sock != PGINVALID_SOCKET)
+ {
+ MyClientSocket = MemoryContextAlloc(TopMemoryContext, sizeof(ClientSocket));
+ memcpy(MyClientSocket, ¶m->client_sock, sizeof(ClientSocket));
+ read_inheritable_socket(&MyClientSocket->sock, ¶m->inh_sock);
+ }
+
+ SetDataDir(param->DataDir);
+
+ MyCancelKey = param->MyCancelKey;
+ MyPMChildSlot = param->MyPMChildSlot;
+
+#ifdef WIN32
+ ShmemProtectiveRegion = param->ShmemProtectiveRegion;
+#endif
+ UsedShmemSegID = param->UsedShmemSegID;
+ UsedShmemSegAddr = param->UsedShmemSegAddr;
+
+ ShmemLock = param->ShmemLock;
+ ShmemVariableCache = param->ShmemVariableCache;
+ ShmemBackendArray = param->ShmemBackendArray;
+
+#ifndef HAVE_SPINLOCKS
+ SpinlockSemaArray = param->SpinlockSemaArray;
+#endif
+ NamedLWLockTrancheRequests = param->NamedLWLockTrancheRequests;
+ NamedLWLockTrancheArray = param->NamedLWLockTrancheArray;
+ MainLWLockArray = param->MainLWLockArray;
+ ProcStructLock = param->ProcStructLock;
+ ProcGlobal = param->ProcGlobal;
+ AuxiliaryProcs = param->AuxiliaryProcs;
+ PreparedXactProcs = param->PreparedXactProcs;
+ PMSignalState = param->PMSignalState;
+
+ PostmasterPid = param->PostmasterPid;
+ PgStartTime = param->PgStartTime;
+ PgReloadTime = param->PgReloadTime;
+ first_syslogger_file_time = param->first_syslogger_file_time;
+
+ redirection_done = param->redirection_done;
+ IsBinaryUpgrade = param->IsBinaryUpgrade;
+ query_id_enabled = param->query_id_enabled;
+ max_safe_fds = param->max_safe_fds;
+
+ MaxBackends = param->MaxBackends;
+
+#ifdef WIN32
+ PostmasterHandle = param->PostmasterHandle;
+ pgwin32_initial_signal_pipe = param->initial_signal_pipe;
+#else
+ memcpy(&postmaster_alive_fds, ¶m->postmaster_alive_fds,
+ sizeof(postmaster_alive_fds));
+#endif
+
+ memcpy(&syslogPipe, ¶m->syslogPipe, sizeof(syslogPipe));
+
+ strlcpy(my_exec_path, param->my_exec_path, MAXPGPATH);
+
+ strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+
+ /*
+ * We need to restore fd.c's counts of externally-opened FDs; to avoid
+ * confusion, be sure to do this after restoring max_safe_fds. (Note:
+ * BackendInitialize will handle this for client_sock->sock.)
+ */
+#ifndef WIN32
+ if (postmaster_alive_fds[0] >= 0)
+ ReserveExternalFD();
+ if (postmaster_alive_fds[1] >= 0)
+ ReserveExternalFD();
+#endif
+}
+
+#endif /* EXEC_BACKEND */
diff --git a/src/backend/postmaster/meson.build b/src/backend/postmaster/meson.build
index cda921fd10b..89ff11beb0a 100644
--- a/src/backend/postmaster/meson.build
+++ b/src/backend/postmaster/meson.build
@@ -6,8 +6,8 @@ backend_sources += files(
'bgworker.c',
'bgwriter.c',
'checkpointer.c',
- 'fork_process.c',
'interrupt.c',
+ 'launch_backend.c',
'pgarch.c',
'postmaster.c',
'startup.c',
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 46af3495644..b82d66fab52 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -36,6 +36,7 @@
#include "lib/binaryheap.h"
#include "libpq/pqsignal.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/interrupt.h"
#include "postmaster/pgarch.h"
#include "storage/fd.h"
@@ -211,8 +212,14 @@ PgArchCanRestart(void)
/* Main entry point for archiver process */
void
-PgArchiverMain(void)
+PgArchiverMain(char *startup_data, size_t startup_data_len)
{
+ Assert(startup_data_len == 0);
+
+ MyAuxProcType = ArchiverProcess;
+ MyBackendType = B_ARCHIVER;
+ AuxiliaryProcessInit();
+
/*
* Ignore all signals usually bound to some action in the postmaster,
* except for SIGHUP, SIGTERM, SIGUSR1, SIGUSR2, and SIGQUIT.
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 1b1a44e3ea8..10a4d9540c4 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -2,9 +2,9 @@
*
* postmaster.c
* This program acts as a clearing house for requests to the
- * POSTGRES system. Frontend programs send a startup message
- * to the Postmaster and the postmaster uses the info in the
- * message to setup a backend process.
+ * POSTGRES system. Frontend programs connect to the Postmaster,
+ * and postmaster forks a new backend process to handle the
+ * connection.
*
* The postmaster also manages system-wide operations such as
* startup and shutdown. The postmaster itself doesn't do those
@@ -110,7 +110,6 @@
#include "postmaster/autovacuum.h"
#include "postmaster/auxprocess.h"
#include "postmaster/bgworker_internals.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
@@ -132,10 +131,6 @@
#include "utils/timestamp.h"
#include "utils/varlena.h"
-#ifdef EXEC_BACKEND
-#include "storage/spin.h"
-#endif
-
/*
* Possible types of a backend. Beyond being the possible bkend_type values in
@@ -188,7 +183,7 @@ typedef struct bkend
static dlist_head BackendList = DLIST_STATIC_INIT(BackendList);
#ifdef EXEC_BACKEND
-static Backend *ShmemBackendArray;
+Backend *ShmemBackendArray;
#endif
BackgroundWorker *MyBgworkerEntry = NULL;
@@ -413,7 +408,7 @@ static void StartupPacketTimeoutHandler(void);
static void CleanupBackend(int pid, int exitstatus);
static bool CleanupBackgroundWorker(int pid, int exitstatus);
static void HandleChildCrash(int pid, int exitstatus, const char *procname);
-static void LogChildExit(int lev, const char *procname,
+static void LogChildExit(int lev, const char *procnamBackendInitializee,
int pid, int exitstatus);
static void PostmasterStateMachine(void);
@@ -428,7 +423,6 @@ typedef enum CAC_state
} CAC_state;
static void BackendInitialize(ClientSocket *client_sock, CAC_state cac);
-static void BackendRun(void) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
static int BackendStartup(ClientSocket *port);
@@ -449,7 +443,7 @@ static int CountChildren(int target);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
-static pid_t StartChildProcess(AuxProcType type);
+static pid_t StartChildProcess(PostmasterChildType type);
static void StartAutovacuumWorker(void);
static void MaybeStartWalReceiver(void);
static void InitPostmasterDeathWatchHandle(void);
@@ -484,95 +478,19 @@ typedef struct
} win32_deadchild_waitinfo;
#endif /* WIN32 */
-static pid_t backend_forkexec(ClientSocket *client_sock, CAC_state cac);
-static pid_t internal_forkexec(int argc, char *argv[], ClientSocket *client_sock);
-
-/* Type for a socket that can be inherited to a client process */
-#ifdef WIN32
-typedef struct
-{
- SOCKET origsocket; /* Original socket value, or PGINVALID_SOCKET
- * if not a socket */
- WSAPROTOCOL_INFO wsainfo;
-} InheritableSocket;
-#else
-typedef int InheritableSocket;
-#endif
-
-/*
- * Structure contains all variables passed to exec:ed backends
- */
-typedef struct
-{
- ClientSocket client_sock;
- InheritableSocket serialized_sock;
- char DataDir[MAXPGPATH];
- int32 MyCancelKey;
- int MyPMChildSlot;
-#ifndef WIN32
- unsigned long UsedShmemSegID;
-#else
- void *ShmemProtectiveRegion;
- HANDLE UsedShmemSegID;
-#endif
- void *UsedShmemSegAddr;
- slock_t *ShmemLock;
- VariableCache ShmemVariableCache;
- Backend *ShmemBackendArray;
-#ifndef HAVE_SPINLOCKS
- PGSemaphore *SpinlockSemaArray;
-#endif
- int NamedLWLockTrancheRequests;
- NamedLWLockTranche *NamedLWLockTrancheArray;
- LWLockPadded *MainLWLockArray;
- slock_t *ProcStructLock;
- PROC_HDR *ProcGlobal;
- PGPROC *AuxiliaryProcs;
- PGPROC *PreparedXactProcs;
- PMSignalData *PMSignalState;
- pid_t PostmasterPid;
- TimestampTz PgStartTime;
- TimestampTz PgReloadTime;
- pg_time_t first_syslogger_file_time;
- bool redirection_done;
- bool IsBinaryUpgrade;
- bool query_id_enabled;
- int max_safe_fds;
- int MaxBackends;
-#ifdef WIN32
- HANDLE PostmasterHandle;
- HANDLE initial_signal_pipe;
- HANDLE syslogPipe[2];
-#else
- int postmaster_alive_fds[2];
- int syslogPipe[2];
-#endif
- char my_exec_path[MAXPGPATH];
- char pkglib_path[MAXPGPATH];
-
- BackgroundWorker MyBgworkerEntry;
-} BackendParameters;
-
-static void read_backend_variables(char *id, ClientSocket *client_sock);
-static void restore_backend_variables(BackendParameters *param, ClientSocket *client_sock);
-
-#ifndef WIN32
-static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock);
-#else
-static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock,
- HANDLE childProcess, pid_t childPid);
-#endif
-
static void ShmemBackendArrayAdd(Backend *bn);
static void ShmemBackendArrayRemove(Backend *bn);
#endif /* EXEC_BACKEND */
-#define StartupDataBase() StartChildProcess(StartupProcess)
-#define StartArchiver() StartChildProcess(ArchiverProcess)
-#define StartBackgroundWriter() StartChildProcess(BgWriterProcess)
-#define StartCheckpointer() StartChildProcess(CheckpointerProcess)
-#define StartWalWriter() StartChildProcess(WalWriterProcess)
-#define StartWalReceiver() StartChildProcess(WalReceiverProcess)
+#define StartupDataBase() StartChildProcess(PMC_STARTUP)
+#define StartArchiver() StartChildProcess(PMC_ARCHIVER)
+#define StartBackgroundWriter() StartChildProcess(PMC_BGWRITER)
+#define StartCheckpointer() StartChildProcess(PMC_CHECKPOINTER)
+#define StartWalWriter() StartChildProcess(PMC_WAL_WRITER)
+#define StartWalReceiver() StartChildProcess(PMC_WAL_RECEIVER)
+
+#define StartAutoVacLauncher() StartChildProcess(PMC_AV_LAUNCHER);
+#define StartAutoVacWorker() StartChildProcess(PMC_AV_WORKER);
/* Macros to check exit status of a child process */
#define EXIT_STATUS_0(st) ((st) == 0)
@@ -1115,11 +1033,11 @@ PostmasterMain(int argc, char *argv[])
/*
* Clean out the temp directory used to transmit parameters to child
- * processes (see internal_forkexec, below). We must do this before
- * launching any child processes, else we have a race condition: we could
- * remove a parameter file before the child can read it. It should be
- * safe to do so now, because we verified earlier that there are no
- * conflicting Postgres processes in this data directory.
+ * processes (see internal_forkexec). We must do this before launching
+ * any child processes, else we have a race condition: we could remove a
+ * parameter file before the child can read it. It should be safe to do
+ * so now, because we verified earlier that there are no conflicting
+ * Postgres processes in this data directory.
*/
RemovePgTempFilesInDir(PG_TEMP_FILES_DIR, true, false);
#endif
@@ -3989,6 +3907,11 @@ TerminateChildren(int signal)
signal_child(PgArchPID, signal);
}
+typedef struct BackendStartupInfo
+{
+ CAC_state canAcceptConnections;
+} BackendStartupInfo;
+
/*
* BackendStartup -- start backend process
*
@@ -4001,7 +3924,7 @@ BackendStartup(ClientSocket *client_sock)
{
Backend *bn; /* for backend cleanup */
pid_t pid;
- CAC_state cac;
+ BackendStartupInfo info;
/*
* Create backend data structure. Better before the fork() so we can
@@ -4030,11 +3953,10 @@ BackendStartup(ClientSocket *client_sock)
return STATUS_ERROR;
}
- bn->cancel_key = MyCancelKey;
-
/* Pass down canAcceptConnections state */
- cac = canAcceptConnections(BACKEND_TYPE_NORMAL);
- bn->dead_end = (cac != CAC_OK);
+ info.canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
+ bn->dead_end = (info.canAcceptConnections != CAC_OK);
+ bn->cancel_key = MyCancelKey;
/*
* Unless it's a dead_end child, assign it a child slot number
@@ -4047,26 +3969,7 @@ BackendStartup(ClientSocket *client_sock)
/* Hasn't asked to be notified about any bgworkers yet */
bn->bgworker_notify = false;
-#ifdef EXEC_BACKEND
- pid = backend_forkexec(client_sock, cac);
-#else /* !EXEC_BACKEND */
- pid = fork_process();
- if (pid == 0) /* child */
- {
- /* Detangle from postmaster */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Perform additional initialization and collect startup packet */
- BackendInitialize(client_sock, cac);
-
- /* And run the backend */
- BackendRun();
- }
-#endif /* EXEC_BACKEND */
-
+ pid = postmaster_child_launch(PMC_BACKEND, (char *) &info, sizeof(info), client_sock);
if (pid < 0)
{
/* in parent, fork failed */
@@ -4103,6 +4006,61 @@ BackendStartup(ClientSocket *client_sock)
return STATUS_OK;
}
+void
+BackendMain(char *startup_data, size_t startup_data_len)
+{
+ BackendStartupInfo *info = (BackendStartupInfo *) startup_data;
+
+ Assert(startup_data_len == sizeof(BackendStartupInfo));
+ Assert(MyClientSocket != NULL);
+
+#ifdef EXEC_BACKEND
+
+ /*
+ * Need to reinitialize the SSL library in the backend, since the context
+ * structures contain function pointers and cannot be passed through the
+ * parameter file.
+ *
+ * If for some reason reload fails (maybe the user installed broken key
+ * files), soldier on without SSL; that's better than all connections
+ * becoming impossible.
+ *
+ * XXX should we do this in all child processes? For the moment it's
+ * enough to do it in backend children.
+ */
+#ifdef USE_SSL
+ if (EnableSSL)
+ {
+ if (secure_initialize(false) == 0)
+ LoadedSSL = true;
+ else
+ ereport(LOG,
+ (errmsg("SSL configuration could not be loaded in child process")));
+ }
+#endif
+#endif
+
+ /* Perform additional initialization and collect startup packet */
+ BackendInitialize(MyClientSocket, info->canAcceptConnections);
+
+ /*
+ * Create a per-backend PGPROC struct in shared memory. We must do this
+ * before we can use LWLocks.
+ */
+ InitProcess();
+
+ /* Attach process to shared data structures */
+ AttachSharedMemoryAndSemaphores();
+
+ /*
+ * Make sure we aren't in PostmasterContext anymore. (We can't delete it
+ * just yet, though, because InitPostgres will need the HBA data.)
+ */
+ MemoryContextSwitchTo(TopMemoryContext);
+
+ PostgresMain(MyProcPort->database_name, MyProcPort->user_name);
+}
+
/*
* Try to report backend fork() failure to client before we close the
* connection. Since we do not care to risk blocking the postmaster on
@@ -4378,732 +4336,161 @@ BackendInitialize(ClientSocket *client_sock, CAC_state cac)
/*
- * BackendRun -- set up the backend's argument list and invoke PostgresMain()
+ * ExitPostmaster -- cleanup
*
- * returns:
- * Doesn't return at all.
+ * Do NOT call exit() directly --- always go through here!
*/
static void
-BackendRun(void)
+ExitPostmaster(int status)
{
+#ifdef HAVE_PTHREAD_IS_THREADED_NP
+
/*
- * Create a per-backend PGPROC struct in shared memory. We must do this
- * before we can use LWLocks (in AttachSharedMemoryAndSemaphores).
+ * There is no known cause for a postmaster to become multithreaded after
+ * startup. Recheck to account for the possibility of unknown causes.
+ * This message uses LOG level, because an unclean shutdown at this point
+ * would usually not look much different from a clean shutdown.
*/
- InitProcess();
+ if (pthread_is_threaded_np() != 0)
+ ereport(LOG,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg_internal("postmaster became multithreaded"),
+ errdetail("Please report this to <%s>.", PACKAGE_BUGREPORT)));
+#endif
- /* Attach process to shared data structures */
- AttachSharedMemoryAndSemaphores();
+ /* should cleanup shared memory and kill all backends */
/*
- * Make sure we aren't in PostmasterContext anymore. (We can't delete it
- * just yet, though, because InitPostgres will need the HBA data.)
+ * Not sure of the semantics here. When the Postmaster dies, should the
+ * backends all be killed? probably not.
+ *
+ * MUST -- vadim 05-10-1999
*/
- MemoryContextSwitchTo(TopMemoryContext);
-
- PostgresMain(MyProcPort->database_name, MyProcPort->user_name);
-}
-
-
-#ifdef EXEC_BACKEND
-
-/*
- * postmaster_forkexec -- fork and exec a postmaster subprocess
- *
- * The caller must have set up the argv array already, except for argv[2]
- * which will be filled with the name of the temp variable file.
- *
- * Returns the child process PID, or -1 on fork failure (a suitable error
- * message has been logged on failure).
- *
- * All uses of this routine will dispatch to SubPostmasterMain in the
- * child process.
- */
-pid_t
-postmaster_forkexec(int argc, char *argv[])
-{
- ClientSocket client_sock;
-
- /* This entry point doesn't pass a client socket */
- memset(&client_sock, 0, sizeof(ClientSocket));
- return internal_forkexec(argc, argv, &client_sock);
-}
-
-/*
- * backend_forkexec -- fork/exec off a backend process
- *
- * Some operating systems (WIN32) don't have fork() so we have to simulate
- * it by storing parameters that need to be passed to the child and
- * then create a new child process.
- *
- * returns the pid of the fork/exec'd process, or -1 on failure
- */
-static pid_t
-backend_forkexec(ClientSocket *client_sock, CAC_state cac)
-{
- char *av[5];
- int ac = 0;
- char cacbuf[10];
- av[ac++] = "postgres";
- av[ac++] = "--forkbackend";
- av[ac++] = NULL; /* filled in by internal_forkexec */
-
- snprintf(cacbuf, sizeof(cacbuf), "%d", (int) cac);
- av[ac++] = cacbuf;
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return internal_forkexec(ac, av, client_sock);
+ proc_exit(status);
}
-#ifndef WIN32
-
/*
- * internal_forkexec non-win32 implementation
- *
- * - writes out backend variables to the parameter file
- * - fork():s, and then exec():s the child process
+ * Handle pmsignal conditions representing requests from backends,
+ * and check for promote and logrotate requests from pg_ctl.
*/
-static pid_t
-internal_forkexec(int argc, char *argv[], ClientSocket *client_sock)
+static void
+process_pm_pmsignal(void)
{
- static unsigned long tmpBackendFileNum = 0;
- pid_t pid;
- char tmpfilename[MAXPGPATH];
- BackendParameters param;
- FILE *fp;
-
- if (!save_backend_variables(¶m, client_sock))
- return -1; /* log made by save_backend_variables */
+ pending_pm_pmsignal = false;
- /* Calculate name for temp file */
- snprintf(tmpfilename, MAXPGPATH, "%s/%s.backend_var.%d.%lu",
- PG_TEMP_FILES_DIR, PG_TEMP_FILE_PREFIX,
- MyProcPid, ++tmpBackendFileNum);
+ ereport(DEBUG2,
+ (errmsg_internal("postmaster received pmsignal signal")));
- /* Open file */
- fp = AllocateFile(tmpfilename, PG_BINARY_W);
- if (!fp)
+ /*
+ * RECOVERY_STARTED and BEGIN_HOT_STANDBY signals are ignored in
+ * unexpected states. If the startup process quickly starts up, completes
+ * recovery, exits, we might process the death of the startup process
+ * first. We don't want to go back to recovery in that case.
+ */
+ if (CheckPostmasterSignal(PMSIGNAL_RECOVERY_STARTED) &&
+ pmState == PM_STARTUP && Shutdown == NoShutdown)
{
+ /* WAL redo has started. We're out of reinitialization. */
+ FatalError = false;
+ AbortStartTime = 0;
+
/*
- * As in OpenTemporaryFileInTablespace, try to make the temp-file
- * directory, ignoring errors.
+ * Start the archiver if we're responsible for (re-)archiving received
+ * files.
*/
- (void) MakePGDirectory(PG_TEMP_FILES_DIR);
+ Assert(PgArchPID == 0);
+ if (XLogArchivingAlways())
+ PgArchPID = StartArchiver();
- fp = AllocateFile(tmpfilename, PG_BINARY_W);
- if (!fp)
+ /*
+ * If we aren't planning to enter hot standby mode later, treat
+ * RECOVERY_STARTED as meaning we're out of startup, and report status
+ * accordingly.
+ */
+ if (!EnableHotStandby)
{
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not create file \"%s\": %m",
- tmpfilename)));
- return -1;
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+#ifdef USE_SYSTEMD
+ sd_notify(0, "READY=1");
+#endif
}
+
+ pmState = PM_RECOVERY;
}
- if (fwrite(¶m, sizeof(param), 1, fp) != 1)
+ if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
+ pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not write to file \"%s\": %m", tmpfilename)));
- FreeFile(fp);
- return -1;
+ (errmsg("database system is ready to accept read-only connections")));
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_READY);
+#ifdef USE_SYSTEMD
+ sd_notify(0, "READY=1");
+#endif
+
+ pmState = PM_HOT_STANDBY;
+ connsAllowed = true;
+
+ /* Some workers may be scheduled to start now */
+ StartWorkerNeeded = true;
}
- /* Release file */
- if (FreeFile(fp))
+ /* Process background worker state changes. */
+ if (CheckPostmasterSignal(PMSIGNAL_BACKGROUND_WORKER_CHANGE))
{
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not write to file \"%s\": %m", tmpfilename)));
- return -1;
+ /* Accept new worker requests only if not stopping. */
+ BackgroundWorkerStateChange(pmState < PM_STOP_BACKENDS);
+ StartWorkerNeeded = true;
}
- /* Make sure caller set up argv properly */
- Assert(argc >= 3);
- Assert(argv[argc] == NULL);
- Assert(strncmp(argv[1], "--fork", 6) == 0);
- Assert(argv[2] == NULL);
-
- /* Insert temp file name after --fork argument */
- argv[2] = tmpfilename;
+ if (StartWorkerNeeded || HaveCrashedWorker)
+ maybe_start_bgworkers();
- /* Fire off execv in child */
- if ((pid = fork_process()) == 0)
+ /* Tell syslogger to rotate logfile if requested */
+ if (SysLoggerPID != 0)
{
- if (execv(postgres_exec_path, argv) < 0)
+ if (CheckLogrotateSignal())
{
- ereport(LOG,
- (errmsg("could not execute server process \"%s\": %m",
- postgres_exec_path)));
- /* We're already in the child process here, can't return */
- exit(1);
+ signal_child(SysLoggerPID, SIGUSR1);
+ RemoveLogrotateSignalFiles();
+ }
+ else if (CheckPostmasterSignal(PMSIGNAL_ROTATE_LOGFILE))
+ {
+ signal_child(SysLoggerPID, SIGUSR1);
}
}
- return pid; /* Parent returns pid, or -1 on fork failure */
-}
-#else /* WIN32 */
-
-/*
- * internal_forkexec win32 implementation
- *
- * - starts backend using CreateProcess(), in suspended state
- * - writes out backend variables to the parameter file
- * - during this, duplicates handles and sockets required for
- * inheritance into the new process
- * - resumes execution of the new process once the backend parameter
- * file is complete.
- */
-static pid_t
-internal_forkexec(int argc, char *argv[], Port *port)
-{
- int retry_count = 0;
- STARTUPINFO si;
- PROCESS_INFORMATION pi;
- int i;
- int j;
- char cmdLine[MAXPGPATH * 2];
- HANDLE paramHandle;
- BackendParameters *param;
- SECURITY_ATTRIBUTES sa;
- char paramHandleStr[32];
- win32_deadchild_waitinfo *childinfo;
-
- /* Make sure caller set up argv properly */
- Assert(argc >= 3);
- Assert(argv[argc] == NULL);
- Assert(strncmp(argv[1], "--fork", 6) == 0);
- Assert(argv[2] == NULL);
-
- /* Resume here if we need to retry */
-retry:
-
- /* Set up shared memory for parameter passing */
- ZeroMemory(&sa, sizeof(sa));
- sa.nLength = sizeof(sa);
- sa.bInheritHandle = TRUE;
- paramHandle = CreateFileMapping(INVALID_HANDLE_VALUE,
- &sa,
- PAGE_READWRITE,
- 0,
- sizeof(BackendParameters),
- NULL);
- if (paramHandle == INVALID_HANDLE_VALUE)
+ if (CheckPostmasterSignal(PMSIGNAL_START_AUTOVAC_LAUNCHER) &&
+ Shutdown <= SmartShutdown && pmState < PM_STOP_BACKENDS)
{
- ereport(LOG,
- (errmsg("could not create backend parameter file mapping: error code %lu",
- GetLastError())));
- return -1;
+ /*
+ * Start one iteration of the autovacuum daemon, even if autovacuuming
+ * is nominally not enabled. This is so we can have an active defense
+ * against transaction ID wraparound. We set a flag for the main loop
+ * to do it rather than trying to do it here --- this is because the
+ * autovac process itself may send the signal, and we want to handle
+ * that by launching another iteration as soon as the current one
+ * completes.
+ */
+ start_autovac_launcher = true;
}
- param = MapViewOfFile(paramHandle, FILE_MAP_WRITE, 0, 0, sizeof(BackendParameters));
- if (!param)
+ if (CheckPostmasterSignal(PMSIGNAL_START_AUTOVAC_WORKER) &&
+ Shutdown <= SmartShutdown && pmState < PM_STOP_BACKENDS)
{
- ereport(LOG,
- (errmsg("could not map backend parameter memory: error code %lu",
- GetLastError())));
- CloseHandle(paramHandle);
- return -1;
+ /* The autovacuum launcher wants us to start a worker process. */
+ StartAutovacuumWorker();
}
- /* Insert temp file name after --fork argument */
-#ifdef _WIN64
- sprintf(paramHandleStr, "%llu", (LONG_PTR) paramHandle);
-#else
- sprintf(paramHandleStr, "%lu", (DWORD) paramHandle);
-#endif
- argv[2] = paramHandleStr;
-
- /* Format the cmd line */
- cmdLine[sizeof(cmdLine) - 1] = '\0';
- cmdLine[sizeof(cmdLine) - 2] = '\0';
- snprintf(cmdLine, sizeof(cmdLine) - 1, "\"%s\"", postgres_exec_path);
- i = 0;
- while (argv[++i] != NULL)
- {
- j = strlen(cmdLine);
- snprintf(cmdLine + j, sizeof(cmdLine) - 1 - j, " \"%s\"", argv[i]);
- }
- if (cmdLine[sizeof(cmdLine) - 2] != '\0')
+ if (CheckPostmasterSignal(PMSIGNAL_START_WALRECEIVER))
{
- ereport(LOG,
- (errmsg("subprocess command line too long")));
- UnmapViewOfFile(param);
- CloseHandle(paramHandle);
- return -1;
- }
-
- memset(&pi, 0, sizeof(pi));
- memset(&si, 0, sizeof(si));
- si.cb = sizeof(si);
-
- /*
- * Create the subprocess in a suspended state. This will be resumed later,
- * once we have written out the parameter file.
- */
- if (!CreateProcess(NULL, cmdLine, NULL, NULL, TRUE, CREATE_SUSPENDED,
- NULL, NULL, &si, &pi))
- {
- ereport(LOG,
- (errmsg("CreateProcess() call failed: %m (error code %lu)",
- GetLastError())));
- UnmapViewOfFile(param);
- CloseHandle(paramHandle);
- return -1;
- }
-
- if (!save_backend_variables(param, port, pi.hProcess, pi.dwProcessId))
- {
- /*
- * log made by save_backend_variables, but we have to clean up the
- * mess with the half-started process
- */
- if (!TerminateProcess(pi.hProcess, 255))
- ereport(LOG,
- (errmsg_internal("could not terminate unstarted process: error code %lu",
- GetLastError())));
- CloseHandle(pi.hProcess);
- CloseHandle(pi.hThread);
- UnmapViewOfFile(param);
- CloseHandle(paramHandle);
- return -1; /* log made by save_backend_variables */
- }
-
- /* Drop the parameter shared memory that is now inherited to the backend */
- if (!UnmapViewOfFile(param))
- ereport(LOG,
- (errmsg("could not unmap view of backend parameter file: error code %lu",
- GetLastError())));
- if (!CloseHandle(paramHandle))
- ereport(LOG,
- (errmsg("could not close handle to backend parameter file: error code %lu",
- GetLastError())));
-
- /*
- * Reserve the memory region used by our main shared memory segment before
- * we resume the child process. Normally this should succeed, but if ASLR
- * is active then it might sometimes fail due to the stack or heap having
- * gotten mapped into that range. In that case, just terminate the
- * process and retry.
- */
- if (!pgwin32_ReserveSharedMemoryRegion(pi.hProcess))
- {
- /* pgwin32_ReserveSharedMemoryRegion already made a log entry */
- if (!TerminateProcess(pi.hProcess, 255))
- ereport(LOG,
- (errmsg_internal("could not terminate process that failed to reserve memory: error code %lu",
- GetLastError())));
- CloseHandle(pi.hProcess);
- CloseHandle(pi.hThread);
- if (++retry_count < 100)
- goto retry;
- ereport(LOG,
- (errmsg("giving up after too many tries to reserve shared memory"),
- errhint("This might be caused by ASLR or antivirus software.")));
- return -1;
- }
-
- /*
- * Now that the backend variables are written out, we start the child
- * thread so it can start initializing while we set up the rest of the
- * parent state.
- */
- if (ResumeThread(pi.hThread) == -1)
- {
- if (!TerminateProcess(pi.hProcess, 255))
- {
- ereport(LOG,
- (errmsg_internal("could not terminate unstartable process: error code %lu",
- GetLastError())));
- CloseHandle(pi.hProcess);
- CloseHandle(pi.hThread);
- return -1;
- }
- CloseHandle(pi.hProcess);
- CloseHandle(pi.hThread);
- ereport(LOG,
- (errmsg_internal("could not resume thread of unstarted process: error code %lu",
- GetLastError())));
- return -1;
- }
-
- /*
- * Queue a waiter to signal when this child dies. The wait will be handled
- * automatically by an operating system thread pool. The memory will be
- * freed by a later call to waitpid().
- */
- childinfo = palloc(sizeof(win32_deadchild_waitinfo));
- childinfo->procHandle = pi.hProcess;
- childinfo->procId = pi.dwProcessId;
-
- if (!RegisterWaitForSingleObject(&childinfo->waitHandle,
- pi.hProcess,
- pgwin32_deadchild_callback,
- childinfo,
- INFINITE,
- WT_EXECUTEONLYONCE | WT_EXECUTEINWAITTHREAD))
- ereport(FATAL,
- (errmsg_internal("could not register process for wait: error code %lu",
- GetLastError())));
-
- /* Don't close pi.hProcess here - waitpid() needs access to it */
-
- CloseHandle(pi.hThread);
-
- return pi.dwProcessId;
-}
-#endif /* WIN32 */
-
-
-/*
- * SubPostmasterMain -- Get the fork/exec'd process into a state equivalent
- * to what it would be if we'd simply forked on Unix, and then
- * dispatch to the appropriate place.
- *
- * The first two command line arguments are expected to be "--forkFOO"
- * (where FOO indicates which postmaster child we are to become), and
- * the name of a variables file that we can read to load data that would
- * have been inherited by fork() on Unix. Remaining arguments go to the
- * subprocess FooMain() routine.
- */
-void
-SubPostmasterMain(int argc, char *argv[])
-{
- ClientSocket client_sock;
-
- /* In EXEC_BACKEND case we will not have inherited these settings */
- IsPostmasterEnvironment = true;
- whereToSendOutput = DestNone;
-
- /* Setup essential subsystems (to ensure elog() behaves sanely) */
- InitializeGUCOptions();
-
- /* Check we got appropriate args */
- if (argc < 3)
- elog(FATAL, "invalid subpostmaster invocation");
-
- /* Read in the variables file */
- memset(&client_sock, 0, sizeof(ClientSocket));
- read_backend_variables(argv[2], &client_sock);
-
- /* Close the postmaster's sockets (as soon as we know them) */
- ClosePostmasterPorts(strcmp(argv[1], "--forklog") == 0);
-
- /* Setup as postmaster child */
- InitPostmasterChild();
-
- /*
- * If appropriate, physically re-attach to shared memory segment. We want
- * to do this before going any further to ensure that we can attach at the
- * same address the postmaster used. On the other hand, if we choose not
- * to re-attach, we may have other cleanup to do.
- *
- * If testing EXEC_BACKEND on Linux, you should run this as root before
- * starting the postmaster:
- *
- * sysctl -w kernel.randomize_va_space=0
- *
- * This prevents using randomized stack and code addresses that cause the
- * child process's memory map to be different from the parent's, making it
- * sometimes impossible to attach to shared memory at the desired address.
- * Return the setting to its old value (usually '1' or '2') when finished.
- */
- if (strcmp(argv[1], "--forkbackend") == 0 ||
- strcmp(argv[1], "--forkavlauncher") == 0 ||
- strcmp(argv[1], "--forkavworker") == 0 ||
- strcmp(argv[1], "--forkaux") == 0 ||
- strncmp(argv[1], "--forkbgworker", 14) == 0)
- PGSharedMemoryReAttach();
- else
- PGSharedMemoryNoReAttach();
-
- /* autovacuum needs this set before calling InitProcess */
- if (strcmp(argv[1], "--forkavlauncher") == 0)
- AutovacuumLauncherIAm();
- if (strcmp(argv[1], "--forkavworker") == 0)
- AutovacuumWorkerIAm();
-
- /* Read in remaining GUC variables */
- read_nondefault_variables();
-
- /*
- * Check that the data directory looks valid, which will also check the
- * privileges on the data directory and update our umask and file/group
- * variables for creating files later. Note: this should really be done
- * before we create any files or directories.
- */
- checkDataDir();
-
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
- /*
- * Reload any libraries that were preloaded by the postmaster. Since we
- * exec'd this process, those libraries didn't come along with us; but we
- * should load them into all child processes to be consistent with the
- * non-EXEC_BACKEND behavior.
- */
- process_shared_preload_libraries();
-
- /* Run backend or appropriate child */
- if (strcmp(argv[1], "--forkbackend") == 0)
- {
- CAC_state cac;
-
- Assert(argc == 4);
- cac = (CAC_state) atoi(argv[3]);
-
- /*
- * Need to reinitialize the SSL library in the backend, since the
- * context structures contain function pointers and cannot be passed
- * through the parameter file.
- *
- * If for some reason reload fails (maybe the user installed broken
- * key files), soldier on without SSL; that's better than all
- * connections becoming impossible.
- *
- * XXX should we do this in all child processes? For the moment it's
- * enough to do it in backend children.
- */
-#ifdef USE_SSL
- if (EnableSSL)
- {
- if (secure_initialize(false) == 0)
- LoadedSSL = true;
- else
- ereport(LOG,
- (errmsg("SSL configuration could not be loaded in child process")));
- }
-#endif
-
- /*
- * Perform additional initialization and collect startup packet.
- *
- * We want to do this before InitProcess() for a couple of reasons: 1.
- * so that we aren't eating up a PGPROC slot while waiting on the
- * client. 2. so that if InitProcess() fails due to being out of
- * PGPROC slots, we have already initialized libpq and are able to
- * report the error to the client.
- */
- BackendInitialize(&client_sock, cac);
-
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* And run the backend */
- BackendRun(); /* does not return */
- }
- if (strcmp(argv[1], "--forkaux") == 0)
- {
- AuxProcType auxtype;
-
- Assert(argc == 4);
-
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- auxtype = atoi(argv[3]);
- AuxiliaryProcessMain(auxtype); /* does not return */
- }
- if (strcmp(argv[1], "--forkavlauncher") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- AutoVacLauncherMain(argc - 2, argv + 2); /* does not return */
- }
- if (strcmp(argv[1], "--forkavworker") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
- }
- if (strncmp(argv[1], "--forkbgworker", 14) == 0)
- {
- /* do this as early as possible; in particular, before InitProcess() */
- IsBackgroundWorker = true;
-
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- StartBackgroundWorker();
- }
- if (strcmp(argv[1], "--forklog") == 0)
- {
- /* Do not want to attach to shared memory */
-
- SysLoggerMain(argc, argv); /* does not return */
- }
-
- abort(); /* shouldn't get here */
-}
-#endif /* EXEC_BACKEND */
-
-
-/*
- * ExitPostmaster -- cleanup
- *
- * Do NOT call exit() directly --- always go through here!
- */
-static void
-ExitPostmaster(int status)
-{
-#ifdef HAVE_PTHREAD_IS_THREADED_NP
-
- /*
- * There is no known cause for a postmaster to become multithreaded after
- * startup. Recheck to account for the possibility of unknown causes.
- * This message uses LOG level, because an unclean shutdown at this point
- * would usually not look much different from a clean shutdown.
- */
- if (pthread_is_threaded_np() != 0)
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg_internal("postmaster became multithreaded"),
- errdetail("Please report this to <%s>.", PACKAGE_BUGREPORT)));
-#endif
-
- /* should cleanup shared memory and kill all backends */
-
- /*
- * Not sure of the semantics here. When the Postmaster dies, should the
- * backends all be killed? probably not.
- *
- * MUST -- vadim 05-10-1999
- */
-
- proc_exit(status);
-}
-
-/*
- * Handle pmsignal conditions representing requests from backends,
- * and check for promote and logrotate requests from pg_ctl.
- */
-static void
-process_pm_pmsignal(void)
-{
- pending_pm_pmsignal = false;
-
- ereport(DEBUG2,
- (errmsg_internal("postmaster received pmsignal signal")));
-
- /*
- * RECOVERY_STARTED and BEGIN_HOT_STANDBY signals are ignored in
- * unexpected states. If the startup process quickly starts up, completes
- * recovery, exits, we might process the death of the startup process
- * first. We don't want to go back to recovery in that case.
- */
- if (CheckPostmasterSignal(PMSIGNAL_RECOVERY_STARTED) &&
- pmState == PM_STARTUP && Shutdown == NoShutdown)
- {
- /* WAL redo has started. We're out of reinitialization. */
- FatalError = false;
- AbortStartTime = 0;
-
- /*
- * Start the archiver if we're responsible for (re-)archiving received
- * files.
- */
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
- PgArchPID = StartArchiver();
-
- /*
- * If we aren't planning to enter hot standby mode later, treat
- * RECOVERY_STARTED as meaning we're out of startup, and report status
- * accordingly.
- */
- if (!EnableHotStandby)
- {
- AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
-#ifdef USE_SYSTEMD
- sd_notify(0, "READY=1");
-#endif
- }
-
- pmState = PM_RECOVERY;
- }
-
- if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
- pmState == PM_RECOVERY && Shutdown == NoShutdown)
- {
- ereport(LOG,
- (errmsg("database system is ready to accept read-only connections")));
-
- /* Report status */
- AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_READY);
-#ifdef USE_SYSTEMD
- sd_notify(0, "READY=1");
-#endif
-
- pmState = PM_HOT_STANDBY;
- connsAllowed = true;
-
- /* Some workers may be scheduled to start now */
- StartWorkerNeeded = true;
- }
-
- /* Process background worker state changes. */
- if (CheckPostmasterSignal(PMSIGNAL_BACKGROUND_WORKER_CHANGE))
- {
- /* Accept new worker requests only if not stopping. */
- BackgroundWorkerStateChange(pmState < PM_STOP_BACKENDS);
- StartWorkerNeeded = true;
- }
-
- if (StartWorkerNeeded || HaveCrashedWorker)
- maybe_start_bgworkers();
-
- /* Tell syslogger to rotate logfile if requested */
- if (SysLoggerPID != 0)
- {
- if (CheckLogrotateSignal())
- {
- signal_child(SysLoggerPID, SIGUSR1);
- RemoveLogrotateSignalFiles();
- }
- else if (CheckPostmasterSignal(PMSIGNAL_ROTATE_LOGFILE))
- {
- signal_child(SysLoggerPID, SIGUSR1);
- }
- }
-
- if (CheckPostmasterSignal(PMSIGNAL_START_AUTOVAC_LAUNCHER) &&
- Shutdown <= SmartShutdown && pmState < PM_STOP_BACKENDS)
- {
- /*
- * Start one iteration of the autovacuum daemon, even if autovacuuming
- * is nominally not enabled. This is so we can have an active defense
- * against transaction ID wraparound. We set a flag for the main loop
- * to do it rather than trying to do it here --- this is because the
- * autovac process itself may send the signal, and we want to handle
- * that by launching another iteration as soon as the current one
- * completes.
- */
- start_autovac_launcher = true;
- }
-
- if (CheckPostmasterSignal(PMSIGNAL_START_AUTOVAC_WORKER) &&
- Shutdown <= SmartShutdown && pmState < PM_STOP_BACKENDS)
- {
- /* The autovacuum launcher wants us to start a worker process. */
- StartAutovacuumWorker();
- }
-
- if (CheckPostmasterSignal(PMSIGNAL_START_WALRECEIVER))
- {
- /* Startup Process wants us to start the walreceiver process. */
- /* Start immediately if possible, else remember request for later. */
- WalReceiverRequested = true;
- MaybeStartWalReceiver();
+ /* Startup Process wants us to start the walreceiver process. */
+ /* Start immediately if possible, else remember request for later. */
+ WalReceiverRequested = true;
+ MaybeStartWalReceiver();
}
/*
@@ -5240,93 +4627,23 @@ CountChildren(int target)
* to start subprocess.
*/
static pid_t
-StartChildProcess(AuxProcType type)
+StartChildProcess(PostmasterChildType type)
{
pid_t pid;
-#ifdef EXEC_BACKEND
- {
- char *av[10];
- int ac = 0;
- char typebuf[32];
-
- /*
- * Set up command-line arguments for subprocess
- */
- av[ac++] = "postgres";
- av[ac++] = "--forkaux";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- snprintf(typebuf, sizeof(typebuf), "%d", type);
- av[ac++] = typebuf;
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- pid = postmaster_forkexec(ac, av);
- }
-#else /* !EXEC_BACKEND */
- pid = fork_process();
-
- if (pid == 0) /* child */
- {
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Release postmaster's working memory context */
- MemoryContextSwitchTo(TopMemoryContext);
- MemoryContextDelete(PostmasterContext);
- PostmasterContext = NULL;
-
- AuxiliaryProcessMain(type); /* does not return */
- }
-#endif /* EXEC_BACKEND */
-
+ pid = postmaster_child_launch(type, NULL, 0, NULL);
if (pid < 0)
{
/* in parent, fork failed */
- int save_errno = errno;
-
- errno = save_errno;
- switch (type)
- {
- case StartupProcess:
- ereport(LOG,
- (errmsg("could not fork startup process: %m")));
- break;
- case ArchiverProcess:
- ereport(LOG,
- (errmsg("could not fork archiver process: %m")));
- break;
- case BgWriterProcess:
- ereport(LOG,
- (errmsg("could not fork background writer process: %m")));
- break;
- case CheckpointerProcess:
- ereport(LOG,
- (errmsg("could not fork checkpointer process: %m")));
- break;
- case WalWriterProcess:
- ereport(LOG,
- (errmsg("could not fork WAL writer process: %m")));
- break;
- case WalReceiverProcess:
- ereport(LOG,
- (errmsg("could not fork WAL receiver process: %m")));
- break;
- default:
- ereport(LOG,
- (errmsg("could not fork process: %m")));
- break;
- }
+ /* XXX: translation? */
+ ereport(LOG,
+ (errmsg("could not fork %s process: %m", PostmasterChildName(type))));
/*
* fork failure is fatal during startup, but there's no need to choke
* immediately if starting other child types fails.
*/
- if (type == StartupProcess)
+ if (type == PMC_STARTUP)
ExitPostmaster(1);
return 0;
}
@@ -5584,32 +4901,6 @@ BackgroundWorkerUnblockSignals(void)
sigprocmask(SIG_SETMASK, &UnBlockSig, NULL);
}
-#ifdef EXEC_BACKEND
-static pid_t
-bgworker_forkexec(BackgroundWorker *worker)
-{
- char *av[10];
- int ac = 0;
- char forkav[MAXPGPATH];
- pid_t result;
-
- snprintf(forkav, MAXPGPATH, "--forkbgworker");
-
- av[ac++] = "postgres";
- av[ac++] = forkav;
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- MyBgworkerEntry = worker;
- result = postmaster_forkexec(ac, av);
- MyBgworkerEntry = NULL;
-
- return result;
-}
-#endif
-
/*
* Start a new bgworker.
* Starting time conditions must have been checked already.
@@ -5646,65 +4937,32 @@ do_start_bgworker(RegisteredBgWorker *rw)
(errmsg_internal("starting background worker process \"%s\"",
rw->rw_worker.bgw_name)));
-#ifdef EXEC_BACKEND
- switch ((worker_pid = bgworker_forkexec(&rw->rw_worker)))
-#else
- switch ((worker_pid = fork_process()))
-#endif
+ worker_pid = postmaster_child_launch(PMC_BGWORKER, (char *) &rw->rw_worker, sizeof(BackgroundWorker), NULL);
+ if (worker_pid == -1)
{
- case -1:
- /* in postmaster, fork failed ... */
- ereport(LOG,
- (errmsg("could not fork worker process: %m")));
- /* undo what assign_backendlist_entry did */
- ReleasePostmasterChildSlot(rw->rw_child_slot);
- rw->rw_child_slot = 0;
- pfree(rw->rw_backend);
- rw->rw_backend = NULL;
- /* mark entry as crashed, so we'll try again later */
- rw->rw_crashed_at = GetCurrentTimestamp();
- break;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /*
- * Before blowing away PostmasterContext, save this bgworker's
- * data where it can find it.
- */
- MyBgworkerEntry = (BackgroundWorker *)
- MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
- memcpy(MyBgworkerEntry, &rw->rw_worker, sizeof(BackgroundWorker));
-
- /* Release postmaster's working memory context */
- MemoryContextSwitchTo(TopMemoryContext);
- MemoryContextDelete(PostmasterContext);
- PostmasterContext = NULL;
-
- BackgroundWorkerMain();
+ /* in postmaster, fork failed ... */
+ ereport(LOG,
+ (errmsg("could not fork worker process: %m")));
+ /* undo what assign_backendlist_entry did */
+ ReleasePostmasterChildSlot(rw->rw_child_slot);
+ rw->rw_child_slot = 0;
+ pfree(rw->rw_backend);
+ rw->rw_backend = NULL;
+ /* mark entry as crashed, so we'll try again later */
+ rw->rw_crashed_at = GetCurrentTimestamp();
+ return false;
+ }
- exit(1); /* should not get here */
- break;
-#endif
- default:
- /* in postmaster, fork successful ... */
- rw->rw_pid = worker_pid;
- rw->rw_backend->pid = rw->rw_pid;
- ReportBackgroundWorkerPID(rw);
- /* add new worker to lists of backends */
- dlist_push_head(&BackendList, &rw->rw_backend->elem);
+ /* in postmaster, fork successful ... */
+ rw->rw_pid = worker_pid;
+ rw->rw_backend->pid = rw->rw_pid;
+ ReportBackgroundWorkerPID(rw);
+ /* add new worker to lists of backends */
+ dlist_push_head(&BackendList, &rw->rw_backend->elem);
#ifdef EXEC_BACKEND
- ShmemBackendArrayAdd(rw->rw_backend);
+ ShmemBackendArrayAdd(rw->rw_backend);
#endif
- return true;
- }
-
- return false;
+ return true;
}
/*
@@ -5956,351 +5214,6 @@ PostmasterMarkPIDForWorkerNotify(int pid)
#ifdef EXEC_BACKEND
-/*
- * The following need to be available to the save/restore_backend_variables
- * functions. They are marked NON_EXEC_STATIC in their home modules.
- */
-extern slock_t *ShmemLock;
-extern slock_t *ProcStructLock;
-extern PGPROC *AuxiliaryProcs;
-extern PMSignalData *PMSignalState;
-extern pg_time_t first_syslogger_file_time;
-
-#ifndef WIN32
-#define write_inheritable_socket(dest, src, childpid) ((*(dest) = (src)), true)
-#define read_inheritable_socket(dest, src) (*(dest) = *(src))
-#else
-static bool write_duplicated_handle(HANDLE *dest, HANDLE src, HANDLE child);
-static bool write_inheritable_socket(InheritableSocket *dest, SOCKET src,
- pid_t childPid);
-static void read_inheritable_socket(SOCKET *dest, InheritableSocket *src);
-#endif
-
-
-/* Save critical backend variables into the BackendParameters struct */
-#ifndef WIN32
-static bool
-save_backend_variables(BackendParameters *param, ClientSocket *client_sock)
-#else
-static bool
-save_backend_variables(BackendParameters *param, ClientSocket *client_sock,
- HANDLE childProcess, pid_t childPid)
-#endif
-{
- memcpy(¶m->client_sock, client_sock, sizeof(ClientSocket));
- if (!write_inheritable_socket(¶m->serialized_sock, client_sock->sock, childPid))
- return false;
-
- strlcpy(param->DataDir, DataDir, MAXPGPATH);
-
- param->MyCancelKey = MyCancelKey;
- param->MyPMChildSlot = MyPMChildSlot;
-
-#ifdef WIN32
- param->ShmemProtectiveRegion = ShmemProtectiveRegion;
-#endif
- param->UsedShmemSegID = UsedShmemSegID;
- param->UsedShmemSegAddr = UsedShmemSegAddr;
-
- param->ShmemLock = ShmemLock;
- param->ShmemVariableCache = ShmemVariableCache;
- param->ShmemBackendArray = ShmemBackendArray;
-
-#ifndef HAVE_SPINLOCKS
- param->SpinlockSemaArray = SpinlockSemaArray;
-#endif
- param->NamedLWLockTrancheRequests = NamedLWLockTrancheRequests;
- param->NamedLWLockTrancheArray = NamedLWLockTrancheArray;
- param->MainLWLockArray = MainLWLockArray;
- param->ProcStructLock = ProcStructLock;
- param->ProcGlobal = ProcGlobal;
- param->AuxiliaryProcs = AuxiliaryProcs;
- param->PreparedXactProcs = PreparedXactProcs;
- param->PMSignalState = PMSignalState;
-
- param->PostmasterPid = PostmasterPid;
- param->PgStartTime = PgStartTime;
- param->PgReloadTime = PgReloadTime;
- param->first_syslogger_file_time = first_syslogger_file_time;
-
- param->redirection_done = redirection_done;
- param->IsBinaryUpgrade = IsBinaryUpgrade;
- param->query_id_enabled = query_id_enabled;
- param->max_safe_fds = max_safe_fds;
-
- param->MaxBackends = MaxBackends;
-
-#ifdef WIN32
- param->PostmasterHandle = PostmasterHandle;
- if (!write_duplicated_handle(¶m->initial_signal_pipe,
- pgwin32_create_signal_listener(childPid),
- childProcess))
- return false;
-#else
- memcpy(¶m->postmaster_alive_fds, &postmaster_alive_fds,
- sizeof(postmaster_alive_fds));
-#endif
-
- memcpy(¶m->syslogPipe, &syslogPipe, sizeof(syslogPipe));
-
- strlcpy(param->my_exec_path, my_exec_path, MAXPGPATH);
-
- strlcpy(param->pkglib_path, pkglib_path, MAXPGPATH);
-
- if (MyBgworkerEntry)
- memcpy(¶m->MyBgworkerEntry, MyBgworkerEntry, sizeof(BackgroundWorker));
- else
- memset(¶m->MyBgworkerEntry, 0, sizeof(BackgroundWorker));
-
- return true;
-}
-
-
-#ifdef WIN32
-/*
- * Duplicate a handle for usage in a child process, and write the child
- * process instance of the handle to the parameter file.
- */
-static bool
-write_duplicated_handle(HANDLE *dest, HANDLE src, HANDLE childProcess)
-{
- HANDLE hChild = INVALID_HANDLE_VALUE;
-
- if (!DuplicateHandle(GetCurrentProcess(),
- src,
- childProcess,
- &hChild,
- 0,
- TRUE,
- DUPLICATE_CLOSE_SOURCE | DUPLICATE_SAME_ACCESS))
- {
- ereport(LOG,
- (errmsg_internal("could not duplicate handle to be written to backend parameter file: error code %lu",
- GetLastError())));
- return false;
- }
-
- *dest = hChild;
- return true;
-}
-
-/*
- * Duplicate a socket for usage in a child process, and write the resulting
- * structure to the parameter file.
- * This is required because a number of LSPs (Layered Service Providers) very
- * common on Windows (antivirus, firewalls, download managers etc) break
- * straight socket inheritance.
- */
-static bool
-write_inheritable_socket(InheritableSocket *dest, SOCKET src, pid_t childpid)
-{
- dest->origsocket = src;
- if (src != 0 && src != PGINVALID_SOCKET)
- {
- /* Actual socket */
- if (WSADuplicateSocket(src, childpid, &dest->wsainfo) != 0)
- {
- ereport(LOG,
- (errmsg("could not duplicate socket %d for use in backend: error code %d",
- (int) src, WSAGetLastError())));
- return false;
- }
- }
- return true;
-}
-
-/*
- * Read a duplicate socket structure back, and get the socket descriptor.
- */
-static void
-read_inheritable_socket(SOCKET *dest, InheritableSocket *src)
-{
- SOCKET s;
-
- if (src->origsocket == PGINVALID_SOCKET || src->origsocket == 0)
- {
- /* Not a real socket! */
- *dest = src->origsocket;
- }
- else
- {
- /* Actual socket, so create from structure */
- s = WSASocket(FROM_PROTOCOL_INFO,
- FROM_PROTOCOL_INFO,
- FROM_PROTOCOL_INFO,
- &src->wsainfo,
- 0,
- 0);
- if (s == INVALID_SOCKET)
- {
- write_stderr("could not create inherited socket: error code %d\n",
- WSAGetLastError());
- exit(1);
- }
- *dest = s;
-
- /*
- * To make sure we don't get two references to the same socket, close
- * the original one. (This would happen when inheritance actually
- * works..
- */
- closesocket(src->origsocket);
- }
-}
-#endif
-
-static void
-read_backend_variables(char *id, ClientSocket *client_sock)
-{
- BackendParameters param;
-
-#ifndef WIN32
- /* Non-win32 implementation reads from file */
- FILE *fp;
-
- /* Open file */
- fp = AllocateFile(id, PG_BINARY_R);
- if (!fp)
- {
- write_stderr("could not open backend variables file \"%s\": %s\n",
- id, strerror(errno));
- exit(1);
- }
-
- if (fread(¶m, sizeof(param), 1, fp) != 1)
- {
- write_stderr("could not read from backend variables file \"%s\": %s\n",
- id, strerror(errno));
- exit(1);
- }
-
- /* Release file */
- FreeFile(fp);
- if (unlink(id) != 0)
- {
- write_stderr("could not remove file \"%s\": %s\n",
- id, strerror(errno));
- exit(1);
- }
-#else
- /* Win32 version uses mapped file */
- HANDLE paramHandle;
- BackendParameters *paramp;
-
-#ifdef _WIN64
- paramHandle = (HANDLE) _atoi64(id);
-#else
- paramHandle = (HANDLE) atol(id);
-#endif
- paramp = MapViewOfFile(paramHandle, FILE_MAP_READ, 0, 0, 0);
- if (!paramp)
- {
- write_stderr("could not map view of backend variables: error code %lu\n",
- GetLastError());
- exit(1);
- }
-
- memcpy(¶m, paramp, sizeof(BackendParameters));
-
- if (!UnmapViewOfFile(paramp))
- {
- write_stderr("could not unmap view of backend variables: error code %lu\n",
- GetLastError());
- exit(1);
- }
-
- if (!CloseHandle(paramHandle))
- {
- write_stderr("could not close handle to backend parameter variables: error code %lu\n",
- GetLastError());
- exit(1);
- }
-#endif
-
- restore_backend_variables(¶m, client_sock);
-}
-
-/* Restore critical backend variables from the BackendParameters struct */
-static void
-restore_backend_variables(BackendParameters *param, ClientSocket *client_sock)
-{
- memcpy(client_sock, ¶m->client_sock, sizeof(ClientSocket));
- read_inheritable_socket(&client_sock->sock, ¶m->serialized_sock);
-
- SetDataDir(param->DataDir);
-
- MyCancelKey = param->MyCancelKey;
- MyPMChildSlot = param->MyPMChildSlot;
-
-#ifdef WIN32
- ShmemProtectiveRegion = param->ShmemProtectiveRegion;
-#endif
- UsedShmemSegID = param->UsedShmemSegID;
- UsedShmemSegAddr = param->UsedShmemSegAddr;
-
- ShmemLock = param->ShmemLock;
- ShmemVariableCache = param->ShmemVariableCache;
- ShmemBackendArray = param->ShmemBackendArray;
-
-#ifndef HAVE_SPINLOCKS
- SpinlockSemaArray = param->SpinlockSemaArray;
-#endif
- NamedLWLockTrancheRequests = param->NamedLWLockTrancheRequests;
- NamedLWLockTrancheArray = param->NamedLWLockTrancheArray;
- MainLWLockArray = param->MainLWLockArray;
- ProcStructLock = param->ProcStructLock;
- ProcGlobal = param->ProcGlobal;
- AuxiliaryProcs = param->AuxiliaryProcs;
- PreparedXactProcs = param->PreparedXactProcs;
- PMSignalState = param->PMSignalState;
-
- PostmasterPid = param->PostmasterPid;
- PgStartTime = param->PgStartTime;
- PgReloadTime = param->PgReloadTime;
- first_syslogger_file_time = param->first_syslogger_file_time;
-
- redirection_done = param->redirection_done;
- IsBinaryUpgrade = param->IsBinaryUpgrade;
- query_id_enabled = param->query_id_enabled;
- max_safe_fds = param->max_safe_fds;
-
- MaxBackends = param->MaxBackends;
-
-#ifdef WIN32
- PostmasterHandle = param->PostmasterHandle;
- pgwin32_initial_signal_pipe = param->initial_signal_pipe;
-#else
- memcpy(&postmaster_alive_fds, ¶m->postmaster_alive_fds,
- sizeof(postmaster_alive_fds));
-#endif
-
- memcpy(&syslogPipe, ¶m->syslogPipe, sizeof(syslogPipe));
-
- strlcpy(my_exec_path, param->my_exec_path, MAXPGPATH);
-
- strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
-
- if (param->MyBgworkerEntry.bgw_name[0] != '\0')
- {
- MyBgworkerEntry = (BackgroundWorker *)
- MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
- memcpy(MyBgworkerEntry, ¶m->MyBgworkerEntry, sizeof(BackgroundWorker));
- }
-
- /*
- * We need to restore fd.c's counts of externally-opened FDs; to avoid
- * confusion, be sure to do this after restoring max_safe_fds. (Note:
- * BackendInitialize will handle this for port->sock.)
- */
-#ifndef WIN32
- if (postmaster_alive_fds[0] >= 0)
- ReserveExternalFD();
- if (postmaster_alive_fds[1] >= 0)
- ReserveExternalFD();
-#endif
-}
-
-
Size
ShmemBackendArraySize(void)
{
@@ -6422,6 +5335,32 @@ pgwin32_deadchild_callback(PVOID lpParameter, BOOLEAN TimerOrWaitFired)
/* Queue SIGCHLD signal. */
pg_queue_signal(SIGCHLD);
}
+
+/*
+ * Queue a waiter to signal when this child dies. The wait will be handled
+ * automatically by an operating system thread pool. The memory will be
+ * freed by a later call to waitpid().
+ */
+void
+pgwin32_register_deadchild_callback(HANDLE procHandle, DWORD procId)
+{
+ win32_deadchild_waitinfo *childinfo;
+
+ childinfo = palloc(sizeof(win32_deadchild_waitinfo));
+ childinfo->procHandle = procHandle;
+ childinfo->procId = procId;
+
+ if (!RegisterWaitForSingleObject(&childinfo->waitHandle,
+ procHandle,
+ pgwin32_deadchild_callback,
+ childinfo,
+ INFINITE,
+ WT_EXECUTEONLYONCE | WT_EXECUTEINWAITTHREAD))
+ ereport(FATAL,
+ (errmsg_internal("could not register process for wait: error code %lu",
+ GetLastError())));
+}
+
#endif /* WIN32 */
/*
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index efc2580536a..0db0843ab8e 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -25,6 +25,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/interrupt.h"
#include "postmaster/startup.h"
#include "storage/ipc.h"
@@ -227,8 +228,14 @@ StartupProcExit(int code, Datum arg)
* ----------------------------------
*/
void
-StartupProcessMain(void)
+StartupProcessMain(char *startup_data, size_t startup_data_len)
{
+ Assert(startup_data_len == 0);
+
+ MyAuxProcType = StartupProcess;
+ MyBackendType = B_STARTUP;
+ AuxiliaryProcessInit();
+
/* Arrange to clean up at startup process exit */
on_shmem_exit(StartupProcExit, 0);
diff --git a/src/backend/postmaster/syslogger.c b/src/backend/postmaster/syslogger.c
index 858a2f6b2b9..de45b9565fe 100644
--- a/src/backend/postmaster/syslogger.c
+++ b/src/backend/postmaster/syslogger.c
@@ -39,7 +39,6 @@
#include "pgstat.h"
#include "pgtime.h"
#include "port/pg_bitutils.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "postmaster/syslogger.h"
@@ -50,6 +49,7 @@
#include "storage/pg_shmem.h"
#include "tcop/tcopprot.h"
#include "utils/guc.h"
+#include "utils/memutils.h"
#include "utils/ps_status.h"
#include "utils/timestamp.h"
@@ -134,10 +134,7 @@ static volatile sig_atomic_t rotation_requested = false;
#ifdef EXEC_BACKEND
static int syslogger_fdget(FILE *file);
static FILE *syslogger_fdopen(int fd);
-static pid_t syslogger_forkexec(void);
-static void syslogger_parseArgs(int argc, char *argv[]);
#endif
-NON_EXEC_STATIC void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
static void process_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static void flush_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static FILE *logfile_open(const char *filename, const char *mode,
@@ -156,13 +153,19 @@ static void set_next_rotation_time(void);
static void sigUsr1Handler(SIGNAL_ARGS);
static void update_metainfo_datafile(void);
+typedef struct
+{
+ int syslogFile;
+ int csvlogFile;
+ int jsonlogFile;
+} syslogger_startup_data;
/*
* Main entry point for syslogger process
* argc/argv parameters are valid only in EXEC_BACKEND case.
*/
-NON_EXEC_STATIC void
-SysLoggerMain(int argc, char *argv[])
+void
+SysLoggerMain(char *startup_data, size_t startup_data_len)
{
#ifndef WIN32
char logbuffer[READ_BUF_SIZE];
@@ -174,11 +177,34 @@ SysLoggerMain(int argc, char *argv[])
pg_time_t now;
WaitEventSet *wes;
- now = MyStartTime;
+ /* Release postmaster's working memory context */
+ if (PostmasterContext)
+ {
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
+ }
+ /*
+ * Re-open the error output files that were opened by SysLogger_Start().
+ *
+ * We expect this will always succeed, which is too optimistic, but if it
+ * fails there's not a lot we can do to report the problem anyway. As
+ * coded, we'll just crash on a null pointer dereference after failure...
+ */
#ifdef EXEC_BACKEND
- syslogger_parseArgs(argc, argv);
-#endif /* EXEC_BACKEND */
+ {
+ syslogger_startup_data *info = (syslogger_startup_data *) startup_data;
+
+ Assert(startup_data_len == sizeof(*info));
+ syslogFile = syslogger_fdopen(info->syslogFile);
+ csvlogFile = syslogger_fdopen(info->csvlogFile);
+ jsonlogFile = syslogger_fdopen(info->jsonlogFile);
+ }
+#else
+ Assert(startup_data_len == 0);
+#endif
+
+ now = MyStartTime;
MyBackendType = B_LOGGER;
init_ps_display(NULL);
@@ -568,6 +594,9 @@ SysLogger_Start(void)
{
pid_t sysloggerPid;
char *filename;
+#ifdef EXEC_BACKEND
+ syslogger_startup_data startup_data;
+#endif /* EXEC_BACKEND */
if (!Logging_collector)
return 0;
@@ -667,112 +696,95 @@ SysLogger_Start(void)
}
#ifdef EXEC_BACKEND
- switch ((sysloggerPid = syslogger_forkexec()))
+ startup_data.syslogFile = syslogger_fdget(syslogFile);
+ startup_data.csvlogFile = syslogger_fdget(csvlogFile);
+ startup_data.jsonlogFile = syslogger_fdget(jsonlogFile);
+ sysloggerPid = postmaster_child_launch(PMC_SYSLOGGER, (char *) &startup_data, sizeof(startup_data), NULL);
#else
- switch ((sysloggerPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork system logger: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(true);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
+ sysloggerPid = postmaster_child_launch(PMC_SYSLOGGER, NULL, 0, NULL);
+#endif /* EXEC_BACKEND */
- /* do the work */
- SysLoggerMain(0, NULL);
- break;
-#endif
+ if (sysloggerPid == -1)
+ {
+ ereport(LOG,
+ (errmsg("could not fork system logger: %m")));
+ return 0;
+ }
- default:
- /* success, in postmaster */
+ /* success, in postmaster */
- /* now we redirect stderr, if not done already */
- if (!redirection_done)
- {
+ /* now we redirect stderr, if not done already */
+ if (!redirection_done)
+ {
#ifdef WIN32
- int fd;
+ int fd;
#endif
- /*
- * Leave a breadcrumb trail when redirecting, in case the user
- * forgets that redirection is active and looks only at the
- * original stderr target file.
- */
- ereport(LOG,
- (errmsg("redirecting log output to logging collector process"),
- errhint("Future log output will appear in directory \"%s\".",
- Log_directory)));
+ /*
+ * Leave a breadcrumb trail when redirecting, in case the user forgets
+ * that redirection is active and looks only at the original stderr
+ * target file.
+ */
+ ereport(LOG,
+ (errmsg("redirecting log output to logging collector process"),
+ errhint("Future log output will appear in directory \"%s\".",
+ Log_directory)));
#ifndef WIN32
- fflush(stdout);
- if (dup2(syslogPipe[1], STDOUT_FILENO) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stdout: %m")));
- fflush(stderr);
- if (dup2(syslogPipe[1], STDERR_FILENO) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stderr: %m")));
- /* Now we are done with the write end of the pipe. */
- close(syslogPipe[1]);
- syslogPipe[1] = -1;
+ fflush(stdout);
+ if (dup2(syslogPipe[1], STDOUT_FILENO) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stdout: %m")));
+ fflush(stderr);
+ if (dup2(syslogPipe[1], STDERR_FILENO) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stderr: %m")));
+ /* Now we are done with the write end of the pipe. */
+ close(syslogPipe[1]);
+ syslogPipe[1] = -1;
#else
- /*
- * open the pipe in binary mode and make sure stderr is binary
- * after it's been dup'ed into, to avoid disturbing the pipe
- * chunking protocol.
- */
- fflush(stderr);
- fd = _open_osfhandle((intptr_t) syslogPipe[1],
- _O_APPEND | _O_BINARY);
- if (dup2(fd, STDERR_FILENO) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stderr: %m")));
- close(fd);
- _setmode(STDERR_FILENO, _O_BINARY);
+ /*
+ * open the pipe in binary mode and make sure stderr is binary after
+ * it's been dup'ed into, to avoid disturbing the pipe chunking
+ * protocol.
+ */
+ fflush(stderr);
+ fd = _open_osfhandle((intptr_t) syslogPipe[1],
+ _O_APPEND | _O_BINARY);
+ if (dup2(fd, STDERR_FILENO) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stderr: %m")));
+ close(fd);
+ _setmode(STDERR_FILENO, _O_BINARY);
- /*
- * Now we are done with the write end of the pipe.
- * CloseHandle() must not be called because the preceding
- * close() closes the underlying handle.
- */
- syslogPipe[1] = 0;
+ /*
+ * Now we are done with the write end of the pipe. CloseHandle() must
+ * not be called because the preceding close() closes the underlying
+ * handle.
+ */
+ syslogPipe[1] = 0;
#endif
- redirection_done = true;
- }
-
- /* postmaster will never write the file(s); close 'em */
- fclose(syslogFile);
- syslogFile = NULL;
- if (csvlogFile != NULL)
- {
- fclose(csvlogFile);
- csvlogFile = NULL;
- }
- if (jsonlogFile != NULL)
- {
- fclose(jsonlogFile);
- jsonlogFile = NULL;
- }
- return (int) sysloggerPid;
+ redirection_done = true;
}
- /* we should never reach here */
- return 0;
+ /* postmaster will never write the file(s); close 'em */
+ fclose(syslogFile);
+ syslogFile = NULL;
+ if (csvlogFile != NULL)
+ {
+ fclose(csvlogFile);
+ csvlogFile = NULL;
+ }
+ if (jsonlogFile != NULL)
+ {
+ fclose(jsonlogFile);
+ jsonlogFile = NULL;
+ }
+ return (int) sysloggerPid;
}
@@ -831,69 +843,6 @@ syslogger_fdopen(int fd)
return file;
}
-
-/*
- * syslogger_forkexec() -
- *
- * Format up the arglist for, then fork and exec, a syslogger process
- */
-static pid_t
-syslogger_forkexec(void)
-{
- char *av[10];
- int ac = 0;
- char filenobuf[32];
- char csvfilenobuf[32];
- char jsonfilenobuf[32];
-
- av[ac++] = "postgres";
- av[ac++] = "--forklog";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- /* static variables (those not passed by write_backend_variables) */
- snprintf(filenobuf, sizeof(filenobuf), "%d",
- syslogger_fdget(syslogFile));
- av[ac++] = filenobuf;
- snprintf(csvfilenobuf, sizeof(csvfilenobuf), "%d",
- syslogger_fdget(csvlogFile));
- av[ac++] = csvfilenobuf;
- snprintf(jsonfilenobuf, sizeof(jsonfilenobuf), "%d",
- syslogger_fdget(jsonlogFile));
- av[ac++] = jsonfilenobuf;
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-
-/*
- * syslogger_parseArgs() -
- *
- * Extract data from the arglist for exec'ed syslogger process
- */
-static void
-syslogger_parseArgs(int argc, char *argv[])
-{
- int fd;
-
- Assert(argc == 6);
- argv += 3;
-
- /*
- * Re-open the error output files that were opened by SysLogger_Start().
- *
- * We expect this will always succeed, which is too optimistic, but if it
- * fails there's not a lot we can do to report the problem anyway. As
- * coded, we'll just crash on a null pointer dereference after failure...
- */
- fd = atoi(*argv++);
- syslogFile = syslogger_fdopen(fd);
- fd = atoi(*argv++);
- csvlogFile = syslogger_fdopen(fd);
- fd = atoi(*argv++);
- jsonlogFile = syslogger_fdopen(fd);
-}
#endif /* EXEC_BACKEND */
diff --git a/src/backend/postmaster/walwriter.c b/src/backend/postmaster/walwriter.c
index 266fbc23399..886c8bd8f6a 100644
--- a/src/backend/postmaster/walwriter.c
+++ b/src/backend/postmaster/walwriter.c
@@ -48,6 +48,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/interrupt.h"
#include "postmaster/walwriter.h"
#include "storage/bufmgr.h"
@@ -88,13 +89,19 @@ static void HandleWalWriterInterrupts(void);
* basic execution environment, but not enabled signals yet.
*/
void
-WalWriterMain(void)
+WalWriterMain(char *startup_data, size_t startup_data_len)
{
sigjmp_buf local_sigjmp_buf;
MemoryContext walwriter_context;
int left_till_hibernate;
bool hibernating;
+ Assert(startup_data_len == 0);
+
+ MyAuxProcType = WalWriterProcess;
+ MyBackendType = B_WAL_WRITER;
+ AuxiliaryProcessInit();
+
/*
* Properly accept or ignore signals the postmaster might send us
*
diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c
index 501910b4454..a4098c23b2b 100644
--- a/src/backend/replication/logical/launcher.c
+++ b/src/backend/replication/logical/launcher.c
@@ -30,7 +30,6 @@
#include "miscadmin.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "replication/logicallauncher.h"
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index 073668d4f2e..0d4bdd976db 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -65,6 +65,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/interrupt.h"
#include "replication/walreceiver.h"
#include "replication/walsender.h"
@@ -184,7 +185,7 @@ ProcessWalRcvInterrupts(void)
/* Main entry point for walreceiver process */
void
-WalReceiverMain(void)
+WalReceiverMain(char *startup_data, size_t startup_data_len)
{
char conninfo[MAXCONNINFO];
char *tmp_conninfo;
@@ -200,6 +201,12 @@ WalReceiverMain(void)
char *sender_host = NULL;
int sender_port = 0;
+ Assert(startup_data_len == 0);
+
+ MyAuxProcType = WalReceiverProcess;
+ MyBackendType = B_WAL_RECEIVER;
+ AuxiliaryProcessInit();
+
/*
* WalRcv should be set up already (if we are a backend, we inherit this
* by fork() or EXEC_BACKEND mechanism from the postmaster).
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 5465fa19646..d65d461340c 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -144,6 +144,8 @@ InitShmemAllocation(void)
/*
* Initialize ShmemVariableCache for transaction manager. (This doesn't
* really belong here, but not worth moving.)
+ *
+ * XXX: we really should move this
*/
ShmemVariableCache = (VariableCache)
ShmemAlloc(sizeof(*ShmemVariableCache));
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 011ec18015a..e7dde1b4e1f 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -44,6 +44,7 @@ volatile uint32 CritSectionCount = 0;
int MyProcPid;
pg_time_t MyStartTime;
TimestampTz MyStartTimestamp;
+struct ClientSocket *MyClientSocket;
struct Port *MyProcPort;
int32 MyCancelKey;
int MyPMChildSlot;
diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c
index 1e671c560c8..c80222112b3 100644
--- a/src/backend/utils/init/miscinit.c
+++ b/src/backend/utils/init/miscinit.c
@@ -85,139 +85,6 @@ bool IgnoreSystemIndexes = false;
* ----------------------------------------------------------------
*/
-/*
- * Initialize the basic environment for a postmaster child
- *
- * Should be called as early as possible after the child's startup. However,
- * on EXEC_BACKEND builds it does need to be after read_backend_variables().
- */
-void
-InitPostmasterChild(void)
-{
- IsUnderPostmaster = true; /* we are a postmaster subprocess now */
-
- /*
- * Start our win32 signal implementation. This has to be done after we
- * read the backend variables, because we need to pick up the signal pipe
- * from the parent process.
- */
-#ifdef WIN32
- pgwin32_signal_initialize();
-#endif
-
- /*
- * Set reference point for stack-depth checking. This might seem
- * redundant in !EXEC_BACKEND builds; but it's not because the postmaster
- * launches its children from signal handlers, so we might be running on
- * an alternative stack.
- */
- (void) set_stack_base();
-
- InitProcessGlobals();
-
- /*
- * make sure stderr is in binary mode before anything can possibly be
- * written to it, in case it's actually the syslogger pipe, so the pipe
- * chunking protocol isn't disturbed. Non-logpipe data gets translated on
- * redirection (e.g. via pg_ctl -l) anyway.
- */
-#ifdef WIN32
- _setmode(fileno(stderr), _O_BINARY);
-#endif
-
- /* We don't want the postmaster's proc_exit() handlers */
- on_exit_reset();
-
- /* In EXEC_BACKEND case we will not have inherited BlockSig etc values */
-#ifdef EXEC_BACKEND
- pqinitmask();
-#endif
-
- /* Initialize process-local latch support */
- InitializeLatchSupport();
- InitProcessLocalLatch();
- InitializeLatchWaitSet();
-
- /*
- * If possible, make this process a group leader, so that the postmaster
- * can signal any child processes too. Not all processes will have
- * children, but for consistency we make all postmaster child processes do
- * this.
- */
-#ifdef HAVE_SETSID
- if (setsid() < 0)
- elog(FATAL, "setsid() failed: %m");
-#endif
-
- /*
- * Every postmaster child process is expected to respond promptly to
- * SIGQUIT at all times. Therefore we centrally remove SIGQUIT from
- * BlockSig and install a suitable signal handler. (Client-facing
- * processes may choose to replace this default choice of handler with
- * quickdie().) All other blockable signals remain blocked for now.
- */
- pqsignal(SIGQUIT, SignalHandlerForCrashExit);
-
- sigdelset(&BlockSig, SIGQUIT);
- sigprocmask(SIG_SETMASK, &BlockSig, NULL);
-
- /* Request a signal if the postmaster dies, if possible. */
- PostmasterDeathSignalInit();
-
- /* Don't give the pipe to subprograms that we execute. */
-#ifndef WIN32
- if (fcntl(postmaster_alive_fds[POSTMASTER_FD_WATCH], F_SETFD, FD_CLOEXEC) < 0)
- ereport(FATAL,
- (errcode_for_socket_access(),
- errmsg_internal("could not set postmaster death monitoring pipe to FD_CLOEXEC mode: %m")));
-#endif
-}
-
-/*
- * Initialize the basic environment for a standalone process.
- *
- * argv0 has to be suitable to find the program's executable.
- */
-void
-InitStandaloneProcess(const char *argv0)
-{
- Assert(!IsPostmasterEnvironment);
-
- MyBackendType = B_STANDALONE_BACKEND;
-
- /*
- * Start our win32 signal implementation
- */
-#ifdef WIN32
- pgwin32_signal_initialize();
-#endif
-
- InitProcessGlobals();
-
- /* Initialize process-local latch support */
- InitializeLatchSupport();
- InitProcessLocalLatch();
- InitializeLatchWaitSet();
-
- /*
- * For consistency with InitPostmasterChild, initialize signal mask here.
- * But we don't unblock SIGQUIT or provide a default handler for it.
- */
- pqinitmask();
- sigprocmask(SIG_SETMASK, &BlockSig, NULL);
-
- /* Compute paths, no postmaster to inherit from */
- if (my_exec_path[0] == '\0')
- {
- if (find_my_exec(argv0, my_exec_path) < 0)
- elog(FATAL, "%s: could not locate my own executable path",
- argv0);
- }
-
- if (pkglib_path[0] == '\0')
- get_pkglib_path(my_exec_path, pkglib_path);
-}
-
void
SwitchToSharedLatch(void)
{
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 9b6d8fc5571..959feeba025 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -132,10 +132,11 @@ typedef struct ClientConnectionInfo
typedef struct Port
{
pgsocket sock; /* File descriptor */
- bool noblock; /* is the socket in non-blocking mode? */
- ProtocolVersion proto; /* FE/BE protocol version */
SockAddr laddr; /* local addr (postmaster) */
SockAddr raddr; /* remote addr (client) */
+
+ bool noblock; /* is the socket in non-blocking mode? */
+ ProtocolVersion proto; /* FE/BE protocol version */
char *remote_host; /* name (or ip addr) of remote host */
char *remote_hostname; /* name (not ip addr) of remote host, if
* available */
@@ -218,11 +219,13 @@ typedef struct Port
* ClientSocket holds a socket for an accepted connection, along with the
* information about the endpoints.
*/
-typedef struct ClientSocket {
+struct ClientSocket
+{
pgsocket sock; /* File descriptor */
SockAddr laddr; /* local addr (postmaster) */
SockAddr raddr; /* remote addr (client) */
-} ClientSocket;
+};
+typedef struct ClientSocket ClientSocket;
#ifdef USE_SSL
/*
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 1cc3712c0ff..a60323cefca 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -317,7 +317,6 @@ extern int trace_recovery(int trace_level);
extern PGDLLIMPORT char *DatabasePath;
/* now in utils/init/miscinit.c */
-extern void InitPostmasterChild(void);
extern void InitStandaloneProcess(const char *argv0);
extern void InitProcessLocalLatch(void);
extern void SwitchToSharedLatch(void);
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 65afd1ea1e8..b6b338a4971 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -55,20 +55,14 @@ extern bool IsAutoVacuumWorkerProcess(void);
#define IsAnyAutoVacuumProcess() \
(IsAutoVacuumLauncherProcess() || IsAutoVacuumWorkerProcess())
-/* Functions to start autovacuum process, called from postmaster */
+/* called from postmaster at server startup */
extern void autovac_init(void);
-extern int StartAutoVacLauncher(void);
-extern int StartAutoVacWorker(void);
/* called from postmaster when a worker could not be forked */
extern void AutoVacWorkerFailed(void);
-#ifdef EXEC_BACKEND
-extern void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
-extern void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
-extern void AutovacuumWorkerIAm(void);
-extern void AutovacuumLauncherIAm(void);
-#endif
+extern void AutoVacLauncherMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
+extern void AutoVacWorkerMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern bool AutoVacuumRequestWork(AutoVacuumWorkItemType type,
Oid relationId, BlockNumber blkno);
diff --git a/src/include/postmaster/auxprocess.h b/src/include/postmaster/auxprocess.h
index 5c2d6527ff6..75394ca0155 100644
--- a/src/include/postmaster/auxprocess.h
+++ b/src/include/postmaster/auxprocess.h
@@ -13,8 +13,6 @@
#ifndef AUXPROCESS_H
#define AUXPROCESS_H
-#include "miscadmin.h"
-
-extern void AuxiliaryProcessMain(AuxProcType auxtype) pg_attribute_noreturn();
+extern void AuxiliaryProcessInit(void);
#endif /* AUXPROCESS_H */
diff --git a/src/include/postmaster/bgworker_internals.h b/src/include/postmaster/bgworker_internals.h
index 323f1e07291..4055d2f5626 100644
--- a/src/include/postmaster/bgworker_internals.h
+++ b/src/include/postmaster/bgworker_internals.h
@@ -55,6 +55,6 @@ extern void ForgetUnstartedBackgroundWorkers(void);
extern void ResetBackgroundWorkerCrashTimes(void);
/* Entry point for background worker processes */
-extern void BackgroundWorkerMain(void) pg_attribute_noreturn();
+extern void BackgroundWorkerMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
#endif /* BGWORKER_INTERNALS_H */
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index a66722873f4..ee54fc401ef 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -27,8 +27,8 @@ extern PGDLLIMPORT int CheckPointTimeout;
extern PGDLLIMPORT int CheckPointWarning;
extern PGDLLIMPORT double CheckPointCompletionTarget;
-extern void BackgroundWriterMain(void) pg_attribute_noreturn();
-extern void CheckpointerMain(void) pg_attribute_noreturn();
+extern void BackgroundWriterMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
+extern void CheckpointerMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern void RequestCheckpoint(int flags);
extern void CheckpointWriteDelay(int flags, double progress);
diff --git a/src/include/postmaster/fork_process.h b/src/include/postmaster/fork_process.h
deleted file mode 100644
index 12decc8133b..00000000000
--- a/src/include/postmaster/fork_process.h
+++ /dev/null
@@ -1,17 +0,0 @@
-/*-------------------------------------------------------------------------
- *
- * fork_process.h
- * Exports from postmaster/fork_process.c.
- *
- * Copyright (c) 1996-2023, PostgreSQL Global Development Group
- *
- * src/include/postmaster/fork_process.h
- *
- *-------------------------------------------------------------------------
- */
-#ifndef FORK_PROCESS_H
-#define FORK_PROCESS_H
-
-extern pid_t fork_process(void);
-
-#endif /* FORK_PROCESS_H */
diff --git a/src/include/postmaster/pgarch.h b/src/include/postmaster/pgarch.h
index 3bd4fac71e5..577fc14e1d0 100644
--- a/src/include/postmaster/pgarch.h
+++ b/src/include/postmaster/pgarch.h
@@ -29,7 +29,7 @@
extern Size PgArchShmemSize(void);
extern void PgArchShmemInit(void);
extern bool PgArchCanRestart(void);
-extern void PgArchiverMain(void) pg_attribute_noreturn();
+extern void PgArchiverMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern void PgArchWakeup(void);
extern void PgArchForceDirScan(void);
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index 3b3889c58c0..1dbb5c991f5 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -58,14 +58,52 @@ extern int MaxLivePostmasterChildren(void);
extern bool PostmasterMarkPIDForWorkerNotify(int);
-#ifdef EXEC_BACKEND
-extern pid_t postmaster_forkexec(int argc, char *argv[]);
-extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
+extern void BackendMain(char *startup_data, size_t startup_data_len);
+#ifdef EXEC_BACKEND
extern Size ShmemBackendArraySize(void);
extern void ShmemBackendArrayAllocation(void);
#endif
+/* in launch_backend.c */
+
+/* this better match the list in launch_backend.c */
+typedef enum PostmasterChildType
+{
+ PMC_BACKEND = 0,
+ PMC_AV_LAUNCHER,
+ PMC_AV_WORKER,
+ PMC_BGWORKER,
+ PMC_SYSLOGGER,
+
+ /*
+ * so-called "aux processes". These access shared memory, but are not
+ * attached to any particular database. Only one of each of these can be
+ * running at a time.
+ */
+ PMC_STARTUP,
+ PMC_BGWRITER,
+ PMC_ARCHIVER,
+ PMC_CHECKPOINTER,
+ PMC_WAL_WRITER,
+ PMC_WAL_RECEIVER,
+} PostmasterChildType;
+
+/* defined in libpq-be.h */
+extern struct ClientSocket *MyClientSocket;
+
+extern pid_t postmaster_child_launch(PostmasterChildType child_type, char *startup_data, size_t startup_data_len, struct ClientSocket *sock);
+
+#ifdef EXEC_BACKEND
+extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
+#endif
+
+#ifdef WIN32
+extern void pgwin32_register_deadchild_callback(HANDLE procHandle, DWORD procId);
+#endif
+
+const char *PostmasterChildName(PostmasterChildType child_type);
+
/*
* Note: MAX_BACKENDS is limited to 2^18-1 because that's the width reserved
* for buffer references in buf_internals.h. This limitation could be lifted
diff --git a/src/include/postmaster/startup.h b/src/include/postmaster/startup.h
index 6a2e4c4526b..ec885063aab 100644
--- a/src/include/postmaster/startup.h
+++ b/src/include/postmaster/startup.h
@@ -26,7 +26,7 @@
extern PGDLLIMPORT int log_startup_progress_interval;
extern void HandleStartupProcInterrupts(void);
-extern void StartupProcessMain(void) pg_attribute_noreturn();
+extern void StartupProcessMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern void PreRestoreCommand(void);
extern void PostRestoreCommand(void);
extern bool IsPromoteSignaled(void);
diff --git a/src/include/postmaster/syslogger.h b/src/include/postmaster/syslogger.h
index 34da778f1ef..7dc41b30e7c 100644
--- a/src/include/postmaster/syslogger.h
+++ b/src/include/postmaster/syslogger.h
@@ -86,9 +86,7 @@ extern int SysLogger_Start(void);
extern void write_syslogger_file(const char *buffer, int count, int destination);
-#ifdef EXEC_BACKEND
-extern void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
+extern void SysLoggerMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern bool CheckLogrotateSignal(void);
extern void RemoveLogrotateSignalFiles(void);
diff --git a/src/include/postmaster/walwriter.h b/src/include/postmaster/walwriter.h
index 6eba7ad79cf..99b7cc07fb2 100644
--- a/src/include/postmaster/walwriter.h
+++ b/src/include/postmaster/walwriter.h
@@ -18,6 +18,6 @@
extern PGDLLIMPORT int WalWriterDelay;
extern PGDLLIMPORT int WalWriterFlushAfter;
-extern void WalWriterMain(void) pg_attribute_noreturn();
+extern void WalWriterMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
#endif /* _WALWRITER_H */
diff --git a/src/include/replication/walreceiver.h b/src/include/replication/walreceiver.h
index 281626fa6f5..92dead4db7b 100644
--- a/src/include/replication/walreceiver.h
+++ b/src/include/replication/walreceiver.h
@@ -457,7 +457,7 @@ walrcv_clear_result(WalRcvExecResult *walres)
}
/* prototypes for functions in walreceiver.c */
-extern void WalReceiverMain(void) pg_attribute_noreturn();
+extern void WalReceiverMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern void ProcessWalRcvInterrupts(void);
extern void WalRcvForceReply(void);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 78a9e0ebbc8..ecc4ea0ee9c 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2111,6 +2111,7 @@ PortalStrategy
PostParseColumnRefHook
PostgresPollingStatusType
PostingItem
+PostmasterChildType
PreParseColumnRefHook
PredClass
PredIterInfo
@@ -3778,6 +3779,7 @@ substitute_actual_parameters_context
substitute_actual_srf_parameters_context
substitute_phv_relids_context
symbol
+syslogger_startup_data
tablespaceinfo
teSection
temp_tablespaces_extra
--
2.39.2
Re: Refactoring backend fork+exec code
On 11/10/2023 14:12, Heikki Linnakangas wrote:
On 11/07/2023 01:50, Andres Freund wrote:
Subject: [PATCH 3/9] Refactor CreateSharedMemoryAndSemaphores.
Moves InitProcess calls a little later in EXEC_BACKEND case.
What's the reason for this part?
The point is that with this commit, InitProcess() is called at same
place in EXEC_BACKEND mode and !EXEC_BACKEND. It feels more consistent
that way.ISTM that we'd really want to get away from plastering duplicated
InitProcess() etc everywhere.Sure, we could do more to reduce the duplication. I think this is a step
in the right direction, though.
Here's another rebased patch set. Compared to previous version, I did a
little more refactoring around CreateSharedMemoryAndSemaphores and
InitProcess:
- patch 1 splits CreateSharedMemoryAndSemaphores into two functions:
CreateSharedMemoryAndSemaphores is now only called at postmaster
startup, and a new function called AttachSharedMemoryStructs() is called
in backends in EXEC_BACKEND mode. I extracted the common parts of those
functions to a new static function. (Some of this refactoring used to be
part of the 3rd patch in the series, but it seems useful on its own, so
I split it out.)
- patch 3 moves the call to AttachSharedMemoryStructs() to
InitProcess(), reducing the boilerplate code a little.
The patches are incrementally useful, so if you don't have time to
review all of them, a review on a subset would be useful too.
--
Heikki Linnakangas
Neon (https://neon.tech)
Attachments:
v3-0001-Refactor-CreateSharedMemoryAndSemaphores.patchtext/x-patch; charset=UTF-8; name=v3-0001-Refactor-CreateSharedMemoryAndSemaphores.patchDownload
From a96b6e92fdeaa947bf32774c425419b8f987b8e2 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Thu, 30 Nov 2023 00:01:25 +0200
Subject: [PATCH v3 1/7] Refactor CreateSharedMemoryAndSemaphores
For clarity, have separate functions for *creating* the shared memory
and semaphores at postmaster or single-user backend startup, and
for *attaching* to existing shared memory structures in EXEC_BACKEND
case. CreateSharedMemoryAndSemaphores() is now called only at
postmaster startup, and a new AttachSharedMemoryStructs() function is
called at backend startup in EXEC_BACKEND mode.
Reviewed-by: Tristan Partin, Andres Freund
Discussion: https://www.postgresql.org/message-id/7a59b073-5b5b-151e-7ed3-8b01ff7ce9ef@iki.fi
---
src/backend/postmaster/postmaster.c | 20 +--
src/backend/replication/walreceiver.c | 3 +-
src/backend/storage/ipc/ipci.c | 178 +++++++++++++++-----------
src/backend/storage/lmgr/proc.c | 2 +-
src/include/storage/ipc.h | 3 +
5 files changed, 117 insertions(+), 89 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 7a5cd06c5c9..b03e88e6702 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -4909,11 +4909,11 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ /* Need a PGPROC to run AttachSharedMemoryStructs */
InitProcess();
/* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
+ AttachSharedMemoryStructs();
/* And run the backend */
BackendRun(&port); /* does not return */
@@ -4927,11 +4927,11 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ /* Need a PGPROC to run AttachSharedMemoryStructs */
InitAuxiliaryProcess();
/* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
+ AttachSharedMemoryStructs();
auxtype = atoi(argv[3]);
AuxiliaryProcessMain(auxtype); /* does not return */
@@ -4941,11 +4941,11 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ /* Need a PGPROC to run AttachSharedMemoryStructs */
InitProcess();
/* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
+ AttachSharedMemoryStructs();
AutoVacLauncherMain(argc - 2, argv + 2); /* does not return */
}
@@ -4954,11 +4954,11 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ /* Need a PGPROC to run AttachSharedMemoryStructs */
InitProcess();
/* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
+ AttachSharedMemoryStructs();
AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
}
@@ -4972,11 +4972,11 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ /* Need a PGPROC to run AttachSharedMemoryStructs */
InitProcess();
/* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
+ AttachSharedMemoryStructs();
/* Fetch MyBgworkerEntry from shared memory */
shmem_slot = atoi(argv[1] + 15);
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index 2398167f495..26ded928a71 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -193,7 +193,7 @@ WalReceiverMain(void)
TimeLineID startpointTLI;
TimeLineID primaryTLI;
bool first_stream;
- WalRcvData *walrcv = WalRcv;
+ WalRcvData *walrcv;
TimestampTz now;
char *err;
char *sender_host = NULL;
@@ -203,6 +203,7 @@ WalReceiverMain(void)
* WalRcv should be set up already (if we are a backend, we inherit this
* by fork() or EXEC_BACKEND mechanism from the postmaster).
*/
+ walrcv = WalRcv;
Assert(walrcv != NULL);
/*
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index a3d8eacb8dc..2225a4a6e60 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -58,6 +58,8 @@ shmem_startup_hook_type shmem_startup_hook = NULL;
static Size total_addin_request = 0;
+static void CreateOrAttachShmemStructs(void);
+
/*
* RequestAddinShmemSpace
* Request that extra shmem space be allocated for use by
@@ -156,9 +158,106 @@ CalculateShmemSize(int *num_semaphores)
return size;
}
+#ifdef EXEC_BACKEND
+/*
+ * AttachSharedMemoryStructs
+ * Initialize a postmaster child process's access to shared memory
+ * structures.
+ *
+ * In !EXEC_BACKEND mode, we inherit everything through the fork, and this
+ * isn't needed.
+ */
+void
+AttachSharedMemoryStructs(void)
+{
+ /* InitProcess must've been called already */
+ Assert(MyProc != NULL);
+ Assert(IsUnderPostmaster);
+
+ CreateOrAttachShmemStructs();
+
+ /*
+ * Now give loadable modules a chance to set up their shmem allocations
+ */
+ if (shmem_startup_hook)
+ shmem_startup_hook();
+}
+#endif
+
/*
* CreateSharedMemoryAndSemaphores
* Creates and initializes shared memory and semaphores.
+ */
+void
+CreateSharedMemoryAndSemaphores(void)
+{
+ PGShmemHeader *shim;
+ PGShmemHeader *seghdr;
+ Size size;
+ int numSemas;
+
+ Assert(!IsUnderPostmaster);
+
+ /* Compute the size of the shared-memory block */
+ size = CalculateShmemSize(&numSemas);
+ elog(DEBUG3, "invoking IpcMemoryCreate(size=%zu)", size);
+
+ /*
+ * Create the shmem segment
+ */
+ seghdr = PGSharedMemoryCreate(size, &shim);
+
+ /*
+ * Make sure that huge pages are never reported as "unknown" while the
+ * server is running.
+ */
+ Assert(strcmp("unknown",
+ GetConfigOption("huge_pages_status", false, false)) != 0);
+
+ InitShmemAccess(seghdr);
+
+ /*
+ * Create semaphores
+ */
+ PGReserveSemaphores(numSemas);
+
+ /*
+ * If spinlocks are disabled, initialize emulation layer (which depends on
+ * semaphores, so the order is important here).
+ */
+#ifndef HAVE_SPINLOCKS
+ SpinlockSemaInit();
+#endif
+
+ /*
+ * Set up shared memory allocation mechanism
+ */
+ InitShmemAllocation();
+
+ /* Initialize subsystems */
+ CreateOrAttachShmemStructs();
+
+#ifdef EXEC_BACKEND
+
+ /*
+ * Alloc the win32 shared backend array
+ */
+ ShmemBackendArrayAllocation();
+#endif
+
+ /* Initialize dynamic shared memory facilities. */
+ dsm_postmaster_startup(shim);
+
+ /*
+ * Now give loadable modules a chance to set up their shmem allocations
+ */
+ if (shmem_startup_hook)
+ shmem_startup_hook();
+}
+
+/*
+ * Initialize various subsystems, setting up their data structures in
+ * shared memory.
*
* This is called by the postmaster or by a standalone backend.
* It is also called by a backend forked from the postmaster in the
@@ -171,65 +270,9 @@ CalculateShmemSize(int *num_semaphores)
* check IsUnderPostmaster, rather than EXEC_BACKEND, to detect this case.
* This is a bit code-wasteful and could be cleaned up.)
*/
-void
-CreateSharedMemoryAndSemaphores(void)
+static void
+CreateOrAttachShmemStructs(void)
{
- PGShmemHeader *shim = NULL;
-
- if (!IsUnderPostmaster)
- {
- PGShmemHeader *seghdr;
- Size size;
- int numSemas;
-
- /* Compute the size of the shared-memory block */
- size = CalculateShmemSize(&numSemas);
- elog(DEBUG3, "invoking IpcMemoryCreate(size=%zu)", size);
-
- /*
- * Create the shmem segment
- */
- seghdr = PGSharedMemoryCreate(size, &shim);
-
- /*
- * Make sure that huge pages are never reported as "unknown" while the
- * server is running.
- */
- Assert(strcmp("unknown",
- GetConfigOption("huge_pages_status", false, false)) != 0);
-
- InitShmemAccess(seghdr);
-
- /*
- * Create semaphores
- */
- PGReserveSemaphores(numSemas);
-
- /*
- * If spinlocks are disabled, initialize emulation layer (which
- * depends on semaphores, so the order is important here).
- */
-#ifndef HAVE_SPINLOCKS
- SpinlockSemaInit();
-#endif
- }
- else
- {
- /*
- * We are reattaching to an existing shared memory segment. This
- * should only be reached in the EXEC_BACKEND case.
- */
-#ifndef EXEC_BACKEND
- elog(PANIC, "should be attached to shared memory already");
-#endif
- }
-
- /*
- * Set up shared memory allocation mechanism
- */
- if (!IsUnderPostmaster)
- InitShmemAllocation();
-
/*
* Now initialize LWLocks, which do shared memory allocation and are
* needed for InitShmemIndex.
@@ -302,25 +345,6 @@ CreateSharedMemoryAndSemaphores(void)
AsyncShmemInit();
StatsShmemInit();
WaitEventExtensionShmemInit();
-
-#ifdef EXEC_BACKEND
-
- /*
- * Alloc the win32 shared backend array
- */
- if (!IsUnderPostmaster)
- ShmemBackendArrayAllocation();
-#endif
-
- /* Initialize dynamic shared memory facilities. */
- if (!IsUnderPostmaster)
- dsm_postmaster_startup(shim);
-
- /*
- * Now give loadable modules a chance to set up their shmem allocations
- */
- if (shmem_startup_hook)
- shmem_startup_hook();
}
/*
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index e9e445bb216..cdfdd6fbe1d 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -468,7 +468,7 @@ InitProcess(void)
*
* This is separate from InitProcess because we can't acquire LWLocks until
* we've created a PGPROC, but in the EXEC_BACKEND case ProcArrayAdd won't
- * work until after we've done CreateSharedMemoryAndSemaphores.
+ * work until after we've done AttachSharedMemoryStructs.
*/
void
InitProcessPhase2(void)
diff --git a/src/include/storage/ipc.h b/src/include/storage/ipc.h
index 888c08b3067..e282dcad962 100644
--- a/src/include/storage/ipc.h
+++ b/src/include/storage/ipc.h
@@ -79,6 +79,9 @@ extern PGDLLIMPORT shmem_startup_hook_type shmem_startup_hook;
extern Size CalculateShmemSize(int *num_semaphores);
extern void CreateSharedMemoryAndSemaphores(void);
+#ifdef EXEC_BACKEND
+extern void AttachSharedMemoryStructs(void);
+#endif
extern void InitializeShmemGUCs(void);
#endif /* IPC_H */
--
2.39.2
v3-0002-Pass-BackgroundWorker-entry-in-the-parameter-file.patchtext/x-patch; charset=UTF-8; name=v3-0002-Pass-BackgroundWorker-entry-in-the-parameter-file.patchDownload
From 3478cafcf74a5c8d649e0287e6c72669a29c0e70 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Thu, 30 Nov 2023 00:02:03 +0200
Subject: [PATCH v3 2/7] Pass BackgroundWorker entry in the parameter file in
EXEC_BACKEND mode
This makes it possible to move InitProcess later in SubPostmasterMain
(in next commit), as we no longer need to access shared memory to read
background worker entry.
Reviewed-by: Tristan Partin, Andres Freund
Discussion: https://www.postgresql.org/message-id/7a59b073-5b5b-151e-7ed3-8b01ff7ce9ef@iki.fi
---
src/backend/postmaster/bgworker.c | 21 ------------
src/backend/postmaster/postmaster.c | 37 ++++++++++++++-------
src/include/postmaster/bgworker_internals.h | 4 ---
3 files changed, 25 insertions(+), 37 deletions(-)
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index 48a9924527e..b3c2e65ba9f 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -631,27 +631,6 @@ ResetBackgroundWorkerCrashTimes(void)
}
}
-#ifdef EXEC_BACKEND
-/*
- * In EXEC_BACKEND mode, workers use this to retrieve their details from
- * shared memory.
- */
-BackgroundWorker *
-BackgroundWorkerEntry(int slotno)
-{
- static BackgroundWorker myEntry;
- BackgroundWorkerSlot *slot;
-
- Assert(slotno < BackgroundWorkerData->total_slots);
- slot = &BackgroundWorkerData->slot[slotno];
- Assert(slot->in_use);
-
- /* must copy this in case we don't intend to retain shmem access */
- memcpy(&myEntry, &slot->worker, sizeof myEntry);
- return &myEntry;
-}
-#endif
-
/*
* Complain about the BackgroundWorker definition using error level elevel.
* Return true if it looks ok, false if not (unless elevel >= ERROR, in
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index b03e88e6702..ac0e2c0a35b 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -540,6 +540,8 @@ typedef struct
#endif
char my_exec_path[MAXPGPATH];
char pkglib_path[MAXPGPATH];
+
+ BackgroundWorker MyBgworkerEntry;
} BackendParameters;
static void read_backend_variables(char *id, Port *port);
@@ -4831,7 +4833,7 @@ SubPostmasterMain(int argc, char *argv[])
strcmp(argv[1], "--forkavlauncher") == 0 ||
strcmp(argv[1], "--forkavworker") == 0 ||
strcmp(argv[1], "--forkaux") == 0 ||
- strncmp(argv[1], "--forkbgworker=", 15) == 0)
+ strncmp(argv[1], "--forkbgworker", 14) == 0)
PGSharedMemoryReAttach();
else
PGSharedMemoryNoReAttach();
@@ -4962,10 +4964,8 @@ SubPostmasterMain(int argc, char *argv[])
AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
}
- if (strncmp(argv[1], "--forkbgworker=", 15) == 0)
+ if (strncmp(argv[1], "--forkbgworker", 14) == 0)
{
- int shmem_slot;
-
/* do this as early as possible; in particular, before InitProcess() */
IsBackgroundWorker = true;
@@ -4978,10 +4978,6 @@ SubPostmasterMain(int argc, char *argv[])
/* Attach process to shared data structures */
AttachSharedMemoryStructs();
- /* Fetch MyBgworkerEntry from shared memory */
- shmem_slot = atoi(argv[1] + 15);
- MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
-
BackgroundWorkerMain();
}
if (strcmp(argv[1], "--forklog") == 0)
@@ -5640,13 +5636,14 @@ BackgroundWorkerUnblockSignals(void)
#ifdef EXEC_BACKEND
static pid_t
-bgworker_forkexec(int shmem_slot)
+bgworker_forkexec(BackgroundWorker *worker)
{
char *av[10];
int ac = 0;
char forkav[MAXPGPATH];
+ pid_t result;
- snprintf(forkav, MAXPGPATH, "--forkbgworker=%d", shmem_slot);
+ snprintf(forkav, MAXPGPATH, "--forkbgworker");
av[ac++] = "postgres";
av[ac++] = forkav;
@@ -5655,7 +5652,11 @@ bgworker_forkexec(int shmem_slot)
Assert(ac < lengthof(av));
- return postmaster_forkexec(ac, av);
+ MyBgworkerEntry = worker;
+ result = postmaster_forkexec(ac, av);
+ MyBgworkerEntry = NULL;
+
+ return result;
}
#endif
@@ -5696,7 +5697,7 @@ do_start_bgworker(RegisteredBgWorker *rw)
rw->rw_worker.bgw_name)));
#ifdef EXEC_BACKEND
- switch ((worker_pid = bgworker_forkexec(rw->rw_shmem_slot)))
+ switch ((worker_pid = bgworker_forkexec(&rw->rw_worker)))
#else
switch ((worker_pid = fork_process()))
#endif
@@ -6096,6 +6097,11 @@ save_backend_variables(BackendParameters *param, Port *port,
strlcpy(param->pkglib_path, pkglib_path, MAXPGPATH);
+ if (MyBgworkerEntry)
+ memcpy(¶m->MyBgworkerEntry, MyBgworkerEntry, sizeof(BackgroundWorker));
+ else
+ memset(¶m->MyBgworkerEntry, 0, sizeof(BackgroundWorker));
+
return true;
}
@@ -6324,6 +6330,13 @@ restore_backend_variables(BackendParameters *param, Port *port)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ if (param->MyBgworkerEntry.bgw_name[0] != '\0')
+ {
+ MyBgworkerEntry = (BackgroundWorker *)
+ MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
+ memcpy(MyBgworkerEntry, ¶m->MyBgworkerEntry, sizeof(BackgroundWorker));
+ }
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/postmaster/bgworker_internals.h b/src/include/postmaster/bgworker_internals.h
index 09df054fcce..323f1e07291 100644
--- a/src/include/postmaster/bgworker_internals.h
+++ b/src/include/postmaster/bgworker_internals.h
@@ -57,8 +57,4 @@ extern void ResetBackgroundWorkerCrashTimes(void);
/* Entry point for background worker processes */
extern void BackgroundWorkerMain(void) pg_attribute_noreturn();
-#ifdef EXEC_BACKEND
-extern BackgroundWorker *BackgroundWorkerEntry(int slotno);
-#endif
-
#endif /* BGWORKER_INTERNALS_H */
--
2.39.2
v3-0003-Refactor-how-InitProcess-is-called.patchtext/x-patch; charset=UTF-8; name=v3-0003-Refactor-how-InitProcess-is-called.patchDownload
From 0d071474e12a70ff8113c7b0731c5b97fec45007 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 29 Nov 2023 23:47:25 +0200
Subject: [PATCH v3 3/7] Refactor how InitProcess is called
The order of process initialization steps is now more consistent
between !EXEC_BACKEND and EXEC_BACKEND modes. InitProcess() is called
at the same place in either mode. We can now also move the
AttachSharedMemoryStructs() call into InitProcess() itself. This
reduces the number of "#ifdef EXEC_BACKEND" blocks.
Reviewed-by: Tristan Partin, Andres Freund
Discussion: https://www.postgresql.org/message-id/7a59b073-5b5b-151e-7ed3-8b01ff7ce9ef@iki.fi
---
src/backend/postmaster/autovacuum.c | 16 +++-------
src/backend/postmaster/auxprocess.c | 5 +---
src/backend/postmaster/bgworker.c | 8 ++---
src/backend/postmaster/postmaster.c | 45 ++++-------------------------
src/backend/storage/lmgr/proc.c | 12 ++++++++
5 files changed, 25 insertions(+), 61 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 86a3b3d8be2..2f54485c217 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -476,14 +476,10 @@ AutoVacLauncherMain(int argc, char *argv[])
pqsignal(SIGCHLD, SIG_DFL);
/*
- * Create a per-backend PGPROC struct in shared memory, except in the
- * EXEC_BACKEND case where this was done in SubPostmasterMain. We must do
- * this before we can use LWLocks (and in the EXEC_BACKEND case we already
- * had to do some stuff with LWLocks).
+ * Create a per-backend PGPROC struct in shared memory. We must do this
+ * before we can use LWLocks or access any shared memory.
*/
-#ifndef EXEC_BACKEND
InitProcess();
-#endif
/* Early initialization */
BaseInit();
@@ -1548,14 +1544,10 @@ AutoVacWorkerMain(int argc, char *argv[])
pqsignal(SIGCHLD, SIG_DFL);
/*
- * Create a per-backend PGPROC struct in shared memory, except in the
- * EXEC_BACKEND case where this was done in SubPostmasterMain. We must do
- * this before we can use LWLocks (and in the EXEC_BACKEND case we already
- * had to do some stuff with LWLocks).
+ * Create a per-backend PGPROC struct in shared memory. We must do this
+ * before we can use LWLocks or access any shared memory.
*/
-#ifndef EXEC_BACKEND
InitProcess();
-#endif
/* Early initialization */
BaseInit();
diff --git a/src/backend/postmaster/auxprocess.c b/src/backend/postmaster/auxprocess.c
index cae6feb3562..bae6f68c402 100644
--- a/src/backend/postmaster/auxprocess.c
+++ b/src/backend/postmaster/auxprocess.c
@@ -97,12 +97,9 @@ AuxiliaryProcessMain(AuxProcType auxtype)
*/
/*
- * Create a PGPROC so we can use LWLocks. In the EXEC_BACKEND case, this
- * was already done by SubPostmasterMain().
+ * Create a PGPROC so we can use LWLocks and access shared memory.
*/
-#ifndef EXEC_BACKEND
InitAuxiliaryProcess();
-#endif
BaseInit();
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index b3c2e65ba9f..3fbc8f503cb 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -810,14 +810,10 @@ BackgroundWorkerMain(void)
PG_exception_stack = &local_sigjmp_buf;
/*
- * Create a per-backend PGPROC struct in shared memory, except in the
- * EXEC_BACKEND case where this was done in SubPostmasterMain. We must do
- * this before we can use LWLocks (and in the EXEC_BACKEND case we already
- * had to do some stuff with LWLocks).
+ * Create a per-backend PGPROC struct in shared memory. We must do this
+ * before we can use LWLocks or access any shared memory.
*/
-#ifndef EXEC_BACKEND
InitProcess();
-#endif
/*
* Early initialization.
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index ac0e2c0a35b..dd290d06e3d 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -4097,15 +4097,6 @@ BackendStartup(Port *port)
/* Perform additional initialization and collect startup packet */
BackendInitialize(port);
- /*
- * Create a per-backend PGPROC struct in shared memory. We must do
- * this before we can use LWLocks. In the !EXEC_BACKEND case (here)
- * this could be delayed a bit further, but EXEC_BACKEND needs to do
- * stuff with LWLocks before PostgresMain(), so we do it here as well
- * for symmetry.
- */
- InitProcess();
-
/* And run the backend */
BackendRun(port);
}
@@ -4416,6 +4407,12 @@ BackendInitialize(Port *port)
static void
BackendRun(Port *port)
{
+ /*
+ * Create a per-backend PGPROC struct in shared memory. We must do this
+ * before we can use LWLocks or access any shared memory.
+ */
+ InitProcess();
+
/*
* Make sure we aren't in PostmasterContext anymore. (We can't delete it
* just yet, though, because InitPostgres will need the HBA data.)
@@ -4911,12 +4908,6 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run AttachSharedMemoryStructs */
- InitProcess();
-
- /* Attach process to shared data structures */
- AttachSharedMemoryStructs();
-
/* And run the backend */
BackendRun(&port); /* does not return */
}
@@ -4929,12 +4920,6 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run AttachSharedMemoryStructs */
- InitAuxiliaryProcess();
-
- /* Attach process to shared data structures */
- AttachSharedMemoryStructs();
-
auxtype = atoi(argv[3]);
AuxiliaryProcessMain(auxtype); /* does not return */
}
@@ -4943,12 +4928,6 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run AttachSharedMemoryStructs */
- InitProcess();
-
- /* Attach process to shared data structures */
- AttachSharedMemoryStructs();
-
AutoVacLauncherMain(argc - 2, argv + 2); /* does not return */
}
if (strcmp(argv[1], "--forkavworker") == 0)
@@ -4956,12 +4935,6 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run AttachSharedMemoryStructs */
- InitProcess();
-
- /* Attach process to shared data structures */
- AttachSharedMemoryStructs();
-
AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
}
if (strncmp(argv[1], "--forkbgworker", 14) == 0)
@@ -4972,12 +4945,6 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run AttachSharedMemoryStructs */
- InitProcess();
-
- /* Attach process to shared data structures */
- AttachSharedMemoryStructs();
-
BackgroundWorkerMain();
}
if (strcmp(argv[1], "--forklog") == 0)
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index cdfdd6fbe1d..6c708777dde 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -461,6 +461,12 @@ InitProcess(void)
*/
InitLWLockAccess();
InitDeadLockChecking();
+
+#ifdef EXEC_BACKEND
+ /* Attach process to shared data structures */
+ if (IsUnderPostmaster)
+ AttachSharedMemoryStructs();
+#endif
}
/*
@@ -614,6 +620,12 @@ InitAuxiliaryProcess(void)
* Arrange to clean up at process exit.
*/
on_shmem_exit(AuxiliaryProcKill, Int32GetDatum(proctype));
+
+#ifdef EXEC_BACKEND
+ /* Attach process to shared data structures */
+ if (IsUnderPostmaster)
+ AttachSharedMemoryStructs();
+#endif
}
/*
--
2.39.2
v3-0004-Pass-CAC-as-argument-to-backend-process.patchtext/x-patch; charset=UTF-8; name=v3-0004-Pass-CAC-as-argument-to-backend-process.patchDownload
From 8886db1ed6bae21bf6d77c9bb1230edbb55e24f9 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Thu, 30 Nov 2023 00:04:22 +0200
Subject: [PATCH v3 4/7] Pass CAC as argument to backend process
We used to smuggle it to the child process in the Port struct, but it seems
better to pass it down as a separate argument.
Reviewed-by: Tristan Partin, Andres Freund
Discussion: https://www.postgresql.org/message-id/7a59b073-5b5b-151e-7ed3-8b01ff7ce9ef@iki.fi
---
src/backend/postmaster/postmaster.c | 43 +++++++++++++++++++++--------
src/include/libpq/libpq-be.h | 12 --------
2 files changed, 31 insertions(+), 24 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index dd290d06e3d..1a72b522997 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -418,7 +418,18 @@ static void HandleChildCrash(int pid, int exitstatus, const char *procname);
static void LogChildExit(int lev, const char *procname,
int pid, int exitstatus);
static void PostmasterStateMachine(void);
-static void BackendInitialize(Port *port);
+
+typedef enum CAC_state
+{
+ CAC_OK,
+ CAC_STARTUP,
+ CAC_SHUTDOWN,
+ CAC_RECOVERY,
+ CAC_NOTCONSISTENT,
+ CAC_TOOMANY,
+} CAC_state;
+
+static void BackendInitialize(Port *port, CAC_state cac);
static void BackendRun(Port *port) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
@@ -475,7 +486,7 @@ typedef struct
} win32_deadchild_waitinfo;
#endif /* WIN32 */
-static pid_t backend_forkexec(Port *port);
+static pid_t backend_forkexec(Port *port, CAC_state cac);
static pid_t internal_forkexec(int argc, char *argv[], Port *port);
/* Type for a socket that can be inherited to a client process */
@@ -4037,6 +4048,7 @@ BackendStartup(Port *port)
{
Backend *bn; /* for backend cleanup */
pid_t pid;
+ CAC_state cac;
/*
* Create backend data structure. Better before the fork() so we can
@@ -4068,8 +4080,8 @@ BackendStartup(Port *port)
bn->cancel_key = MyCancelKey;
/* Pass down canAcceptConnections state */
- port->canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
- bn->dead_end = (port->canAcceptConnections != CAC_OK);
+ cac = canAcceptConnections(BACKEND_TYPE_NORMAL);
+ bn->dead_end = (cac != CAC_OK);
/*
* Unless it's a dead_end child, assign it a child slot number
@@ -4083,7 +4095,7 @@ BackendStartup(Port *port)
bn->bgworker_notify = false;
#ifdef EXEC_BACKEND
- pid = backend_forkexec(port);
+ pid = backend_forkexec(port, cac);
#else /* !EXEC_BACKEND */
pid = fork_process();
if (pid == 0) /* child */
@@ -4095,7 +4107,7 @@ BackendStartup(Port *port)
ClosePostmasterPorts(false);
/* Perform additional initialization and collect startup packet */
- BackendInitialize(port);
+ BackendInitialize(port, cac);
/* And run the backend */
BackendRun(port);
@@ -4182,7 +4194,7 @@ report_fork_failure_to_client(Port *port, int errnum)
* but have not yet set up most of our local pointers to shmem structures.
*/
static void
-BackendInitialize(Port *port)
+BackendInitialize(Port *port, CAC_state cac)
{
int status;
int ret;
@@ -4315,7 +4327,7 @@ BackendInitialize(Port *port)
* now instead of wasting cycles on an authentication exchange. (This also
* allows a pg_ping utility to be written.)
*/
- switch (port->canAcceptConnections)
+ switch (cac)
{
case CAC_STARTUP:
ereport(FATAL,
@@ -4457,15 +4469,19 @@ postmaster_forkexec(int argc, char *argv[])
* returns the pid of the fork/exec'd process, or -1 on failure
*/
static pid_t
-backend_forkexec(Port *port)
+backend_forkexec(Port *port, CAC_state cac)
{
- char *av[4];
+ char *av[5];
int ac = 0;
+ char cacbuf[10];
av[ac++] = "postgres";
av[ac++] = "--forkbackend";
av[ac++] = NULL; /* filled in by internal_forkexec */
+ snprintf(cacbuf, sizeof(cacbuf), "%d", (int) cac);
+ av[ac++] = cacbuf;
+
av[ac] = NULL;
Assert(ac < lengthof(av));
@@ -4869,7 +4885,10 @@ SubPostmasterMain(int argc, char *argv[])
/* Run backend or appropriate child */
if (strcmp(argv[1], "--forkbackend") == 0)
{
- Assert(argc == 3); /* shouldn't be any more args */
+ CAC_state cac;
+
+ Assert(argc == 4);
+ cac = (CAC_state) atoi(argv[3]);
/*
* Need to reinitialize the SSL library in the backend, since the
@@ -4903,7 +4922,7 @@ SubPostmasterMain(int argc, char *argv[])
* PGPROC slots, we have already initialized libpq and are able to
* report the error to the client.
*/
- BackendInitialize(&port);
+ BackendInitialize(&port, cac);
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index c57ed12fb6d..335cb2de44a 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -58,17 +58,6 @@ typedef struct
#include "libpq/pqcomm.h"
-typedef enum CAC_state
-{
- CAC_OK,
- CAC_STARTUP,
- CAC_SHUTDOWN,
- CAC_RECOVERY,
- CAC_NOTCONSISTENT,
- CAC_TOOMANY,
-} CAC_state;
-
-
/*
* GSSAPI specific state information
*/
@@ -156,7 +145,6 @@ typedef struct Port
int remote_hostname_resolv; /* see above */
int remote_hostname_errcode; /* see above */
char *remote_port; /* text rep of remote port */
- CAC_state canAcceptConnections; /* postmaster connection status */
/*
* Information that needs to be saved from the startup packet and passed
--
2.39.2
v3-0005-Remove-ConnCreate-and-ConnFree-and-allocate-Port-.patchtext/x-patch; charset=UTF-8; name=v3-0005-Remove-ConnCreate-and-ConnFree-and-allocate-Port-.patchDownload
From 98f8397b32a0b36e221475b32697c9c5bbca86a0 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 11 Oct 2023 13:38:06 +0300
Subject: [PATCH v3 5/7] Remove ConnCreate and ConnFree, and allocate Port in
stack
By allocating Port in stack, we don't need to deal with out-of-memory
errors, which makes the code a little less verbose.
XXX: This should perhaps be squashed with the next commit. It changes
how the Port structure is allocated again. But maybe it's easier to
review separately.
Reviewed-by: Tristan Partin, Andres Freund
Discussion: https://www.postgresql.org/message-id/7a59b073-5b5b-151e-7ed3-8b01ff7ce9ef@iki.fi
---
src/backend/postmaster/postmaster.c | 68 +++++------------------------
src/backend/tcop/postgres.c | 2 +-
2 files changed, 11 insertions(+), 59 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 1a72b522997..dcec983e7d9 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -399,8 +399,6 @@ static void CloseServerPorts(int status, Datum arg);
static void unlink_external_pid_file(int status, Datum arg);
static void getInstallationPaths(const char *argv0);
static void checkControlFile(void);
-static Port *ConnCreate(int serverFd);
-static void ConnFree(Port *port);
static void handle_pm_pmsignal_signal(SIGNAL_ARGS);
static void handle_pm_child_exit_signal(SIGNAL_ARGS);
static void handle_pm_reload_request_signal(SIGNAL_ARGS);
@@ -1772,20 +1770,18 @@ ServerLoop(void)
if (events[i].events & WL_SOCKET_ACCEPT)
{
- Port *port;
+ Port port;
- port = ConnCreate(events[i].fd);
- if (port)
- {
- BackendStartup(port);
+ memset(&port, 0, sizeof(port));
+ if (StreamConnection(events[i].fd, &port) == STATUS_OK)
+ BackendStartup(&port);
- /*
- * We no longer need the open socket or port structure in
- * this process
- */
- StreamClose(port->sock);
- ConnFree(port);
- }
+ /*
+ * We no longer need the open socket or port structure in this
+ * process
+ */
+ if (port.sock != PGINVALID_SOCKET)
+ StreamClose(port.sock);
}
}
@@ -2447,50 +2443,6 @@ canAcceptConnections(int backend_type)
return result;
}
-
-/*
- * ConnCreate -- create a local connection data structure
- *
- * Returns NULL on failure, other than out-of-memory which is fatal.
- */
-static Port *
-ConnCreate(int serverFd)
-{
- Port *port;
-
- if (!(port = (Port *) calloc(1, sizeof(Port))))
- {
- ereport(LOG,
- (errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("out of memory")));
- ExitPostmaster(1);
- }
-
- if (StreamConnection(serverFd, port) != STATUS_OK)
- {
- if (port->sock != PGINVALID_SOCKET)
- StreamClose(port->sock);
- ConnFree(port);
- return NULL;
- }
-
- return port;
-}
-
-
-/*
- * ConnFree -- free a local connection data structure
- *
- * Caller has already closed the socket if any, so there's not much
- * to do here.
- */
-static void
-ConnFree(Port *port)
-{
- free(port);
-}
-
-
/*
* ClosePostmasterPorts -- close all the postmaster's open sockets
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index e415cf1f34a..fdf51ce281a 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -4214,7 +4214,7 @@ PostgresMain(const char *dbname, const char *username)
/*
* If the PostmasterContext is still around, recycle the space; we don't
* need it anymore after InitPostgres completes. Note this does not trash
- * *MyProcPort, because ConnCreate() allocated that space with malloc()
+ * *MyProcPort, because that space is allocated in stack
* ... else we'd need to copy the Port data first. Also, subsidiary data
* such as the username isn't lost either; see ProcessStartupPacket().
*/
--
2.39.2
v3-0006-Introduce-ClientSocket-rename-some-funcs.patchtext/x-patch; charset=UTF-8; name=v3-0006-Introduce-ClientSocket-rename-some-funcs.patchDownload
From 79aab42705a8cb0e16e61c33052fc56fdd4fca76 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 11 Oct 2023 13:38:10 +0300
Subject: [PATCH v3 6/7] Introduce ClientSocket, rename some funcs
- Move more of the work on a client socket to the child process.
- Reduce the amount of data that needs to be passed from postmaster to
child. (Used to pass a full Port struct, although most of the fields were
empty. Now we pass the much slimmer ClientSocket.)
Reviewed-by: Tristan Partin, Andres Freund
Discussion: https://www.postgresql.org/message-id/7a59b073-5b5b-151e-7ed3-8b01ff7ce9ef@iki.fi
---
src/backend/libpq/pqcomm.c | 93 +++++++++--------
src/backend/postmaster/postmaster.c | 149 +++++++++++++++-------------
src/backend/tcop/postgres.c | 5 +-
src/include/libpq/libpq-be.h | 19 ++--
src/include/libpq/libpq.h | 6 +-
src/tools/pgindent/typedefs.list | 1 +
6 files changed, 149 insertions(+), 124 deletions(-)
diff --git a/src/backend/libpq/pqcomm.c b/src/backend/libpq/pqcomm.c
index 2802efc63fc..ccdf1d2140f 100644
--- a/src/backend/libpq/pqcomm.c
+++ b/src/backend/libpq/pqcomm.c
@@ -29,9 +29,9 @@
* INTERFACE ROUTINES
*
* setup/teardown:
- * StreamServerPort - Open postmaster's server port
- * StreamConnection - Create new connection with client
- * StreamClose - Close a client/backend connection
+ * ListenServerPort - Open postmaster's server port
+ * AcceptClientConnection - Create new connection with client
+ * StreamConnection - Initialize a client connection
* TouchSocketFiles - Protect socket files against /tmp cleaners
* pq_init - initialize libpq at backend startup
* socket_comm_reset - reset libpq during error recovery
@@ -304,7 +304,7 @@ socket_close(int code, Datum arg)
/*
- * StreamServerPort -- open a "listening" port to accept connections.
+ * ListenServerPort -- open a "listening" port to accept connections.
*
* family should be AF_UNIX or AF_UNSPEC; portNumber is the port number.
* For AF_UNIX ports, hostName should be NULL and unixSocketDir must be
@@ -319,7 +319,7 @@ socket_close(int code, Datum arg)
* RETURNS: STATUS_OK or STATUS_ERROR
*/
int
-StreamServerPort(int family, const char *hostName, unsigned short portNumber,
+ListenServerPort(int family, const char *hostName, unsigned short portNumber,
const char *unixSocketDir,
pgsocket ListenSockets[], int *NumListenSockets, int MaxListen)
{
@@ -685,8 +685,9 @@ Setup_AF_UNIX(const char *sock_path)
/*
- * StreamConnection -- create a new connection with client using
- * server port. Set port->sock to the FD of the new connection.
+ * AcceptClientConnection -- accept a new connection with client using
+ * server port. Fills *client_sock with the FD and endpoint info
+ * of the new connection.
*
* ASSUME: that this doesn't need to be non-blocking because
* the Postmaster waits for the socket to be ready to accept().
@@ -694,13 +695,13 @@ Setup_AF_UNIX(const char *sock_path)
* RETURNS: STATUS_OK or STATUS_ERROR
*/
int
-StreamConnection(pgsocket server_fd, Port *port)
+AcceptClientConnection(pgsocket server_fd, ClientSocket *client_sock)
{
/* accept connection and fill in the client (remote) address */
- port->raddr.salen = sizeof(port->raddr.addr);
- if ((port->sock = accept(server_fd,
- (struct sockaddr *) &port->raddr.addr,
- &port->raddr.salen)) == PGINVALID_SOCKET)
+ client_sock->raddr.salen = sizeof(client_sock->raddr.addr);
+ if ((client_sock->sock = accept(server_fd,
+ (struct sockaddr *) &client_sock->raddr.addr,
+ &client_sock->raddr.salen)) == PGINVALID_SOCKET)
{
ereport(LOG,
(errcode_for_socket_access(),
@@ -718,10 +719,10 @@ StreamConnection(pgsocket server_fd, Port *port)
}
/* fill in the server (local) address */
- port->laddr.salen = sizeof(port->laddr.addr);
- if (getsockname(port->sock,
- (struct sockaddr *) &port->laddr.addr,
- &port->laddr.salen) < 0)
+ client_sock->laddr.salen = sizeof(client_sock->laddr.addr);
+ if (getsockname(client_sock->sock,
+ (struct sockaddr *) &client_sock->laddr.addr,
+ &client_sock->laddr.salen) < 0)
{
ereport(LOG,
(errmsg("%s() failed: %m", "getsockname")));
@@ -729,7 +730,7 @@ StreamConnection(pgsocket server_fd, Port *port)
}
/* select NODELAY and KEEPALIVE options if it's a TCP connection */
- if (port->laddr.addr.ss_family != AF_UNIX)
+ if (client_sock->laddr.addr.ss_family != AF_UNIX)
{
int on;
#ifdef WIN32
@@ -740,7 +741,7 @@ StreamConnection(pgsocket server_fd, Port *port)
#ifdef TCP_NODELAY
on = 1;
- if (setsockopt(port->sock, IPPROTO_TCP, TCP_NODELAY,
+ if (setsockopt(client_sock->sock, IPPROTO_TCP, TCP_NODELAY,
(char *) &on, sizeof(on)) < 0)
{
ereport(LOG,
@@ -749,7 +750,7 @@ StreamConnection(pgsocket server_fd, Port *port)
}
#endif
on = 1;
- if (setsockopt(port->sock, SOL_SOCKET, SO_KEEPALIVE,
+ if (setsockopt(client_sock->sock, SOL_SOCKET, SO_KEEPALIVE,
(char *) &on, sizeof(on)) < 0)
{
ereport(LOG,
@@ -781,7 +782,7 @@ StreamConnection(pgsocket server_fd, Port *port)
* https://msdn.microsoft.com/en-us/library/bb736549%28v=vs.85%29.aspx
*/
optlen = sizeof(oldopt);
- if (getsockopt(port->sock, SOL_SOCKET, SO_SNDBUF, (char *) &oldopt,
+ if (getsockopt(client_sock->sock, SOL_SOCKET, SO_SNDBUF, (char *) &oldopt,
&optlen) < 0)
{
ereport(LOG,
@@ -791,7 +792,7 @@ StreamConnection(pgsocket server_fd, Port *port)
newopt = PQ_SEND_BUFFER_SIZE * 4;
if (oldopt < newopt)
{
- if (setsockopt(port->sock, SOL_SOCKET, SO_SNDBUF, (char *) &newopt,
+ if (setsockopt(client_sock->sock, SOL_SOCKET, SO_SNDBUF, (char *) &newopt,
sizeof(newopt)) < 0)
{
ereport(LOG,
@@ -800,13 +801,34 @@ StreamConnection(pgsocket server_fd, Port *port)
}
}
#endif
+ }
+ return STATUS_OK;
+}
+
+/*
+ * StreamConnection -- create a new connection from the given socket.
+ *
+ * This runs in the backend process.
+ */
+Port *
+StreamConnection(ClientSocket *client_sock)
+{
+ Port *port;
+
+ port = palloc0(sizeof(Port));
+ port->sock = client_sock->sock;
+ port->laddr = client_sock->laddr;
+ port->raddr = client_sock->raddr;
+ /* Apply the current keepalive parameters if it's a TCP connection */
+ if (port->laddr.addr.ss_family != AF_UNIX)
+ {
/*
- * Also apply the current keepalive parameters. If we fail to set a
- * parameter, don't error out, because these aren't universally
- * supported. (Note: you might think we need to reset the GUC
- * variables to 0 in such a case, but it's not necessary because the
- * show hooks for these variables report the truth anyway.)
+ * If we fail to set a parameter, don't error out, because these
+ * aren't universally supported. (Note: you might think we need to
+ * reset the GUC variables to 0 in such a case, but it's not necessary
+ * because the show hooks for these variables report the truth
+ * anyway.)
*/
(void) pq_setkeepalivesidle(tcp_keepalives_idle, port);
(void) pq_setkeepalivesinterval(tcp_keepalives_interval, port);
@@ -814,24 +836,7 @@ StreamConnection(pgsocket server_fd, Port *port)
(void) pq_settcpusertimeout(tcp_user_timeout, port);
}
- return STATUS_OK;
-}
-
-/*
- * StreamClose -- close a client/backend connection
- *
- * NOTE: this is NOT used to terminate a session; it is just used to release
- * the file descriptor in a process that should no longer have the socket
- * open. (For example, the postmaster calls this after passing ownership
- * of the connection to a child process.) It is expected that someone else
- * still has the socket open. So, we only want to close the descriptor,
- * we do NOT want to send anything to the far end.
- */
-void
-StreamClose(pgsocket sock)
-{
- if (closesocket(sock) != 0)
- elog(LOG, "could not close client or listen socket: %m");
+ return port;
}
/*
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index dcec983e7d9..550136009a8 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -427,15 +427,15 @@ typedef enum CAC_state
CAC_TOOMANY,
} CAC_state;
-static void BackendInitialize(Port *port, CAC_state cac);
-static void BackendRun(Port *port) pg_attribute_noreturn();
+static void BackendInitialize(ClientSocket *client_sock, CAC_state cac);
+static void BackendRun(void) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
-static int BackendStartup(Port *port);
+static int BackendStartup(ClientSocket *port);
static int ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done);
static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
static void processCancelRequest(Port *port, void *pkt);
-static void report_fork_failure_to_client(Port *port, int errnum);
+static void report_fork_failure_to_client(ClientSocket *client_sock, int errnum);
static CAC_state canAcceptConnections(int backend_type);
static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
@@ -484,8 +484,8 @@ typedef struct
} win32_deadchild_waitinfo;
#endif /* WIN32 */
-static pid_t backend_forkexec(Port *port, CAC_state cac);
-static pid_t internal_forkexec(int argc, char *argv[], Port *port);
+static pid_t backend_forkexec(ClientSocket *client_sock, CAC_state cac);
+static pid_t internal_forkexec(int argc, char *argv[], ClientSocket *client_sock);
/* Type for a socket that can be inherited to a client process */
#ifdef WIN32
@@ -504,8 +504,8 @@ typedef int InheritableSocket;
*/
typedef struct
{
- Port port;
- InheritableSocket portsocket;
+ ClientSocket client_sock;
+ InheritableSocket serialized_sock;
char DataDir[MAXPGPATH];
int32 MyCancelKey;
int MyPMChildSlot;
@@ -553,13 +553,13 @@ typedef struct
BackgroundWorker MyBgworkerEntry;
} BackendParameters;
-static void read_backend_variables(char *id, Port *port);
-static void restore_backend_variables(BackendParameters *param, Port *port);
+static void read_backend_variables(char *id, ClientSocket *client_sock);
+static void restore_backend_variables(BackendParameters *param, ClientSocket *client_sock);
#ifndef WIN32
-static bool save_backend_variables(BackendParameters *param, Port *port);
+static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock);
#else
-static bool save_backend_variables(BackendParameters *param, Port *port,
+static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock,
HANDLE childProcess, pid_t childPid);
#endif
@@ -1219,14 +1219,14 @@ PostmasterMain(int argc, char *argv[])
char *curhost = (char *) lfirst(l);
if (strcmp(curhost, "*") == 0)
- status = StreamServerPort(AF_UNSPEC, NULL,
+ status = ListenServerPort(AF_UNSPEC, NULL,
(unsigned short) PostPortNumber,
NULL,
ListenSockets,
&NumListenSockets,
MAXLISTEN);
else
- status = StreamServerPort(AF_UNSPEC, curhost,
+ status = ListenServerPort(AF_UNSPEC, curhost,
(unsigned short) PostPortNumber,
NULL,
ListenSockets,
@@ -1320,7 +1320,7 @@ PostmasterMain(int argc, char *argv[])
{
char *socketdir = (char *) lfirst(l);
- status = StreamServerPort(AF_UNIX, NULL,
+ status = ListenServerPort(AF_UNIX, NULL,
(unsigned short) PostPortNumber,
socketdir,
ListenSockets,
@@ -1500,7 +1500,7 @@ CloseServerPorts(int status, Datum arg)
* condition if a new postmaster wants to re-use the TCP port number.
*/
for (i = 0; i < NumListenSockets; i++)
- StreamClose(ListenSockets[i]);
+ closesocket(ListenSockets[i]);
NumListenSockets = 0;
/*
@@ -1770,18 +1770,20 @@ ServerLoop(void)
if (events[i].events & WL_SOCKET_ACCEPT)
{
- Port port;
+ ClientSocket s;
- memset(&port, 0, sizeof(port));
- if (StreamConnection(events[i].fd, &port) == STATUS_OK)
- BackendStartup(&port);
+ if (AcceptClientConnection(events[i].fd, &s) == STATUS_OK)
+ BackendStartup(&s);
/*
* We no longer need the open socket or port structure in this
* process
*/
- if (port.sock != PGINVALID_SOCKET)
- StreamClose(port.sock);
+ if (s.sock != PGINVALID_SOCKET)
+ {
+ if (closesocket(s.sock) != 0)
+ elog(LOG, "could not close client socket: %m");
+ }
}
}
@@ -2146,11 +2148,7 @@ retry1:
/*
* Now fetch parameters out of startup packet and save them into the Port
- * structure. All data structures attached to the Port struct must be
- * allocated in TopMemoryContext so that they will remain available in a
- * running backend (even after PostmasterContext is destroyed). We need
- * not worry about leaking this storage on failure, since we aren't in the
- * postmaster process anymore.
+ * structure.
*/
oldcontext = MemoryContextSwitchTo(TopMemoryContext);
@@ -2286,7 +2284,7 @@ retry1:
port->database_name[0] = '\0';
/*
- * Done putting stuff in TopMemoryContext.
+ * Done filling the Port structure
*/
MemoryContextSwitchTo(oldcontext);
@@ -2490,7 +2488,10 @@ ClosePostmasterPorts(bool am_syslogger)
if (ListenSockets)
{
for (int i = 0; i < NumListenSockets; i++)
- StreamClose(ListenSockets[i]);
+ {
+ if (closesocket(ListenSockets[i]) != 0)
+ elog(LOG, "could not close listen socket: %m");
+ }
pfree(ListenSockets);
}
NumListenSockets = 0;
@@ -3996,7 +3997,7 @@ TerminateChildren(int signal)
* Note: if you change this code, also consider StartAutovacuumWorker.
*/
static int
-BackendStartup(Port *port)
+BackendStartup(ClientSocket *client_sock)
{
Backend *bn; /* for backend cleanup */
pid_t pid;
@@ -4047,7 +4048,7 @@ BackendStartup(Port *port)
bn->bgworker_notify = false;
#ifdef EXEC_BACKEND
- pid = backend_forkexec(port, cac);
+ pid = backend_forkexec(client_sock, cac);
#else /* !EXEC_BACKEND */
pid = fork_process();
if (pid == 0) /* child */
@@ -4059,10 +4060,10 @@ BackendStartup(Port *port)
ClosePostmasterPorts(false);
/* Perform additional initialization and collect startup packet */
- BackendInitialize(port, cac);
+ BackendInitialize(client_sock, cac);
/* And run the backend */
- BackendRun(port);
+ BackendRun();
}
#endif /* EXEC_BACKEND */
@@ -4077,14 +4078,14 @@ BackendStartup(Port *port)
errno = save_errno;
ereport(LOG,
(errmsg("could not fork new process for connection: %m")));
- report_fork_failure_to_client(port, save_errno);
+ report_fork_failure_to_client(client_sock, save_errno);
return STATUS_ERROR;
}
/* in parent, successful fork */
ereport(DEBUG2,
(errmsg_internal("forked new backend, pid=%d socket=%d",
- (int) pid, (int) port->sock)));
+ (int) pid, (int) client_sock->sock)));
/*
* Everything's been successful, it's safe to add this backend to our list
@@ -4111,7 +4112,7 @@ BackendStartup(Port *port)
* it's not up and running.
*/
static void
-report_fork_failure_to_client(Port *port, int errnum)
+report_fork_failure_to_client(ClientSocket *client_sock, int errnum)
{
char buffer[1000];
int rc;
@@ -4122,13 +4123,13 @@ report_fork_failure_to_client(Port *port, int errnum)
strerror(errnum));
/* Set port to non-blocking. Don't do send() if this fails */
- if (!pg_set_noblock(port->sock))
+ if (!pg_set_noblock(client_sock->sock))
return;
/* We'll retry after EINTR, but ignore all other failures */
do
{
- rc = send(port->sock, buffer, strlen(buffer) + 1, 0);
+ rc = send(client_sock->sock, buffer, strlen(buffer) + 1, 0);
} while (rc < 0 && errno == EINTR);
}
@@ -4146,16 +4147,28 @@ report_fork_failure_to_client(Port *port, int errnum)
* but have not yet set up most of our local pointers to shmem structures.
*/
static void
-BackendInitialize(Port *port, CAC_state cac)
+BackendInitialize(ClientSocket *client_sock, CAC_state cac)
{
int status;
int ret;
+ Port *port;
char remote_host[NI_MAXHOST];
char remote_port[NI_MAXSERV];
StringInfoData ps_data;
+ MemoryContext oldcontext;
- /* Save port etc. for ps status */
+ /*
+ * Create the Port structure.
+ *
+ * The Port structure and all data structures attached to it are allocated
+ * in TopMemoryContext, so that they survive into PostgresMain execution.
+ * We need not worry about leaking this storage on failure, since we
+ * aren't in the postmaster process anymore.
+ */
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
+ port = StreamConnection(client_sock);
MyProcPort = port;
+ MemoryContextSwitchTo(oldcontext);
/* Tell fd.c about the long-lived FD associated with the port */
ReserveExternalFD();
@@ -4216,8 +4229,9 @@ BackendInitialize(Port *port, CAC_state cac)
* Save remote_host and remote_port in port structure (after this, they
* will appear in log_line_prefix data for log messages).
*/
- port->remote_host = strdup(remote_host);
- port->remote_port = strdup(remote_port);
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
+ port->remote_host = pstrdup(remote_host);
+ port->remote_port = pstrdup(remote_port);
/* And now we can issue the Log_connections message, if wanted */
if (Log_connections)
@@ -4248,7 +4262,8 @@ BackendInitialize(Port *port, CAC_state cac)
ret == 0 &&
strspn(remote_host, "0123456789.") < strlen(remote_host) &&
strspn(remote_host, "0123456789ABCDEFabcdef:") < strlen(remote_host))
- port->remote_hostname = strdup(remote_host);
+ port->remote_hostname = pstrdup(remote_host);
+ MemoryContextSwitchTo(oldcontext);
/*
* Ready to begin client interaction. We will give up and _exit(1) after
@@ -4369,7 +4384,7 @@ BackendInitialize(Port *port, CAC_state cac)
* Doesn't return at all.
*/
static void
-BackendRun(Port *port)
+BackendRun(void)
{
/*
* Create a per-backend PGPROC struct in shared memory. We must do this
@@ -4383,7 +4398,7 @@ BackendRun(Port *port)
*/
MemoryContextSwitchTo(TopMemoryContext);
- PostgresMain(port->database_name, port->user_name);
+ PostgresMain(MyProcPort->database_name, MyProcPort->user_name);
}
@@ -4404,11 +4419,11 @@ BackendRun(Port *port)
pid_t
postmaster_forkexec(int argc, char *argv[])
{
- Port port;
+ ClientSocket client_sock;
- /* This entry point passes dummy values for the Port variables */
- memset(&port, 0, sizeof(port));
- return internal_forkexec(argc, argv, &port);
+ /* This entry point doesn't pass a client socket */
+ memset(&client_sock, 0, sizeof(ClientSocket));
+ return internal_forkexec(argc, argv, &client_sock);
}
/*
@@ -4421,7 +4436,7 @@ postmaster_forkexec(int argc, char *argv[])
* returns the pid of the fork/exec'd process, or -1 on failure
*/
static pid_t
-backend_forkexec(Port *port, CAC_state cac)
+backend_forkexec(ClientSocket *client_sock, CAC_state cac)
{
char *av[5];
int ac = 0;
@@ -4437,7 +4452,7 @@ backend_forkexec(Port *port, CAC_state cac)
av[ac] = NULL;
Assert(ac < lengthof(av));
- return internal_forkexec(ac, av, port);
+ return internal_forkexec(ac, av, client_sock);
}
#ifndef WIN32
@@ -4449,7 +4464,7 @@ backend_forkexec(Port *port, CAC_state cac)
* - fork():s, and then exec():s the child process
*/
static pid_t
-internal_forkexec(int argc, char *argv[], Port *port)
+internal_forkexec(int argc, char *argv[], ClientSocket *client_sock)
{
static unsigned long tmpBackendFileNum = 0;
pid_t pid;
@@ -4457,7 +4472,7 @@ internal_forkexec(int argc, char *argv[], Port *port)
BackendParameters param;
FILE *fp;
- if (!save_backend_variables(¶m, port))
+ if (!save_backend_variables(¶m, client_sock))
return -1; /* log made by save_backend_variables */
/* Calculate name for temp file */
@@ -4755,7 +4770,7 @@ retry:
void
SubPostmasterMain(int argc, char *argv[])
{
- Port port;
+ ClientSocket client_sock;
/* In EXEC_BACKEND case we will not have inherited these settings */
IsPostmasterEnvironment = true;
@@ -4769,8 +4784,8 @@ SubPostmasterMain(int argc, char *argv[])
elog(FATAL, "invalid subpostmaster invocation");
/* Read in the variables file */
- memset(&port, 0, sizeof(Port));
- read_backend_variables(argv[2], &port);
+ memset(&client_sock, 0, sizeof(ClientSocket));
+ read_backend_variables(argv[2], &client_sock);
/* Close the postmaster's sockets (as soon as we know them) */
ClosePostmasterPorts(strcmp(argv[1], "--forklog") == 0);
@@ -4874,13 +4889,13 @@ SubPostmasterMain(int argc, char *argv[])
* PGPROC slots, we have already initialized libpq and are able to
* report the error to the client.
*/
- BackendInitialize(&port, cac);
+ BackendInitialize(&client_sock, cac);
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
/* And run the backend */
- BackendRun(&port); /* does not return */
+ BackendRun(); /* does not return */
}
if (strcmp(argv[1], "--forkaux") == 0)
{
@@ -5968,15 +5983,15 @@ static void read_inheritable_socket(SOCKET *dest, InheritableSocket *src);
/* Save critical backend variables into the BackendParameters struct */
#ifndef WIN32
static bool
-save_backend_variables(BackendParameters *param, Port *port)
+save_backend_variables(BackendParameters *param, ClientSocket *client_sock)
#else
static bool
-save_backend_variables(BackendParameters *param, Port *port,
+save_backend_variables(BackendParameters *param, ClientSocket *client_sock,
HANDLE childProcess, pid_t childPid)
#endif
{
- memcpy(¶m->port, port, sizeof(Port));
- if (!write_inheritable_socket(¶m->portsocket, port->sock, childPid))
+ memcpy(¶m->client_sock, client_sock, sizeof(ClientSocket));
+ if (!write_inheritable_socket(¶m->serialized_sock, client_sock->sock, childPid))
return false;
strlcpy(param->DataDir, DataDir, MAXPGPATH);
@@ -6138,7 +6153,7 @@ read_inheritable_socket(SOCKET *dest, InheritableSocket *src)
#endif
static void
-read_backend_variables(char *id, Port *port)
+read_backend_variables(char *id, ClientSocket *client_sock)
{
BackendParameters param;
@@ -6205,15 +6220,15 @@ read_backend_variables(char *id, Port *port)
}
#endif
- restore_backend_variables(¶m, port);
+ restore_backend_variables(¶m, client_sock);
}
/* Restore critical backend variables from the BackendParameters struct */
static void
-restore_backend_variables(BackendParameters *param, Port *port)
+restore_backend_variables(BackendParameters *param, ClientSocket *client_sock)
{
- memcpy(port, ¶m->port, sizeof(Port));
- read_inheritable_socket(&port->sock, ¶m->portsocket);
+ memcpy(client_sock, ¶m->client_sock, sizeof(ClientSocket));
+ read_inheritable_socket(&client_sock->sock, ¶m->serialized_sock);
SetDataDir(param->DataDir);
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index fdf51ce281a..9a9a1e70039 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -4213,10 +4213,7 @@ PostgresMain(const char *dbname, const char *username)
/*
* If the PostmasterContext is still around, recycle the space; we don't
- * need it anymore after InitPostgres completes. Note this does not trash
- * *MyProcPort, because that space is allocated in stack
- * ... else we'd need to copy the Port data first. Also, subsidiary data
- * such as the username isn't lost either; see ProcessStartupPacket().
+ * need it anymore after InitPostgres completes.
*/
if (PostmasterContext)
{
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 335cb2de44a..9b6d8fc5571 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -110,12 +110,9 @@ typedef struct ClientConnectionInfo
} ClientConnectionInfo;
/*
- * This is used by the postmaster in its communication with frontends. It
- * contains all state information needed during this communication before the
- * backend is run. The Port structure is kept in malloc'd memory and is
- * still available when a backend is running (see MyProcPort). The data
- * it points to must also be malloc'd, or else palloc'd in TopMemoryContext,
- * so that it survives into PostgresMain execution!
+ * The Port structure holds state information about a client connection in a
+ * backend process. It is available in the global variable MyProcPort. The
+ * struct and all the data it points are kept in TopMemoryContext.
*
* remote_hostname is set if we did a successful reverse lookup of the
* client's IP address during connection setup.
@@ -217,6 +214,16 @@ typedef struct Port
#endif
} Port;
+/*
+ * ClientSocket holds a socket for an accepted connection, along with the
+ * information about the endpoints.
+ */
+typedef struct ClientSocket {
+ pgsocket sock; /* File descriptor */
+ SockAddr laddr; /* local addr (postmaster) */
+ SockAddr raddr; /* remote addr (client) */
+} ClientSocket;
+
#ifdef USE_SSL
/*
* Hardcoded DH parameters, used in ephemeral DH keying. (See also
diff --git a/src/include/libpq/libpq.h b/src/include/libpq/libpq.h
index a6104d8cd02..889e86a7f61 100644
--- a/src/include/libpq/libpq.h
+++ b/src/include/libpq/libpq.h
@@ -64,11 +64,11 @@ extern PGDLLIMPORT WaitEventSet *FeBeWaitSet;
#define FeBeWaitSetLatchPos 1
#define FeBeWaitSetNEvents 3
-extern int StreamServerPort(int family, const char *hostName,
+extern int ListenServerPort(int family, const char *hostName,
unsigned short portNumber, const char *unixSocketDir,
pgsocket ListenSocket[], int *NumListenSockets, int MaxListen);
-extern int StreamConnection(pgsocket server_fd, Port *port);
-extern void StreamClose(pgsocket sock);
+extern int AcceptClientConnection(pgsocket server_fd, ClientSocket *client_sock);
+extern Port *StreamConnection(ClientSocket *client_sock);
extern void TouchSocketFiles(void);
extern void RemoveSocketFiles(void);
extern void pq_init(void);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index d659adbfd6c..00649608e46 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -382,6 +382,7 @@ ClientCertMode
ClientCertName
ClientConnectionInfo
ClientData
+ClientSocket
ClonePtrType
ClosePortalStmt
ClosePtrType
--
2.39.2
v3-0007-Refactor-postmaster-child-process-launching.patchtext/x-patch; charset=UTF-8; name=v3-0007-Refactor-postmaster-child-process-launching.patchDownload
From ce51876f87f1e4317e25baf64184749448fcd033 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Thu, 30 Nov 2023 00:07:34 +0200
Subject: [PATCH v3 7/7] Refactor postmaster child process launching
- Move code related to launching backend processes to new source file,
launch_backend.c
- Introduce new postmaster_child_launch() function that deals with the
differences between EXEC_BACKEND and fork mode.
- Refactor the mechanism of passing information from the parent to
child process. Instead of using different command-line arguments when
launching the child process in EXEC_BACKEND mode, pass a
variable-length blob of data along with all the global variables. The
contents of that blob depend on the kind of child process being
launched. In !EXEC_BACKEND mode, we use the same blob, but it's simply
inherited from the parent to child process.
Reviewed-by: Tristan Partin, Andres Freund
Discussion: https://www.postgresql.org/message-id/7a59b073-5b5b-151e-7ed3-8b01ff7ce9ef@iki.fi
---
src/backend/postmaster/Makefile | 2 +-
src/backend/postmaster/autovacuum.c | 173 +--
src/backend/postmaster/auxprocess.c | 72 +-
src/backend/postmaster/bgworker.c | 14 +-
src/backend/postmaster/bgwriter.c | 9 +-
src/backend/postmaster/checkpointer.c | 9 +-
src/backend/postmaster/fork_process.c | 126 --
src/backend/postmaster/launch_backend.c | 1272 +++++++++++++++
src/backend/postmaster/meson.build | 2 +-
src/backend/postmaster/pgarch.c | 9 +-
src/backend/postmaster/postmaster.c | 1553 +++----------------
src/backend/postmaster/startup.c | 9 +-
src/backend/postmaster/syslogger.c | 275 ++--
src/backend/postmaster/walwriter.c | 9 +-
src/backend/replication/logical/launcher.c | 1 -
src/backend/replication/walreceiver.c | 9 +-
src/backend/storage/ipc/shmem.c | 2 +
src/backend/utils/init/globals.c | 1 +
src/backend/utils/init/miscinit.c | 133 --
src/include/libpq/libpq-be.h | 11 +-
src/include/miscadmin.h | 1 -
src/include/postmaster/autovacuum.h | 12 +-
src/include/postmaster/auxprocess.h | 4 +-
src/include/postmaster/bgworker_internals.h | 2 +-
src/include/postmaster/bgwriter.h | 4 +-
src/include/postmaster/fork_process.h | 17 -
src/include/postmaster/pgarch.h | 2 +-
src/include/postmaster/postmaster.h | 44 +-
src/include/postmaster/startup.h | 2 +-
src/include/postmaster/syslogger.h | 4 +-
src/include/postmaster/walwriter.h | 2 +-
src/include/replication/walreceiver.h | 2 +-
src/tools/pgindent/typedefs.list | 2 +
33 files changed, 1787 insertions(+), 2002 deletions(-)
delete mode 100644 src/backend/postmaster/fork_process.c
create mode 100644 src/backend/postmaster/launch_backend.c
delete mode 100644 src/include/postmaster/fork_process.h
diff --git a/src/backend/postmaster/Makefile b/src/backend/postmaster/Makefile
index 047448b34eb..fc88f5bae0b 100644
--- a/src/backend/postmaster/Makefile
+++ b/src/backend/postmaster/Makefile
@@ -18,8 +18,8 @@ OBJS = \
bgworker.o \
bgwriter.o \
checkpointer.o \
- fork_process.o \
interrupt.o \
+ launch_backend.o \
pgarch.o \
postmaster.o \
startup.o \
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 2f54485c217..0dda47ba300 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -84,7 +84,6 @@
#include "nodes/makefuncs.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "storage/bufmgr.h"
@@ -315,13 +314,6 @@ static WorkerInfo MyWorkerInfo = NULL;
/* PID of launcher, valid only in worker while shutting down */
int AutovacuumLauncherPid = 0;
-#ifdef EXEC_BACKEND
-static pid_t avlauncher_forkexec(void);
-static pid_t avworker_forkexec(void);
-#endif
-NON_EXEC_STATIC void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
-NON_EXEC_STATIC void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
-
static Oid do_start_worker(void);
static void HandleAutoVacLauncherInterrupts(void);
static void AutoVacLauncherShutdown(void) pg_attribute_noreturn();
@@ -365,85 +357,21 @@ static void avl_sigusr2_handler(SIGNAL_ARGS);
* AUTOVACUUM LAUNCHER CODE
********************************************************************/
-#ifdef EXEC_BACKEND
/*
- * forkexec routine for the autovacuum launcher process.
- *
- * Format up the arglist, then fork and exec.
- */
-static pid_t
-avlauncher_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkavlauncher";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-
-/*
- * We need this set from the outside, before InitProcess is called
+ * Main loop for the autovacuum launcher process.
*/
void
-AutovacuumLauncherIAm(void)
-{
- am_autovacuum_launcher = true;
-}
-#endif
-
-/*
- * Main entry point for autovacuum launcher process, to be called from the
- * postmaster.
- */
-int
-StartAutoVacLauncher(void)
+AutoVacLauncherMain(char *startup_data, size_t startup_data_len)
{
- pid_t AutoVacPID;
+ sigjmp_buf local_sigjmp_buf;
-#ifdef EXEC_BACKEND
- switch ((AutoVacPID = avlauncher_forkexec()))
-#else
- switch ((AutoVacPID = fork_process()))
-#endif
+ /* Release postmaster's working memory context */
+ if (PostmasterContext)
{
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum launcher process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacLauncherMain(0, NULL);
- break;
-#endif
- default:
- return (int) AutoVacPID;
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
}
- /* shouldn't get here */
- return 0;
-}
-
-/*
- * Main loop for the autovacuum launcher process.
- */
-NON_EXEC_STATIC void
-AutoVacLauncherMain(int argc, char *argv[])
-{
- sigjmp_buf local_sigjmp_buf;
-
am_autovacuum_launcher = true;
MyBackendType = B_AUTOVAC_LAUNCHER;
@@ -475,6 +403,9 @@ AutoVacLauncherMain(int argc, char *argv[])
pqsignal(SIGFPE, FloatExceptionHandler);
pqsignal(SIGCHLD, SIG_DFL);
+ /* autovacuum needs this set before calling InitProcess */
+ am_autovacuum_launcher = true;
+
/*
* Create a per-backend PGPROC struct in shared memory. We must do this
* before we can use LWLocks or access any shared memory.
@@ -1432,87 +1363,22 @@ avl_sigusr2_handler(SIGNAL_ARGS)
* AUTOVACUUM WORKER CODE
********************************************************************/
-#ifdef EXEC_BACKEND
-/*
- * forkexec routines for the autovacuum worker.
- *
- * Format up the arglist, then fork and exec.
- */
-static pid_t
-avworker_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkavworker";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-
/*
- * We need this set from the outside, before InitProcess is called
+ * AutoVacWorkerMain
*/
void
-AutovacuumWorkerIAm(void)
+AutoVacWorkerMain(char *startup_data, size_t startup_data_len)
{
- am_autovacuum_worker = true;
-}
-#endif
-
-/*
- * Main entry point for autovacuum worker process.
- *
- * This code is heavily based on pgarch.c, q.v.
- */
-int
-StartAutoVacWorker(void)
-{
- pid_t worker_pid;
+ sigjmp_buf local_sigjmp_buf;
+ Oid dbid;
-#ifdef EXEC_BACKEND
- switch ((worker_pid = avworker_forkexec()))
-#else
- switch ((worker_pid = fork_process()))
-#endif
+ /* Release postmaster's working memory context */
+ if (PostmasterContext)
{
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum worker process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacWorkerMain(0, NULL);
- break;
-#endif
- default:
- return (int) worker_pid;
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
}
- /* shouldn't get here */
- return 0;
-}
-
-/*
- * AutoVacWorkerMain
- */
-NON_EXEC_STATIC void
-AutoVacWorkerMain(int argc, char *argv[])
-{
- sigjmp_buf local_sigjmp_buf;
- Oid dbid;
-
am_autovacuum_worker = true;
MyBackendType = B_AUTOVAC_WORKER;
@@ -1543,6 +1409,9 @@ AutoVacWorkerMain(int argc, char *argv[])
pqsignal(SIGFPE, FloatExceptionHandler);
pqsignal(SIGCHLD, SIG_DFL);
+ /* autovacuum needs this set before calling InitProcess */
+ am_autovacuum_worker = true;
+
/*
* Create a per-backend PGPROC struct in shared memory. We must do this
* before we can use LWLocks or access any shared memory.
diff --git a/src/backend/postmaster/auxprocess.c b/src/backend/postmaster/auxprocess.c
index bae6f68c402..d4acd5146f4 100644
--- a/src/backend/postmaster/auxprocess.c
+++ b/src/backend/postmaster/auxprocess.c
@@ -46,45 +46,23 @@ AuxProcType MyAuxProcType = NotAnAuxProcess; /* declared in miscadmin.h */
/*
- * AuxiliaryProcessMain
+ * AuxiliaryProcessInit
*
- * The main entry point for auxiliary processes, such as the bgwriter,
+ * Common initialization code for auxiliary processes, such as the bgwriter,
* walwriter, walreceiver, bootstrapper and the shared memory checker code.
- *
- * This code is here just because of historical reasons.
*/
void
-AuxiliaryProcessMain(AuxProcType auxtype)
+AuxiliaryProcessInit(void)
{
- Assert(IsUnderPostmaster);
-
- MyAuxProcType = auxtype;
-
- switch (MyAuxProcType)
+ /* Release postmaster's working memory context */
+ if (PostmasterContext)
{
- case StartupProcess:
- MyBackendType = B_STARTUP;
- break;
- case ArchiverProcess:
- MyBackendType = B_ARCHIVER;
- break;
- case BgWriterProcess:
- MyBackendType = B_BG_WRITER;
- break;
- case CheckpointerProcess:
- MyBackendType = B_CHECKPOINTER;
- break;
- case WalWriterProcess:
- MyBackendType = B_WAL_WRITER;
- break;
- case WalReceiverProcess:
- MyBackendType = B_WAL_RECEIVER;
- break;
- default:
- elog(PANIC, "unrecognized process type: %d", (int) MyAuxProcType);
- MyBackendType = B_INVALID;
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
}
+ Assert(IsUnderPostmaster);
+
init_ps_display(NULL);
SetProcessingMode(BootstrapProcessing);
@@ -122,7 +100,6 @@ AuxiliaryProcessMain(AuxProcType auxtype)
*/
CreateAuxProcessResourceOwner();
-
/* Initialize backend status information */
pgstat_beinit();
pgstat_bestart();
@@ -131,37 +108,6 @@ AuxiliaryProcessMain(AuxProcType auxtype)
before_shmem_exit(ShutdownAuxiliaryProcess, 0);
SetProcessingMode(NormalProcessing);
-
- switch (MyAuxProcType)
- {
- case StartupProcess:
- StartupProcessMain();
- proc_exit(1);
-
- case ArchiverProcess:
- PgArchiverMain();
- proc_exit(1);
-
- case BgWriterProcess:
- BackgroundWriterMain();
- proc_exit(1);
-
- case CheckpointerProcess:
- CheckpointerMain();
- proc_exit(1);
-
- case WalWriterProcess:
- WalWriterMain();
- proc_exit(1);
-
- case WalReceiverProcess:
- WalReceiverMain();
- proc_exit(1);
-
- default:
- elog(PANIC, "unrecognized process type: %d", (int) MyAuxProcType);
- proc_exit(1);
- }
}
/*
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index 3fbc8f503cb..a27a658c025 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -723,17 +723,27 @@ bgworker_die(SIGNAL_ARGS)
* Main entry point for background worker processes.
*/
void
-BackgroundWorkerMain(void)
+BackgroundWorkerMain(char *startup_data, size_t startup_data_len)
{
sigjmp_buf local_sigjmp_buf;
- BackgroundWorker *worker = MyBgworkerEntry;
+ BackgroundWorker *worker;
bgworker_main_type entrypt;
+ /* Release postmaster's working memory context */
+ if (PostmasterContext)
+ {
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
+ }
+
+ Assert(startup_data_len == sizeof(BackgroundWorker));
+ worker = (BackgroundWorker *) startup_data;
if (worker == NULL)
elog(FATAL, "unable to find bgworker entry");
IsBackgroundWorker = true;
+ MyBgworkerEntry = worker;
MyBackendType = B_BG_WORKER;
init_ps_display(worker->bgw_name);
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index d02dc17b9c1..027590f824b 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -36,6 +36,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/bgwriter.h"
#include "postmaster/interrupt.h"
#include "storage/buf_internals.h"
@@ -88,13 +89,19 @@ static XLogRecPtr last_snapshot_lsn = InvalidXLogRecPtr;
* basic execution environment, but not enabled signals yet.
*/
void
-BackgroundWriterMain(void)
+BackgroundWriterMain(char *startup_data, size_t startup_data_len)
{
sigjmp_buf local_sigjmp_buf;
MemoryContext bgwriter_context;
bool prev_hibernate;
WritebackContext wb_context;
+ Assert(startup_data_len == 0);
+
+ MyAuxProcType = BgWriterProcess;
+ MyBackendType = B_BG_WRITER;
+ AuxiliaryProcessInit();
+
/*
* Properly accept or ignore signals that might be sent to us.
*/
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 42c807d3528..363c105cee6 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -42,6 +42,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/bgwriter.h"
#include "postmaster/interrupt.h"
#include "replication/syncrep.h"
@@ -169,11 +170,17 @@ static void ReqCheckpointHandler(SIGNAL_ARGS);
* basic execution environment, but not enabled signals yet.
*/
void
-CheckpointerMain(void)
+CheckpointerMain(char *startup_data, size_t startup_data_len)
{
sigjmp_buf local_sigjmp_buf;
MemoryContext checkpointer_context;
+ Assert(startup_data_len == 0);
+
+ MyAuxProcType = CheckpointerProcess;
+ MyBackendType = B_CHECKPOINTER;
+ AuxiliaryProcessInit();
+
CheckpointerShmem->checkpointer_pid = MyProcPid;
/*
diff --git a/src/backend/postmaster/fork_process.c b/src/backend/postmaster/fork_process.c
deleted file mode 100644
index 6f9c2765d68..00000000000
--- a/src/backend/postmaster/fork_process.c
+++ /dev/null
@@ -1,126 +0,0 @@
-/*
- * fork_process.c
- * A simple wrapper on top of fork(). This does not handle the
- * EXEC_BACKEND case; it might be extended to do so, but it would be
- * considerably more complex.
- *
- * Copyright (c) 1996-2023, PostgreSQL Global Development Group
- *
- * IDENTIFICATION
- * src/backend/postmaster/fork_process.c
- */
-#include "postgres.h"
-
-#include <fcntl.h>
-#include <signal.h>
-#include <time.h>
-#include <sys/stat.h>
-#include <sys/time.h>
-#include <unistd.h>
-
-#include "libpq/pqsignal.h"
-#include "postmaster/fork_process.h"
-
-#ifndef WIN32
-/*
- * Wrapper for fork(). Return values are the same as those for fork():
- * -1 if the fork failed, 0 in the child process, and the PID of the
- * child in the parent process. Signals are blocked while forking, so
- * the child must unblock.
- */
-pid_t
-fork_process(void)
-{
- pid_t result;
- const char *oomfilename;
- sigset_t save_mask;
-
-#ifdef LINUX_PROFILE
- struct itimerval prof_itimer;
-#endif
-
- /*
- * Flush stdio channels just before fork, to avoid double-output problems.
- */
- fflush(NULL);
-
-#ifdef LINUX_PROFILE
-
- /*
- * Linux's fork() resets the profiling timer in the child process. If we
- * want to profile child processes then we need to save and restore the
- * timer setting. This is a waste of time if not profiling, however, so
- * only do it if commanded by specific -DLINUX_PROFILE switch.
- */
- getitimer(ITIMER_PROF, &prof_itimer);
-#endif
-
- /*
- * We start postmaster children with signals blocked. This allows them to
- * install their own handlers before unblocking, to avoid races where they
- * might run the postmaster's handler and miss an important control
- * signal. With more analysis this could potentially be relaxed.
- */
- sigprocmask(SIG_SETMASK, &BlockSig, &save_mask);
- result = fork();
- if (result == 0)
- {
- /* fork succeeded, in child */
-#ifdef LINUX_PROFILE
- setitimer(ITIMER_PROF, &prof_itimer, NULL);
-#endif
-
- /*
- * By default, Linux tends to kill the postmaster in out-of-memory
- * situations, because it blames the postmaster for the sum of child
- * process sizes *including shared memory*. (This is unbelievably
- * stupid, but the kernel hackers seem uninterested in improving it.)
- * Therefore it's often a good idea to protect the postmaster by
- * setting its OOM score adjustment negative (which has to be done in
- * a root-owned startup script). Since the adjustment is inherited by
- * child processes, this would ordinarily mean that all the
- * postmaster's children are equally protected against OOM kill, which
- * is not such a good idea. So we provide this code to allow the
- * children to change their OOM score adjustments again. Both the
- * file name to write to and the value to write are controlled by
- * environment variables, which can be set by the same startup script
- * that did the original adjustment.
- */
- oomfilename = getenv("PG_OOM_ADJUST_FILE");
-
- if (oomfilename != NULL)
- {
- /*
- * Use open() not stdio, to ensure we control the open flags. Some
- * Linux security environments reject anything but O_WRONLY.
- */
- int fd = open(oomfilename, O_WRONLY, 0);
-
- /* We ignore all errors */
- if (fd >= 0)
- {
- const char *oomvalue = getenv("PG_OOM_ADJUST_VALUE");
- int rc;
-
- if (oomvalue == NULL) /* supply a useful default */
- oomvalue = "0";
-
- rc = write(fd, oomvalue, strlen(oomvalue));
- (void) rc;
- close(fd);
- }
- }
-
- /* do post-fork initialization for random number generation */
- pg_strong_random_init();
- }
- else
- {
- /* in parent, restore signal mask */
- sigprocmask(SIG_SETMASK, &save_mask, NULL);
- }
-
- return result;
-}
-
-#endif /* ! WIN32 */
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
new file mode 100644
index 00000000000..fac658ba687
--- /dev/null
+++ b/src/backend/postmaster/launch_backend.c
@@ -0,0 +1,1272 @@
+/*-------------------------------------------------------------------------
+ *
+ * launch_backend.c
+ * Functions for launching backends and other postmaster child
+ * processes.
+ *
+ * On Unix systems, a new child process is launched with fork(). It inherits
+ * all the global variables and data structures that had been initialized in
+ * the postmaster. After forking, the child process closes the file
+ * descriptors that are not needed in the child process, and sets up the
+ * mechanism to detect death of the parent postmaster process, etc. After
+ * that, it calls the right Main function depending on the kind of child
+ * process.
+ *
+ * In EXEC_BACKEND mode, which is used on Windows but can be enabled on other
+ * platforms for testing, the child process is launched by fork() + exec() (or
+ * CreateProcess() on Windows). It does not inherit the state from the
+ * postmaster, so it needs to re-attach to the shared memory, re-initialize
+ * global variables, re-load the config file etc.
+ *
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/postmaster/launch_backend.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include <unistd.h>
+#include <signal.h>
+#include <time.h>
+#include <sys/wait.h>
+#include <sys/stat.h>
+#include <sys/socket.h>
+#include <fcntl.h>
+#include <sys/param.h>
+#include <netdb.h>
+#include <limits.h>
+
+#include "access/transam.h"
+#include "access/xlog.h"
+#include "common/file_utils.h"
+#include "libpq/libpq-be.h"
+#include "libpq/pqsignal.h"
+#include "miscadmin.h"
+#include "nodes/queryjumble.h"
+#include "port.h"
+#include "postmaster/autovacuum.h"
+#include "postmaster/auxprocess.h"
+#include "postmaster/bgworker_internals.h"
+#include "postmaster/bgwriter.h"
+#include "postmaster/interrupt.h"
+#include "postmaster/pgarch.h"
+#include "postmaster/postmaster.h"
+#include "postmaster/startup.h"
+#include "postmaster/syslogger.h"
+#include "postmaster/walwriter.h"
+#include "replication/walreceiver.h"
+#include "storage/fd.h"
+#include "storage/ipc.h"
+#include "storage/pg_shmem.h"
+#include "storage/pmsignal.h"
+#include "storage/proc.h"
+#include "tcop/tcopprot.h"
+#include "utils/builtins.h"
+#include "utils/datetime.h"
+#include "utils/guc.h"
+#include "utils/memutils.h"
+#include "utils/ps_status.h"
+#include "utils/timeout.h"
+#include "utils/timestamp.h"
+
+#ifdef EXEC_BACKEND
+#include "storage/spin.h"
+#endif
+
+
+/* Type for a socket that can be inherited to a client process */
+#ifdef WIN32
+typedef struct
+{
+ SOCKET origsocket; /* Original socket value, or PGINVALID_SOCKET
+ * if not a socket */
+ WSAPROTOCOL_INFO wsainfo;
+} InheritableSocket;
+#else
+typedef int InheritableSocket;
+#endif
+
+#ifdef EXEC_BACKEND
+
+/*
+ * Structure contains all global variables passed to exec:ed backends
+ */
+typedef struct
+{
+ char DataDir[MAXPGPATH];
+ int32 MyCancelKey;
+ int MyPMChildSlot;
+#ifndef WIN32
+ unsigned long UsedShmemSegID;
+#else
+ void *ShmemProtectiveRegion;
+ HANDLE UsedShmemSegID;
+#endif
+ void *UsedShmemSegAddr;
+ slock_t *ShmemLock;
+ VariableCache ShmemVariableCache;
+ struct bkend *ShmemBackendArray;
+#ifndef HAVE_SPINLOCKS
+ PGSemaphore *SpinlockSemaArray;
+#endif
+ int NamedLWLockTrancheRequests;
+ NamedLWLockTranche *NamedLWLockTrancheArray;
+ LWLockPadded *MainLWLockArray;
+ slock_t *ProcStructLock;
+ PROC_HDR *ProcGlobal;
+ PGPROC *AuxiliaryProcs;
+ PGPROC *PreparedXactProcs;
+ PMSignalData *PMSignalState;
+ pid_t PostmasterPid;
+ TimestampTz PgStartTime;
+ TimestampTz PgReloadTime;
+ pg_time_t first_syslogger_file_time;
+ bool redirection_done;
+ bool IsBinaryUpgrade;
+ bool query_id_enabled;
+ int max_safe_fds;
+ int MaxBackends;
+#ifdef WIN32
+ HANDLE PostmasterHandle;
+ HANDLE initial_signal_pipe;
+ HANDLE syslogPipe[2];
+#else
+ int postmaster_alive_fds[2];
+ int syslogPipe[2];
+#endif
+ char my_exec_path[MAXPGPATH];
+ char pkglib_path[MAXPGPATH];
+
+ /*
+ * These are only used by backend processes, but it's here because passing
+ * a socket needs some special handling on Windows. 'client_sock' is an
+ * explicit argument to postmaster_child_launch, but is stored in
+ * MyClientSocket in the child process.
+ */
+ ClientSocket client_sock;
+ InheritableSocket inh_sock;
+
+ size_t startup_data_len;
+ /* startup data follows */
+ char startup_data[FLEXIBLE_ARRAY_MEMBER];
+} BackendParameters;
+
+#define SizeOfBackendParameters(startup_data_len) (offsetof(BackendParameters, startup_data) + startup_data_len)
+
+static void read_backend_variables(char *id, char **startup_data, size_t *startup_data_len);
+static void restore_backend_variables(BackendParameters *param);
+
+static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock,
+#ifdef WIN32
+ HANDLE childProcess, pid_t childPid,
+#endif
+ char *startup_data, size_t startup_data_len);
+
+#endif /* EXEC_BACKEND */
+
+static void InitPostmasterChild(bool am_syslogger);
+
+
+#ifdef EXEC_BACKEND
+static pid_t internal_forkexec(const char *main_fn_name, char *startup_data, size_t startup_data_len, ClientSocket *client_sock);
+#endif /* EXEC_BACKEND */
+
+#ifndef WIN32
+static pid_t fork_process(void);
+#endif
+
+/*
+ * Information needed to launch different kinds of child processes.
+ */
+static const struct
+{
+ const char *name;
+ void (*main_fn) (char *startup_data, size_t startup_data_len);
+ bool shmem_attach;
+} entry_kinds[] = {
+ [PMC_BACKEND] = {"backend", BackendMain, true},
+
+ [PMC_AV_LAUNCHER] = {"autovacuum launcher", AutoVacLauncherMain, true},
+ [PMC_AV_WORKER] = {"autovacuum worker", AutoVacWorkerMain, true},
+ [PMC_BGWORKER] = {"bgworker", BackgroundWorkerMain, true},
+ [PMC_SYSLOGGER] = {"syslogger", SysLoggerMain, false},
+
+ [PMC_STARTUP] = {"startup", StartupProcessMain, true},
+ [PMC_BGWRITER] = {"bgwriter", BackgroundWriterMain, true},
+ [PMC_ARCHIVER] = {"archiver", PgArchiverMain, true},
+ [PMC_CHECKPOINTER] = {"checkpointer", CheckpointerMain, true},
+ [PMC_WAL_WRITER] = {"wal_writer", WalWriterMain, true},
+ [PMC_WAL_RECEIVER] = {"wal_receiver", WalReceiverMain, true},
+};
+
+const char *
+PostmasterChildName(PostmasterChildType child_type)
+{
+ Assert(child_type >= 0 && child_type < lengthof(entry_kinds));
+ return entry_kinds[child_type].name;
+}
+
+/*
+ * Start a new postmaster child process.
+ *
+ * The child process will be restored to roughly the same state, whether
+ * EXEC_BACKEND is used or not: it will be attached to shared memory, and fds
+ * and other resources that we've inherited from postmaster that are not
+ * needed in a child process have been closed.
+ *
+ * 'startup_data' is an optional contiguous chunk of data that is passed to
+ * the child process.
+ */
+pid_t
+postmaster_child_launch(PostmasterChildType child_type, char *startup_data, size_t startup_data_len, ClientSocket *client_sock)
+{
+ pid_t pid;
+
+ Assert(child_type >= 0 && child_type < lengthof(entry_kinds));
+ Assert(IsPostmasterEnvironment && !IsUnderPostmaster);
+
+#ifdef EXEC_BACKEND
+ pid = internal_forkexec(entry_kinds[child_type].name, startup_data, startup_data_len, client_sock);
+ /* the child process will arrive in SubPostmasterMain */
+#else /* !EXEC_BACKEND */
+ pid = fork_process();
+ if (pid == 0) /* child */
+ {
+ /* Detangle from postmaster */
+ InitPostmasterChild(child_type == PMC_SYSLOGGER);
+
+ /*
+ * Before blowing away PostmasterContext (in the Main function), save
+ * the startup data.
+ */
+ MemoryContextSwitchTo(TopMemoryContext);
+ if (startup_data != NULL)
+ {
+ char *cp = palloc(startup_data_len);
+
+ memcpy(cp, startup_data, startup_data_len);
+ startup_data = cp;
+ }
+
+ if (client_sock)
+ {
+ MyClientSocket = palloc(sizeof(ClientSocket));
+ memcpy(MyClientSocket, client_sock, sizeof(ClientSocket));
+ }
+
+ entry_kinds[child_type].main_fn(startup_data, startup_data_len);
+ Assert(false);
+ }
+#endif /* EXEC_BACKEND */
+ return pid;
+}
+
+#ifndef WIN32
+/*
+ * Wrapper for fork(). Performs initialization steps that are the same in
+ * !EXEC_BACKEND and EXEC_BACKEND modes.
+ *
+ * Return values are the same as those for fork(): -1 if the fork failed, 0 in
+ * the child process, and the PID of the child in the parent process. Signals
+ * are blocked while forking, so the child must unblock.
+ */
+static pid_t
+fork_process(void)
+{
+ pid_t result;
+ const char *oomfilename;
+ sigset_t save_mask;
+
+#ifdef LINUX_PROFILE
+ struct itimerval prof_itimer;
+#endif
+
+ /*
+ * Flush stdio channels just before fork, to avoid double-output problems.
+ */
+ fflush(NULL);
+
+#ifdef LINUX_PROFILE
+
+ /*
+ * Linux's fork() resets the profiling timer in the child process. If we
+ * want to profile child processes then we need to save and restore the
+ * timer setting. This is a waste of time if not profiling, however, so
+ * only do it if commanded by specific -DLINUX_PROFILE switch.
+ */
+ getitimer(ITIMER_PROF, &prof_itimer);
+#endif
+
+ /*
+ * We start postmaster children with signals blocked. This allows them to
+ * install their own handlers before unblocking, to avoid races where they
+ * might run the postmaster's handler and miss an important control
+ * signal. With more analysis this could potentially be relaxed.
+ */
+ sigprocmask(SIG_SETMASK, &BlockSig, &save_mask);
+ result = fork();
+ if (result == 0)
+ {
+ /* fork succeeded, in child */
+#ifdef LINUX_PROFILE
+ setitimer(ITIMER_PROF, &prof_itimer, NULL);
+#endif
+
+ /*
+ * By default, Linux tends to kill the postmaster in out-of-memory
+ * situations, because it blames the postmaster for the sum of child
+ * process sizes *including shared memory*. (This is unbelievably
+ * stupid, but the kernel hackers seem uninterested in improving it.)
+ * Therefore it's often a good idea to protect the postmaster by
+ * setting its OOM score adjustment negative (which has to be done in
+ * a root-owned startup script). Since the adjustment is inherited by
+ * child processes, this would ordinarily mean that all the
+ * postmaster's children are equally protected against OOM kill, which
+ * is not such a good idea. So we provide this code to allow the
+ * children to change their OOM score adjustments again. Both the
+ * file name to write to and the value to write are controlled by
+ * environment variables, which can be set by the same startup script
+ * that did the original adjustment.
+ */
+ oomfilename = getenv("PG_OOM_ADJUST_FILE");
+
+ if (oomfilename != NULL)
+ {
+ /*
+ * Use open() not stdio, to ensure we control the open flags. Some
+ * Linux security environments reject anything but O_WRONLY.
+ */
+ int fd = open(oomfilename, O_WRONLY, 0);
+
+ /* We ignore all errors */
+ if (fd >= 0)
+ {
+ const char *oomvalue = getenv("PG_OOM_ADJUST_VALUE");
+ int rc;
+
+ if (oomvalue == NULL) /* supply a useful default */
+ oomvalue = "0";
+
+ rc = write(fd, oomvalue, strlen(oomvalue));
+ (void) rc;
+ close(fd);
+ }
+ }
+
+ /* do post-fork initialization for random number generation */
+ pg_strong_random_init();
+ }
+ else
+ {
+ /* in parent, restore signal mask */
+ sigprocmask(SIG_SETMASK, &save_mask, NULL);
+ }
+
+ return result;
+}
+
+#endif /* !WIN32 */
+
+#ifdef EXEC_BACKEND
+#ifndef WIN32
+
+/*
+ * internal_forkexec non-win32 implementation
+ *
+ * - writes out backend variables to the parameter file
+ * - fork():s, and then exec():s the child process
+ */
+static pid_t
+internal_forkexec(const char *entry_name, char *startup_data, size_t startup_data_len, ClientSocket *client_sock)
+{
+ static unsigned long tmpBackendFileNum = 0;
+ pid_t pid;
+ char tmpfilename[MAXPGPATH];
+ size_t paramsz;
+ BackendParameters *param;
+ FILE *fp;
+ char *argv[4];
+ char forkav[MAXPGPATH];
+
+ paramsz = SizeOfBackendParameters(startup_data_len);
+ param = palloc(paramsz);
+ if (!save_backend_variables(param, client_sock, startup_data, startup_data_len))
+ {
+ pfree(param);
+ return -1; /* log made by save_backend_variables */
+ }
+
+ /* Calculate name for temp file */
+ snprintf(tmpfilename, MAXPGPATH, "%s/%s.backend_var.%d.%lu",
+ PG_TEMP_FILES_DIR, PG_TEMP_FILE_PREFIX,
+ MyProcPid, ++tmpBackendFileNum);
+
+ /* Open file */
+ fp = AllocateFile(tmpfilename, PG_BINARY_W);
+ if (!fp)
+ {
+ /*
+ * As in OpenTemporaryFileInTablespace, try to make the temp-file
+ * directory, ignoring errors.
+ */
+ (void) MakePGDirectory(PG_TEMP_FILES_DIR);
+
+ fp = AllocateFile(tmpfilename, PG_BINARY_W);
+ if (!fp)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ tmpfilename)));
+ return -1;
+ }
+ }
+
+ if (fwrite(param, paramsz, 1, fp) != 1)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not write to file \"%s\": %m", tmpfilename)));
+ FreeFile(fp);
+ return -1;
+ }
+
+ /* Release file */
+ if (FreeFile(fp))
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not write to file \"%s\": %m", tmpfilename)));
+ return -1;
+ }
+
+ /* set up argv properly */
+ argv[0] = "postgres";
+ snprintf(forkav, MAXPGPATH, "--forkchild=%s", entry_name);
+ argv[1] = forkav;
+ /* Insert temp file name after --forkchild argument */
+ argv[2] = tmpfilename;
+ argv[3] = NULL;
+
+ /* Fire off execv in child */
+ if ((pid = fork_process()) == 0)
+ {
+ if (execv(postgres_exec_path, argv) < 0)
+ {
+ ereport(LOG,
+ (errmsg("could not execute server process \"%s\": %m",
+ postgres_exec_path)));
+ /* We're already in the child process here, can't return */
+ exit(1);
+ }
+ }
+
+ return pid; /* Parent returns pid, or -1 on fork failure */
+}
+#else /* WIN32 */
+
+/*
+ * internal_forkexec win32 implementation
+ *
+ * - starts backend using CreateProcess(), in suspended state
+ * - writes out backend variables to the parameter file
+ * - during this, duplicates handles and sockets required for
+ * inheritance into the new process
+ * - resumes execution of the new process once the backend parameter
+ * file is complete.
+ */
+static pid_t
+internal_forkexec(const char *entry_name, char *startup_data, size_t startup_data_len, ClientSocket *client_sock)
+{
+ int retry_count = 0;
+ STARTUPINFO si;
+ PROCESS_INFORMATION pi;
+ char cmdLine[MAXPGPATH * 2];
+ HANDLE paramHandle;
+ BackendParameters *param;
+ SECURITY_ATTRIBUTES sa;
+ size_t paramsz;
+ char paramHandleStr[32];
+ int l;
+
+ paramsz = SizeOfBackendParameters(startup_data_len);
+
+ /* Resume here if we need to retry */
+retry:
+
+ /* Set up shared memory for parameter passing */
+ ZeroMemory(&sa, sizeof(sa));
+ sa.nLength = sizeof(sa);
+ sa.bInheritHandle = TRUE;
+ paramHandle = CreateFileMapping(INVALID_HANDLE_VALUE,
+ &sa,
+ PAGE_READWRITE,
+ 0,
+ paramsz,
+ NULL);
+ if (paramHandle == INVALID_HANDLE_VALUE)
+ {
+ ereport(LOG,
+ (errmsg("could not create backend parameter file mapping: error code %lu",
+ GetLastError())));
+ return -1;
+ }
+ param = MapViewOfFile(paramHandle, FILE_MAP_WRITE, 0, 0, paramsz);
+ if (!param)
+ {
+ ereport(LOG,
+ (errmsg("could not map backend parameter memory: error code %lu",
+ GetLastError())));
+ CloseHandle(paramHandle);
+ return -1;
+ }
+
+ /* Format the cmd line */
+#ifdef _WIN64
+ sprintf(paramHandleStr, "%llu", (LONG_PTR) paramHandle);
+#else
+ sprintf(paramHandleStr, "%lu", (DWORD) paramHandle);
+#endif
+ l = snprintf(cmdLine, sizeof(cmdLine) - 1, "\"%s\" --forkchild=\"%s\" %s",
+ postgres_exec_path, entry_name, paramHandleStr);
+ if (l >= sizeof(cmdLine))
+ {
+ ereport(LOG,
+ (errmsg("subprocess command line too long")));
+ UnmapViewOfFile(param);
+ CloseHandle(paramHandle);
+ return -1;
+ }
+
+ memset(&pi, 0, sizeof(pi));
+ memset(&si, 0, sizeof(si));
+ si.cb = sizeof(si);
+
+ /*
+ * Create the subprocess in a suspended state. This will be resumed later,
+ * once we have written out the parameter file.
+ */
+ if (!CreateProcess(NULL, cmdLine, NULL, NULL, TRUE, CREATE_SUSPENDED,
+ NULL, NULL, &si, &pi))
+ {
+ ereport(LOG,
+ (errmsg("CreateProcess() call failed: %m (error code %lu)",
+ GetLastError())));
+ UnmapViewOfFile(param);
+ CloseHandle(paramHandle);
+ return -1;
+ }
+
+ if (!save_backend_variables(param, client_sock, pi.hProcess, pi.dwProcessId, startup_data, startup_data_len))
+ {
+ /*
+ * log made by save_backend_variables, but we have to clean up the
+ * mess with the half-started process
+ */
+ if (!TerminateProcess(pi.hProcess, 255))
+ ereport(LOG,
+ (errmsg_internal("could not terminate unstarted process: error code %lu",
+ GetLastError())));
+ CloseHandle(pi.hProcess);
+ CloseHandle(pi.hThread);
+ UnmapViewOfFile(param);
+ CloseHandle(paramHandle);
+ return -1; /* log made by save_backend_variables */
+ }
+
+ /* Drop the parameter shared memory that is now inherited to the backend */
+ if (!UnmapViewOfFile(param))
+ ereport(LOG,
+ (errmsg("could not unmap view of backend parameter file: error code %lu",
+ GetLastError())));
+ if (!CloseHandle(paramHandle))
+ ereport(LOG,
+ (errmsg("could not close handle to backend parameter file: error code %lu",
+ GetLastError())));
+
+ /*
+ * Reserve the memory region used by our main shared memory segment before
+ * we resume the child process. Normally this should succeed, but if ASLR
+ * is active then it might sometimes fail due to the stack or heap having
+ * gotten mapped into that range. In that case, just terminate the
+ * process and retry.
+ */
+ if (!pgwin32_ReserveSharedMemoryRegion(pi.hProcess))
+ {
+ /* pgwin32_ReserveSharedMemoryRegion already made a log entry */
+ if (!TerminateProcess(pi.hProcess, 255))
+ ereport(LOG,
+ (errmsg_internal("could not terminate process that failed to reserve memory: error code %lu",
+ GetLastError())));
+ CloseHandle(pi.hProcess);
+ CloseHandle(pi.hThread);
+ if (++retry_count < 100)
+ goto retry;
+ ereport(LOG,
+ (errmsg("giving up after too many tries to reserve shared memory"),
+ errhint("This might be caused by ASLR or antivirus software.")));
+ return -1;
+ }
+
+ /*
+ * Now that the backend variables are written out, we start the child
+ * thread so it can start initializing while we set up the rest of the
+ * parent state.
+ */
+ if (ResumeThread(pi.hThread) == -1)
+ {
+ if (!TerminateProcess(pi.hProcess, 255))
+ {
+ ereport(LOG,
+ (errmsg_internal("could not terminate unstartable process: error code %lu",
+ GetLastError())));
+ CloseHandle(pi.hProcess);
+ CloseHandle(pi.hThread);
+ return -1;
+ }
+ CloseHandle(pi.hProcess);
+ CloseHandle(pi.hThread);
+ ereport(LOG,
+ (errmsg_internal("could not resume thread of unstarted process: error code %lu",
+ GetLastError())));
+ return -1;
+ }
+
+ pgwin32_register_deadchild_callback(pi.hProcess, pi.dwProcessId);
+
+ /* Don't close pi.hProcess here - waitpid() needs access to it */
+ CloseHandle(pi.hThread);
+
+ return pi.dwProcessId;
+}
+#endif /* WIN32 */
+
+/*
+ * SubPostmasterMain -- Get the fork/exec'd process into a state equivalent
+ * to what it would be if we'd simply forked on Unix, and then
+ * dispatch to the appropriate place.
+ *
+ * The first two command line arguments are expected to be "--forkchild=<name>",
+ * where <name> indicates which postmaster child we are to become, and
+ * the name of a variables file that we can read to load data that would
+ * have been inherited by fork() on Unix.
+ */
+void
+SubPostmasterMain(int argc, char *argv[])
+{
+ PostmasterChildType child_type;
+ char *startup_data;
+ size_t startup_data_len;
+ char *entry_name;
+ bool found = false;
+
+ /* In EXEC_BACKEND case we will not have inherited these settings */
+ IsPostmasterEnvironment = true;
+ whereToSendOutput = DestNone;
+
+ /* Setup essential subsystems (to ensure elog() behaves sanely) */
+ InitializeGUCOptions();
+
+ /* Check we got appropriate args */
+ if (argc != 3)
+ elog(FATAL, "invalid subpostmaster invocation");
+
+ if (strncmp(argv[1], "--forkchild=", 12) != 0)
+ elog(FATAL, "invalid subpostmaster invocation (--forkchild argument missing)");
+ entry_name = argv[1] + 12;
+ found = false;
+ for (int idx = 0; idx < lengthof(entry_kinds); idx++)
+ {
+ if (strcmp(entry_kinds[idx].name, entry_name) == 0)
+ {
+ child_type = idx;
+ found = true;
+ break;
+ }
+ }
+ if (!found)
+ elog(ERROR, "unknown child kind %s", entry_name);
+
+ /* Read in the variables file */
+ read_backend_variables(argv[2], &startup_data, &startup_data_len);
+
+ /* Setup as postmaster child */
+ InitPostmasterChild(child_type == PMC_SYSLOGGER);
+
+ /*
+ * If appropriate, physically re-attach to shared memory segment. We want
+ * to do this before going any further to ensure that we can attach at the
+ * same address the postmaster used. On the other hand, if we choose not
+ * to re-attach, we may have other cleanup to do.
+ *
+ * If testing EXEC_BACKEND on Linux, you should run this as root before
+ * starting the postmaster:
+ *
+ * sysctl -w kernel.randomize_va_space=0
+ *
+ * This prevents using randomized stack and code addresses that cause the
+ * child process's memory map to be different from the parent's, making it
+ * sometimes impossible to attach to shared memory at the desired address.
+ * Return the setting to its old value (usually '1' or '2') when finished.
+ */
+ if (entry_kinds[child_type].shmem_attach)
+ PGSharedMemoryReAttach();
+ else
+ PGSharedMemoryNoReAttach();
+
+ /* Read in remaining GUC variables */
+ read_nondefault_variables();
+
+ /*
+ * Check that the data directory looks valid, which will also check the
+ * privileges on the data directory and update our umask and file/group
+ * variables for creating files later. Note: this should really be done
+ * before we create any files or directories.
+ */
+ checkDataDir();
+
+ /*
+ * (re-)read control file, as it contains config. The postmaster will
+ * already have read this, but this process doesn't know about that.
+ */
+ LocalProcessControlFile(false);
+
+ /*
+ * Reload any libraries that were preloaded by the postmaster. Since we
+ * exec'd this process, those libraries didn't come along with us; but we
+ * should load them into all child processes to be consistent with the
+ * non-EXEC_BACKEND behavior.
+ */
+ process_shared_preload_libraries();
+
+ /* Restore basic shared memory pointers */
+ if (UsedShmemSegAddr != NULL)
+ InitShmemAccess(UsedShmemSegAddr);
+
+ /* Run backend or appropriate child */
+ entry_kinds[child_type].main_fn(startup_data, startup_data_len);
+
+ abort(); /* shouldn't get here */
+}
+#endif /* EXEC_BACKEND */
+
+/* ----------------------------------------------------------------
+ * common process startup code
+ * ----------------------------------------------------------------
+ */
+
+/*
+ * Initialize the basic environment for a postmaster child
+ *
+ * Should be called as early as possible after the child's startup. However,
+ * on EXEC_BACKEND builds it does need to be after read_backend_variables().
+ */
+static void
+InitPostmasterChild(bool am_syslogger)
+{
+ /* Close the postmaster's sockets (as soon as we know them) */
+ ClosePostmasterPorts(am_syslogger);
+
+ IsUnderPostmaster = true; /* we are a postmaster subprocess now */
+
+ /*
+ * Start our win32 signal implementation. This has to be done after we
+ * read the backend variables, because we need to pick up the signal pipe
+ * from the parent process.
+ */
+#ifdef WIN32
+ pgwin32_signal_initialize();
+#endif
+
+ /*
+ * Set reference point for stack-depth checking. This might seem
+ * redundant in !EXEC_BACKEND builds; but it's not because the postmaster
+ * launches its children from signal handlers, so we might be running on
+ * an alternative stack. XXX still true?
+ */
+ (void) set_stack_base();
+
+ InitProcessGlobals();
+
+ /*
+ * make sure stderr is in binary mode before anything can possibly be
+ * written to it, in case it's actually the syslogger pipe, so the pipe
+ * chunking protocol isn't disturbed. Non-logpipe data gets translated on
+ * redirection (e.g. via pg_ctl -l) anyway.
+ */
+#ifdef WIN32
+ _setmode(fileno(stderr), _O_BINARY);
+#endif
+
+ /* We don't want the postmaster's proc_exit() handlers */
+ on_exit_reset();
+
+ /* In EXEC_BACKEND case we will not have inherited BlockSig etc values */
+#ifdef EXEC_BACKEND
+ pqinitmask();
+#endif
+
+ /* Initialize process-local latch support */
+ InitializeLatchSupport();
+ InitProcessLocalLatch();
+ InitializeLatchWaitSet();
+
+ /*
+ * If possible, make this process a group leader, so that the postmaster
+ * can signal any child processes too. Not all processes will have
+ * children, but for consistency we make all postmaster child processes do
+ * this.
+ */
+#ifdef HAVE_SETSID
+ if (setsid() < 0)
+ elog(FATAL, "setsid() failed: %m");
+#endif
+
+ /*
+ * Every postmaster child process is expected to respond promptly to
+ * SIGQUIT at all times. Therefore we centrally remove SIGQUIT from
+ * BlockSig and install a suitable signal handler. (Client-facing
+ * processes may choose to replace this default choice of handler with
+ * quickdie().) All other blockable signals remain blocked for now.
+ */
+ pqsignal(SIGQUIT, SignalHandlerForCrashExit);
+
+ sigdelset(&BlockSig, SIGQUIT);
+ sigprocmask(SIG_SETMASK, &BlockSig, NULL);
+
+ /* Request a signal if the postmaster dies, if possible. */
+ PostmasterDeathSignalInit();
+
+ /* Don't give the pipe to subprograms that we execute. */
+#ifndef WIN32
+ if (fcntl(postmaster_alive_fds[POSTMASTER_FD_WATCH], F_SETFD, FD_CLOEXEC) < 0)
+ ereport(FATAL,
+ (errcode_for_socket_access(),
+ errmsg_internal("could not set postmaster death monitoring pipe to FD_CLOEXEC mode: %m")));
+#endif
+}
+
+/*
+ * Initialize the basic environment for a standalone process.
+ *
+ * argv0 has to be suitable to find the program's executable.
+ */
+void
+InitStandaloneProcess(const char *argv0)
+{
+ Assert(!IsPostmasterEnvironment);
+
+ MyBackendType = B_STANDALONE_BACKEND;
+
+ /*
+ * Start our win32 signal implementation
+ */
+#ifdef WIN32
+ pgwin32_signal_initialize();
+#endif
+
+ InitProcessGlobals();
+
+ /* Initialize process-local latch support */
+ InitializeLatchSupport();
+ InitProcessLocalLatch();
+ InitializeLatchWaitSet();
+
+ /*
+ * For consistency with InitPostmasterChild, initialize signal mask here.
+ * But we don't unblock SIGQUIT or provide a default handler for it.
+ */
+ pqinitmask();
+ sigprocmask(SIG_SETMASK, &BlockSig, NULL);
+
+ /* Compute paths, no postmaster to inherit from */
+ if (my_exec_path[0] == '\0')
+ {
+ if (find_my_exec(argv0, my_exec_path) < 0)
+ elog(FATAL, "%s: could not locate my own executable path",
+ argv0);
+ }
+
+ if (pkglib_path[0] == '\0')
+ get_pkglib_path(my_exec_path, pkglib_path);
+}
+
+
+#ifdef EXEC_BACKEND
+
+/*
+ * The following need to be available to the save/restore_backend_variables
+ * functions. They are marked NON_EXEC_STATIC in their home modules.
+ */
+extern slock_t *ShmemLock;
+extern slock_t *ProcStructLock;
+extern PGPROC *AuxiliaryProcs;
+extern PMSignalData *PMSignalState;
+extern pg_time_t first_syslogger_file_time;
+extern struct bkend *ShmemBackendArray;
+extern bool redirection_done;
+
+#ifndef WIN32
+#define write_inheritable_socket(dest, src, childpid) ((*(dest) = (src)), true)
+#define read_inheritable_socket(dest, src) (*(dest) = *(src))
+#else
+static bool write_duplicated_handle(HANDLE *dest, HANDLE src, HANDLE child);
+static bool write_inheritable_socket(InheritableSocket *dest, SOCKET src,
+ pid_t childPid);
+static void read_inheritable_socket(SOCKET *dest, InheritableSocket *src);
+#endif
+
+
+/* Save critical backend variables into the BackendParameters struct */
+static bool
+save_backend_variables(BackendParameters *param, ClientSocket *client_sock,
+#ifdef WIN32
+ HANDLE childProcess, pid_t childPid,
+#endif
+ char *startup_data, size_t startup_data_len)
+{
+ if (client_sock)
+ memcpy(¶m->client_sock, client_sock, sizeof(ClientSocket));
+ else
+ memset(¶m->client_sock, 0, sizeof(ClientSocket));
+ if (!write_inheritable_socket(¶m->inh_sock,
+ client_sock ? client_sock->sock : PGINVALID_SOCKET,
+ childPid))
+ return false;
+
+ strlcpy(param->DataDir, DataDir, MAXPGPATH);
+
+ param->MyCancelKey = MyCancelKey;
+ param->MyPMChildSlot = MyPMChildSlot;
+
+#ifdef WIN32
+ param->ShmemProtectiveRegion = ShmemProtectiveRegion;
+#endif
+ param->UsedShmemSegID = UsedShmemSegID;
+ param->UsedShmemSegAddr = UsedShmemSegAddr;
+
+ param->ShmemLock = ShmemLock;
+ param->ShmemVariableCache = ShmemVariableCache;
+ param->ShmemBackendArray = ShmemBackendArray;
+
+#ifndef HAVE_SPINLOCKS
+ param->SpinlockSemaArray = SpinlockSemaArray;
+#endif
+ param->NamedLWLockTrancheRequests = NamedLWLockTrancheRequests;
+ param->NamedLWLockTrancheArray = NamedLWLockTrancheArray;
+ param->MainLWLockArray = MainLWLockArray;
+ param->ProcStructLock = ProcStructLock;
+ param->ProcGlobal = ProcGlobal;
+ param->AuxiliaryProcs = AuxiliaryProcs;
+ param->PreparedXactProcs = PreparedXactProcs;
+ param->PMSignalState = PMSignalState;
+
+ param->PostmasterPid = PostmasterPid;
+ param->PgStartTime = PgStartTime;
+ param->PgReloadTime = PgReloadTime;
+ param->first_syslogger_file_time = first_syslogger_file_time;
+
+ param->redirection_done = redirection_done;
+ param->IsBinaryUpgrade = IsBinaryUpgrade;
+ param->query_id_enabled = query_id_enabled;
+ param->max_safe_fds = max_safe_fds;
+
+ param->MaxBackends = MaxBackends;
+
+#ifdef WIN32
+ param->PostmasterHandle = PostmasterHandle;
+ if (!write_duplicated_handle(¶m->initial_signal_pipe,
+ pgwin32_create_signal_listener(childPid),
+ childProcess))
+ return false;
+#else
+ memcpy(¶m->postmaster_alive_fds, &postmaster_alive_fds,
+ sizeof(postmaster_alive_fds));
+#endif
+
+ memcpy(¶m->syslogPipe, &syslogPipe, sizeof(syslogPipe));
+
+ strlcpy(param->my_exec_path, my_exec_path, MAXPGPATH);
+
+ strlcpy(param->pkglib_path, pkglib_path, MAXPGPATH);
+
+ param->startup_data_len = startup_data_len;
+ memcpy(param->startup_data, startup_data, startup_data_len);
+
+ return true;
+}
+
+
+#ifdef WIN32
+/*
+ * Duplicate a handle for usage in a child process, and write the child
+ * process instance of the handle to the parameter file.
+ */
+static bool
+write_duplicated_handle(HANDLE *dest, HANDLE src, HANDLE childProcess)
+{
+ HANDLE hChild = INVALID_HANDLE_VALUE;
+
+ if (!DuplicateHandle(GetCurrentProcess(),
+ src,
+ childProcess,
+ &hChild,
+ 0,
+ TRUE,
+ DUPLICATE_CLOSE_SOURCE | DUPLICATE_SAME_ACCESS))
+ {
+ ereport(LOG,
+ (errmsg_internal("could not duplicate handle to be written to backend parameter file: error code %lu",
+ GetLastError())));
+ return false;
+ }
+
+ *dest = hChild;
+ return true;
+}
+
+/*
+ * Duplicate a socket for usage in a child process, and write the resulting
+ * structure to the parameter file.
+ * This is required because a number of LSPs (Layered Service Providers) very
+ * common on Windows (antivirus, firewalls, download managers etc) break
+ * straight socket inheritance.
+ */
+static bool
+write_inheritable_socket(InheritableSocket *dest, SOCKET src, pid_t childpid)
+{
+ dest->origsocket = src;
+ if (src != 0 && src != PGINVALID_SOCKET)
+ {
+ /* Actual socket */
+ if (WSADuplicateSocket(src, childpid, &dest->wsainfo) != 0)
+ {
+ ereport(LOG,
+ (errmsg("could not duplicate socket %d for use in backend: error code %d",
+ (int) src, WSAGetLastError())));
+ return false;
+ }
+ }
+ return true;
+}
+
+/*
+ * Read a duplicate socket structure back, and get the socket descriptor.
+ */
+static void
+read_inheritable_socket(SOCKET *dest, InheritableSocket *src)
+{
+ SOCKET s;
+
+ if (src->origsocket == PGINVALID_SOCKET || src->origsocket == 0)
+ {
+ /* Not a real socket! */
+ *dest = src->origsocket;
+ }
+ else
+ {
+ /* Actual socket, so create from structure */
+ s = WSASocket(FROM_PROTOCOL_INFO,
+ FROM_PROTOCOL_INFO,
+ FROM_PROTOCOL_INFO,
+ &src->wsainfo,
+ 0,
+ 0);
+ if (s == INVALID_SOCKET)
+ {
+ write_stderr("could not create inherited socket: error code %d\n",
+ WSAGetLastError());
+ exit(1);
+ }
+ *dest = s;
+
+ /*
+ * To make sure we don't get two references to the same socket, close
+ * the original one. (This would happen when inheritance actually
+ * works..
+ */
+ closesocket(src->origsocket);
+ }
+}
+#endif
+
+static void
+read_backend_variables(char *id, char **startup_data, size_t *startup_data_len)
+{
+ BackendParameters param;
+
+#ifndef WIN32
+ /* Non-win32 implementation reads from file */
+ FILE *fp;
+
+ /* Open file */
+ fp = AllocateFile(id, PG_BINARY_R);
+ if (!fp)
+ {
+ write_stderr("could not open backend variables file \"%s\": %s\n",
+ id, strerror(errno));
+ exit(1);
+ }
+
+ if (fread(¶m, sizeof(param), 1, fp) != 1)
+ {
+ write_stderr("could not read from backend variables file \"%s\": %s\n",
+ id, strerror(errno));
+ exit(1);
+ }
+
+ /* read startup data */
+ *startup_data_len = param.startup_data_len;
+ if (param.startup_data_len > 0)
+ {
+ *startup_data = palloc(*startup_data_len);
+ if (fread(*startup_data, *startup_data_len, 1, fp) != 1)
+ {
+ write_stderr("could not read startup data from backend variables file \"%s\": %s\n",
+ id, strerror(errno));
+ exit(1);
+ }
+ }
+ else
+ *startup_data = NULL;
+
+ /* Release file */
+ FreeFile(fp);
+ if (unlink(id) != 0)
+ {
+ write_stderr("could not remove file \"%s\": %s\n",
+ id, strerror(errno));
+ exit(1);
+ }
+#else
+ /* Win32 version uses mapped file */
+ HANDLE paramHandle;
+ BackendParameters *filep;
+
+#ifdef _WIN64
+ paramHandle = (HANDLE) _atoi64(id);
+#else
+ paramHandle = (HANDLE) atol(id);
+#endif
+ filep = MapViewOfFile(paramHandle, FILE_MAP_READ, 0, 0, 0);
+ if (!filep)
+ {
+ write_stderr("could not map view of backend variables: error code %lu\n",
+ GetLastError());
+ exit(1);
+ }
+
+ memcpy(¶m, filep, sizeof(BackendParameters));
+
+ /* read startup data */
+ *startup_data_len = param.startup_data_len;
+ if (param.startup_data_len > 0)
+ {
+ *startup_data = palloc(param.startup_data_len);
+ memcpy(*startup_data, filep->startup_data, param.startup_data_len);
+ }
+ else
+ *startup_data = NULL;
+
+ if (!UnmapViewOfFile(filep))
+ {
+ write_stderr("could not unmap view of backend variables: error code %lu\n",
+ GetLastError());
+ exit(1);
+ }
+
+ if (!CloseHandle(paramHandle))
+ {
+ write_stderr("could not close handle to backend parameter variables: error code %lu\n",
+ GetLastError());
+ exit(1);
+ }
+#endif
+
+ restore_backend_variables(¶m);
+}
+
+/* Restore critical backend variables from the BackendParameters struct */
+static void
+restore_backend_variables(BackendParameters *param)
+{
+ if (param->client_sock.sock != PGINVALID_SOCKET)
+ {
+ MyClientSocket = MemoryContextAlloc(TopMemoryContext, sizeof(ClientSocket));
+ memcpy(MyClientSocket, ¶m->client_sock, sizeof(ClientSocket));
+ read_inheritable_socket(&MyClientSocket->sock, ¶m->inh_sock);
+ }
+
+ SetDataDir(param->DataDir);
+
+ MyCancelKey = param->MyCancelKey;
+ MyPMChildSlot = param->MyPMChildSlot;
+
+#ifdef WIN32
+ ShmemProtectiveRegion = param->ShmemProtectiveRegion;
+#endif
+ UsedShmemSegID = param->UsedShmemSegID;
+ UsedShmemSegAddr = param->UsedShmemSegAddr;
+
+ ShmemLock = param->ShmemLock;
+ ShmemVariableCache = param->ShmemVariableCache;
+ ShmemBackendArray = param->ShmemBackendArray;
+
+#ifndef HAVE_SPINLOCKS
+ SpinlockSemaArray = param->SpinlockSemaArray;
+#endif
+ NamedLWLockTrancheRequests = param->NamedLWLockTrancheRequests;
+ NamedLWLockTrancheArray = param->NamedLWLockTrancheArray;
+ MainLWLockArray = param->MainLWLockArray;
+ ProcStructLock = param->ProcStructLock;
+ ProcGlobal = param->ProcGlobal;
+ AuxiliaryProcs = param->AuxiliaryProcs;
+ PreparedXactProcs = param->PreparedXactProcs;
+ PMSignalState = param->PMSignalState;
+
+ PostmasterPid = param->PostmasterPid;
+ PgStartTime = param->PgStartTime;
+ PgReloadTime = param->PgReloadTime;
+ first_syslogger_file_time = param->first_syslogger_file_time;
+
+ redirection_done = param->redirection_done;
+ IsBinaryUpgrade = param->IsBinaryUpgrade;
+ query_id_enabled = param->query_id_enabled;
+ max_safe_fds = param->max_safe_fds;
+
+ MaxBackends = param->MaxBackends;
+
+#ifdef WIN32
+ PostmasterHandle = param->PostmasterHandle;
+ pgwin32_initial_signal_pipe = param->initial_signal_pipe;
+#else
+ memcpy(&postmaster_alive_fds, ¶m->postmaster_alive_fds,
+ sizeof(postmaster_alive_fds));
+#endif
+
+ memcpy(&syslogPipe, ¶m->syslogPipe, sizeof(syslogPipe));
+
+ strlcpy(my_exec_path, param->my_exec_path, MAXPGPATH);
+
+ strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+
+ /*
+ * We need to restore fd.c's counts of externally-opened FDs; to avoid
+ * confusion, be sure to do this after restoring max_safe_fds. (Note:
+ * BackendInitialize will handle this for client_sock->sock.)
+ */
+#ifndef WIN32
+ if (postmaster_alive_fds[0] >= 0)
+ ReserveExternalFD();
+ if (postmaster_alive_fds[1] >= 0)
+ ReserveExternalFD();
+#endif
+}
+
+#endif /* EXEC_BACKEND */
diff --git a/src/backend/postmaster/meson.build b/src/backend/postmaster/meson.build
index cda921fd10b..89ff11beb0a 100644
--- a/src/backend/postmaster/meson.build
+++ b/src/backend/postmaster/meson.build
@@ -6,8 +6,8 @@ backend_sources += files(
'bgworker.c',
'bgwriter.c',
'checkpointer.c',
- 'fork_process.c',
'interrupt.c',
+ 'launch_backend.c',
'pgarch.c',
'postmaster.c',
'startup.c',
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 46af3495644..b82d66fab52 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -36,6 +36,7 @@
#include "lib/binaryheap.h"
#include "libpq/pqsignal.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/interrupt.h"
#include "postmaster/pgarch.h"
#include "storage/fd.h"
@@ -211,8 +212,14 @@ PgArchCanRestart(void)
/* Main entry point for archiver process */
void
-PgArchiverMain(void)
+PgArchiverMain(char *startup_data, size_t startup_data_len)
{
+ Assert(startup_data_len == 0);
+
+ MyAuxProcType = ArchiverProcess;
+ MyBackendType = B_ARCHIVER;
+ AuxiliaryProcessInit();
+
/*
* Ignore all signals usually bound to some action in the postmaster,
* except for SIGHUP, SIGTERM, SIGUSR1, SIGUSR2, and SIGQUIT.
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 550136009a8..60ea739f7aa 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -2,9 +2,9 @@
*
* postmaster.c
* This program acts as a clearing house for requests to the
- * POSTGRES system. Frontend programs send a startup message
- * to the Postmaster and the postmaster uses the info in the
- * message to setup a backend process.
+ * POSTGRES system. Frontend programs connect to the Postmaster,
+ * and postmaster forks a new backend process to handle the
+ * connection.
*
* The postmaster also manages system-wide operations such as
* startup and shutdown. The postmaster itself doesn't do those
@@ -110,7 +110,6 @@
#include "postmaster/autovacuum.h"
#include "postmaster/auxprocess.h"
#include "postmaster/bgworker_internals.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
@@ -132,10 +131,6 @@
#include "utils/timestamp.h"
#include "utils/varlena.h"
-#ifdef EXEC_BACKEND
-#include "storage/spin.h"
-#endif
-
/*
* Possible types of a backend. Beyond being the possible bkend_type values in
@@ -188,7 +183,7 @@ typedef struct bkend
static dlist_head BackendList = DLIST_STATIC_INIT(BackendList);
#ifdef EXEC_BACKEND
-static Backend *ShmemBackendArray;
+Backend *ShmemBackendArray;
#endif
BackgroundWorker *MyBgworkerEntry = NULL;
@@ -413,7 +408,7 @@ static void StartupPacketTimeoutHandler(void);
static void CleanupBackend(int pid, int exitstatus);
static bool CleanupBackgroundWorker(int pid, int exitstatus);
static void HandleChildCrash(int pid, int exitstatus, const char *procname);
-static void LogChildExit(int lev, const char *procname,
+static void LogChildExit(int lev, const char *procnamBackendInitializee,
int pid, int exitstatus);
static void PostmasterStateMachine(void);
@@ -428,7 +423,6 @@ typedef enum CAC_state
} CAC_state;
static void BackendInitialize(ClientSocket *client_sock, CAC_state cac);
-static void BackendRun(void) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
static int BackendStartup(ClientSocket *port);
@@ -449,7 +443,7 @@ static int CountChildren(int target);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
-static pid_t StartChildProcess(AuxProcType type);
+static pid_t StartChildProcess(PostmasterChildType type);
static void StartAutovacuumWorker(void);
static void MaybeStartWalReceiver(void);
static void InitPostmasterDeathWatchHandle(void);
@@ -484,95 +478,19 @@ typedef struct
} win32_deadchild_waitinfo;
#endif /* WIN32 */
-static pid_t backend_forkexec(ClientSocket *client_sock, CAC_state cac);
-static pid_t internal_forkexec(int argc, char *argv[], ClientSocket *client_sock);
-
-/* Type for a socket that can be inherited to a client process */
-#ifdef WIN32
-typedef struct
-{
- SOCKET origsocket; /* Original socket value, or PGINVALID_SOCKET
- * if not a socket */
- WSAPROTOCOL_INFO wsainfo;
-} InheritableSocket;
-#else
-typedef int InheritableSocket;
-#endif
-
-/*
- * Structure contains all variables passed to exec:ed backends
- */
-typedef struct
-{
- ClientSocket client_sock;
- InheritableSocket serialized_sock;
- char DataDir[MAXPGPATH];
- int32 MyCancelKey;
- int MyPMChildSlot;
-#ifndef WIN32
- unsigned long UsedShmemSegID;
-#else
- void *ShmemProtectiveRegion;
- HANDLE UsedShmemSegID;
-#endif
- void *UsedShmemSegAddr;
- slock_t *ShmemLock;
- VariableCache ShmemVariableCache;
- Backend *ShmemBackendArray;
-#ifndef HAVE_SPINLOCKS
- PGSemaphore *SpinlockSemaArray;
-#endif
- int NamedLWLockTrancheRequests;
- NamedLWLockTranche *NamedLWLockTrancheArray;
- LWLockPadded *MainLWLockArray;
- slock_t *ProcStructLock;
- PROC_HDR *ProcGlobal;
- PGPROC *AuxiliaryProcs;
- PGPROC *PreparedXactProcs;
- PMSignalData *PMSignalState;
- pid_t PostmasterPid;
- TimestampTz PgStartTime;
- TimestampTz PgReloadTime;
- pg_time_t first_syslogger_file_time;
- bool redirection_done;
- bool IsBinaryUpgrade;
- bool query_id_enabled;
- int max_safe_fds;
- int MaxBackends;
-#ifdef WIN32
- HANDLE PostmasterHandle;
- HANDLE initial_signal_pipe;
- HANDLE syslogPipe[2];
-#else
- int postmaster_alive_fds[2];
- int syslogPipe[2];
-#endif
- char my_exec_path[MAXPGPATH];
- char pkglib_path[MAXPGPATH];
-
- BackgroundWorker MyBgworkerEntry;
-} BackendParameters;
-
-static void read_backend_variables(char *id, ClientSocket *client_sock);
-static void restore_backend_variables(BackendParameters *param, ClientSocket *client_sock);
-
-#ifndef WIN32
-static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock);
-#else
-static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock,
- HANDLE childProcess, pid_t childPid);
-#endif
-
static void ShmemBackendArrayAdd(Backend *bn);
static void ShmemBackendArrayRemove(Backend *bn);
#endif /* EXEC_BACKEND */
-#define StartupDataBase() StartChildProcess(StartupProcess)
-#define StartArchiver() StartChildProcess(ArchiverProcess)
-#define StartBackgroundWriter() StartChildProcess(BgWriterProcess)
-#define StartCheckpointer() StartChildProcess(CheckpointerProcess)
-#define StartWalWriter() StartChildProcess(WalWriterProcess)
-#define StartWalReceiver() StartChildProcess(WalReceiverProcess)
+#define StartupDataBase() StartChildProcess(PMC_STARTUP)
+#define StartArchiver() StartChildProcess(PMC_ARCHIVER)
+#define StartBackgroundWriter() StartChildProcess(PMC_BGWRITER)
+#define StartCheckpointer() StartChildProcess(PMC_CHECKPOINTER)
+#define StartWalWriter() StartChildProcess(PMC_WAL_WRITER)
+#define StartWalReceiver() StartChildProcess(PMC_WAL_RECEIVER)
+
+#define StartAutoVacLauncher() StartChildProcess(PMC_AV_LAUNCHER);
+#define StartAutoVacWorker() StartChildProcess(PMC_AV_WORKER);
/* Macros to check exit status of a child process */
#define EXIT_STATUS_0(st) ((st) == 0)
@@ -1115,11 +1033,11 @@ PostmasterMain(int argc, char *argv[])
/*
* Clean out the temp directory used to transmit parameters to child
- * processes (see internal_forkexec, below). We must do this before
- * launching any child processes, else we have a race condition: we could
- * remove a parameter file before the child can read it. It should be
- * safe to do so now, because we verified earlier that there are no
- * conflicting Postgres processes in this data directory.
+ * processes (see internal_forkexec). We must do this before launching
+ * any child processes, else we have a race condition: we could remove a
+ * parameter file before the child can read it. It should be safe to do
+ * so now, because we verified earlier that there are no conflicting
+ * Postgres processes in this data directory.
*/
RemovePgTempFilesInDir(PG_TEMP_FILES_DIR, true, false);
#endif
@@ -3989,6 +3907,11 @@ TerminateChildren(int signal)
signal_child(PgArchPID, signal);
}
+typedef struct BackendStartupInfo
+{
+ CAC_state canAcceptConnections;
+} BackendStartupInfo;
+
/*
* BackendStartup -- start backend process
*
@@ -4001,7 +3924,7 @@ BackendStartup(ClientSocket *client_sock)
{
Backend *bn; /* for backend cleanup */
pid_t pid;
- CAC_state cac;
+ BackendStartupInfo info;
/*
* Create backend data structure. Better before the fork() so we can
@@ -4030,11 +3953,10 @@ BackendStartup(ClientSocket *client_sock)
return STATUS_ERROR;
}
- bn->cancel_key = MyCancelKey;
-
/* Pass down canAcceptConnections state */
- cac = canAcceptConnections(BACKEND_TYPE_NORMAL);
- bn->dead_end = (cac != CAC_OK);
+ info.canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
+ bn->dead_end = (info.canAcceptConnections != CAC_OK);
+ bn->cancel_key = MyCancelKey;
/*
* Unless it's a dead_end child, assign it a child slot number
@@ -4047,26 +3969,7 @@ BackendStartup(ClientSocket *client_sock)
/* Hasn't asked to be notified about any bgworkers yet */
bn->bgworker_notify = false;
-#ifdef EXEC_BACKEND
- pid = backend_forkexec(client_sock, cac);
-#else /* !EXEC_BACKEND */
- pid = fork_process();
- if (pid == 0) /* child */
- {
- /* Detangle from postmaster */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Perform additional initialization and collect startup packet */
- BackendInitialize(client_sock, cac);
-
- /* And run the backend */
- BackendRun();
- }
-#endif /* EXEC_BACKEND */
-
+ pid = postmaster_child_launch(PMC_BACKEND, (char *) &info, sizeof(info), client_sock);
if (pid < 0)
{
/* in parent, fork failed */
@@ -4103,6 +4006,58 @@ BackendStartup(ClientSocket *client_sock)
return STATUS_OK;
}
+void
+BackendMain(char *startup_data, size_t startup_data_len)
+{
+ BackendStartupInfo *info = (BackendStartupInfo *) startup_data;
+
+ Assert(startup_data_len == sizeof(BackendStartupInfo));
+ Assert(MyClientSocket != NULL);
+
+#ifdef EXEC_BACKEND
+
+ /*
+ * Need to reinitialize the SSL library in the backend, since the context
+ * structures contain function pointers and cannot be passed through the
+ * parameter file.
+ *
+ * If for some reason reload fails (maybe the user installed broken key
+ * files), soldier on without SSL; that's better than all connections
+ * becoming impossible.
+ *
+ * XXX should we do this in all child processes? For the moment it's
+ * enough to do it in backend children.
+ */
+#ifdef USE_SSL
+ if (EnableSSL)
+ {
+ if (secure_initialize(false) == 0)
+ LoadedSSL = true;
+ else
+ ereport(LOG,
+ (errmsg("SSL configuration could not be loaded in child process")));
+ }
+#endif
+#endif
+
+ /* Perform additional initialization and collect startup packet */
+ BackendInitialize(MyClientSocket, info->canAcceptConnections);
+
+ /*
+ * Create a per-backend PGPROC struct in shared memory. We must do this
+ * before we can use LWLocks or access any shared memory.
+ */
+ InitProcess();
+
+ /*
+ * Make sure we aren't in PostmasterContext anymore. (We can't delete it
+ * just yet, though, because InitPostgres will need the HBA data.)
+ */
+ MemoryContextSwitchTo(TopMemoryContext);
+
+ PostgresMain(MyProcPort->database_name, MyProcPort->user_name);
+}
+
/*
* Try to report backend fork() failure to client before we close the
* connection. Since we do not care to risk blocking the postmaster on
@@ -4378,729 +4333,161 @@ BackendInitialize(ClientSocket *client_sock, CAC_state cac)
/*
- * BackendRun -- set up the backend's argument list and invoke PostgresMain()
+ * ExitPostmaster -- cleanup
*
- * returns:
- * Doesn't return at all.
+ * Do NOT call exit() directly --- always go through here!
*/
static void
-BackendRun(void)
+ExitPostmaster(int status)
{
- /*
- * Create a per-backend PGPROC struct in shared memory. We must do this
- * before we can use LWLocks or access any shared memory.
- */
- InitProcess();
+#ifdef HAVE_PTHREAD_IS_THREADED_NP
/*
- * Make sure we aren't in PostmasterContext anymore. (We can't delete it
- * just yet, though, because InitPostgres will need the HBA data.)
+ * There is no known cause for a postmaster to become multithreaded after
+ * startup. Recheck to account for the possibility of unknown causes.
+ * This message uses LOG level, because an unclean shutdown at this point
+ * would usually not look much different from a clean shutdown.
*/
- MemoryContextSwitchTo(TopMemoryContext);
-
- PostgresMain(MyProcPort->database_name, MyProcPort->user_name);
-}
-
-
-#ifdef EXEC_BACKEND
-
-/*
- * postmaster_forkexec -- fork and exec a postmaster subprocess
- *
- * The caller must have set up the argv array already, except for argv[2]
- * which will be filled with the name of the temp variable file.
- *
- * Returns the child process PID, or -1 on fork failure (a suitable error
- * message has been logged on failure).
- *
- * All uses of this routine will dispatch to SubPostmasterMain in the
- * child process.
- */
-pid_t
-postmaster_forkexec(int argc, char *argv[])
-{
- ClientSocket client_sock;
-
- /* This entry point doesn't pass a client socket */
- memset(&client_sock, 0, sizeof(ClientSocket));
- return internal_forkexec(argc, argv, &client_sock);
-}
-
-/*
- * backend_forkexec -- fork/exec off a backend process
- *
- * Some operating systems (WIN32) don't have fork() so we have to simulate
- * it by storing parameters that need to be passed to the child and
- * then create a new child process.
- *
- * returns the pid of the fork/exec'd process, or -1 on failure
- */
-static pid_t
-backend_forkexec(ClientSocket *client_sock, CAC_state cac)
-{
- char *av[5];
- int ac = 0;
- char cacbuf[10];
-
- av[ac++] = "postgres";
- av[ac++] = "--forkbackend";
- av[ac++] = NULL; /* filled in by internal_forkexec */
+ if (pthread_is_threaded_np() != 0)
+ ereport(LOG,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg_internal("postmaster became multithreaded"),
+ errdetail("Please report this to <%s>.", PACKAGE_BUGREPORT)));
+#endif
- snprintf(cacbuf, sizeof(cacbuf), "%d", (int) cac);
- av[ac++] = cacbuf;
+ /* should cleanup shared memory and kill all backends */
- av[ac] = NULL;
- Assert(ac < lengthof(av));
+ /*
+ * Not sure of the semantics here. When the Postmaster dies, should the
+ * backends all be killed? probably not.
+ *
+ * MUST -- vadim 05-10-1999
+ */
- return internal_forkexec(ac, av, client_sock);
+ proc_exit(status);
}
-#ifndef WIN32
-
/*
- * internal_forkexec non-win32 implementation
- *
- * - writes out backend variables to the parameter file
- * - fork():s, and then exec():s the child process
+ * Handle pmsignal conditions representing requests from backends,
+ * and check for promote and logrotate requests from pg_ctl.
*/
-static pid_t
-internal_forkexec(int argc, char *argv[], ClientSocket *client_sock)
+static void
+process_pm_pmsignal(void)
{
- static unsigned long tmpBackendFileNum = 0;
- pid_t pid;
- char tmpfilename[MAXPGPATH];
- BackendParameters param;
- FILE *fp;
-
- if (!save_backend_variables(¶m, client_sock))
- return -1; /* log made by save_backend_variables */
+ pending_pm_pmsignal = false;
- /* Calculate name for temp file */
- snprintf(tmpfilename, MAXPGPATH, "%s/%s.backend_var.%d.%lu",
- PG_TEMP_FILES_DIR, PG_TEMP_FILE_PREFIX,
- MyProcPid, ++tmpBackendFileNum);
+ ereport(DEBUG2,
+ (errmsg_internal("postmaster received pmsignal signal")));
- /* Open file */
- fp = AllocateFile(tmpfilename, PG_BINARY_W);
- if (!fp)
+ /*
+ * RECOVERY_STARTED and BEGIN_HOT_STANDBY signals are ignored in
+ * unexpected states. If the startup process quickly starts up, completes
+ * recovery, exits, we might process the death of the startup process
+ * first. We don't want to go back to recovery in that case.
+ */
+ if (CheckPostmasterSignal(PMSIGNAL_RECOVERY_STARTED) &&
+ pmState == PM_STARTUP && Shutdown == NoShutdown)
{
+ /* WAL redo has started. We're out of reinitialization. */
+ FatalError = false;
+ AbortStartTime = 0;
+
/*
- * As in OpenTemporaryFileInTablespace, try to make the temp-file
- * directory, ignoring errors.
+ * Start the archiver if we're responsible for (re-)archiving received
+ * files.
*/
- (void) MakePGDirectory(PG_TEMP_FILES_DIR);
+ Assert(PgArchPID == 0);
+ if (XLogArchivingAlways())
+ PgArchPID = StartArchiver();
- fp = AllocateFile(tmpfilename, PG_BINARY_W);
- if (!fp)
+ /*
+ * If we aren't planning to enter hot standby mode later, treat
+ * RECOVERY_STARTED as meaning we're out of startup, and report status
+ * accordingly.
+ */
+ if (!EnableHotStandby)
{
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not create file \"%s\": %m",
- tmpfilename)));
- return -1;
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
+#ifdef USE_SYSTEMD
+ sd_notify(0, "READY=1");
+#endif
}
+
+ pmState = PM_RECOVERY;
}
- if (fwrite(¶m, sizeof(param), 1, fp) != 1)
+ if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
+ pmState == PM_RECOVERY && Shutdown == NoShutdown)
{
ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not write to file \"%s\": %m", tmpfilename)));
- FreeFile(fp);
- return -1;
+ (errmsg("database system is ready to accept read-only connections")));
+
+ /* Report status */
+ AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_READY);
+#ifdef USE_SYSTEMD
+ sd_notify(0, "READY=1");
+#endif
+
+ pmState = PM_HOT_STANDBY;
+ connsAllowed = true;
+
+ /* Some workers may be scheduled to start now */
+ StartWorkerNeeded = true;
}
- /* Release file */
- if (FreeFile(fp))
+ /* Process background worker state changes. */
+ if (CheckPostmasterSignal(PMSIGNAL_BACKGROUND_WORKER_CHANGE))
{
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not write to file \"%s\": %m", tmpfilename)));
- return -1;
+ /* Accept new worker requests only if not stopping. */
+ BackgroundWorkerStateChange(pmState < PM_STOP_BACKENDS);
+ StartWorkerNeeded = true;
}
- /* Make sure caller set up argv properly */
- Assert(argc >= 3);
- Assert(argv[argc] == NULL);
- Assert(strncmp(argv[1], "--fork", 6) == 0);
- Assert(argv[2] == NULL);
-
- /* Insert temp file name after --fork argument */
- argv[2] = tmpfilename;
+ if (StartWorkerNeeded || HaveCrashedWorker)
+ maybe_start_bgworkers();
- /* Fire off execv in child */
- if ((pid = fork_process()) == 0)
+ /* Tell syslogger to rotate logfile if requested */
+ if (SysLoggerPID != 0)
{
- if (execv(postgres_exec_path, argv) < 0)
+ if (CheckLogrotateSignal())
{
- ereport(LOG,
- (errmsg("could not execute server process \"%s\": %m",
- postgres_exec_path)));
- /* We're already in the child process here, can't return */
- exit(1);
+ signal_child(SysLoggerPID, SIGUSR1);
+ RemoveLogrotateSignalFiles();
+ }
+ else if (CheckPostmasterSignal(PMSIGNAL_ROTATE_LOGFILE))
+ {
+ signal_child(SysLoggerPID, SIGUSR1);
}
}
- return pid; /* Parent returns pid, or -1 on fork failure */
-}
-#else /* WIN32 */
-
-/*
- * internal_forkexec win32 implementation
- *
- * - starts backend using CreateProcess(), in suspended state
- * - writes out backend variables to the parameter file
- * - during this, duplicates handles and sockets required for
- * inheritance into the new process
- * - resumes execution of the new process once the backend parameter
- * file is complete.
- */
-static pid_t
-internal_forkexec(int argc, char *argv[], Port *port)
-{
- int retry_count = 0;
- STARTUPINFO si;
- PROCESS_INFORMATION pi;
- int i;
- int j;
- char cmdLine[MAXPGPATH * 2];
- HANDLE paramHandle;
- BackendParameters *param;
- SECURITY_ATTRIBUTES sa;
- char paramHandleStr[32];
- win32_deadchild_waitinfo *childinfo;
-
- /* Make sure caller set up argv properly */
- Assert(argc >= 3);
- Assert(argv[argc] == NULL);
- Assert(strncmp(argv[1], "--fork", 6) == 0);
- Assert(argv[2] == NULL);
-
- /* Resume here if we need to retry */
-retry:
-
- /* Set up shared memory for parameter passing */
- ZeroMemory(&sa, sizeof(sa));
- sa.nLength = sizeof(sa);
- sa.bInheritHandle = TRUE;
- paramHandle = CreateFileMapping(INVALID_HANDLE_VALUE,
- &sa,
- PAGE_READWRITE,
- 0,
- sizeof(BackendParameters),
- NULL);
- if (paramHandle == INVALID_HANDLE_VALUE)
+ if (CheckPostmasterSignal(PMSIGNAL_START_AUTOVAC_LAUNCHER) &&
+ Shutdown <= SmartShutdown && pmState < PM_STOP_BACKENDS)
{
- ereport(LOG,
- (errmsg("could not create backend parameter file mapping: error code %lu",
- GetLastError())));
- return -1;
+ /*
+ * Start one iteration of the autovacuum daemon, even if autovacuuming
+ * is nominally not enabled. This is so we can have an active defense
+ * against transaction ID wraparound. We set a flag for the main loop
+ * to do it rather than trying to do it here --- this is because the
+ * autovac process itself may send the signal, and we want to handle
+ * that by launching another iteration as soon as the current one
+ * completes.
+ */
+ start_autovac_launcher = true;
}
- param = MapViewOfFile(paramHandle, FILE_MAP_WRITE, 0, 0, sizeof(BackendParameters));
- if (!param)
+ if (CheckPostmasterSignal(PMSIGNAL_START_AUTOVAC_WORKER) &&
+ Shutdown <= SmartShutdown && pmState < PM_STOP_BACKENDS)
{
- ereport(LOG,
- (errmsg("could not map backend parameter memory: error code %lu",
- GetLastError())));
- CloseHandle(paramHandle);
- return -1;
+ /* The autovacuum launcher wants us to start a worker process. */
+ StartAutovacuumWorker();
}
- /* Insert temp file name after --fork argument */
-#ifdef _WIN64
- sprintf(paramHandleStr, "%llu", (LONG_PTR) paramHandle);
-#else
- sprintf(paramHandleStr, "%lu", (DWORD) paramHandle);
-#endif
- argv[2] = paramHandleStr;
-
- /* Format the cmd line */
- cmdLine[sizeof(cmdLine) - 1] = '\0';
- cmdLine[sizeof(cmdLine) - 2] = '\0';
- snprintf(cmdLine, sizeof(cmdLine) - 1, "\"%s\"", postgres_exec_path);
- i = 0;
- while (argv[++i] != NULL)
- {
- j = strlen(cmdLine);
- snprintf(cmdLine + j, sizeof(cmdLine) - 1 - j, " \"%s\"", argv[i]);
- }
- if (cmdLine[sizeof(cmdLine) - 2] != '\0')
+ if (CheckPostmasterSignal(PMSIGNAL_START_WALRECEIVER))
{
- ereport(LOG,
- (errmsg("subprocess command line too long")));
- UnmapViewOfFile(param);
- CloseHandle(paramHandle);
- return -1;
- }
-
- memset(&pi, 0, sizeof(pi));
- memset(&si, 0, sizeof(si));
- si.cb = sizeof(si);
-
- /*
- * Create the subprocess in a suspended state. This will be resumed later,
- * once we have written out the parameter file.
- */
- if (!CreateProcess(NULL, cmdLine, NULL, NULL, TRUE, CREATE_SUSPENDED,
- NULL, NULL, &si, &pi))
- {
- ereport(LOG,
- (errmsg("CreateProcess() call failed: %m (error code %lu)",
- GetLastError())));
- UnmapViewOfFile(param);
- CloseHandle(paramHandle);
- return -1;
- }
-
- if (!save_backend_variables(param, port, pi.hProcess, pi.dwProcessId))
- {
- /*
- * log made by save_backend_variables, but we have to clean up the
- * mess with the half-started process
- */
- if (!TerminateProcess(pi.hProcess, 255))
- ereport(LOG,
- (errmsg_internal("could not terminate unstarted process: error code %lu",
- GetLastError())));
- CloseHandle(pi.hProcess);
- CloseHandle(pi.hThread);
- UnmapViewOfFile(param);
- CloseHandle(paramHandle);
- return -1; /* log made by save_backend_variables */
- }
-
- /* Drop the parameter shared memory that is now inherited to the backend */
- if (!UnmapViewOfFile(param))
- ereport(LOG,
- (errmsg("could not unmap view of backend parameter file: error code %lu",
- GetLastError())));
- if (!CloseHandle(paramHandle))
- ereport(LOG,
- (errmsg("could not close handle to backend parameter file: error code %lu",
- GetLastError())));
-
- /*
- * Reserve the memory region used by our main shared memory segment before
- * we resume the child process. Normally this should succeed, but if ASLR
- * is active then it might sometimes fail due to the stack or heap having
- * gotten mapped into that range. In that case, just terminate the
- * process and retry.
- */
- if (!pgwin32_ReserveSharedMemoryRegion(pi.hProcess))
- {
- /* pgwin32_ReserveSharedMemoryRegion already made a log entry */
- if (!TerminateProcess(pi.hProcess, 255))
- ereport(LOG,
- (errmsg_internal("could not terminate process that failed to reserve memory: error code %lu",
- GetLastError())));
- CloseHandle(pi.hProcess);
- CloseHandle(pi.hThread);
- if (++retry_count < 100)
- goto retry;
- ereport(LOG,
- (errmsg("giving up after too many tries to reserve shared memory"),
- errhint("This might be caused by ASLR or antivirus software.")));
- return -1;
- }
-
- /*
- * Now that the backend variables are written out, we start the child
- * thread so it can start initializing while we set up the rest of the
- * parent state.
- */
- if (ResumeThread(pi.hThread) == -1)
- {
- if (!TerminateProcess(pi.hProcess, 255))
- {
- ereport(LOG,
- (errmsg_internal("could not terminate unstartable process: error code %lu",
- GetLastError())));
- CloseHandle(pi.hProcess);
- CloseHandle(pi.hThread);
- return -1;
- }
- CloseHandle(pi.hProcess);
- CloseHandle(pi.hThread);
- ereport(LOG,
- (errmsg_internal("could not resume thread of unstarted process: error code %lu",
- GetLastError())));
- return -1;
- }
-
- /*
- * Queue a waiter to signal when this child dies. The wait will be handled
- * automatically by an operating system thread pool. The memory will be
- * freed by a later call to waitpid().
- */
- childinfo = palloc(sizeof(win32_deadchild_waitinfo));
- childinfo->procHandle = pi.hProcess;
- childinfo->procId = pi.dwProcessId;
-
- if (!RegisterWaitForSingleObject(&childinfo->waitHandle,
- pi.hProcess,
- pgwin32_deadchild_callback,
- childinfo,
- INFINITE,
- WT_EXECUTEONLYONCE | WT_EXECUTEINWAITTHREAD))
- ereport(FATAL,
- (errmsg_internal("could not register process for wait: error code %lu",
- GetLastError())));
-
- /* Don't close pi.hProcess here - waitpid() needs access to it */
-
- CloseHandle(pi.hThread);
-
- return pi.dwProcessId;
-}
-#endif /* WIN32 */
-
-
-/*
- * SubPostmasterMain -- Get the fork/exec'd process into a state equivalent
- * to what it would be if we'd simply forked on Unix, and then
- * dispatch to the appropriate place.
- *
- * The first two command line arguments are expected to be "--forkFOO"
- * (where FOO indicates which postmaster child we are to become), and
- * the name of a variables file that we can read to load data that would
- * have been inherited by fork() on Unix. Remaining arguments go to the
- * subprocess FooMain() routine.
- */
-void
-SubPostmasterMain(int argc, char *argv[])
-{
- ClientSocket client_sock;
-
- /* In EXEC_BACKEND case we will not have inherited these settings */
- IsPostmasterEnvironment = true;
- whereToSendOutput = DestNone;
-
- /* Setup essential subsystems (to ensure elog() behaves sanely) */
- InitializeGUCOptions();
-
- /* Check we got appropriate args */
- if (argc < 3)
- elog(FATAL, "invalid subpostmaster invocation");
-
- /* Read in the variables file */
- memset(&client_sock, 0, sizeof(ClientSocket));
- read_backend_variables(argv[2], &client_sock);
-
- /* Close the postmaster's sockets (as soon as we know them) */
- ClosePostmasterPorts(strcmp(argv[1], "--forklog") == 0);
-
- /* Setup as postmaster child */
- InitPostmasterChild();
-
- /*
- * If appropriate, physically re-attach to shared memory segment. We want
- * to do this before going any further to ensure that we can attach at the
- * same address the postmaster used. On the other hand, if we choose not
- * to re-attach, we may have other cleanup to do.
- *
- * If testing EXEC_BACKEND on Linux, you should run this as root before
- * starting the postmaster:
- *
- * sysctl -w kernel.randomize_va_space=0
- *
- * This prevents using randomized stack and code addresses that cause the
- * child process's memory map to be different from the parent's, making it
- * sometimes impossible to attach to shared memory at the desired address.
- * Return the setting to its old value (usually '1' or '2') when finished.
- */
- if (strcmp(argv[1], "--forkbackend") == 0 ||
- strcmp(argv[1], "--forkavlauncher") == 0 ||
- strcmp(argv[1], "--forkavworker") == 0 ||
- strcmp(argv[1], "--forkaux") == 0 ||
- strncmp(argv[1], "--forkbgworker", 14) == 0)
- PGSharedMemoryReAttach();
- else
- PGSharedMemoryNoReAttach();
-
- /* autovacuum needs this set before calling InitProcess */
- if (strcmp(argv[1], "--forkavlauncher") == 0)
- AutovacuumLauncherIAm();
- if (strcmp(argv[1], "--forkavworker") == 0)
- AutovacuumWorkerIAm();
-
- /* Read in remaining GUC variables */
- read_nondefault_variables();
-
- /*
- * Check that the data directory looks valid, which will also check the
- * privileges on the data directory and update our umask and file/group
- * variables for creating files later. Note: this should really be done
- * before we create any files or directories.
- */
- checkDataDir();
-
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
- */
- LocalProcessControlFile(false);
-
- /*
- * Reload any libraries that were preloaded by the postmaster. Since we
- * exec'd this process, those libraries didn't come along with us; but we
- * should load them into all child processes to be consistent with the
- * non-EXEC_BACKEND behavior.
- */
- process_shared_preload_libraries();
-
- /* Run backend or appropriate child */
- if (strcmp(argv[1], "--forkbackend") == 0)
- {
- CAC_state cac;
-
- Assert(argc == 4);
- cac = (CAC_state) atoi(argv[3]);
-
- /*
- * Need to reinitialize the SSL library in the backend, since the
- * context structures contain function pointers and cannot be passed
- * through the parameter file.
- *
- * If for some reason reload fails (maybe the user installed broken
- * key files), soldier on without SSL; that's better than all
- * connections becoming impossible.
- *
- * XXX should we do this in all child processes? For the moment it's
- * enough to do it in backend children.
- */
-#ifdef USE_SSL
- if (EnableSSL)
- {
- if (secure_initialize(false) == 0)
- LoadedSSL = true;
- else
- ereport(LOG,
- (errmsg("SSL configuration could not be loaded in child process")));
- }
-#endif
-
- /*
- * Perform additional initialization and collect startup packet.
- *
- * We want to do this before InitProcess() for a couple of reasons: 1.
- * so that we aren't eating up a PGPROC slot while waiting on the
- * client. 2. so that if InitProcess() fails due to being out of
- * PGPROC slots, we have already initialized libpq and are able to
- * report the error to the client.
- */
- BackendInitialize(&client_sock, cac);
-
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* And run the backend */
- BackendRun(); /* does not return */
- }
- if (strcmp(argv[1], "--forkaux") == 0)
- {
- AuxProcType auxtype;
-
- Assert(argc == 4);
-
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- auxtype = atoi(argv[3]);
- AuxiliaryProcessMain(auxtype); /* does not return */
- }
- if (strcmp(argv[1], "--forkavlauncher") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- AutoVacLauncherMain(argc - 2, argv + 2); /* does not return */
- }
- if (strcmp(argv[1], "--forkavworker") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
- }
- if (strncmp(argv[1], "--forkbgworker", 14) == 0)
- {
- /* do this as early as possible; in particular, before InitProcess() */
- IsBackgroundWorker = true;
-
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- BackgroundWorkerMain();
- }
- if (strcmp(argv[1], "--forklog") == 0)
- {
- /* Do not want to attach to shared memory */
-
- SysLoggerMain(argc, argv); /* does not return */
- }
-
- abort(); /* shouldn't get here */
-}
-#endif /* EXEC_BACKEND */
-
-
-/*
- * ExitPostmaster -- cleanup
- *
- * Do NOT call exit() directly --- always go through here!
- */
-static void
-ExitPostmaster(int status)
-{
-#ifdef HAVE_PTHREAD_IS_THREADED_NP
-
- /*
- * There is no known cause for a postmaster to become multithreaded after
- * startup. Recheck to account for the possibility of unknown causes.
- * This message uses LOG level, because an unclean shutdown at this point
- * would usually not look much different from a clean shutdown.
- */
- if (pthread_is_threaded_np() != 0)
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg_internal("postmaster became multithreaded"),
- errdetail("Please report this to <%s>.", PACKAGE_BUGREPORT)));
-#endif
-
- /* should cleanup shared memory and kill all backends */
-
- /*
- * Not sure of the semantics here. When the Postmaster dies, should the
- * backends all be killed? probably not.
- *
- * MUST -- vadim 05-10-1999
- */
-
- proc_exit(status);
-}
-
-/*
- * Handle pmsignal conditions representing requests from backends,
- * and check for promote and logrotate requests from pg_ctl.
- */
-static void
-process_pm_pmsignal(void)
-{
- pending_pm_pmsignal = false;
-
- ereport(DEBUG2,
- (errmsg_internal("postmaster received pmsignal signal")));
-
- /*
- * RECOVERY_STARTED and BEGIN_HOT_STANDBY signals are ignored in
- * unexpected states. If the startup process quickly starts up, completes
- * recovery, exits, we might process the death of the startup process
- * first. We don't want to go back to recovery in that case.
- */
- if (CheckPostmasterSignal(PMSIGNAL_RECOVERY_STARTED) &&
- pmState == PM_STARTUP && Shutdown == NoShutdown)
- {
- /* WAL redo has started. We're out of reinitialization. */
- FatalError = false;
- AbortStartTime = 0;
-
- /*
- * Start the archiver if we're responsible for (re-)archiving received
- * files.
- */
- Assert(PgArchPID == 0);
- if (XLogArchivingAlways())
- PgArchPID = StartArchiver();
-
- /*
- * If we aren't planning to enter hot standby mode later, treat
- * RECOVERY_STARTED as meaning we're out of startup, and report status
- * accordingly.
- */
- if (!EnableHotStandby)
- {
- AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
-#ifdef USE_SYSTEMD
- sd_notify(0, "READY=1");
-#endif
- }
-
- pmState = PM_RECOVERY;
- }
-
- if (CheckPostmasterSignal(PMSIGNAL_BEGIN_HOT_STANDBY) &&
- pmState == PM_RECOVERY && Shutdown == NoShutdown)
- {
- ereport(LOG,
- (errmsg("database system is ready to accept read-only connections")));
-
- /* Report status */
- AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_READY);
-#ifdef USE_SYSTEMD
- sd_notify(0, "READY=1");
-#endif
-
- pmState = PM_HOT_STANDBY;
- connsAllowed = true;
-
- /* Some workers may be scheduled to start now */
- StartWorkerNeeded = true;
- }
-
- /* Process background worker state changes. */
- if (CheckPostmasterSignal(PMSIGNAL_BACKGROUND_WORKER_CHANGE))
- {
- /* Accept new worker requests only if not stopping. */
- BackgroundWorkerStateChange(pmState < PM_STOP_BACKENDS);
- StartWorkerNeeded = true;
- }
-
- if (StartWorkerNeeded || HaveCrashedWorker)
- maybe_start_bgworkers();
-
- /* Tell syslogger to rotate logfile if requested */
- if (SysLoggerPID != 0)
- {
- if (CheckLogrotateSignal())
- {
- signal_child(SysLoggerPID, SIGUSR1);
- RemoveLogrotateSignalFiles();
- }
- else if (CheckPostmasterSignal(PMSIGNAL_ROTATE_LOGFILE))
- {
- signal_child(SysLoggerPID, SIGUSR1);
- }
- }
-
- if (CheckPostmasterSignal(PMSIGNAL_START_AUTOVAC_LAUNCHER) &&
- Shutdown <= SmartShutdown && pmState < PM_STOP_BACKENDS)
- {
- /*
- * Start one iteration of the autovacuum daemon, even if autovacuuming
- * is nominally not enabled. This is so we can have an active defense
- * against transaction ID wraparound. We set a flag for the main loop
- * to do it rather than trying to do it here --- this is because the
- * autovac process itself may send the signal, and we want to handle
- * that by launching another iteration as soon as the current one
- * completes.
- */
- start_autovac_launcher = true;
- }
-
- if (CheckPostmasterSignal(PMSIGNAL_START_AUTOVAC_WORKER) &&
- Shutdown <= SmartShutdown && pmState < PM_STOP_BACKENDS)
- {
- /* The autovacuum launcher wants us to start a worker process. */
- StartAutovacuumWorker();
- }
-
- if (CheckPostmasterSignal(PMSIGNAL_START_WALRECEIVER))
- {
- /* Startup Process wants us to start the walreceiver process. */
- /* Start immediately if possible, else remember request for later. */
- WalReceiverRequested = true;
- MaybeStartWalReceiver();
+ /* Startup Process wants us to start the walreceiver process. */
+ /* Start immediately if possible, else remember request for later. */
+ WalReceiverRequested = true;
+ MaybeStartWalReceiver();
}
/*
@@ -5237,93 +4624,23 @@ CountChildren(int target)
* to start subprocess.
*/
static pid_t
-StartChildProcess(AuxProcType type)
+StartChildProcess(PostmasterChildType type)
{
pid_t pid;
-#ifdef EXEC_BACKEND
- {
- char *av[10];
- int ac = 0;
- char typebuf[32];
-
- /*
- * Set up command-line arguments for subprocess
- */
- av[ac++] = "postgres";
- av[ac++] = "--forkaux";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- snprintf(typebuf, sizeof(typebuf), "%d", type);
- av[ac++] = typebuf;
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- pid = postmaster_forkexec(ac, av);
- }
-#else /* !EXEC_BACKEND */
- pid = fork_process();
-
- if (pid == 0) /* child */
- {
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Release postmaster's working memory context */
- MemoryContextSwitchTo(TopMemoryContext);
- MemoryContextDelete(PostmasterContext);
- PostmasterContext = NULL;
-
- AuxiliaryProcessMain(type); /* does not return */
- }
-#endif /* EXEC_BACKEND */
-
+ pid = postmaster_child_launch(type, NULL, 0, NULL);
if (pid < 0)
{
/* in parent, fork failed */
- int save_errno = errno;
-
- errno = save_errno;
- switch (type)
- {
- case StartupProcess:
- ereport(LOG,
- (errmsg("could not fork startup process: %m")));
- break;
- case ArchiverProcess:
- ereport(LOG,
- (errmsg("could not fork archiver process: %m")));
- break;
- case BgWriterProcess:
- ereport(LOG,
- (errmsg("could not fork background writer process: %m")));
- break;
- case CheckpointerProcess:
- ereport(LOG,
- (errmsg("could not fork checkpointer process: %m")));
- break;
- case WalWriterProcess:
- ereport(LOG,
- (errmsg("could not fork WAL writer process: %m")));
- break;
- case WalReceiverProcess:
- ereport(LOG,
- (errmsg("could not fork WAL receiver process: %m")));
- break;
- default:
- ereport(LOG,
- (errmsg("could not fork process: %m")));
- break;
- }
+ /* XXX: translation? */
+ ereport(LOG,
+ (errmsg("could not fork %s process: %m", PostmasterChildName(type))));
/*
* fork failure is fatal during startup, but there's no need to choke
* immediately if starting other child types fails.
*/
- if (type == StartupProcess)
+ if (type == PMC_STARTUP)
ExitPostmaster(1);
return 0;
}
@@ -5587,32 +4904,6 @@ BackgroundWorkerUnblockSignals(void)
sigprocmask(SIG_SETMASK, &UnBlockSig, NULL);
}
-#ifdef EXEC_BACKEND
-static pid_t
-bgworker_forkexec(BackgroundWorker *worker)
-{
- char *av[10];
- int ac = 0;
- char forkav[MAXPGPATH];
- pid_t result;
-
- snprintf(forkav, MAXPGPATH, "--forkbgworker");
-
- av[ac++] = "postgres";
- av[ac++] = forkav;
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- MyBgworkerEntry = worker;
- result = postmaster_forkexec(ac, av);
- MyBgworkerEntry = NULL;
-
- return result;
-}
-#endif
-
/*
* Start a new bgworker.
* Starting time conditions must have been checked already.
@@ -5649,65 +4940,32 @@ do_start_bgworker(RegisteredBgWorker *rw)
(errmsg_internal("starting background worker process \"%s\"",
rw->rw_worker.bgw_name)));
-#ifdef EXEC_BACKEND
- switch ((worker_pid = bgworker_forkexec(&rw->rw_worker)))
-#else
- switch ((worker_pid = fork_process()))
-#endif
+ worker_pid = postmaster_child_launch(PMC_BGWORKER, (char *) &rw->rw_worker, sizeof(BackgroundWorker), NULL);
+ if (worker_pid == -1)
{
- case -1:
- /* in postmaster, fork failed ... */
- ereport(LOG,
- (errmsg("could not fork worker process: %m")));
- /* undo what assign_backendlist_entry did */
- ReleasePostmasterChildSlot(rw->rw_child_slot);
- rw->rw_child_slot = 0;
- pfree(rw->rw_backend);
- rw->rw_backend = NULL;
- /* mark entry as crashed, so we'll try again later */
- rw->rw_crashed_at = GetCurrentTimestamp();
- break;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /*
- * Before blowing away PostmasterContext, save this bgworker's
- * data where it can find it.
- */
- MyBgworkerEntry = (BackgroundWorker *)
- MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
- memcpy(MyBgworkerEntry, &rw->rw_worker, sizeof(BackgroundWorker));
-
- /* Release postmaster's working memory context */
- MemoryContextSwitchTo(TopMemoryContext);
- MemoryContextDelete(PostmasterContext);
- PostmasterContext = NULL;
-
- BackgroundWorkerMain();
+ /* in postmaster, fork failed ... */
+ ereport(LOG,
+ (errmsg("could not fork worker process: %m")));
+ /* undo what assign_backendlist_entry did */
+ ReleasePostmasterChildSlot(rw->rw_child_slot);
+ rw->rw_child_slot = 0;
+ pfree(rw->rw_backend);
+ rw->rw_backend = NULL;
+ /* mark entry as crashed, so we'll try again later */
+ rw->rw_crashed_at = GetCurrentTimestamp();
+ return false;
+ }
- exit(1); /* should not get here */
- break;
-#endif
- default:
- /* in postmaster, fork successful ... */
- rw->rw_pid = worker_pid;
- rw->rw_backend->pid = rw->rw_pid;
- ReportBackgroundWorkerPID(rw);
- /* add new worker to lists of backends */
- dlist_push_head(&BackendList, &rw->rw_backend->elem);
+ /* in postmaster, fork successful ... */
+ rw->rw_pid = worker_pid;
+ rw->rw_backend->pid = rw->rw_pid;
+ ReportBackgroundWorkerPID(rw);
+ /* add new worker to lists of backends */
+ dlist_push_head(&BackendList, &rw->rw_backend->elem);
#ifdef EXEC_BACKEND
- ShmemBackendArrayAdd(rw->rw_backend);
+ ShmemBackendArrayAdd(rw->rw_backend);
#endif
- return true;
- }
-
- return false;
+ return true;
}
/*
@@ -5959,351 +5217,6 @@ PostmasterMarkPIDForWorkerNotify(int pid)
#ifdef EXEC_BACKEND
-/*
- * The following need to be available to the save/restore_backend_variables
- * functions. They are marked NON_EXEC_STATIC in their home modules.
- */
-extern slock_t *ShmemLock;
-extern slock_t *ProcStructLock;
-extern PGPROC *AuxiliaryProcs;
-extern PMSignalData *PMSignalState;
-extern pg_time_t first_syslogger_file_time;
-
-#ifndef WIN32
-#define write_inheritable_socket(dest, src, childpid) ((*(dest) = (src)), true)
-#define read_inheritable_socket(dest, src) (*(dest) = *(src))
-#else
-static bool write_duplicated_handle(HANDLE *dest, HANDLE src, HANDLE child);
-static bool write_inheritable_socket(InheritableSocket *dest, SOCKET src,
- pid_t childPid);
-static void read_inheritable_socket(SOCKET *dest, InheritableSocket *src);
-#endif
-
-
-/* Save critical backend variables into the BackendParameters struct */
-#ifndef WIN32
-static bool
-save_backend_variables(BackendParameters *param, ClientSocket *client_sock)
-#else
-static bool
-save_backend_variables(BackendParameters *param, ClientSocket *client_sock,
- HANDLE childProcess, pid_t childPid)
-#endif
-{
- memcpy(¶m->client_sock, client_sock, sizeof(ClientSocket));
- if (!write_inheritable_socket(¶m->serialized_sock, client_sock->sock, childPid))
- return false;
-
- strlcpy(param->DataDir, DataDir, MAXPGPATH);
-
- param->MyCancelKey = MyCancelKey;
- param->MyPMChildSlot = MyPMChildSlot;
-
-#ifdef WIN32
- param->ShmemProtectiveRegion = ShmemProtectiveRegion;
-#endif
- param->UsedShmemSegID = UsedShmemSegID;
- param->UsedShmemSegAddr = UsedShmemSegAddr;
-
- param->ShmemLock = ShmemLock;
- param->ShmemVariableCache = ShmemVariableCache;
- param->ShmemBackendArray = ShmemBackendArray;
-
-#ifndef HAVE_SPINLOCKS
- param->SpinlockSemaArray = SpinlockSemaArray;
-#endif
- param->NamedLWLockTrancheRequests = NamedLWLockTrancheRequests;
- param->NamedLWLockTrancheArray = NamedLWLockTrancheArray;
- param->MainLWLockArray = MainLWLockArray;
- param->ProcStructLock = ProcStructLock;
- param->ProcGlobal = ProcGlobal;
- param->AuxiliaryProcs = AuxiliaryProcs;
- param->PreparedXactProcs = PreparedXactProcs;
- param->PMSignalState = PMSignalState;
-
- param->PostmasterPid = PostmasterPid;
- param->PgStartTime = PgStartTime;
- param->PgReloadTime = PgReloadTime;
- param->first_syslogger_file_time = first_syslogger_file_time;
-
- param->redirection_done = redirection_done;
- param->IsBinaryUpgrade = IsBinaryUpgrade;
- param->query_id_enabled = query_id_enabled;
- param->max_safe_fds = max_safe_fds;
-
- param->MaxBackends = MaxBackends;
-
-#ifdef WIN32
- param->PostmasterHandle = PostmasterHandle;
- if (!write_duplicated_handle(¶m->initial_signal_pipe,
- pgwin32_create_signal_listener(childPid),
- childProcess))
- return false;
-#else
- memcpy(¶m->postmaster_alive_fds, &postmaster_alive_fds,
- sizeof(postmaster_alive_fds));
-#endif
-
- memcpy(¶m->syslogPipe, &syslogPipe, sizeof(syslogPipe));
-
- strlcpy(param->my_exec_path, my_exec_path, MAXPGPATH);
-
- strlcpy(param->pkglib_path, pkglib_path, MAXPGPATH);
-
- if (MyBgworkerEntry)
- memcpy(¶m->MyBgworkerEntry, MyBgworkerEntry, sizeof(BackgroundWorker));
- else
- memset(¶m->MyBgworkerEntry, 0, sizeof(BackgroundWorker));
-
- return true;
-}
-
-
-#ifdef WIN32
-/*
- * Duplicate a handle for usage in a child process, and write the child
- * process instance of the handle to the parameter file.
- */
-static bool
-write_duplicated_handle(HANDLE *dest, HANDLE src, HANDLE childProcess)
-{
- HANDLE hChild = INVALID_HANDLE_VALUE;
-
- if (!DuplicateHandle(GetCurrentProcess(),
- src,
- childProcess,
- &hChild,
- 0,
- TRUE,
- DUPLICATE_CLOSE_SOURCE | DUPLICATE_SAME_ACCESS))
- {
- ereport(LOG,
- (errmsg_internal("could not duplicate handle to be written to backend parameter file: error code %lu",
- GetLastError())));
- return false;
- }
-
- *dest = hChild;
- return true;
-}
-
-/*
- * Duplicate a socket for usage in a child process, and write the resulting
- * structure to the parameter file.
- * This is required because a number of LSPs (Layered Service Providers) very
- * common on Windows (antivirus, firewalls, download managers etc) break
- * straight socket inheritance.
- */
-static bool
-write_inheritable_socket(InheritableSocket *dest, SOCKET src, pid_t childpid)
-{
- dest->origsocket = src;
- if (src != 0 && src != PGINVALID_SOCKET)
- {
- /* Actual socket */
- if (WSADuplicateSocket(src, childpid, &dest->wsainfo) != 0)
- {
- ereport(LOG,
- (errmsg("could not duplicate socket %d for use in backend: error code %d",
- (int) src, WSAGetLastError())));
- return false;
- }
- }
- return true;
-}
-
-/*
- * Read a duplicate socket structure back, and get the socket descriptor.
- */
-static void
-read_inheritable_socket(SOCKET *dest, InheritableSocket *src)
-{
- SOCKET s;
-
- if (src->origsocket == PGINVALID_SOCKET || src->origsocket == 0)
- {
- /* Not a real socket! */
- *dest = src->origsocket;
- }
- else
- {
- /* Actual socket, so create from structure */
- s = WSASocket(FROM_PROTOCOL_INFO,
- FROM_PROTOCOL_INFO,
- FROM_PROTOCOL_INFO,
- &src->wsainfo,
- 0,
- 0);
- if (s == INVALID_SOCKET)
- {
- write_stderr("could not create inherited socket: error code %d\n",
- WSAGetLastError());
- exit(1);
- }
- *dest = s;
-
- /*
- * To make sure we don't get two references to the same socket, close
- * the original one. (This would happen when inheritance actually
- * works..
- */
- closesocket(src->origsocket);
- }
-}
-#endif
-
-static void
-read_backend_variables(char *id, ClientSocket *client_sock)
-{
- BackendParameters param;
-
-#ifndef WIN32
- /* Non-win32 implementation reads from file */
- FILE *fp;
-
- /* Open file */
- fp = AllocateFile(id, PG_BINARY_R);
- if (!fp)
- {
- write_stderr("could not open backend variables file \"%s\": %s\n",
- id, strerror(errno));
- exit(1);
- }
-
- if (fread(¶m, sizeof(param), 1, fp) != 1)
- {
- write_stderr("could not read from backend variables file \"%s\": %s\n",
- id, strerror(errno));
- exit(1);
- }
-
- /* Release file */
- FreeFile(fp);
- if (unlink(id) != 0)
- {
- write_stderr("could not remove file \"%s\": %s\n",
- id, strerror(errno));
- exit(1);
- }
-#else
- /* Win32 version uses mapped file */
- HANDLE paramHandle;
- BackendParameters *paramp;
-
-#ifdef _WIN64
- paramHandle = (HANDLE) _atoi64(id);
-#else
- paramHandle = (HANDLE) atol(id);
-#endif
- paramp = MapViewOfFile(paramHandle, FILE_MAP_READ, 0, 0, 0);
- if (!paramp)
- {
- write_stderr("could not map view of backend variables: error code %lu\n",
- GetLastError());
- exit(1);
- }
-
- memcpy(¶m, paramp, sizeof(BackendParameters));
-
- if (!UnmapViewOfFile(paramp))
- {
- write_stderr("could not unmap view of backend variables: error code %lu\n",
- GetLastError());
- exit(1);
- }
-
- if (!CloseHandle(paramHandle))
- {
- write_stderr("could not close handle to backend parameter variables: error code %lu\n",
- GetLastError());
- exit(1);
- }
-#endif
-
- restore_backend_variables(¶m, client_sock);
-}
-
-/* Restore critical backend variables from the BackendParameters struct */
-static void
-restore_backend_variables(BackendParameters *param, ClientSocket *client_sock)
-{
- memcpy(client_sock, ¶m->client_sock, sizeof(ClientSocket));
- read_inheritable_socket(&client_sock->sock, ¶m->serialized_sock);
-
- SetDataDir(param->DataDir);
-
- MyCancelKey = param->MyCancelKey;
- MyPMChildSlot = param->MyPMChildSlot;
-
-#ifdef WIN32
- ShmemProtectiveRegion = param->ShmemProtectiveRegion;
-#endif
- UsedShmemSegID = param->UsedShmemSegID;
- UsedShmemSegAddr = param->UsedShmemSegAddr;
-
- ShmemLock = param->ShmemLock;
- ShmemVariableCache = param->ShmemVariableCache;
- ShmemBackendArray = param->ShmemBackendArray;
-
-#ifndef HAVE_SPINLOCKS
- SpinlockSemaArray = param->SpinlockSemaArray;
-#endif
- NamedLWLockTrancheRequests = param->NamedLWLockTrancheRequests;
- NamedLWLockTrancheArray = param->NamedLWLockTrancheArray;
- MainLWLockArray = param->MainLWLockArray;
- ProcStructLock = param->ProcStructLock;
- ProcGlobal = param->ProcGlobal;
- AuxiliaryProcs = param->AuxiliaryProcs;
- PreparedXactProcs = param->PreparedXactProcs;
- PMSignalState = param->PMSignalState;
-
- PostmasterPid = param->PostmasterPid;
- PgStartTime = param->PgStartTime;
- PgReloadTime = param->PgReloadTime;
- first_syslogger_file_time = param->first_syslogger_file_time;
-
- redirection_done = param->redirection_done;
- IsBinaryUpgrade = param->IsBinaryUpgrade;
- query_id_enabled = param->query_id_enabled;
- max_safe_fds = param->max_safe_fds;
-
- MaxBackends = param->MaxBackends;
-
-#ifdef WIN32
- PostmasterHandle = param->PostmasterHandle;
- pgwin32_initial_signal_pipe = param->initial_signal_pipe;
-#else
- memcpy(&postmaster_alive_fds, ¶m->postmaster_alive_fds,
- sizeof(postmaster_alive_fds));
-#endif
-
- memcpy(&syslogPipe, ¶m->syslogPipe, sizeof(syslogPipe));
-
- strlcpy(my_exec_path, param->my_exec_path, MAXPGPATH);
-
- strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
-
- if (param->MyBgworkerEntry.bgw_name[0] != '\0')
- {
- MyBgworkerEntry = (BackgroundWorker *)
- MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
- memcpy(MyBgworkerEntry, ¶m->MyBgworkerEntry, sizeof(BackgroundWorker));
- }
-
- /*
- * We need to restore fd.c's counts of externally-opened FDs; to avoid
- * confusion, be sure to do this after restoring max_safe_fds. (Note:
- * BackendInitialize will handle this for port->sock.)
- */
-#ifndef WIN32
- if (postmaster_alive_fds[0] >= 0)
- ReserveExternalFD();
- if (postmaster_alive_fds[1] >= 0)
- ReserveExternalFD();
-#endif
-}
-
-
Size
ShmemBackendArraySize(void)
{
@@ -6425,6 +5338,32 @@ pgwin32_deadchild_callback(PVOID lpParameter, BOOLEAN TimerOrWaitFired)
/* Queue SIGCHLD signal. */
pg_queue_signal(SIGCHLD);
}
+
+/*
+ * Queue a waiter to signal when this child dies. The wait will be handled
+ * automatically by an operating system thread pool. The memory will be
+ * freed by a later call to waitpid().
+ */
+void
+pgwin32_register_deadchild_callback(HANDLE procHandle, DWORD procId)
+{
+ win32_deadchild_waitinfo *childinfo;
+
+ childinfo = palloc(sizeof(win32_deadchild_waitinfo));
+ childinfo->procHandle = procHandle;
+ childinfo->procId = procId;
+
+ if (!RegisterWaitForSingleObject(&childinfo->waitHandle,
+ procHandle,
+ pgwin32_deadchild_callback,
+ childinfo,
+ INFINITE,
+ WT_EXECUTEONLYONCE | WT_EXECUTEINWAITTHREAD))
+ ereport(FATAL,
+ (errmsg_internal("could not register process for wait: error code %lu",
+ GetLastError())));
+}
+
#endif /* WIN32 */
/*
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index 0e7de26bc28..6b38acf922a 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -27,6 +27,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/interrupt.h"
#include "postmaster/startup.h"
#include "storage/ipc.h"
@@ -242,8 +243,14 @@ StartupProcExit(int code, Datum arg)
* ----------------------------------
*/
void
-StartupProcessMain(void)
+StartupProcessMain(char *startup_data, size_t startup_data_len)
{
+ Assert(startup_data_len == 0);
+
+ MyAuxProcType = StartupProcess;
+ MyBackendType = B_STARTUP;
+ AuxiliaryProcessInit();
+
/* Arrange to clean up at startup process exit */
on_shmem_exit(StartupProcExit, 0);
diff --git a/src/backend/postmaster/syslogger.c b/src/backend/postmaster/syslogger.c
index 96dd03d9e06..c923843532f 100644
--- a/src/backend/postmaster/syslogger.c
+++ b/src/backend/postmaster/syslogger.c
@@ -39,7 +39,6 @@
#include "pgstat.h"
#include "pgtime.h"
#include "port/pg_bitutils.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "postmaster/syslogger.h"
@@ -50,6 +49,7 @@
#include "storage/pg_shmem.h"
#include "tcop/tcopprot.h"
#include "utils/guc.h"
+#include "utils/memutils.h"
#include "utils/ps_status.h"
#include "utils/timestamp.h"
@@ -134,10 +134,7 @@ static volatile sig_atomic_t rotation_requested = false;
#ifdef EXEC_BACKEND
static int syslogger_fdget(FILE *file);
static FILE *syslogger_fdopen(int fd);
-static pid_t syslogger_forkexec(void);
-static void syslogger_parseArgs(int argc, char *argv[]);
#endif
-NON_EXEC_STATIC void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
static void process_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static void flush_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static FILE *logfile_open(const char *filename, const char *mode,
@@ -156,13 +153,19 @@ static void set_next_rotation_time(void);
static void sigUsr1Handler(SIGNAL_ARGS);
static void update_metainfo_datafile(void);
+typedef struct
+{
+ int syslogFile;
+ int csvlogFile;
+ int jsonlogFile;
+} syslogger_startup_data;
/*
* Main entry point for syslogger process
* argc/argv parameters are valid only in EXEC_BACKEND case.
*/
-NON_EXEC_STATIC void
-SysLoggerMain(int argc, char *argv[])
+void
+SysLoggerMain(char *startup_data, size_t startup_data_len)
{
#ifndef WIN32
char logbuffer[READ_BUF_SIZE];
@@ -174,11 +177,34 @@ SysLoggerMain(int argc, char *argv[])
pg_time_t now;
WaitEventSet *wes;
- now = MyStartTime;
+ /* Release postmaster's working memory context */
+ if (PostmasterContext)
+ {
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
+ }
+ /*
+ * Re-open the error output files that were opened by SysLogger_Start().
+ *
+ * We expect this will always succeed, which is too optimistic, but if it
+ * fails there's not a lot we can do to report the problem anyway. As
+ * coded, we'll just crash on a null pointer dereference after failure...
+ */
#ifdef EXEC_BACKEND
- syslogger_parseArgs(argc, argv);
-#endif /* EXEC_BACKEND */
+ {
+ syslogger_startup_data *info = (syslogger_startup_data *) startup_data;
+
+ Assert(startup_data_len == sizeof(*info));
+ syslogFile = syslogger_fdopen(info->syslogFile);
+ csvlogFile = syslogger_fdopen(info->csvlogFile);
+ jsonlogFile = syslogger_fdopen(info->jsonlogFile);
+ }
+#else
+ Assert(startup_data_len == 0);
+#endif
+
+ now = MyStartTime;
MyBackendType = B_LOGGER;
init_ps_display(NULL);
@@ -568,6 +594,9 @@ SysLogger_Start(void)
{
pid_t sysloggerPid;
char *filename;
+#ifdef EXEC_BACKEND
+ syslogger_startup_data startup_data;
+#endif /* EXEC_BACKEND */
if (!Logging_collector)
return 0;
@@ -667,112 +696,95 @@ SysLogger_Start(void)
}
#ifdef EXEC_BACKEND
- switch ((sysloggerPid = syslogger_forkexec()))
+ startup_data.syslogFile = syslogger_fdget(syslogFile);
+ startup_data.csvlogFile = syslogger_fdget(csvlogFile);
+ startup_data.jsonlogFile = syslogger_fdget(jsonlogFile);
+ sysloggerPid = postmaster_child_launch(PMC_SYSLOGGER, (char *) &startup_data, sizeof(startup_data), NULL);
#else
- switch ((sysloggerPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork system logger: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(true);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
+ sysloggerPid = postmaster_child_launch(PMC_SYSLOGGER, NULL, 0, NULL);
+#endif /* EXEC_BACKEND */
- /* do the work */
- SysLoggerMain(0, NULL);
- break;
-#endif
+ if (sysloggerPid == -1)
+ {
+ ereport(LOG,
+ (errmsg("could not fork system logger: %m")));
+ return 0;
+ }
- default:
- /* success, in postmaster */
+ /* success, in postmaster */
- /* now we redirect stderr, if not done already */
- if (!redirection_done)
- {
+ /* now we redirect stderr, if not done already */
+ if (!redirection_done)
+ {
#ifdef WIN32
- int fd;
+ int fd;
#endif
- /*
- * Leave a breadcrumb trail when redirecting, in case the user
- * forgets that redirection is active and looks only at the
- * original stderr target file.
- */
- ereport(LOG,
- (errmsg("redirecting log output to logging collector process"),
- errhint("Future log output will appear in directory \"%s\".",
- Log_directory)));
+ /*
+ * Leave a breadcrumb trail when redirecting, in case the user forgets
+ * that redirection is active and looks only at the original stderr
+ * target file.
+ */
+ ereport(LOG,
+ (errmsg("redirecting log output to logging collector process"),
+ errhint("Future log output will appear in directory \"%s\".",
+ Log_directory)));
#ifndef WIN32
- fflush(stdout);
- if (dup2(syslogPipe[1], STDOUT_FILENO) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stdout: %m")));
- fflush(stderr);
- if (dup2(syslogPipe[1], STDERR_FILENO) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stderr: %m")));
- /* Now we are done with the write end of the pipe. */
- close(syslogPipe[1]);
- syslogPipe[1] = -1;
+ fflush(stdout);
+ if (dup2(syslogPipe[1], STDOUT_FILENO) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stdout: %m")));
+ fflush(stderr);
+ if (dup2(syslogPipe[1], STDERR_FILENO) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stderr: %m")));
+ /* Now we are done with the write end of the pipe. */
+ close(syslogPipe[1]);
+ syslogPipe[1] = -1;
#else
- /*
- * open the pipe in binary mode and make sure stderr is binary
- * after it's been dup'ed into, to avoid disturbing the pipe
- * chunking protocol.
- */
- fflush(stderr);
- fd = _open_osfhandle((intptr_t) syslogPipe[1],
- _O_APPEND | _O_BINARY);
- if (dup2(fd, STDERR_FILENO) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stderr: %m")));
- close(fd);
- _setmode(STDERR_FILENO, _O_BINARY);
+ /*
+ * open the pipe in binary mode and make sure stderr is binary after
+ * it's been dup'ed into, to avoid disturbing the pipe chunking
+ * protocol.
+ */
+ fflush(stderr);
+ fd = _open_osfhandle((intptr_t) syslogPipe[1],
+ _O_APPEND | _O_BINARY);
+ if (dup2(fd, STDERR_FILENO) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stderr: %m")));
+ close(fd);
+ _setmode(STDERR_FILENO, _O_BINARY);
- /*
- * Now we are done with the write end of the pipe.
- * CloseHandle() must not be called because the preceding
- * close() closes the underlying handle.
- */
- syslogPipe[1] = 0;
+ /*
+ * Now we are done with the write end of the pipe. CloseHandle() must
+ * not be called because the preceding close() closes the underlying
+ * handle.
+ */
+ syslogPipe[1] = 0;
#endif
- redirection_done = true;
- }
-
- /* postmaster will never write the file(s); close 'em */
- fclose(syslogFile);
- syslogFile = NULL;
- if (csvlogFile != NULL)
- {
- fclose(csvlogFile);
- csvlogFile = NULL;
- }
- if (jsonlogFile != NULL)
- {
- fclose(jsonlogFile);
- jsonlogFile = NULL;
- }
- return (int) sysloggerPid;
+ redirection_done = true;
}
- /* we should never reach here */
- return 0;
+ /* postmaster will never write the file(s); close 'em */
+ fclose(syslogFile);
+ syslogFile = NULL;
+ if (csvlogFile != NULL)
+ {
+ fclose(csvlogFile);
+ csvlogFile = NULL;
+ }
+ if (jsonlogFile != NULL)
+ {
+ fclose(jsonlogFile);
+ jsonlogFile = NULL;
+ }
+ return (int) sysloggerPid;
}
@@ -831,69 +843,6 @@ syslogger_fdopen(int fd)
return file;
}
-
-/*
- * syslogger_forkexec() -
- *
- * Format up the arglist for, then fork and exec, a syslogger process
- */
-static pid_t
-syslogger_forkexec(void)
-{
- char *av[10];
- int ac = 0;
- char filenobuf[32];
- char csvfilenobuf[32];
- char jsonfilenobuf[32];
-
- av[ac++] = "postgres";
- av[ac++] = "--forklog";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- /* static variables (those not passed by write_backend_variables) */
- snprintf(filenobuf, sizeof(filenobuf), "%d",
- syslogger_fdget(syslogFile));
- av[ac++] = filenobuf;
- snprintf(csvfilenobuf, sizeof(csvfilenobuf), "%d",
- syslogger_fdget(csvlogFile));
- av[ac++] = csvfilenobuf;
- snprintf(jsonfilenobuf, sizeof(jsonfilenobuf), "%d",
- syslogger_fdget(jsonlogFile));
- av[ac++] = jsonfilenobuf;
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-
-/*
- * syslogger_parseArgs() -
- *
- * Extract data from the arglist for exec'ed syslogger process
- */
-static void
-syslogger_parseArgs(int argc, char *argv[])
-{
- int fd;
-
- Assert(argc == 6);
- argv += 3;
-
- /*
- * Re-open the error output files that were opened by SysLogger_Start().
- *
- * We expect this will always succeed, which is too optimistic, but if it
- * fails there's not a lot we can do to report the problem anyway. As
- * coded, we'll just crash on a null pointer dereference after failure...
- */
- fd = atoi(*argv++);
- syslogFile = syslogger_fdopen(fd);
- fd = atoi(*argv++);
- csvlogFile = syslogger_fdopen(fd);
- fd = atoi(*argv++);
- jsonlogFile = syslogger_fdopen(fd);
-}
#endif /* EXEC_BACKEND */
diff --git a/src/backend/postmaster/walwriter.c b/src/backend/postmaster/walwriter.c
index 48bc92205b5..89950350aee 100644
--- a/src/backend/postmaster/walwriter.c
+++ b/src/backend/postmaster/walwriter.c
@@ -48,6 +48,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/interrupt.h"
#include "postmaster/walwriter.h"
#include "storage/bufmgr.h"
@@ -88,13 +89,19 @@ static void HandleWalWriterInterrupts(void);
* basic execution environment, but not enabled signals yet.
*/
void
-WalWriterMain(void)
+WalWriterMain(char *startup_data, size_t startup_data_len)
{
sigjmp_buf local_sigjmp_buf;
MemoryContext walwriter_context;
int left_till_hibernate;
bool hibernating;
+ Assert(startup_data_len == 0);
+
+ MyAuxProcType = WalWriterProcess;
+ MyBackendType = B_WAL_WRITER;
+ AuxiliaryProcessInit();
+
/*
* Properly accept or ignore signals the postmaster might send us
*
diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c
index 501910b4454..a4098c23b2b 100644
--- a/src/backend/replication/logical/launcher.c
+++ b/src/backend/replication/logical/launcher.c
@@ -30,7 +30,6 @@
#include "miscadmin.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "replication/logicallauncher.h"
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index 26ded928a71..9b01132704b 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -65,6 +65,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/interrupt.h"
#include "replication/walreceiver.h"
#include "replication/walsender.h"
@@ -183,7 +184,7 @@ ProcessWalRcvInterrupts(void)
/* Main entry point for walreceiver process */
void
-WalReceiverMain(void)
+WalReceiverMain(char *startup_data, size_t startup_data_len)
{
char conninfo[MAXCONNINFO];
char *tmp_conninfo;
@@ -199,6 +200,12 @@ WalReceiverMain(void)
char *sender_host = NULL;
int sender_port = 0;
+ Assert(startup_data_len == 0);
+
+ MyAuxProcType = WalReceiverProcess;
+ MyBackendType = B_WAL_RECEIVER;
+ AuxiliaryProcessInit();
+
/*
* WalRcv should be set up already (if we are a backend, we inherit this
* by fork() or EXEC_BACKEND mechanism from the postmaster).
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 5465fa19646..d65d461340c 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -144,6 +144,8 @@ InitShmemAllocation(void)
/*
* Initialize ShmemVariableCache for transaction manager. (This doesn't
* really belong here, but not worth moving.)
+ *
+ * XXX: we really should move this
*/
ShmemVariableCache = (VariableCache)
ShmemAlloc(sizeof(*ShmemVariableCache));
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 60bc1217fb4..b6c3055027e 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -44,6 +44,7 @@ volatile uint32 CritSectionCount = 0;
int MyProcPid;
pg_time_t MyStartTime;
TimestampTz MyStartTimestamp;
+struct ClientSocket *MyClientSocket;
struct Port *MyProcPort;
int32 MyCancelKey;
int MyPMChildSlot;
diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c
index cfc5afaa6fd..0cec9e54861 100644
--- a/src/backend/utils/init/miscinit.c
+++ b/src/backend/utils/init/miscinit.c
@@ -85,139 +85,6 @@ bool IgnoreSystemIndexes = false;
* ----------------------------------------------------------------
*/
-/*
- * Initialize the basic environment for a postmaster child
- *
- * Should be called as early as possible after the child's startup. However,
- * on EXEC_BACKEND builds it does need to be after read_backend_variables().
- */
-void
-InitPostmasterChild(void)
-{
- IsUnderPostmaster = true; /* we are a postmaster subprocess now */
-
- /*
- * Start our win32 signal implementation. This has to be done after we
- * read the backend variables, because we need to pick up the signal pipe
- * from the parent process.
- */
-#ifdef WIN32
- pgwin32_signal_initialize();
-#endif
-
- /*
- * Set reference point for stack-depth checking. This might seem
- * redundant in !EXEC_BACKEND builds; but it's not because the postmaster
- * launches its children from signal handlers, so we might be running on
- * an alternative stack.
- */
- (void) set_stack_base();
-
- InitProcessGlobals();
-
- /*
- * make sure stderr is in binary mode before anything can possibly be
- * written to it, in case it's actually the syslogger pipe, so the pipe
- * chunking protocol isn't disturbed. Non-logpipe data gets translated on
- * redirection (e.g. via pg_ctl -l) anyway.
- */
-#ifdef WIN32
- _setmode(fileno(stderr), _O_BINARY);
-#endif
-
- /* We don't want the postmaster's proc_exit() handlers */
- on_exit_reset();
-
- /* In EXEC_BACKEND case we will not have inherited BlockSig etc values */
-#ifdef EXEC_BACKEND
- pqinitmask();
-#endif
-
- /* Initialize process-local latch support */
- InitializeLatchSupport();
- InitProcessLocalLatch();
- InitializeLatchWaitSet();
-
- /*
- * If possible, make this process a group leader, so that the postmaster
- * can signal any child processes too. Not all processes will have
- * children, but for consistency we make all postmaster child processes do
- * this.
- */
-#ifdef HAVE_SETSID
- if (setsid() < 0)
- elog(FATAL, "setsid() failed: %m");
-#endif
-
- /*
- * Every postmaster child process is expected to respond promptly to
- * SIGQUIT at all times. Therefore we centrally remove SIGQUIT from
- * BlockSig and install a suitable signal handler. (Client-facing
- * processes may choose to replace this default choice of handler with
- * quickdie().) All other blockable signals remain blocked for now.
- */
- pqsignal(SIGQUIT, SignalHandlerForCrashExit);
-
- sigdelset(&BlockSig, SIGQUIT);
- sigprocmask(SIG_SETMASK, &BlockSig, NULL);
-
- /* Request a signal if the postmaster dies, if possible. */
- PostmasterDeathSignalInit();
-
- /* Don't give the pipe to subprograms that we execute. */
-#ifndef WIN32
- if (fcntl(postmaster_alive_fds[POSTMASTER_FD_WATCH], F_SETFD, FD_CLOEXEC) < 0)
- ereport(FATAL,
- (errcode_for_socket_access(),
- errmsg_internal("could not set postmaster death monitoring pipe to FD_CLOEXEC mode: %m")));
-#endif
-}
-
-/*
- * Initialize the basic environment for a standalone process.
- *
- * argv0 has to be suitable to find the program's executable.
- */
-void
-InitStandaloneProcess(const char *argv0)
-{
- Assert(!IsPostmasterEnvironment);
-
- MyBackendType = B_STANDALONE_BACKEND;
-
- /*
- * Start our win32 signal implementation
- */
-#ifdef WIN32
- pgwin32_signal_initialize();
-#endif
-
- InitProcessGlobals();
-
- /* Initialize process-local latch support */
- InitializeLatchSupport();
- InitProcessLocalLatch();
- InitializeLatchWaitSet();
-
- /*
- * For consistency with InitPostmasterChild, initialize signal mask here.
- * But we don't unblock SIGQUIT or provide a default handler for it.
- */
- pqinitmask();
- sigprocmask(SIG_SETMASK, &BlockSig, NULL);
-
- /* Compute paths, no postmaster to inherit from */
- if (my_exec_path[0] == '\0')
- {
- if (find_my_exec(argv0, my_exec_path) < 0)
- elog(FATAL, "%s: could not locate my own executable path",
- argv0);
- }
-
- if (pkglib_path[0] == '\0')
- get_pkglib_path(my_exec_path, pkglib_path);
-}
-
void
SwitchToSharedLatch(void)
{
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 9b6d8fc5571..959feeba025 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -132,10 +132,11 @@ typedef struct ClientConnectionInfo
typedef struct Port
{
pgsocket sock; /* File descriptor */
- bool noblock; /* is the socket in non-blocking mode? */
- ProtocolVersion proto; /* FE/BE protocol version */
SockAddr laddr; /* local addr (postmaster) */
SockAddr raddr; /* remote addr (client) */
+
+ bool noblock; /* is the socket in non-blocking mode? */
+ ProtocolVersion proto; /* FE/BE protocol version */
char *remote_host; /* name (or ip addr) of remote host */
char *remote_hostname; /* name (not ip addr) of remote host, if
* available */
@@ -218,11 +219,13 @@ typedef struct Port
* ClientSocket holds a socket for an accepted connection, along with the
* information about the endpoints.
*/
-typedef struct ClientSocket {
+struct ClientSocket
+{
pgsocket sock; /* File descriptor */
SockAddr laddr; /* local addr (postmaster) */
SockAddr raddr; /* remote addr (client) */
-} ClientSocket;
+};
+typedef struct ClientSocket ClientSocket;
#ifdef USE_SSL
/*
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index f0cc651435c..a179cb39c40 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -319,7 +319,6 @@ extern int trace_recovery(int trace_level);
extern PGDLLIMPORT char *DatabasePath;
/* now in utils/init/miscinit.c */
-extern void InitPostmasterChild(void);
extern void InitStandaloneProcess(const char *argv0);
extern void InitProcessLocalLatch(void);
extern void SwitchToSharedLatch(void);
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index b553e858ad4..ee12e477f11 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -55,20 +55,14 @@ extern bool IsAutoVacuumWorkerProcess(void);
#define IsAnyAutoVacuumProcess() \
(IsAutoVacuumLauncherProcess() || IsAutoVacuumWorkerProcess())
-/* Functions to start autovacuum process, called from postmaster */
+/* called from postmaster at server startup */
extern void autovac_init(void);
-extern int StartAutoVacLauncher(void);
-extern int StartAutoVacWorker(void);
/* called from postmaster when a worker could not be forked */
extern void AutoVacWorkerFailed(void);
-#ifdef EXEC_BACKEND
-extern void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
-extern void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
-extern void AutovacuumWorkerIAm(void);
-extern void AutovacuumLauncherIAm(void);
-#endif
+extern void AutoVacLauncherMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
+extern void AutoVacWorkerMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern bool AutoVacuumRequestWork(AutoVacuumWorkItemType type,
Oid relationId, BlockNumber blkno);
diff --git a/src/include/postmaster/auxprocess.h b/src/include/postmaster/auxprocess.h
index 5c2d6527ff6..75394ca0155 100644
--- a/src/include/postmaster/auxprocess.h
+++ b/src/include/postmaster/auxprocess.h
@@ -13,8 +13,6 @@
#ifndef AUXPROCESS_H
#define AUXPROCESS_H
-#include "miscadmin.h"
-
-extern void AuxiliaryProcessMain(AuxProcType auxtype) pg_attribute_noreturn();
+extern void AuxiliaryProcessInit(void);
#endif /* AUXPROCESS_H */
diff --git a/src/include/postmaster/bgworker_internals.h b/src/include/postmaster/bgworker_internals.h
index 323f1e07291..4055d2f5626 100644
--- a/src/include/postmaster/bgworker_internals.h
+++ b/src/include/postmaster/bgworker_internals.h
@@ -55,6 +55,6 @@ extern void ForgetUnstartedBackgroundWorkers(void);
extern void ResetBackgroundWorkerCrashTimes(void);
/* Entry point for background worker processes */
-extern void BackgroundWorkerMain(void) pg_attribute_noreturn();
+extern void BackgroundWorkerMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
#endif /* BGWORKER_INTERNALS_H */
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index a66722873f4..ee54fc401ef 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -27,8 +27,8 @@ extern PGDLLIMPORT int CheckPointTimeout;
extern PGDLLIMPORT int CheckPointWarning;
extern PGDLLIMPORT double CheckPointCompletionTarget;
-extern void BackgroundWriterMain(void) pg_attribute_noreturn();
-extern void CheckpointerMain(void) pg_attribute_noreturn();
+extern void BackgroundWriterMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
+extern void CheckpointerMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern void RequestCheckpoint(int flags);
extern void CheckpointWriteDelay(int flags, double progress);
diff --git a/src/include/postmaster/fork_process.h b/src/include/postmaster/fork_process.h
deleted file mode 100644
index 12decc8133b..00000000000
--- a/src/include/postmaster/fork_process.h
+++ /dev/null
@@ -1,17 +0,0 @@
-/*-------------------------------------------------------------------------
- *
- * fork_process.h
- * Exports from postmaster/fork_process.c.
- *
- * Copyright (c) 1996-2023, PostgreSQL Global Development Group
- *
- * src/include/postmaster/fork_process.h
- *
- *-------------------------------------------------------------------------
- */
-#ifndef FORK_PROCESS_H
-#define FORK_PROCESS_H
-
-extern pid_t fork_process(void);
-
-#endif /* FORK_PROCESS_H */
diff --git a/src/include/postmaster/pgarch.h b/src/include/postmaster/pgarch.h
index 3bd4fac71e5..577fc14e1d0 100644
--- a/src/include/postmaster/pgarch.h
+++ b/src/include/postmaster/pgarch.h
@@ -29,7 +29,7 @@
extern Size PgArchShmemSize(void);
extern void PgArchShmemInit(void);
extern bool PgArchCanRestart(void);
-extern void PgArchiverMain(void) pg_attribute_noreturn();
+extern void PgArchiverMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern void PgArchWakeup(void);
extern void PgArchForceDirScan(void);
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index 3b3889c58c0..1dbb5c991f5 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -58,14 +58,52 @@ extern int MaxLivePostmasterChildren(void);
extern bool PostmasterMarkPIDForWorkerNotify(int);
-#ifdef EXEC_BACKEND
-extern pid_t postmaster_forkexec(int argc, char *argv[]);
-extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
+extern void BackendMain(char *startup_data, size_t startup_data_len);
+#ifdef EXEC_BACKEND
extern Size ShmemBackendArraySize(void);
extern void ShmemBackendArrayAllocation(void);
#endif
+/* in launch_backend.c */
+
+/* this better match the list in launch_backend.c */
+typedef enum PostmasterChildType
+{
+ PMC_BACKEND = 0,
+ PMC_AV_LAUNCHER,
+ PMC_AV_WORKER,
+ PMC_BGWORKER,
+ PMC_SYSLOGGER,
+
+ /*
+ * so-called "aux processes". These access shared memory, but are not
+ * attached to any particular database. Only one of each of these can be
+ * running at a time.
+ */
+ PMC_STARTUP,
+ PMC_BGWRITER,
+ PMC_ARCHIVER,
+ PMC_CHECKPOINTER,
+ PMC_WAL_WRITER,
+ PMC_WAL_RECEIVER,
+} PostmasterChildType;
+
+/* defined in libpq-be.h */
+extern struct ClientSocket *MyClientSocket;
+
+extern pid_t postmaster_child_launch(PostmasterChildType child_type, char *startup_data, size_t startup_data_len, struct ClientSocket *sock);
+
+#ifdef EXEC_BACKEND
+extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
+#endif
+
+#ifdef WIN32
+extern void pgwin32_register_deadchild_callback(HANDLE procHandle, DWORD procId);
+#endif
+
+const char *PostmasterChildName(PostmasterChildType child_type);
+
/*
* Note: MAX_BACKENDS is limited to 2^18-1 because that's the width reserved
* for buffer references in buf_internals.h. This limitation could be lifted
diff --git a/src/include/postmaster/startup.h b/src/include/postmaster/startup.h
index 6a2e4c4526b..ec885063aab 100644
--- a/src/include/postmaster/startup.h
+++ b/src/include/postmaster/startup.h
@@ -26,7 +26,7 @@
extern PGDLLIMPORT int log_startup_progress_interval;
extern void HandleStartupProcInterrupts(void);
-extern void StartupProcessMain(void) pg_attribute_noreturn();
+extern void StartupProcessMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern void PreRestoreCommand(void);
extern void PostRestoreCommand(void);
extern bool IsPromoteSignaled(void);
diff --git a/src/include/postmaster/syslogger.h b/src/include/postmaster/syslogger.h
index 34da778f1ef..7dc41b30e7c 100644
--- a/src/include/postmaster/syslogger.h
+++ b/src/include/postmaster/syslogger.h
@@ -86,9 +86,7 @@ extern int SysLogger_Start(void);
extern void write_syslogger_file(const char *buffer, int count, int destination);
-#ifdef EXEC_BACKEND
-extern void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
+extern void SysLoggerMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern bool CheckLogrotateSignal(void);
extern void RemoveLogrotateSignalFiles(void);
diff --git a/src/include/postmaster/walwriter.h b/src/include/postmaster/walwriter.h
index 6eba7ad79cf..99b7cc07fb2 100644
--- a/src/include/postmaster/walwriter.h
+++ b/src/include/postmaster/walwriter.h
@@ -18,6 +18,6 @@
extern PGDLLIMPORT int WalWriterDelay;
extern PGDLLIMPORT int WalWriterFlushAfter;
-extern void WalWriterMain(void) pg_attribute_noreturn();
+extern void WalWriterMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
#endif /* _WALWRITER_H */
diff --git a/src/include/replication/walreceiver.h b/src/include/replication/walreceiver.h
index 04b439dc503..4b76062255b 100644
--- a/src/include/replication/walreceiver.h
+++ b/src/include/replication/walreceiver.h
@@ -457,7 +457,7 @@ walrcv_clear_result(WalRcvExecResult *walres)
}
/* prototypes for functions in walreceiver.c */
-extern void WalReceiverMain(void) pg_attribute_noreturn();
+extern void WalReceiverMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern void ProcessWalRcvInterrupts(void);
extern void WalRcvForceReply(void);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 00649608e46..cfc3f259c95 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2121,6 +2121,7 @@ PortalStrategy
PostParseColumnRefHook
PostgresPollingStatusType
PostingItem
+PostmasterChildType
PreParseColumnRefHook
PredClass
PredIterInfo
@@ -3793,6 +3794,7 @@ substitute_actual_parameters_context
substitute_actual_srf_parameters_context
substitute_phv_relids_context
symbol
+syslogger_startup_data
tablespaceinfo
teSection
temp_tablespaces_extra
--
2.39.2
Re: Refactoring backend fork+exec code
On Wed Nov 29, 2023 at 5:36 PM CST, Heikki Linnakangas wrote:
On 11/10/2023 14:12, Heikki Linnakangas wrote:
Here's another rebased patch set. Compared to previous version, I did a
little more refactoring around CreateSharedMemoryAndSemaphores and
InitProcess:- patch 1 splits CreateSharedMemoryAndSemaphores into two functions:
CreateSharedMemoryAndSemaphores is now only called at postmaster
startup, and a new function called AttachSharedMemoryStructs() is called
in backends in EXEC_BACKEND mode. I extracted the common parts of those
functions to a new static function. (Some of this refactoring used to be
part of the 3rd patch in the series, but it seems useful on its own, so
I split it out.)- patch 3 moves the call to AttachSharedMemoryStructs() to
InitProcess(), reducing the boilerplate code a little.The patches are incrementally useful, so if you don't have time to
review all of them, a review on a subset would be useful too.
From 8886db1ed6bae21bf6d77c9bb1230edbb55e24f9 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Thu, 30 Nov 2023 00:04:22 +0200
Subject: [PATCH v3 4/7] Pass CAC as argument to backend process
For me, being new to the code, it would be nice to have more of an
explanation as to why this is "better." I don't doubt it; it would just
help me and future readers of this commit in the future. More of an
explanation in the commit message would suffice.
My other comment on this commit is that we now seem to have lost the
context on what CAC stands for. Before we had the member variable to
explain it. A comment on the enum would be great or changing cac named
variables to canAcceptConnections. I did notice in patch 7 that there
are still some variables named canAcceptConnections around, so I'll
leave this comment up to you.
From 98f8397b32a0b36e221475b32697c9c5bbca86a0 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 11 Oct 2023 13:38:06 +0300
Subject: [PATCH v3 5/7] Remove ConnCreate and ConnFree, and allocate Port in
stack
I like it separate.
From 79aab42705a8cb0e16e61c33052fc56fdd4fca76 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 11 Oct 2023 13:38:10 +0300
Subject: [PATCH v3 6/7] Introduce ClientSocket, rename some funcs
+static int BackendStartup(ClientSocket *port);
s/port/client_sock
- port->remote_hostname = strdup(remote_host); + port->remote_hostname = pstrdup(remote_host); + MemoryContextSwitchTo(oldcontext);
Something funky with the whitespace here, but my eyes might also be
playing tricks on me. Mixing spaces in tabs like what do in this
codebase makes it difficult to review :).
From ce51876f87f1e4317e25baf64184749448fcd033 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Thu, 30 Nov 2023 00:07:34 +0200
Subject: [PATCH v3 7/7] Refactor postmaster child process launching
+ entry_kinds[child_type].main_fn(startup_data, startup_data_len); + Assert(false);
Seems like you want the pg_unreachable() macro here instead of
Assert(false). Similar comment at the end of SubPostmasterMain().
+ if (fwrite(param, paramsz, 1, fp) != 1) + { + ereport(LOG, + (errcode_for_file_access(), + errmsg("could not write to file \"%s\": %m", tmpfilename))); + FreeFile(fp); + return -1; + } + + /* Release file */ + if (FreeFile(fp)) + { + ereport(LOG, + (errcode_for_file_access(), + errmsg("could not write to file \"%s\": %m", tmpfilename))); + return -1; + }
Two pieces of feedback here. I generally find write(2) more useful than
fwrite(3) because write(2) will report a useful errno, whereas fwrite(2)
just uses ferror(3). The additional errno information might be valuable
context in the log message. Up to you if you think it is also valuable.
The log message if FreeFile() fails doesn't seem to make sense to me.
I didn't see any file writing in that code path, but it is possible that
I missed something.
+ /* + * Set reference point for stack-depth checking. This might seem + * redundant in !EXEC_BACKEND builds; but it's not because the postmaster + * launches its children from signal handlers, so we might be running on + * an alternative stack. XXX still true? + */ + (void) set_stack_base();
Looks like there is still this XXX left. Can't say I completely
understand the second sentence either.
+ /* + * make sure stderr is in binary mode before anything can possibly be + * written to it, in case it's actually the syslogger pipe, so the pipe + * chunking protocol isn't disturbed. Non-logpipe data gets translated on + * redirection (e.g. via pg_ctl -l) anyway. + */
Nit: The 'm' in the first "make" should be capitalized.
+ if (fread(¶m, sizeof(param), 1, fp) != 1) + { + write_stderr("could not read from backend variables file \"%s\": %s\n", + id, strerror(errno)); + exit(1); + } + + /* read startup data */ + *startup_data_len = param.startup_data_len; + if (param.startup_data_len > 0) + { + *startup_data = palloc(*startup_data_len); + if (fread(*startup_data, *startup_data_len, 1, fp) != 1) + { + write_stderr("could not read startup data from backend variables file \"%s\": %s\n", + id, strerror(errno)); + exit(1); + } + }
fread(3) doesn't set errno. I would probably switch these to read(2) for
the reason I wrote in a previous comment.
+ /* + * Need to reinitialize the SSL library in the backend, since the context + * structures contain function pointers and cannot be passed through the + * parameter file. + * + * If for some reason reload fails (maybe the user installed broken key + * files), soldier on without SSL; that's better than all connections + * becoming impossible. + * + * XXX should we do this in all child processes? For the moment it's + * enough to do it in backend children. + */ +#ifdef USE_SSL + if (EnableSSL) + { + if (secure_initialize(false) == 0) + LoadedSSL = true; + else + ereport(LOG, + (errmsg("SSL configuration could not be loaded in child process"))); + } +#endif
Do other child process types do any non-local communication?
-typedef struct ClientSocket { +struct ClientSocket +{ pgsocket sock; /* File descriptor */ SockAddr laddr; /* local addr (postmaster) */ SockAddr raddr; /* remote addr (client) */ -} ClientSocket; +}; +typedef struct ClientSocket ClientSocket;
Can't say I completely understand the reason for this change given it
was added in your series.
I didn't look too hard at the Windows-specific code, so maybe someone
who knows Windows will have something to say, but it also might've just
been copy-paste that I didn't realize.
There were a few more XXXs that probably should be figured out before
committing. Though perhaps some of them were already there.
Patches 1-3 seem committable as-is. I only had minor comments on
everything but 7, so after taking a look at those, they could be
committed.
Overall, this seems liked a marked improvement :).
--
Tristan Partin
Neon (https://neon.tech)
Re: Refactoring backend fork+exec code
Hi,
On 2023-11-30 01:36:25 +0200, Heikki Linnakangas wrote:
- patch 1 splits CreateSharedMemoryAndSemaphores into two functions:
CreateSharedMemoryAndSemaphores is now only called at postmaster startup,
and a new function called AttachSharedMemoryStructs() is called in backends
in EXEC_BACKEND mode. I extracted the common parts of those functions to a
new static function. (Some of this refactoring used to be part of the 3rd
patch in the series, but it seems useful on its own, so I split it out.)
I like that idea.
From a96b6e92fdeaa947bf32774c425419b8f987b8e2 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Thu, 30 Nov 2023 00:01:25 +0200
Subject: [PATCH v3 1/7] Refactor CreateSharedMemoryAndSemaphoresFor clarity, have separate functions for *creating* the shared memory
and semaphores at postmaster or single-user backend startup, and
for *attaching* to existing shared memory structures in EXEC_BACKEND
case. CreateSharedMemoryAndSemaphores() is now called only at
postmaster startup, and a new AttachSharedMemoryStructs() function is
called at backend startup in EXEC_BACKEND mode.
I assume CreateSharedMemoryAndSemaphores() is still called during crash
restart? I wonder if it shouldn't split three ways:
1) create
2) initialize
3) attach
From 3478cafcf74a5c8d649e0287e6c72669a29c0e70 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Thu, 30 Nov 2023 00:02:03 +0200
Subject: [PATCH v3 2/7] Pass BackgroundWorker entry in the parameter file in
EXEC_BACKEND modeThis makes it possible to move InitProcess later in SubPostmasterMain
(in next commit), as we no longer need to access shared memory to read
background worker entry.
static void read_backend_variables(char *id, Port *port); @@ -4831,7 +4833,7 @@ SubPostmasterMain(int argc, char *argv[]) strcmp(argv[1], "--forkavlauncher") == 0 || strcmp(argv[1], "--forkavworker") == 0 || strcmp(argv[1], "--forkaux") == 0 || - strncmp(argv[1], "--forkbgworker=", 15) == 0) + strncmp(argv[1], "--forkbgworker", 14) == 0) PGSharedMemoryReAttach(); else PGSharedMemoryNoReAttach(); @@ -4962,10 +4964,8 @@ SubPostmasterMain(int argc, char *argv[])AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */ } - if (strncmp(argv[1], "--forkbgworker=", 15) == 0) + if (strncmp(argv[1], "--forkbgworker", 14) == 0)
Now that we don't need to look at parameters anymore, these should probably be
just a strcmp(), like the other cases?
From 0d071474e12a70ff8113c7b0731c5b97fec45007 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 29 Nov 2023 23:47:25 +0200
Subject: [PATCH v3 3/7] Refactor how InitProcess is calledThe order of process initialization steps is now more consistent
between !EXEC_BACKEND and EXEC_BACKEND modes. InitProcess() is called
at the same place in either mode. We can now also move the
AttachSharedMemoryStructs() call into InitProcess() itself. This
reduces the number of "#ifdef EXEC_BACKEND" blocks.
Yay.
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c index cdfdd6fbe1d..6c708777dde 100644 --- a/src/backend/storage/lmgr/proc.c +++ b/src/backend/storage/lmgr/proc.c @@ -461,6 +461,12 @@ InitProcess(void) */ InitLWLockAccess(); InitDeadLockChecking(); + +#ifdef EXEC_BACKEND + /* Attach process to shared data structures */ + if (IsUnderPostmaster) + AttachSharedMemoryStructs(); +#endif }/* @@ -614,6 +620,12 @@ InitAuxiliaryProcess(void) * Arrange to clean up at process exit. */ on_shmem_exit(AuxiliaryProcKill, Int32GetDatum(proctype)); + +#ifdef EXEC_BACKEND + /* Attach process to shared data structures */ + if (IsUnderPostmaster) + AttachSharedMemoryStructs(); +#endif }
Aside: Somewhat odd that InitAuxiliaryProcess() doesn't call
InitLWLockAccess().
I think a short comment explaining why we can attach to shmem structs after
already accessing shared memory earlier in the function would be worthwhile.
From ce51876f87f1e4317e25baf64184749448fcd033 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Thu, 30 Nov 2023 00:07:34 +0200
Subject: [PATCH v3 7/7] Refactor postmaster child process launching- Move code related to launching backend processes to new source file,
launch_backend.c- Introduce new postmaster_child_launch() function that deals with the
differences between EXEC_BACKEND and fork mode.- Refactor the mechanism of passing information from the parent to
child process. Instead of using different command-line arguments when
launching the child process in EXEC_BACKEND mode, pass a
variable-length blob of data along with all the global variables. The
contents of that blob depend on the kind of child process being
launched. In !EXEC_BACKEND mode, we use the same blob, but it's simply
inherited from the parent to child process.[...]
33 files changed, 1787 insertions(+), 2002 deletions(-)
Well, that's not small...
I think it may be worth splitting some of the file renaming out into a
separate commit, makes it harder to see what changed here.
+AutoVacLauncherMain(char *startup_data, size_t startup_data_len) { - pid_t AutoVacPID; + sigjmp_buf local_sigjmp_buf;-#ifdef EXEC_BACKEND - switch ((AutoVacPID = avlauncher_forkexec())) -#else - switch ((AutoVacPID = fork_process())) -#endif + /* Release postmaster's working memory context */ + if (PostmasterContext) { - case -1: - ereport(LOG, - (errmsg("could not fork autovacuum launcher process: %m"))); - return 0; - -#ifndef EXEC_BACKEND - case 0: - /* in postmaster child ... */ - InitPostmasterChild(); - - /* Close the postmaster's sockets */ - ClosePostmasterPorts(false); - - AutoVacLauncherMain(0, NULL); - break; -#endif - default: - return (int) AutoVacPID; + MemoryContextDelete(PostmasterContext); + PostmasterContext = NULL; }- /* shouldn't get here */
- return 0;
-}
This if (PostmasterContext) ... else "shouldn't get here" business seems
pretty silly, more likely to hide problems than to help.
+/* + * Information needed to launch different kinds of child processes. + */ +static const struct +{ + const char *name; + void (*main_fn) (char *startup_data, size_t startup_data_len); + bool shmem_attach; +} entry_kinds[] = { + [PMC_BACKEND] = {"backend", BackendMain, true},
Personally I'd give the struct an actual name - makes the debugging experience
a bit nicer than anonymous structs that you can't even reference by a typedef.
+ [PMC_AV_LAUNCHER] = {"autovacuum launcher", AutoVacLauncherMain, true}, + [PMC_AV_WORKER] = {"autovacuum worker", AutoVacWorkerMain, true}, + [PMC_BGWORKER] = {"bgworker", BackgroundWorkerMain, true}, + [PMC_SYSLOGGER] = {"syslogger", SysLoggerMain, false}, + + [PMC_STARTUP] = {"startup", StartupProcessMain, true}, + [PMC_BGWRITER] = {"bgwriter", BackgroundWriterMain, true}, + [PMC_ARCHIVER] = {"archiver", PgArchiverMain, true}, + [PMC_CHECKPOINTER] = {"checkpointer", CheckpointerMain, true}, + [PMC_WAL_WRITER] = {"wal_writer", WalWriterMain, true}, + [PMC_WAL_RECEIVER] = {"wal_receiver", WalReceiverMain, true}, +};
It feels like we have too many different ways of documenting the type of a
process. This new PMC_ stuff, enum AuxProcType, enum BackendType. Which then
leads to code like this:
-CheckpointerMain(void) +CheckpointerMain(char *startup_data, size_t startup_data_len) { sigjmp_buf local_sigjmp_buf; MemoryContext checkpointer_context;+ Assert(startup_data_len == 0); + + MyAuxProcType = CheckpointerProcess; + MyBackendType = B_CHECKPOINTER; + AuxiliaryProcessInit(); +
For each type of child process. That seems a bit too redundant. Can't we
unify this at least somewhat? Can't we just reuse BackendType here? Sure,
there'd be pointless entry for B_INVALID, but that doesn't seem like a
problem, could even be useful, by pointing it to a function raising an error.
At the very least this shouldn't deviate from the naming pattern of
BackendType.
+/* + * SubPostmasterMain -- Get the fork/exec'd process into a state equivalent + * to what it would be if we'd simply forked on Unix, and then + * dispatch to the appropriate place. + * + * The first two command line arguments are expected to be "--forkchild=<name>", + * where <name> indicates which postmaster child we are to become, and + * the name of a variables file that we can read to load data that would + * have been inherited by fork() on Unix. + */ +void +SubPostmasterMain(int argc, char *argv[]) +{ + PostmasterChildType child_type; + char *startup_data; + size_t startup_data_len; + char *entry_name; + bool found = false; + + /* In EXEC_BACKEND case we will not have inherited these settings */ + IsPostmasterEnvironment = true; + whereToSendOutput = DestNone; + + /* Setup essential subsystems (to ensure elog() behaves sanely) */ + InitializeGUCOptions(); + + /* Check we got appropriate args */ + if (argc != 3) + elog(FATAL, "invalid subpostmaster invocation"); + + if (strncmp(argv[1], "--forkchild=", 12) != 0) + elog(FATAL, "invalid subpostmaster invocation (--forkchild argument missing)"); + entry_name = argv[1] + 12; + found = false; + for (int idx = 0; idx < lengthof(entry_kinds); idx++) + { + if (strcmp(entry_kinds[idx].name, entry_name) == 0) + { + child_type = idx; + found = true; + break; + } + } + if (!found) + elog(ERROR, "unknown child kind %s", entry_name);
If we then have to search linearly, why don't we just pass the index into the
array?
-#define StartupDataBase() StartChildProcess(StartupProcess) -#define StartArchiver() StartChildProcess(ArchiverProcess) -#define StartBackgroundWriter() StartChildProcess(BgWriterProcess) -#define StartCheckpointer() StartChildProcess(CheckpointerProcess) -#define StartWalWriter() StartChildProcess(WalWriterProcess) -#define StartWalReceiver() StartChildProcess(WalReceiverProcess) +#define StartupDataBase() StartChildProcess(PMC_STARTUP) +#define StartArchiver() StartChildProcess(PMC_ARCHIVER) +#define StartBackgroundWriter() StartChildProcess(PMC_BGWRITER) +#define StartCheckpointer() StartChildProcess(PMC_CHECKPOINTER) +#define StartWalWriter() StartChildProcess(PMC_WAL_WRITER) +#define StartWalReceiver() StartChildProcess(PMC_WAL_RECEIVER) + +#define StartAutoVacLauncher() StartChildProcess(PMC_AV_LAUNCHER); +#define StartAutoVacWorker() StartChildProcess(PMC_AV_WORKER);
Obviously not your fault, but these macros are so pointless... Making it
harder to find where we start child processes, all to save a a few characters
in one place, while adding considerably more in others.
+void +BackendMain(char *startup_data, size_t startup_data_len) +{
Is there any remaining reason for this to live in postmaster.c? Given that
other backend types don't, that seems oddly assymmetrical.
Greetings,
Andres Freund
Re: Refactoring backend fork+exec code
Hi,
On 2023-11-30 12:44:33 -0600, Tristan Partin wrote:
+ /* + * Set reference point for stack-depth checking. This might seem + * redundant in !EXEC_BACKEND builds; but it's not because the postmaster + * launches its children from signal handlers, so we might be running on + * an alternative stack. XXX still true? + */ + (void) set_stack_base();Looks like there is still this XXX left. Can't say I completely understand
the second sentence either.
We used to start some child processes of postmaster in signal handlers. That
was fixed in
commit 7389aad6366
Author: Thomas Munro <tmunro@postgresql.org>
Date: 2023-01-12 12:34:23 +1300
Use WaitEventSet API for postmaster's event loop.
In some cases signal handlers run on a separate stack, which meant that the
set_stack_base() we did in postmaster would yield a completely bogus stack
depth estimation. So this comment should likely have been removed. Thomas?
Greetings,
Andres Freund
Re: Refactoring backend fork+exec code
On Fri, Dec 1, 2023 at 9:31 AM Andres Freund <andres@anarazel.de> wrote:
On 2023-11-30 12:44:33 -0600, Tristan Partin wrote:
+ /* + * Set reference point for stack-depth checking. This might seem + * redundant in !EXEC_BACKEND builds; but it's not because the postmaster + * launches its children from signal handlers, so we might be running on + * an alternative stack. XXX still true? + */ + (void) set_stack_base();Looks like there is still this XXX left. Can't say I completely understand
the second sentence either.We used to start some child processes of postmaster in signal handlers. That
was fixed incommit 7389aad6366
Author: Thomas Munro <tmunro@postgresql.org>
Date: 2023-01-12 12:34:23 +1300Use WaitEventSet API for postmaster's event loop.
In some cases signal handlers run on a separate stack, which meant that the
set_stack_base() we did in postmaster would yield a completely bogus stack
depth estimation. So this comment should likely have been removed. Thomas?
Right, I should delete that comment in master and 16. While wondering
what to write instead, my first thought is that it is better to leave
the actual call there though, because otherwise there is a small
difference in stack reference point between EXEC_BACKEND and
!EXEC_BACKEND builds, consumed by a few postmaster stack frames. So
the new comment would just say that.
(I did idly wonder if there is a longjmp trick to zap those frames
post-fork, not looked into and probably doesn't really improve
anything we care about...)
Re: Refactoring backend fork+exec code
On 30/11/2023 22:26, Andres Freund wrote:
Aside: Somewhat odd that InitAuxiliaryProcess() doesn't call
InitLWLockAccess().
Yeah that caught my eye too.
It seems to have been an oversight in commit 1c6821be31f. Before that,
in 9.4, the lwlock stats were printed for aux processes too, on shutdown.
Committed a fix for that to master.
--
Heikki Linnakangas
Neon (https://neon.tech)
Re: Refactoring backend fork+exec code
On 30/11/2023 22:26, Andres Freund wrote:
On 2023-11-30 01:36:25 +0200, Heikki Linnakangas wrote:
From a96b6e92fdeaa947bf32774c425419b8f987b8e2 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Thu, 30 Nov 2023 00:01:25 +0200
Subject: [PATCH v3 1/7] Refactor CreateSharedMemoryAndSemaphoresFor clarity, have separate functions for *creating* the shared memory
and semaphores at postmaster or single-user backend startup, and
for *attaching* to existing shared memory structures in EXEC_BACKEND
case. CreateSharedMemoryAndSemaphores() is now called only at
postmaster startup, and a new AttachSharedMemoryStructs() function is
called at backend startup in EXEC_BACKEND mode.I assume CreateSharedMemoryAndSemaphores() is still called during crash
restart?
Yes.
I wonder if it shouldn't split three ways:
1) create
2) initialize
3) attach
Why? What would be the difference between create and initialize phases?
--
Heikki Linnakangas
Neon (https://neon.tech)
Re: Refactoring backend fork+exec code
Hi,
On 2023-12-01 01:36:13 +0200, Heikki Linnakangas wrote:
On 30/11/2023 22:26, Andres Freund wrote:
On 2023-11-30 01:36:25 +0200, Heikki Linnakangas wrote:
From a96b6e92fdeaa947bf32774c425419b8f987b8e2 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Thu, 30 Nov 2023 00:01:25 +0200
Subject: [PATCH v3 1/7] Refactor CreateSharedMemoryAndSemaphoresFor clarity, have separate functions for *creating* the shared memory
and semaphores at postmaster or single-user backend startup, and
for *attaching* to existing shared memory structures in EXEC_BACKEND
case. CreateSharedMemoryAndSemaphores() is now called only at
postmaster startup, and a new AttachSharedMemoryStructs() function is
called at backend startup in EXEC_BACKEND mode.I assume CreateSharedMemoryAndSemaphores() is still called during crash
restart?Yes.
I wonder if it shouldn't split three ways:
1) create
2) initialize
3) attachWhy? What would be the difference between create and initialize phases?
Mainly because I somehow mis-remembered how we deal with the shared memory
allocation when crashing. I somehow had remembered that we reused the same
allocation across restarts, but reinitialized it from scratch. There's a
kernel of truth to that, because we can end up re-attaching to an existing
sysv shared memory segment. But not more. Perhaps I was confusing things with
the listen sockets?
Andres
Re: Refactoring backend fork+exec code
On 30/11/2023 20:44, Tristan Partin wrote:
Patches 1-3 seem committable as-is.
Thanks for the review! I'm focusing on patches 1-3 now, and will come
back to the rest after committing 1-3.
There was one test failure with EXEC_BACKEND from patch 2, in
'test_shm_mq'. In restore_backend_variables() I checked if 'bgw_name' is
empty to decide if the BackgroundWorker struct is filled in or not, but
it turns out that 'test_shm_mq' doesn't fill in bgw_name. It probably
should, I think that's an oversight in 'test_shm_mq', but that's a
separate issue.
I did some more refactoring of patch 2, to fix that and to improve it in
general. The BackgroundWorker struct is now passed through the
fork-related functions similarly to the Port struct. That seems more
consistent.
Attached is new version of these patches. For easier review, I made the
new refactorings compared in a new commit 0003. I will squash that
before pushing, but this makes it easier to see what changed.
Barring any new feedback or issues, I will commit these.
--
Heikki Linnakangas
Neon (https://neon.tech)
Attachments:
v4-0001-Refactor-CreateSharedMemoryAndSemaphores.patchtext/x-patch; charset=UTF-8; name=v4-0001-Refactor-CreateSharedMemoryAndSemaphores.patchDownload
From 6dd44d7057e535eb441c9ab8fda1bbdd8079f2fb Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Thu, 30 Nov 2023 00:01:25 +0200
Subject: [PATCH v4 1/4] Refactor CreateSharedMemoryAndSemaphores
For clarity, have separate functions for *creating* the shared memory
and semaphores at postmaster or single-user backend startup, and
for *attaching* to existing shared memory structures in EXEC_BACKEND
case. CreateSharedMemoryAndSemaphores() is now called only at
postmaster startup, and a new AttachSharedMemoryStructs() function is
called at backend startup in EXEC_BACKEND mode.
Reviewed-by: Tristan Partin, Andres Freund
Discussion: https://www.postgresql.org/message-id/7a59b073-5b5b-151e-7ed3-8b01ff7ce9ef@iki.fi
---
src/backend/postmaster/postmaster.c | 20 +--
src/backend/replication/walreceiver.c | 3 +-
src/backend/storage/ipc/ipci.c | 178 +++++++++++++++-----------
src/backend/storage/lmgr/proc.c | 2 +-
src/include/storage/ipc.h | 3 +
5 files changed, 117 insertions(+), 89 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 7a5cd06c5c9..b03e88e6702 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -4909,11 +4909,11 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ /* Need a PGPROC to run AttachSharedMemoryStructs */
InitProcess();
/* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
+ AttachSharedMemoryStructs();
/* And run the backend */
BackendRun(&port); /* does not return */
@@ -4927,11 +4927,11 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ /* Need a PGPROC to run AttachSharedMemoryStructs */
InitAuxiliaryProcess();
/* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
+ AttachSharedMemoryStructs();
auxtype = atoi(argv[3]);
AuxiliaryProcessMain(auxtype); /* does not return */
@@ -4941,11 +4941,11 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ /* Need a PGPROC to run AttachSharedMemoryStructs */
InitProcess();
/* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
+ AttachSharedMemoryStructs();
AutoVacLauncherMain(argc - 2, argv + 2); /* does not return */
}
@@ -4954,11 +4954,11 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ /* Need a PGPROC to run AttachSharedMemoryStructs */
InitProcess();
/* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
+ AttachSharedMemoryStructs();
AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
}
@@ -4972,11 +4972,11 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ /* Need a PGPROC to run AttachSharedMemoryStructs */
InitProcess();
/* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
+ AttachSharedMemoryStructs();
/* Fetch MyBgworkerEntry from shared memory */
shmem_slot = atoi(argv[1] + 15);
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index 2398167f495..26ded928a71 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -193,7 +193,7 @@ WalReceiverMain(void)
TimeLineID startpointTLI;
TimeLineID primaryTLI;
bool first_stream;
- WalRcvData *walrcv = WalRcv;
+ WalRcvData *walrcv;
TimestampTz now;
char *err;
char *sender_host = NULL;
@@ -203,6 +203,7 @@ WalReceiverMain(void)
* WalRcv should be set up already (if we are a backend, we inherit this
* by fork() or EXEC_BACKEND mechanism from the postmaster).
*/
+ walrcv = WalRcv;
Assert(walrcv != NULL);
/*
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index a3d8eacb8dc..2225a4a6e60 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -58,6 +58,8 @@ shmem_startup_hook_type shmem_startup_hook = NULL;
static Size total_addin_request = 0;
+static void CreateOrAttachShmemStructs(void);
+
/*
* RequestAddinShmemSpace
* Request that extra shmem space be allocated for use by
@@ -156,9 +158,106 @@ CalculateShmemSize(int *num_semaphores)
return size;
}
+#ifdef EXEC_BACKEND
+/*
+ * AttachSharedMemoryStructs
+ * Initialize a postmaster child process's access to shared memory
+ * structures.
+ *
+ * In !EXEC_BACKEND mode, we inherit everything through the fork, and this
+ * isn't needed.
+ */
+void
+AttachSharedMemoryStructs(void)
+{
+ /* InitProcess must've been called already */
+ Assert(MyProc != NULL);
+ Assert(IsUnderPostmaster);
+
+ CreateOrAttachShmemStructs();
+
+ /*
+ * Now give loadable modules a chance to set up their shmem allocations
+ */
+ if (shmem_startup_hook)
+ shmem_startup_hook();
+}
+#endif
+
/*
* CreateSharedMemoryAndSemaphores
* Creates and initializes shared memory and semaphores.
+ */
+void
+CreateSharedMemoryAndSemaphores(void)
+{
+ PGShmemHeader *shim;
+ PGShmemHeader *seghdr;
+ Size size;
+ int numSemas;
+
+ Assert(!IsUnderPostmaster);
+
+ /* Compute the size of the shared-memory block */
+ size = CalculateShmemSize(&numSemas);
+ elog(DEBUG3, "invoking IpcMemoryCreate(size=%zu)", size);
+
+ /*
+ * Create the shmem segment
+ */
+ seghdr = PGSharedMemoryCreate(size, &shim);
+
+ /*
+ * Make sure that huge pages are never reported as "unknown" while the
+ * server is running.
+ */
+ Assert(strcmp("unknown",
+ GetConfigOption("huge_pages_status", false, false)) != 0);
+
+ InitShmemAccess(seghdr);
+
+ /*
+ * Create semaphores
+ */
+ PGReserveSemaphores(numSemas);
+
+ /*
+ * If spinlocks are disabled, initialize emulation layer (which depends on
+ * semaphores, so the order is important here).
+ */
+#ifndef HAVE_SPINLOCKS
+ SpinlockSemaInit();
+#endif
+
+ /*
+ * Set up shared memory allocation mechanism
+ */
+ InitShmemAllocation();
+
+ /* Initialize subsystems */
+ CreateOrAttachShmemStructs();
+
+#ifdef EXEC_BACKEND
+
+ /*
+ * Alloc the win32 shared backend array
+ */
+ ShmemBackendArrayAllocation();
+#endif
+
+ /* Initialize dynamic shared memory facilities. */
+ dsm_postmaster_startup(shim);
+
+ /*
+ * Now give loadable modules a chance to set up their shmem allocations
+ */
+ if (shmem_startup_hook)
+ shmem_startup_hook();
+}
+
+/*
+ * Initialize various subsystems, setting up their data structures in
+ * shared memory.
*
* This is called by the postmaster or by a standalone backend.
* It is also called by a backend forked from the postmaster in the
@@ -171,65 +270,9 @@ CalculateShmemSize(int *num_semaphores)
* check IsUnderPostmaster, rather than EXEC_BACKEND, to detect this case.
* This is a bit code-wasteful and could be cleaned up.)
*/
-void
-CreateSharedMemoryAndSemaphores(void)
+static void
+CreateOrAttachShmemStructs(void)
{
- PGShmemHeader *shim = NULL;
-
- if (!IsUnderPostmaster)
- {
- PGShmemHeader *seghdr;
- Size size;
- int numSemas;
-
- /* Compute the size of the shared-memory block */
- size = CalculateShmemSize(&numSemas);
- elog(DEBUG3, "invoking IpcMemoryCreate(size=%zu)", size);
-
- /*
- * Create the shmem segment
- */
- seghdr = PGSharedMemoryCreate(size, &shim);
-
- /*
- * Make sure that huge pages are never reported as "unknown" while the
- * server is running.
- */
- Assert(strcmp("unknown",
- GetConfigOption("huge_pages_status", false, false)) != 0);
-
- InitShmemAccess(seghdr);
-
- /*
- * Create semaphores
- */
- PGReserveSemaphores(numSemas);
-
- /*
- * If spinlocks are disabled, initialize emulation layer (which
- * depends on semaphores, so the order is important here).
- */
-#ifndef HAVE_SPINLOCKS
- SpinlockSemaInit();
-#endif
- }
- else
- {
- /*
- * We are reattaching to an existing shared memory segment. This
- * should only be reached in the EXEC_BACKEND case.
- */
-#ifndef EXEC_BACKEND
- elog(PANIC, "should be attached to shared memory already");
-#endif
- }
-
- /*
- * Set up shared memory allocation mechanism
- */
- if (!IsUnderPostmaster)
- InitShmemAllocation();
-
/*
* Now initialize LWLocks, which do shared memory allocation and are
* needed for InitShmemIndex.
@@ -302,25 +345,6 @@ CreateSharedMemoryAndSemaphores(void)
AsyncShmemInit();
StatsShmemInit();
WaitEventExtensionShmemInit();
-
-#ifdef EXEC_BACKEND
-
- /*
- * Alloc the win32 shared backend array
- */
- if (!IsUnderPostmaster)
- ShmemBackendArrayAllocation();
-#endif
-
- /* Initialize dynamic shared memory facilities. */
- if (!IsUnderPostmaster)
- dsm_postmaster_startup(shim);
-
- /*
- * Now give loadable modules a chance to set up their shmem allocations
- */
- if (shmem_startup_hook)
- shmem_startup_hook();
}
/*
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index 01f7019b10c..6648c6e5e7e 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -468,7 +468,7 @@ InitProcess(void)
*
* This is separate from InitProcess because we can't acquire LWLocks until
* we've created a PGPROC, but in the EXEC_BACKEND case ProcArrayAdd won't
- * work until after we've done CreateSharedMemoryAndSemaphores.
+ * work until after we've done AttachSharedMemoryStructs.
*/
void
InitProcessPhase2(void)
diff --git a/src/include/storage/ipc.h b/src/include/storage/ipc.h
index 888c08b3067..e282dcad962 100644
--- a/src/include/storage/ipc.h
+++ b/src/include/storage/ipc.h
@@ -79,6 +79,9 @@ extern PGDLLIMPORT shmem_startup_hook_type shmem_startup_hook;
extern Size CalculateShmemSize(int *num_semaphores);
extern void CreateSharedMemoryAndSemaphores(void);
+#ifdef EXEC_BACKEND
+extern void AttachSharedMemoryStructs(void);
+#endif
extern void InitializeShmemGUCs(void);
#endif /* IPC_H */
--
2.39.2
v4-0002-Pass-BackgroundWorker-entry-in-the-parameter-file.patchtext/x-patch; charset=UTF-8; name=v4-0002-Pass-BackgroundWorker-entry-in-the-parameter-file.patchDownload
From f4c1efd7a565d0f5856dd0e4fb7bcfb583190185 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Fri, 1 Dec 2023 11:04:01 +0200
Subject: [PATCH v4 2/4] Pass BackgroundWorker entry in the parameter file in
EXEC_BACKEND mode
This makes it possible to move InitProcess later in SubPostmasterMain
(in next commit), as we no longer need to access shared memory to read
background worker entry.
Reviewed-by: Tristan Partin, Andres Freund
Discussion: https://www.postgresql.org/message-id/7a59b073-5b5b-151e-7ed3-8b01ff7ce9ef@iki.fi
---
src/backend/postmaster/bgworker.c | 21 -----------
src/backend/postmaster/postmaster.c | 40 +++++++++++++--------
src/include/postmaster/bgworker_internals.h | 4 ---
3 files changed, 25 insertions(+), 40 deletions(-)
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index 911bf24a7cb..c4e8cdb45d2 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -631,27 +631,6 @@ ResetBackgroundWorkerCrashTimes(void)
}
}
-#ifdef EXEC_BACKEND
-/*
- * In EXEC_BACKEND mode, workers use this to retrieve their details from
- * shared memory.
- */
-BackgroundWorker *
-BackgroundWorkerEntry(int slotno)
-{
- static BackgroundWorker myEntry;
- BackgroundWorkerSlot *slot;
-
- Assert(slotno < BackgroundWorkerData->total_slots);
- slot = &BackgroundWorkerData->slot[slotno];
- Assert(slot->in_use);
-
- /* must copy this in case we don't intend to retain shmem access */
- memcpy(&myEntry, &slot->worker, sizeof myEntry);
- return &myEntry;
-}
-#endif
-
/*
* Complain about the BackgroundWorker definition using error level elevel.
* Return true if it looks ok, false if not (unless elevel >= ERROR, in
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index b03e88e6702..135dc5a5f49 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -540,6 +540,8 @@ typedef struct
#endif
char my_exec_path[MAXPGPATH];
char pkglib_path[MAXPGPATH];
+
+ BackgroundWorker MyBgworkerEntry;
} BackendParameters;
static void read_backend_variables(char *id, Port *port);
@@ -4831,7 +4833,7 @@ SubPostmasterMain(int argc, char *argv[])
strcmp(argv[1], "--forkavlauncher") == 0 ||
strcmp(argv[1], "--forkavworker") == 0 ||
strcmp(argv[1], "--forkaux") == 0 ||
- strncmp(argv[1], "--forkbgworker=", 15) == 0)
+ strcmp(argv[1], "--forkbgworker") == 0)
PGSharedMemoryReAttach();
else
PGSharedMemoryNoReAttach();
@@ -4962,10 +4964,8 @@ SubPostmasterMain(int argc, char *argv[])
AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
}
- if (strncmp(argv[1], "--forkbgworker=", 15) == 0)
+ if (strcmp(argv[1], "--forkbgworker") == 0)
{
- int shmem_slot;
-
/* do this as early as possible; in particular, before InitProcess() */
IsBackgroundWorker = true;
@@ -4978,10 +4978,6 @@ SubPostmasterMain(int argc, char *argv[])
/* Attach process to shared data structures */
AttachSharedMemoryStructs();
- /* Fetch MyBgworkerEntry from shared memory */
- shmem_slot = atoi(argv[1] + 15);
- MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
-
BackgroundWorkerMain();
}
if (strcmp(argv[1], "--forklog") == 0)
@@ -5640,22 +5636,24 @@ BackgroundWorkerUnblockSignals(void)
#ifdef EXEC_BACKEND
static pid_t
-bgworker_forkexec(int shmem_slot)
+bgworker_forkexec(BackgroundWorker *worker)
{
char *av[10];
int ac = 0;
- char forkav[MAXPGPATH];
-
- snprintf(forkav, MAXPGPATH, "--forkbgworker=%d", shmem_slot);
+ pid_t result;
av[ac++] = "postgres";
- av[ac++] = forkav;
+ av[ac++] = "--forkbgworker";
av[ac++] = NULL; /* filled in by postmaster_forkexec */
av[ac] = NULL;
Assert(ac < lengthof(av));
- return postmaster_forkexec(ac, av);
+ MyBgworkerEntry = worker;
+ result = postmaster_forkexec(ac, av);
+ MyBgworkerEntry = NULL;
+
+ return result;
}
#endif
@@ -5696,7 +5694,7 @@ do_start_bgworker(RegisteredBgWorker *rw)
rw->rw_worker.bgw_name)));
#ifdef EXEC_BACKEND
- switch ((worker_pid = bgworker_forkexec(rw->rw_shmem_slot)))
+ switch ((worker_pid = bgworker_forkexec(&rw->rw_worker)))
#else
switch ((worker_pid = fork_process()))
#endif
@@ -6096,6 +6094,11 @@ save_backend_variables(BackendParameters *param, Port *port,
strlcpy(param->pkglib_path, pkglib_path, MAXPGPATH);
+ if (MyBgworkerEntry)
+ memcpy(¶m->MyBgworkerEntry, MyBgworkerEntry, sizeof(BackgroundWorker));
+ else
+ memset(¶m->MyBgworkerEntry, 0, sizeof(BackgroundWorker));
+
return true;
}
@@ -6324,6 +6327,13 @@ restore_backend_variables(BackendParameters *param, Port *port)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+ if (param->MyBgworkerEntry.bgw_name[0] != '\0')
+ {
+ MyBgworkerEntry = (BackgroundWorker *)
+ MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
+ memcpy(MyBgworkerEntry, ¶m->MyBgworkerEntry, sizeof(BackgroundWorker));
+ }
+
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
diff --git a/src/include/postmaster/bgworker_internals.h b/src/include/postmaster/bgworker_internals.h
index 09df054fcce..323f1e07291 100644
--- a/src/include/postmaster/bgworker_internals.h
+++ b/src/include/postmaster/bgworker_internals.h
@@ -57,8 +57,4 @@ extern void ResetBackgroundWorkerCrashTimes(void);
/* Entry point for background worker processes */
extern void BackgroundWorkerMain(void) pg_attribute_noreturn();
-#ifdef EXEC_BACKEND
-extern BackgroundWorker *BackgroundWorkerEntry(int slotno);
-#endif
-
#endif /* BGWORKER_INTERNALS_H */
--
2.39.2
v4-0003-Refactor-how-MyBgWorkerEntry-is-passed-through-th.patchtext/x-patch; charset=UTF-8; name=v4-0003-Refactor-how-MyBgWorkerEntry-is-passed-through-th.patchDownload
From bb079f9c87dacddb6ee896ee62c0bc4ce19c7ccb Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Fri, 1 Dec 2023 12:51:50 +0200
Subject: [PATCH v4 3/4] Refactor how 'MyBgWorkerEntry' is passed through the
fork functions.
The BackgroundWorker struct is now passed the same way as the Port
struct. Seems more consistent.
Also add explicit 'has_port' and 'has_worker' flags to
BackendParameters. Previously, I looked at 'bgw_name[0]' to determine
if BackgroundWorker the struct is valid, but that caused regression
failures in test_shm_mq. Turns out that test_shm_mq leaves bgw_name
empty. That should probably be fixed, and perhaps we should even
assert that it's not empty, because I think it's bad form to not set
it and is almost certainly an oversight. bgw_name is only used in log
messages and such, so it cause any serious consequences
though. But having explicit 'has_port' and 'has_worker' flags is
cheap, so I'm inclined to add them anyway.
XXX: to be squashed with previous commit. I'm having this as a
separate commit here to show the diff from v3 of this patch set, for
easier review.
---
src/backend/postmaster/postmaster.c | 118 ++++++++++++++++------------
1 file changed, 67 insertions(+), 51 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 135dc5a5f49..444dd8c6892 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -476,7 +476,7 @@ typedef struct
#endif /* WIN32 */
static pid_t backend_forkexec(Port *port);
-static pid_t internal_forkexec(int argc, char *argv[], Port *port);
+static pid_t internal_forkexec(int argc, char *argv[], Port *port, BackgroundWorker *worker);
/* Type for a socket that can be inherited to a client process */
#ifdef WIN32
@@ -495,8 +495,11 @@ typedef int InheritableSocket;
*/
typedef struct
{
+ bool has_port;
Port port;
InheritableSocket portsocket;
+ bool has_worker;
+ BackgroundWorker MyBgworkerEntry;
char DataDir[MAXPGPATH];
int32 MyCancelKey;
int MyPMChildSlot;
@@ -540,17 +543,15 @@ typedef struct
#endif
char my_exec_path[MAXPGPATH];
char pkglib_path[MAXPGPATH];
-
- BackgroundWorker MyBgworkerEntry;
} BackendParameters;
-static void read_backend_variables(char *id, Port *port);
-static void restore_backend_variables(BackendParameters *param, Port *port);
+static void read_backend_variables(char *id, Port **port, BackgroundWorker **worker);
+static void restore_backend_variables(BackendParameters *param, Port **port, BackgroundWorker **worker);
#ifndef WIN32
-static bool save_backend_variables(BackendParameters *param, Port *port);
+static bool save_backend_variables(BackendParameters *param, Port *port, BackgroundWorker *worker);
#else
-static bool save_backend_variables(BackendParameters *param, Port *port,
+static bool save_backend_variables(BackendParameters *param, Port *port, BackgroundWorker *worker,
HANDLE childProcess, pid_t childPid);
#endif
@@ -4443,11 +4444,7 @@ BackendRun(Port *port)
pid_t
postmaster_forkexec(int argc, char *argv[])
{
- Port port;
-
- /* This entry point passes dummy values for the Port variables */
- memset(&port, 0, sizeof(port));
- return internal_forkexec(argc, argv, &port);
+ return internal_forkexec(argc, argv, NULL, NULL);
}
/*
@@ -4472,7 +4469,7 @@ backend_forkexec(Port *port)
av[ac] = NULL;
Assert(ac < lengthof(av));
- return internal_forkexec(ac, av, port);
+ return internal_forkexec(ac, av, port, NULL);
}
#ifndef WIN32
@@ -4484,7 +4481,7 @@ backend_forkexec(Port *port)
* - fork():s, and then exec():s the child process
*/
static pid_t
-internal_forkexec(int argc, char *argv[], Port *port)
+internal_forkexec(int argc, char *argv[], Port *port, BackgroundWorker *worker)
{
static unsigned long tmpBackendFileNum = 0;
pid_t pid;
@@ -4492,7 +4489,7 @@ internal_forkexec(int argc, char *argv[], Port *port)
BackendParameters param;
FILE *fp;
- if (!save_backend_variables(¶m, port))
+ if (!save_backend_variables(¶m, port, worker))
return -1; /* log made by save_backend_variables */
/* Calculate name for temp file */
@@ -4576,7 +4573,7 @@ internal_forkexec(int argc, char *argv[], Port *port)
* file is complete.
*/
static pid_t
-internal_forkexec(int argc, char *argv[], Port *port)
+internal_forkexec(int argc, char *argv[], Port *port, BackgroundWorker *worker)
{
int retry_count = 0;
STARTUPINFO si;
@@ -4673,7 +4670,7 @@ retry:
return -1;
}
- if (!save_backend_variables(param, port, pi.hProcess, pi.dwProcessId))
+ if (!save_backend_variables(param, port, worker, pi.hProcess, pi.dwProcessId))
{
/*
* log made by save_backend_variables, but we have to clean up the
@@ -4790,7 +4787,8 @@ retry:
void
SubPostmasterMain(int argc, char *argv[])
{
- Port port;
+ Port *port;
+ BackgroundWorker *worker;
/* In EXEC_BACKEND case we will not have inherited these settings */
IsPostmasterEnvironment = true;
@@ -4804,8 +4802,7 @@ SubPostmasterMain(int argc, char *argv[])
elog(FATAL, "invalid subpostmaster invocation");
/* Read in the variables file */
- memset(&port, 0, sizeof(Port));
- read_backend_variables(argv[2], &port);
+ read_backend_variables(argv[2], &port, &worker);
/* Close the postmaster's sockets (as soon as we know them) */
ClosePostmasterPorts(strcmp(argv[1], "--forklog") == 0);
@@ -4906,7 +4903,7 @@ SubPostmasterMain(int argc, char *argv[])
* PGPROC slots, we have already initialized libpq and are able to
* report the error to the client.
*/
- BackendInitialize(&port);
+ BackendInitialize(port);
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
@@ -4918,7 +4915,7 @@ SubPostmasterMain(int argc, char *argv[])
AttachSharedMemoryStructs();
/* And run the backend */
- BackendRun(&port); /* does not return */
+ BackendRun(port); /* does not return */
}
if (strcmp(argv[1], "--forkaux") == 0)
{
@@ -4978,6 +4975,7 @@ SubPostmasterMain(int argc, char *argv[])
/* Attach process to shared data structures */
AttachSharedMemoryStructs();
+ MyBgworkerEntry = worker;
BackgroundWorkerMain();
}
if (strcmp(argv[1], "--forklog") == 0)
@@ -5640,20 +5638,15 @@ bgworker_forkexec(BackgroundWorker *worker)
{
char *av[10];
int ac = 0;
- pid_t result;
av[ac++] = "postgres";
av[ac++] = "--forkbgworker";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
+ av[ac++] = NULL; /* filled in by internal_forkexec */
av[ac] = NULL;
Assert(ac < lengthof(av));
- MyBgworkerEntry = worker;
- result = postmaster_forkexec(ac, av);
- MyBgworkerEntry = NULL;
-
- return result;
+ return internal_forkexec(ac, av, NULL, worker);
}
#endif
@@ -6027,16 +6020,36 @@ static void read_inheritable_socket(SOCKET *dest, InheritableSocket *src);
/* Save critical backend variables into the BackendParameters struct */
#ifndef WIN32
static bool
-save_backend_variables(BackendParameters *param, Port *port)
+save_backend_variables(BackendParameters *param, Port *port, BackgroundWorker *worker)
#else
static bool
-save_backend_variables(BackendParameters *param, Port *port,
+save_backend_variables(BackendParameters *param, Port *port, BackgroundWorker *worker,
HANDLE childProcess, pid_t childPid)
#endif
{
- memcpy(¶m->port, port, sizeof(Port));
- if (!write_inheritable_socket(¶m->portsocket, port->sock, childPid))
- return false;
+ if (port)
+ {
+ memcpy(¶m->port, port, sizeof(Port));
+ if (!write_inheritable_socket(¶m->portsocket, port->sock, childPid))
+ return false;
+ param->has_port = true;
+ }
+ else
+ {
+ memset(¶m->port, 0, sizeof(Port));
+ param->has_port = false;
+ }
+
+ if (worker)
+ {
+ memcpy(¶m->MyBgworkerEntry, worker, sizeof(BackgroundWorker));
+ param->has_worker = true;
+ }
+ else
+ {
+ memset(¶m->MyBgworkerEntry, 0, sizeof(BackgroundWorker));
+ param->has_worker = false;
+ }
strlcpy(param->DataDir, DataDir, MAXPGPATH);
@@ -6094,11 +6107,6 @@ save_backend_variables(BackendParameters *param, Port *port,
strlcpy(param->pkglib_path, pkglib_path, MAXPGPATH);
- if (MyBgworkerEntry)
- memcpy(¶m->MyBgworkerEntry, MyBgworkerEntry, sizeof(BackgroundWorker));
- else
- memset(¶m->MyBgworkerEntry, 0, sizeof(BackgroundWorker));
-
return true;
}
@@ -6197,7 +6205,7 @@ read_inheritable_socket(SOCKET *dest, InheritableSocket *src)
#endif
static void
-read_backend_variables(char *id, Port *port)
+read_backend_variables(char *id, Port **port, BackgroundWorker **worker)
{
BackendParameters param;
@@ -6264,15 +6272,30 @@ read_backend_variables(char *id, Port *port)
}
#endif
- restore_backend_variables(¶m, port);
+ restore_backend_variables(¶m, port, worker);
}
/* Restore critical backend variables from the BackendParameters struct */
static void
-restore_backend_variables(BackendParameters *param, Port *port)
+restore_backend_variables(BackendParameters *param, Port **port, BackgroundWorker **worker)
{
- memcpy(port, ¶m->port, sizeof(Port));
- read_inheritable_socket(&port->sock, ¶m->portsocket);
+ if (param->has_port)
+ {
+ *port = (Port *) MemoryContextAlloc(TopMemoryContext, sizeof(Port));
+ memcpy(*port, ¶m->port, sizeof(Port));
+ read_inheritable_socket(&(*port)->sock, ¶m->portsocket);
+ }
+ else
+ *port = NULL;
+
+ if (param->has_worker)
+ {
+ *worker = (BackgroundWorker *)
+ MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
+ memcpy(*worker, ¶m->MyBgworkerEntry, sizeof(BackgroundWorker));
+ }
+ else
+ *worker = NULL;
SetDataDir(param->DataDir);
@@ -6327,13 +6350,6 @@ restore_backend_variables(BackendParameters *param, Port *port)
strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
- if (param->MyBgworkerEntry.bgw_name[0] != '\0')
- {
- MyBgworkerEntry = (BackgroundWorker *)
- MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
- memcpy(MyBgworkerEntry, ¶m->MyBgworkerEntry, sizeof(BackgroundWorker));
- }
-
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
--
2.39.2
v4-0004-Refactor-how-InitProcess-is-called.patchtext/x-patch; charset=UTF-8; name=v4-0004-Refactor-how-InitProcess-is-called.patchDownload
From e4d30ce1366e6d6a42089d0f394195b824bb1560 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Fri, 1 Dec 2023 11:28:24 +0200
Subject: [PATCH v4 4/4] Refactor how InitProcess is called
The order of process initialization steps is now more consistent
between !EXEC_BACKEND and EXEC_BACKEND modes. InitProcess() is called
at the same place in either mode. We can now also move the
AttachSharedMemoryStructs() call into InitProcess() itself. This
reduces the number of "#ifdef EXEC_BACKEND" blocks.
Reviewed-by: Tristan Partin, Andres Freund
Discussion: https://www.postgresql.org/message-id/7a59b073-5b5b-151e-7ed3-8b01ff7ce9ef@iki.fi
---
src/backend/postmaster/autovacuum.c | 16 +++-------
src/backend/postmaster/auxprocess.c | 5 +---
src/backend/postmaster/bgworker.c | 8 ++---
src/backend/postmaster/postmaster.c | 45 ++++-------------------------
src/backend/storage/lmgr/proc.c | 24 +++++++++++++--
5 files changed, 35 insertions(+), 63 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 86a3b3d8be2..2f54485c217 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -476,14 +476,10 @@ AutoVacLauncherMain(int argc, char *argv[])
pqsignal(SIGCHLD, SIG_DFL);
/*
- * Create a per-backend PGPROC struct in shared memory, except in the
- * EXEC_BACKEND case where this was done in SubPostmasterMain. We must do
- * this before we can use LWLocks (and in the EXEC_BACKEND case we already
- * had to do some stuff with LWLocks).
+ * Create a per-backend PGPROC struct in shared memory. We must do this
+ * before we can use LWLocks or access any shared memory.
*/
-#ifndef EXEC_BACKEND
InitProcess();
-#endif
/* Early initialization */
BaseInit();
@@ -1548,14 +1544,10 @@ AutoVacWorkerMain(int argc, char *argv[])
pqsignal(SIGCHLD, SIG_DFL);
/*
- * Create a per-backend PGPROC struct in shared memory, except in the
- * EXEC_BACKEND case where this was done in SubPostmasterMain. We must do
- * this before we can use LWLocks (and in the EXEC_BACKEND case we already
- * had to do some stuff with LWLocks).
+ * Create a per-backend PGPROC struct in shared memory. We must do this
+ * before we can use LWLocks or access any shared memory.
*/
-#ifndef EXEC_BACKEND
InitProcess();
-#endif
/* Early initialization */
BaseInit();
diff --git a/src/backend/postmaster/auxprocess.c b/src/backend/postmaster/auxprocess.c
index cae6feb3562..bae6f68c402 100644
--- a/src/backend/postmaster/auxprocess.c
+++ b/src/backend/postmaster/auxprocess.c
@@ -97,12 +97,9 @@ AuxiliaryProcessMain(AuxProcType auxtype)
*/
/*
- * Create a PGPROC so we can use LWLocks. In the EXEC_BACKEND case, this
- * was already done by SubPostmasterMain().
+ * Create a PGPROC so we can use LWLocks and access shared memory.
*/
-#ifndef EXEC_BACKEND
InitAuxiliaryProcess();
-#endif
BaseInit();
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index c4e8cdb45d2..00656fcc19e 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -810,14 +810,10 @@ BackgroundWorkerMain(void)
PG_exception_stack = &local_sigjmp_buf;
/*
- * Create a per-backend PGPROC struct in shared memory, except in the
- * EXEC_BACKEND case where this was done in SubPostmasterMain. We must do
- * this before we can use LWLocks (and in the EXEC_BACKEND case we already
- * had to do some stuff with LWLocks).
+ * Create a per-backend PGPROC struct in shared memory. We must do this
+ * before we can use LWLocks or access any shared memory.
*/
-#ifndef EXEC_BACKEND
InitProcess();
-#endif
/*
* Early initialization.
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 444dd8c6892..c127a748d71 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -4098,15 +4098,6 @@ BackendStartup(Port *port)
/* Perform additional initialization and collect startup packet */
BackendInitialize(port);
- /*
- * Create a per-backend PGPROC struct in shared memory. We must do
- * this before we can use LWLocks. In the !EXEC_BACKEND case (here)
- * this could be delayed a bit further, but EXEC_BACKEND needs to do
- * stuff with LWLocks before PostgresMain(), so we do it here as well
- * for symmetry.
- */
- InitProcess();
-
/* And run the backend */
BackendRun(port);
}
@@ -4417,6 +4408,12 @@ BackendInitialize(Port *port)
static void
BackendRun(Port *port)
{
+ /*
+ * Create a per-backend PGPROC struct in shared memory. We must do this
+ * before we can use LWLocks or access any shared memory.
+ */
+ InitProcess();
+
/*
* Make sure we aren't in PostmasterContext anymore. (We can't delete it
* just yet, though, because InitPostgres will need the HBA data.)
@@ -4908,12 +4905,6 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run AttachSharedMemoryStructs */
- InitProcess();
-
- /* Attach process to shared data structures */
- AttachSharedMemoryStructs();
-
/* And run the backend */
BackendRun(port); /* does not return */
}
@@ -4926,12 +4917,6 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run AttachSharedMemoryStructs */
- InitAuxiliaryProcess();
-
- /* Attach process to shared data structures */
- AttachSharedMemoryStructs();
-
auxtype = atoi(argv[3]);
AuxiliaryProcessMain(auxtype); /* does not return */
}
@@ -4940,12 +4925,6 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run AttachSharedMemoryStructs */
- InitProcess();
-
- /* Attach process to shared data structures */
- AttachSharedMemoryStructs();
-
AutoVacLauncherMain(argc - 2, argv + 2); /* does not return */
}
if (strcmp(argv[1], "--forkavworker") == 0)
@@ -4953,12 +4932,6 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run AttachSharedMemoryStructs */
- InitProcess();
-
- /* Attach process to shared data structures */
- AttachSharedMemoryStructs();
-
AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
}
if (strcmp(argv[1], "--forkbgworker") == 0)
@@ -4969,12 +4942,6 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run AttachSharedMemoryStructs */
- InitProcess();
-
- /* Attach process to shared data structures */
- AttachSharedMemoryStructs();
-
MyBgworkerEntry = worker;
BackgroundWorkerMain();
}
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index 6648c6e5e7e..b6451d9d083 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -291,7 +291,7 @@ InitProcGlobal(void)
}
/*
- * InitProcess -- initialize a per-process data structure for this backend
+ * InitProcess -- initialize a per-process PGPROC entry for this backend
*/
void
InitProcess(void)
@@ -461,6 +461,16 @@ InitProcess(void)
*/
InitLWLockAccess();
InitDeadLockChecking();
+
+#ifdef EXEC_BACKEND
+
+ /*
+ * Initialize backend-local pointers to all the shared data structures.
+ * (We couldn't do this until now because it needs LWLocks.)
+ */
+ if (IsUnderPostmaster)
+ AttachSharedMemoryStructs();
+#endif
}
/*
@@ -487,7 +497,7 @@ InitProcessPhase2(void)
}
/*
- * InitAuxiliaryProcess -- create a per-auxiliary-process data structure
+ * InitAuxiliaryProcess -- create a PGPROC entry for an auxiliary process
*
* This is called by bgwriter and similar processes so that they will have a
* MyProc value that's real enough to let them wait for LWLocks. The PGPROC
@@ -621,6 +631,16 @@ InitAuxiliaryProcess(void)
* acquired in aux processes.)
*/
InitLWLockAccess();
+
+#ifdef EXEC_BACKEND
+
+ /*
+ * Initialize backend-local pointers to all the shared data structures.
+ * (We couldn't do this until now because it needs LWLocks.)
+ */
+ if (IsUnderPostmaster)
+ AttachSharedMemoryStructs();
+#endif
}
/*
--
2.39.2
Re: Refactoring backend fork+exec code
Hello Heikki,
01.12.2023 15:10, Heikki Linnakangas wrote:
Attached is new version of these patches. For easier review, I made the new refactorings compared in a new commit
0003. I will squash that before pushing, but this makes it easier to see what changed.Barring any new feedback or issues, I will commit these.
Maybe you could look at issues with running `make check` under Valgrind
when server built with CPPFLAGS="-DUSE_VALGRIND -DEXEC_BACKEND":
# +++ regress check in src/test/regress +++
# initializing database system by copying initdb template
# postmaster failed, examine ".../src/test/regress/log/postmaster.log" for the reason
Bail out!make[1]: ***
...
2023-12-01 16:48:39.136 MSK postmaster[1307988] LOG: listening on Unix socket "/tmp/pg_regress-pPFNk0/.s.PGSQL.55312"
==00:00:00:01.692 1259396== Syscall param write(buf) points to uninitialised byte(s)
==00:00:00:01.692 1259396== at 0x5245A37: write (write.c:26)
==00:00:00:01.692 1259396== by 0x51BBF6C: _IO_file_write@@GLIBC_2.2.5 (fileops.c:1180)
==00:00:00:01.692 1259396== by 0x51BC84F: new_do_write (fileops.c:448)
==00:00:00:01.692 1259396== by 0x51BC84F: _IO_new_file_xsputn (fileops.c:1254)
==00:00:00:01.692 1259396== by 0x51BC84F: _IO_file_xsputn@@GLIBC_2.2.5 (fileops.c:1196)
==00:00:00:01.692 1259396== by 0x51B1056: fwrite (iofwrite.c:39)
==00:00:00:01.692 1259396== by 0x552E21: internal_forkexec (postmaster.c:4518)
==00:00:00:01.692 1259396== by 0x5546A1: postmaster_forkexec (postmaster.c:4444)
==00:00:00:01.692 1259396== by 0x55471C: StartChildProcess (postmaster.c:5275)
==00:00:00:01.692 1259396== by 0x557B61: PostmasterMain (postmaster.c:1454)
==00:00:00:01.692 1259396== by 0x472136: main (main.c:198)
==00:00:00:01.692 1259396== Address 0x1ffeffdc11 is on thread 1's stack
==00:00:00:01.692 1259396== in frame #4, created by internal_forkexec (postmaster.c:4482)
==00:00:00:01.692 1259396==
With memset(param, 0, sizeof(*param)); added at the beginning of
save_backend_variables(), server starts successfully, but then
`make check` fails with other Valgrind error.
Best regards,
Alexander
Re: Refactoring backend fork+exec code
On Fri Dec 1, 2023 at 6:10 AM CST, Heikki Linnakangas wrote:
On 30/11/2023 20:44, Tristan Partin wrote:
Patches 1-3 seem committable as-is.
Thanks for the review! I'm focusing on patches 1-3 now, and will come
back to the rest after committing 1-3.There was one test failure with EXEC_BACKEND from patch 2, in
'test_shm_mq'. In restore_backend_variables() I checked if 'bgw_name' is
empty to decide if the BackgroundWorker struct is filled in or not, but
it turns out that 'test_shm_mq' doesn't fill in bgw_name. It probably
should, I think that's an oversight in 'test_shm_mq', but that's a
separate issue.I did some more refactoring of patch 2, to fix that and to improve it in
general. The BackgroundWorker struct is now passed through the
fork-related functions similarly to the Port struct. That seems more
consistent.Attached is new version of these patches. For easier review, I made the
new refactorings compared in a new commit 0003. I will squash that
before pushing, but this makes it easier to see what changed.Barring any new feedback or issues, I will commit these.
My only thought is that perhaps has_bg_worker is a better name than
has_worker, but I agree that having a flag is better than checking
bgw_name.
--
Tristan Partin
Neon (https://neon.tech)
Re: Refactoring backend fork+exec code
On 01/12/2023 16:00, Alexander Lakhin wrote:
Maybe you could look at issues with running `make check` under Valgrind
when server built with CPPFLAGS="-DUSE_VALGRIND -DEXEC_BACKEND":
# +++ regress check in src/test/regress +++
# initializing database system by copying initdb template
# postmaster failed, examine ".../src/test/regress/log/postmaster.log" for the reason
Bail out!make[1]: ***...
2023-12-01 16:48:39.136 MSK postmaster[1307988] LOG: listening on Unix socket "/tmp/pg_regress-pPFNk0/.s.PGSQL.55312"
==00:00:00:01.692 1259396== Syscall param write(buf) points to uninitialised byte(s)
==00:00:00:01.692 1259396== at 0x5245A37: write (write.c:26)
==00:00:00:01.692 1259396== by 0x51BBF6C: _IO_file_write@@GLIBC_2.2.5 (fileops.c:1180)
==00:00:00:01.692 1259396== by 0x51BC84F: new_do_write (fileops.c:448)
==00:00:00:01.692 1259396== by 0x51BC84F: _IO_new_file_xsputn (fileops.c:1254)
==00:00:00:01.692 1259396== by 0x51BC84F: _IO_file_xsputn@@GLIBC_2.2.5 (fileops.c:1196)
==00:00:00:01.692 1259396== by 0x51B1056: fwrite (iofwrite.c:39)
==00:00:00:01.692 1259396== by 0x552E21: internal_forkexec (postmaster.c:4518)
==00:00:00:01.692 1259396== by 0x5546A1: postmaster_forkexec (postmaster.c:4444)
==00:00:00:01.692 1259396== by 0x55471C: StartChildProcess (postmaster.c:5275)
==00:00:00:01.692 1259396== by 0x557B61: PostmasterMain (postmaster.c:1454)
==00:00:00:01.692 1259396== by 0x472136: main (main.c:198)
==00:00:00:01.692 1259396== Address 0x1ffeffdc11 is on thread 1's stack
==00:00:00:01.692 1259396== in frame #4, created by internal_forkexec (postmaster.c:4482)
==00:00:00:01.692 1259396==With memset(param, 0, sizeof(*param)); added at the beginning of
save_backend_variables(), server starts successfully, but then
`make check` fails with other Valgrind error.
That's actually a pre-existing issue, I'm seeing that even on unpatched
'master'.
In a nutshell, the problem is that the uninitialized padding bytes in
BackendParameters are written to the file. When we read the file back,
we don't access the padding bytes, so that's harmless. But Valgrind
doesn't know that.
On Windows, the file is created with
CreateFileMapping(INVALID_HANDLE_VALUE, ...) and we write the variables
directly to the mapping. If I understand the Windows API docs correctly,
it is guaranteed to be initialized to zeros, so we don't have this
problem on Windows, only on other platforms with EXEC_BACKEND. I think
it makes sense to clear the memory on other platforms too, since that's
what we do on Windows.
Committed a fix with memset(). I'm not sure what our policy with
backpatching this kind of issues is. This goes back to all supported
versions, but given the lack of complaints, I chose to not backpatch.
--
Heikki Linnakangas
Neon (https://neon.tech)
Re: Refactoring backend fork+exec code
On Fri Dec 1, 2023 at 2:44 PM CST, Heikki Linnakangas wrote:
On 01/12/2023 16:00, Alexander Lakhin wrote:
Maybe you could look at issues with running `make check` under Valgrind
when server built with CPPFLAGS="-DUSE_VALGRIND -DEXEC_BACKEND":
# +++ regress check in src/test/regress +++
# initializing database system by copying initdb template
# postmaster failed, examine ".../src/test/regress/log/postmaster.log" for the reason
Bail out!make[1]: ***...
2023-12-01 16:48:39.136 MSK postmaster[1307988] LOG: listening on Unix socket "/tmp/pg_regress-pPFNk0/.s.PGSQL.55312"
==00:00:00:01.692 1259396== Syscall param write(buf) points to uninitialised byte(s)
==00:00:00:01.692 1259396== at 0x5245A37: write (write.c:26)
==00:00:00:01.692 1259396== by 0x51BBF6C: _IO_file_write@@GLIBC_2.2.5 (fileops.c:1180)
==00:00:00:01.692 1259396== by 0x51BC84F: new_do_write (fileops.c:448)
==00:00:00:01.692 1259396== by 0x51BC84F: _IO_new_file_xsputn (fileops.c:1254)
==00:00:00:01.692 1259396== by 0x51BC84F: _IO_file_xsputn@@GLIBC_2.2.5 (fileops.c:1196)
==00:00:00:01.692 1259396== by 0x51B1056: fwrite (iofwrite.c:39)
==00:00:00:01.692 1259396== by 0x552E21: internal_forkexec (postmaster.c:4518)
==00:00:00:01.692 1259396== by 0x5546A1: postmaster_forkexec (postmaster.c:4444)
==00:00:00:01.692 1259396== by 0x55471C: StartChildProcess (postmaster.c:5275)
==00:00:00:01.692 1259396== by 0x557B61: PostmasterMain (postmaster.c:1454)
==00:00:00:01.692 1259396== by 0x472136: main (main.c:198)
==00:00:00:01.692 1259396== Address 0x1ffeffdc11 is on thread 1's stack
==00:00:00:01.692 1259396== in frame #4, created by internal_forkexec (postmaster.c:4482)
==00:00:00:01.692 1259396==With memset(param, 0, sizeof(*param)); added at the beginning of
save_backend_variables(), server starts successfully, but then
`make check` fails with other Valgrind error.That's actually a pre-existing issue, I'm seeing that even on unpatched
'master'.In a nutshell, the problem is that the uninitialized padding bytes in
BackendParameters are written to the file. When we read the file back,
we don't access the padding bytes, so that's harmless. But Valgrind
doesn't know that.On Windows, the file is created with
CreateFileMapping(INVALID_HANDLE_VALUE, ...) and we write the variables
directly to the mapping. If I understand the Windows API docs correctly,
it is guaranteed to be initialized to zeros, so we don't have this
problem on Windows, only on other platforms with EXEC_BACKEND. I think
it makes sense to clear the memory on other platforms too, since that's
what we do on Windows.Committed a fix with memset(). I'm not sure what our policy with
backpatching this kind of issues is. This goes back to all supported
versions, but given the lack of complaints, I chose to not backpatch.
Seems like a harmless think to backpatch. It is conceivable that someone
might want to run Valgrind on something other than HEAD.
--
Tristan Partin
Neon (https://neon.tech)
Re: Refactoring backend fork+exec code
"Tristan Partin" <tristan@neon.tech> writes:
On Fri Dec 1, 2023 at 2:44 PM CST, Heikki Linnakangas wrote:
Committed a fix with memset(). I'm not sure what our policy with
backpatching this kind of issues is. This goes back to all supported
versions, but given the lack of complaints, I chose to not backpatch.
Seems like a harmless think to backpatch. It is conceivable that someone
might want to run Valgrind on something other than HEAD.
FWIW, I agree with Heikki's conclusion. EXEC_BACKEND on non-Windows
is already a niche developer-only setup, and given the lack of complaints,
nobody's that interested in running Valgrind with it. Fixing it on HEAD
seems like plenty.
regards, tom lane
Re: Refactoring backend fork+exec code
Hello Heikki,
01.12.2023 23:44, Heikki Linnakangas wrote:
With memset(param, 0, sizeof(*param)); added at the beginning of
save_backend_variables(), server starts successfully, but then
`make check` fails with other Valgrind error.That's actually a pre-existing issue, I'm seeing that even on unpatched 'master'.
Thank you for fixing that!
Yes, I had discovered it before, but yesterday I decided to check whether
your patches improve the situation...
What bothered me additionally, is an error detected after server start. I
couldn't see it without the patches applied. I mean, on HEAD I now see
`make check` passing, but with the patches it fails:
...
# parallel group (20 tests): interval date numerology polygon box macaddr8 macaddr multirangetypes line timestamp
timetz timestamptz time circle strings lseg inet md5 path point
not ok 22 + strings 1048 ms
# (test process exited with exit code 2)
not ok 23 + md5 1052 ms
# (test process exited with exit code 2)
...
src/test/regress/log/postmaster.log contains:
==00:00:00:30.730 1713480== Syscall param write(buf) points to uninitialised byte(s)
==00:00:00:30.730 1713480== at 0x5245A37: write (write.c:26)
==00:00:00:30.730 1713480== by 0x51BBF6C: _IO_file_write@@GLIBC_2.2.5 (fileops.c:1180)
==00:00:00:30.730 1713480== by 0x51BC84F: new_do_write (fileops.c:448)
==00:00:00:30.730 1713480== by 0x51BC84F: _IO_new_file_xsputn (fileops.c:1254)
==00:00:00:30.730 1713480== by 0x51BC84F: _IO_file_xsputn@@GLIBC_2.2.5 (fileops.c:1196)
==00:00:00:30.730 1713480== by 0x51B1056: fwrite (iofwrite.c:39)
==00:00:00:30.730 1713480== by 0x5540CF: internal_forkexec (postmaster.c:4526)
==00:00:00:30.730 1713480== by 0x5543C0: bgworker_forkexec (postmaster.c:5624)
==00:00:00:30.730 1713480== by 0x555477: do_start_bgworker (postmaster.c:5665)
==00:00:00:30.730 1713480== by 0x555738: maybe_start_bgworkers (postmaster.c:5928)
==00:00:00:30.730 1713480== by 0x556072: process_pm_pmsignal (postmaster.c:5080)
==00:00:00:30.730 1713480== by 0x556610: ServerLoop (postmaster.c:1761)
==00:00:00:30.730 1713480== by 0x557BE2: PostmasterMain (postmaster.c:1469)
==00:00:00:30.730 1713480== by 0x47216B: main (main.c:198)
==00:00:00:30.730 1713480== Address 0x1ffeffd8c0 is on thread 1's stack
==00:00:00:30.730 1713480== in frame #4, created by internal_forkexec (postmaster.c:4482)
==00:00:00:30.730 1713480==
...
2023-12-02 05:14:30.751 MSK client backend[1713740] pg_regress/rangetypes FATAL: terminating connection due to
unexpected postmaster exit
2023-12-02 05:14:31.033 MSK client backend[1713734] pg_regress/numeric FATAL: postmaster exited during a parallel
transaction
TRAP: failed Assert("!IsTransactionOrTransactionBlock()"), File: "pgstat.c", Line: 591, PID: 1713734
I haven't looked deeper yet, but it seems that we see two issues here (and
Assert is not directly caused by the patches set.)
Best regards,
Alexander
Re: Refactoring backend fork+exec code
On 02/12/2023 05:00, Alexander Lakhin wrote:
What bothered me additionally, is an error detected after server start. I
couldn't see it without the patches applied. I mean, on HEAD I now see
`make check` passing, but with the patches it fails:
...
# parallel group (20 tests): interval date numerology polygon box macaddr8 macaddr multirangetypes line timestamp
timetz timestamptz time circle strings lseg inet md5 path point
not ok 22 + strings 1048 ms
# (test process exited with exit code 2)
not ok 23 + md5 1052 ms
# (test process exited with exit code 2)
...
src/test/regress/log/postmaster.log contains:
==00:00:00:30.730 1713480== Syscall param write(buf) points to uninitialised byte(s)
==00:00:00:30.730 1713480== at 0x5245A37: write (write.c:26)
==00:00:00:30.730 1713480== by 0x51BBF6C: _IO_file_write@@GLIBC_2.2.5 (fileops.c:1180)
==00:00:00:30.730 1713480== by 0x51BC84F: new_do_write (fileops.c:448)
==00:00:00:30.730 1713480== by 0x51BC84F: _IO_new_file_xsputn (fileops.c:1254)
==00:00:00:30.730 1713480== by 0x51BC84F: _IO_file_xsputn@@GLIBC_2.2.5 (fileops.c:1196)
==00:00:00:30.730 1713480== by 0x51B1056: fwrite (iofwrite.c:39)
==00:00:00:30.730 1713480== by 0x5540CF: internal_forkexec (postmaster.c:4526)
==00:00:00:30.730 1713480== by 0x5543C0: bgworker_forkexec (postmaster.c:5624)
==00:00:00:30.730 1713480== by 0x555477: do_start_bgworker (postmaster.c:5665)
==00:00:00:30.730 1713480== by 0x555738: maybe_start_bgworkers (postmaster.c:5928)
==00:00:00:30.730 1713480== by 0x556072: process_pm_pmsignal (postmaster.c:5080)
==00:00:00:30.730 1713480== by 0x556610: ServerLoop (postmaster.c:1761)
==00:00:00:30.730 1713480== by 0x557BE2: PostmasterMain (postmaster.c:1469)
==00:00:00:30.730 1713480== by 0x47216B: main (main.c:198)
==00:00:00:30.730 1713480== Address 0x1ffeffd8c0 is on thread 1's stack
==00:00:00:30.730 1713480== in frame #4, created by internal_forkexec (postmaster.c:4482)
==00:00:00:30.730 1713480==
...
Ack, I see this too. I fixed it by adding MCXT_ALLOC_ZERO to the
allocation of the BackendWorker struct. That's a little heavy-handed,
like with the previous failures the uninitialized padding bytes are
written to the file and read back, but not accessed after that. But it
seems like the simplest fix. This isn't performance critical after all.
I also renamed the 'has_worker' field to 'has_bgworker', per Tristan's
suggestion. Pushed with those changes.
Thanks for the reviews!
2023-12-02 05:14:30.751 MSK client backend[1713740] pg_regress/rangetypes FATAL: terminating connection due to
unexpected postmaster exit
2023-12-02 05:14:31.033 MSK client backend[1713734] pg_regress/numeric FATAL: postmaster exited during a parallel
transaction
TRAP: failed Assert("!IsTransactionOrTransactionBlock()"), File: "pgstat.c", Line: 591, PID: 1713734I haven't looked deeper yet, but it seems that we see two issues here (and
Assert is not directly caused by the patches set.)
I have not been able to reproduce this one.
--
Heikki Linnakangas
Neon (https://neon.tech)
Re: Refactoring backend fork+exec code
On 30/11/2023 22:26, Andres Freund wrote:
On 2023-11-30 01:36:25 +0200, Heikki Linnakangas wrote:
[...]
33 files changed, 1787 insertions(+), 2002 deletions(-)Well, that's not small...
I think it may be worth splitting some of the file renaming out into a
separate commit, makes it harder to see what change here.
Here you are (details at end of this email)
+ [PMC_AV_LAUNCHER] = {"autovacuum launcher", AutoVacLauncherMain, true}, + [PMC_AV_WORKER] = {"autovacuum worker", AutoVacWorkerMain, true}, + [PMC_BGWORKER] = {"bgworker", BackgroundWorkerMain, true}, + [PMC_SYSLOGGER] = {"syslogger", SysLoggerMain, false}, + + [PMC_STARTUP] = {"startup", StartupProcessMain, true}, + [PMC_BGWRITER] = {"bgwriter", BackgroundWriterMain, true}, + [PMC_ARCHIVER] = {"archiver", PgArchiverMain, true}, + [PMC_CHECKPOINTER] = {"checkpointer", CheckpointerMain, true}, + [PMC_WAL_WRITER] = {"wal_writer", WalWriterMain, true}, + [PMC_WAL_RECEIVER] = {"wal_receiver", WalReceiverMain, true}, +};It feels like we have too many different ways of documenting the type of a
process. This new PMC_ stuff, enum AuxProcType, enum BackendType.
Agreed. And "am_walsender" and such variables.
Which then leads to code like this:
-CheckpointerMain(void) +CheckpointerMain(char *startup_data, size_t startup_data_len) { sigjmp_buf local_sigjmp_buf; MemoryContext checkpointer_context;+ Assert(startup_data_len == 0); + + MyAuxProcType = CheckpointerProcess; + MyBackendType = B_CHECKPOINTER; + AuxiliaryProcessInit(); +For each type of child process. That seems a bit too redundant. Can't we
unify this at least somewhat? Can't we just reuse BackendType here? Sure,
there'd be pointless entry for B_INVALID, but that doesn't seem like a
problem, could even be useful, by pointing it to a function raising an error.
There are a few differences: B_INVALID (and B_STANDALONE_BACKEND) are
pointless for this array as you noted. But also, we don't know if the
backend is a regular backend or WAL sender until authentication, so for
a WAL sender, we'd need to change MyBackendType from B_BACKEND to
B_WAL_SENDER after forking. Maybe that's ok.
I didn't do anything about this yet, but I'll give it some more thought.
+ if (strncmp(argv[1], "--forkchild=", 12) != 0) + elog(FATAL, "invalid subpostmaster invocation (--forkchild argument missing)"); + entry_name = argv[1] + 12; + found = false; + for (int idx = 0; idx < lengthof(entry_kinds); idx++) + { + if (strcmp(entry_kinds[idx].name, entry_name) == 0) + { + child_type = idx; + found = true; + break; + } + } + if (!found) + elog(ERROR, "unknown child kind %s", entry_name);If we then have to search linearly, why don't we just pass the index into the
array?
We could. I like the idea of a human-readable name on the command line,
although I'm not sure if it's really visible anywhere.
+void +BackendMain(char *startup_data, size_t startup_data_len) +{Is there any remaining reason for this to live in postmaster.c? Given that
other backend types don't, that seems oddly assymmetrical.
Gee, another yak to shave, thanks ;-). You're right, that makes a lot of
sense. I added another patch that moves that to a new file,
src/backend/tcop/backend_startup.c. ProcessStartupPacket() and friends
go there too. It might make sense to do this before the other patches,
but it's the last patch in the series now.
I kept processCancelRequest() in postmaster.c because it looks at
BackendList/ShmemBackendArray, which are static in postmaster.c. Some
more refactoring might be in order there, perhaps moving those to a
different file too. But that can be done separately, this split is
pretty OK as is.
On 30/11/2023 20:44, Tristan Partin wrote:
From 8886db1ed6bae21bf6d77c9bb1230edbb55e24f9 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Thu, 30 Nov 2023 00:04:22 +0200
Subject: [PATCH v3 4/7] Pass CAC as argument to backend processFor me, being new to the code, it would be nice to have more of an
explanation as to why this is "better." I don't doubt it; it would just
help me and future readers of this commit in the future. More of an
explanation in the commit message would suffice.
Updated the commit message. It's mainly to pave the way for the next
patches, which move the initialization of Port to the backend process,
after forking. And that in turn paves the way for the patches after
that. But also, very subjectively, it feels more natural to me.
My other comment on this commit is that we now seem to have lost the
context on what CAC stands for. Before we had the member variable to
explain it. A comment on the enum would be great or changing cac named
variables to canAcceptConnections. I did notice in patch 7 that there
are still some variables named canAcceptConnections around, so I'll
leave this comment up to you.
Good point. The last patch in this series - which is new compared to
previous patch version - moves CAC_state to a different header file
again. I added a comment there.
+ if (fwrite(param, paramsz, 1, fp) != 1) + { + ereport(LOG, + (errcode_for_file_access(), + errmsg("could not write to file \"%s\": %m", tmpfilename))); + FreeFile(fp); + return -1; + } + + /* Release file */ + if (FreeFile(fp)) + { + ereport(LOG, + (errcode_for_file_access(), + errmsg("could not write to file \"%s\": %m", tmpfilename))); + return -1; + }Two pieces of feedback here. I generally find write(2) more useful than
fwrite(3) because write(2) will report a useful errno, whereas fwrite(2)
just uses ferror(3). The additional errno information might be valuable
context in the log message. Up to you if you think it is also valuable.
In general I agree. This patch just moves existing code though, so I
left it as is.
The log message if FreeFile() fails doesn't seem to make sense to me.
I didn't see any file writing in that code path, but it is possible that
I missed something.
FreeFile() calls fclose(), which flushes the buffer. If fclose() fails,
it's most likely because the write() to flush the buffer failed, so
"could not write" is usually appropriate. (It feels ugly to me too,
error handling with the buffered i/o functions is a bit messy. As you
said, plain open()/write() is more clear.)
+ /* + * Need to reinitialize the SSL library in the backend, since the context + * structures contain function pointers and cannot be passed through the + * parameter file. + * + * If for some reason reload fails (maybe the user installed broken key + * files), soldier on without SSL; that's better than all connections + * becoming impossible. + * + * XXX should we do this in all child processes? For the moment it's + * enough to do it in backend children. + */ +#ifdef USE_SSL + if (EnableSSL) + { + if (secure_initialize(false) == 0) + LoadedSSL = true; + else + ereport(LOG, + (errmsg("SSL configuration could not be loaded in child process"))); + } +#endifDo other child process types do any non-local communication?
No. Although in theory an extension-defined background worker could do
whatever, including opening TLS connections. It's not clear if such a
background worker would want the same initialization that we do in
secure_initialize(), or something else.
Here is a new patch set:
v5-0001-Pass-CAC-as-argument-to-backend-process.patch
v5-0002-Remove-ConnCreate-and-ConnFree-and-allocate-Port-.patch
v5-0003-Move-initialization-of-Port-struct-to-child-proce.patch
These patches form a pretty well-contained unit. The gist is to move the
initialization of the Port struct to after forking the backend process
(in patch 3).
I plan to polish and commit these next, so any final reviews on these
are welcome.
v5-0004-Extract-registration-of-Win32-deadchild-callback-.patch
v5-0005-Move-some-functions-from-postmaster.c-to-new-sour.patch
v5-0006-Refactor-AuxProcess-startup.patch
v5-0007-Refactor-postmaster-child-process-launching.patch
Patches 4-6 are refactorings that don't do much good on their own, but
they help to make patch 7 much smaller and easier to review.
I left out some of the code-moving that I had in previous patch versions:
- Previously I moved fork_process() function from fork_process.c to the
new launch_backend.c file. That might still make sense, there is nothing
else in fork_process.c and the only caller is in launch_backend.c. But
I'm not sure, and it can be done separately.
- Previously I moved InitPostmasterChild from miscinit.c to the new
launch_backend.c file. That might also still make sense, but I'm not
100% sure it's an improvement, and it can be done later if we want to.
v5-0008-Move-code-for-backend-startup-to-separate-file.patch
This moves BackendMain() and friends from postmaster.c to a new file,
per Andres's suggestion.
--
Heikki Linnakangas
Neon (https://neon.tech)
Attachments:
v5-0001-Pass-CAC-as-argument-to-backend-process.patchtext/x-patch; charset=UTF-8; name=v5-0001-Pass-CAC-as-argument-to-backend-process.patchDownload
From 197f6d4f5db7924975762cb1de333ca73c9eba0a Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Thu, 30 Nov 2023 00:04:22 +0200
Subject: [PATCH v5 1/8] Pass CAC as argument to backend process
We used to smuggle it to the child process in the Port struct, but it
seems better to pass it down as a separate argument. This paves the
way for the next commits, in which we move the initialization of the
Port struct to the backend process, after forking.
Reviewed-by: Tristan Partin, Andres Freund
Discussion: https://www.postgresql.org/message-id/7a59b073-5b5b-151e-7ed3-8b01ff7ce9ef@iki.fi
---
src/backend/postmaster/postmaster.c | 43 +++++++++++++++++++++--------
src/include/libpq/libpq-be.h | 12 --------
2 files changed, 31 insertions(+), 24 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 651b85ea746..e53057e83a7 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -416,7 +416,18 @@ static void HandleChildCrash(int pid, int exitstatus, const char *procname);
static void LogChildExit(int lev, const char *procname,
int pid, int exitstatus);
static void PostmasterStateMachine(void);
-static void BackendInitialize(Port *port);
+
+typedef enum CAC_state
+{
+ CAC_OK,
+ CAC_STARTUP,
+ CAC_SHUTDOWN,
+ CAC_RECOVERY,
+ CAC_NOTCONSISTENT,
+ CAC_TOOMANY,
+} CAC_state;
+
+static void BackendInitialize(Port *port, CAC_state cac);
static void BackendRun(Port *port) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
@@ -473,7 +484,7 @@ typedef struct
} win32_deadchild_waitinfo;
#endif /* WIN32 */
-static pid_t backend_forkexec(Port *port);
+static pid_t backend_forkexec(Port *port, CAC_state cac);
static pid_t internal_forkexec(int argc, char *argv[], Port *port, BackgroundWorker *worker);
/* Type for a socket that can be inherited to a client process */
@@ -4037,6 +4048,7 @@ BackendStartup(Port *port)
{
Backend *bn; /* for backend cleanup */
pid_t pid;
+ CAC_state cac;
/*
* Create backend data structure. Better before the fork() so we can
@@ -4068,8 +4080,8 @@ BackendStartup(Port *port)
bn->cancel_key = MyCancelKey;
/* Pass down canAcceptConnections state */
- port->canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
- bn->dead_end = (port->canAcceptConnections != CAC_OK);
+ cac = canAcceptConnections(BACKEND_TYPE_NORMAL);
+ bn->dead_end = (cac != CAC_OK);
/*
* Unless it's a dead_end child, assign it a child slot number
@@ -4083,7 +4095,7 @@ BackendStartup(Port *port)
bn->bgworker_notify = false;
#ifdef EXEC_BACKEND
- pid = backend_forkexec(port);
+ pid = backend_forkexec(port, cac);
#else /* !EXEC_BACKEND */
pid = fork_process();
if (pid == 0) /* child */
@@ -4095,7 +4107,7 @@ BackendStartup(Port *port)
ClosePostmasterPorts(false);
/* Perform additional initialization and collect startup packet */
- BackendInitialize(port);
+ BackendInitialize(port, cac);
/* And run the backend */
BackendRun(port);
@@ -4182,7 +4194,7 @@ report_fork_failure_to_client(Port *port, int errnum)
* but have not yet set up most of our local pointers to shmem structures.
*/
static void
-BackendInitialize(Port *port)
+BackendInitialize(Port *port, CAC_state cac)
{
int status;
int ret;
@@ -4315,7 +4327,7 @@ BackendInitialize(Port *port)
* now instead of wasting cycles on an authentication exchange. (This also
* allows a pg_ping utility to be written.)
*/
- switch (port->canAcceptConnections)
+ switch (cac)
{
case CAC_STARTUP:
ereport(FATAL,
@@ -4453,15 +4465,19 @@ postmaster_forkexec(int argc, char *argv[])
* returns the pid of the fork/exec'd process, or -1 on failure
*/
static pid_t
-backend_forkexec(Port *port)
+backend_forkexec(Port *port, CAC_state cac)
{
- char *av[4];
+ char *av[5];
int ac = 0;
+ char cacbuf[10];
av[ac++] = "postgres";
av[ac++] = "--forkbackend";
av[ac++] = NULL; /* filled in by internal_forkexec */
+ snprintf(cacbuf, sizeof(cacbuf), "%d", (int) cac);
+ av[ac++] = cacbuf;
+
av[ac] = NULL;
Assert(ac < lengthof(av));
@@ -4867,7 +4883,10 @@ SubPostmasterMain(int argc, char *argv[])
/* Run backend or appropriate child */
if (strcmp(argv[1], "--forkbackend") == 0)
{
- Assert(argc == 3); /* shouldn't be any more args */
+ CAC_state cac;
+
+ Assert(argc == 4);
+ cac = (CAC_state) atoi(argv[3]);
/*
* Need to reinitialize the SSL library in the backend, since the
@@ -4901,7 +4920,7 @@ SubPostmasterMain(int argc, char *argv[])
* PGPROC slots, we have already initialized libpq and are able to
* report the error to the client.
*/
- BackendInitialize(port);
+ BackendInitialize(port, cac);
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index c57ed12fb6d..335cb2de44a 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -58,17 +58,6 @@ typedef struct
#include "libpq/pqcomm.h"
-typedef enum CAC_state
-{
- CAC_OK,
- CAC_STARTUP,
- CAC_SHUTDOWN,
- CAC_RECOVERY,
- CAC_NOTCONSISTENT,
- CAC_TOOMANY,
-} CAC_state;
-
-
/*
* GSSAPI specific state information
*/
@@ -156,7 +145,6 @@ typedef struct Port
int remote_hostname_resolv; /* see above */
int remote_hostname_errcode; /* see above */
char *remote_port; /* text rep of remote port */
- CAC_state canAcceptConnections; /* postmaster connection status */
/*
* Information that needs to be saved from the startup packet and passed
--
2.39.2
v5-0002-Remove-ConnCreate-and-ConnFree-and-allocate-Port-.patchtext/x-patch; charset=UTF-8; name=v5-0002-Remove-ConnCreate-and-ConnFree-and-allocate-Port-.patchDownload
From ec97a0b68de76126508507fcb4305e6cad09a20a Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Fri, 8 Dec 2023 10:30:37 +0200
Subject: [PATCH v5 2/8] Remove ConnCreate and ConnFree, and allocate Port in
stack
By allocating Port in stack, we don't need to deal with out-of-memory
errors, which makes the code a little less verbose.
XXX: This should perhaps be squashed with the next commit. It changes
how the Port structure is allocated again. But maybe it's easier to
review separately.
Reviewed-by: Tristan Partin, Andres Freund
Discussion: https://www.postgresql.org/message-id/7a59b073-5b5b-151e-7ed3-8b01ff7ce9ef@iki.fi
---
src/backend/postmaster/postmaster.c | 68 +++++------------------------
src/backend/tcop/postgres.c | 6 +--
2 files changed, 13 insertions(+), 61 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index e53057e83a7..8a12ca696de 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -397,8 +397,6 @@ static void CloseServerPorts(int status, Datum arg);
static void unlink_external_pid_file(int status, Datum arg);
static void getInstallationPaths(const char *argv0);
static void checkControlFile(void);
-static Port *ConnCreate(int serverFd);
-static void ConnFree(Port *port);
static void handle_pm_pmsignal_signal(SIGNAL_ARGS);
static void handle_pm_child_exit_signal(SIGNAL_ARGS);
static void handle_pm_reload_request_signal(SIGNAL_ARGS);
@@ -1772,20 +1770,18 @@ ServerLoop(void)
if (events[i].events & WL_SOCKET_ACCEPT)
{
- Port *port;
+ Port port;
- port = ConnCreate(events[i].fd);
- if (port)
- {
- BackendStartup(port);
+ memset(&port, 0, sizeof(port));
+ if (StreamConnection(events[i].fd, &port) == STATUS_OK)
+ BackendStartup(&port);
- /*
- * We no longer need the open socket or port structure in
- * this process
- */
- StreamClose(port->sock);
- ConnFree(port);
- }
+ /*
+ * We no longer need the open socket or port structure in this
+ * process
+ */
+ if (port.sock != PGINVALID_SOCKET)
+ StreamClose(port.sock);
}
}
@@ -2447,50 +2443,6 @@ canAcceptConnections(int backend_type)
return result;
}
-
-/*
- * ConnCreate -- create a local connection data structure
- *
- * Returns NULL on failure, other than out-of-memory which is fatal.
- */
-static Port *
-ConnCreate(int serverFd)
-{
- Port *port;
-
- if (!(port = (Port *) calloc(1, sizeof(Port))))
- {
- ereport(LOG,
- (errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("out of memory")));
- ExitPostmaster(1);
- }
-
- if (StreamConnection(serverFd, port) != STATUS_OK)
- {
- if (port->sock != PGINVALID_SOCKET)
- StreamClose(port->sock);
- ConnFree(port);
- return NULL;
- }
-
- return port;
-}
-
-
-/*
- * ConnFree -- free a local connection data structure
- *
- * Caller has already closed the socket if any, so there's not much
- * to do here.
- */
-static void
-ConnFree(Port *port)
-{
- free(port);
-}
-
-
/*
* ClosePostmasterPorts -- close all the postmaster's open sockets
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 7298a187d18..0dc2132dac5 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -4214,9 +4214,9 @@ PostgresMain(const char *dbname, const char *username)
/*
* If the PostmasterContext is still around, recycle the space; we don't
* need it anymore after InitPostgres completes. Note this does not trash
- * *MyProcPort, because ConnCreate() allocated that space with malloc()
- * ... else we'd need to copy the Port data first. Also, subsidiary data
- * such as the username isn't lost either; see ProcessStartupPacket().
+ * *MyProcPort, because that space is allocated in stack ... else we'd
+ * need to copy the Port data first. Also, subsidiary data such as the
+ * username isn't lost either; see ProcessStartupPacket().
*/
if (PostmasterContext)
{
--
2.39.2
v5-0003-Move-initialization-of-Port-struct-to-child-proce.patchtext/x-patch; charset=UTF-8; name=v5-0003-Move-initialization-of-Port-struct-to-child-proce.patchDownload
From c9fc5a3f55e8d6c0e6933e49a29c0183c5f2ca26 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Fri, 8 Dec 2023 13:54:24 +0200
Subject: [PATCH v5 3/8] Move initialization of Port struct to child process
In postmaster, use a more lightweight ClientSocket struct that
encapsulates just the socket itself and the endpoint addresses that
you get from accept() call. ClientSocket is passed to the child
process, which initializes the bigger Port struct. This makes it more
clear what information postmaster initializes, and what is left to the
child process.
Rename the StreamServerPort, StreamConnection functions to make it
more clear what they do. Remove StreamClose, replacing it with plain
closesocket() calls.
Reviewed-by: Tristan Partin, Andres Freund
Discussion: https://www.postgresql.org/message-id/7a59b073-5b5b-151e-7ed3-8b01ff7ce9ef@iki.fi
---
src/backend/libpq/pqcomm.c | 95 ++++++++--------
src/backend/postmaster/postmaster.c | 163 ++++++++++++++++------------
src/backend/tcop/postgres.c | 5 +-
src/include/libpq/libpq-be.h | 20 +++-
src/include/libpq/libpq.h | 6 +-
src/tools/pgindent/typedefs.list | 1 +
6 files changed, 160 insertions(+), 130 deletions(-)
diff --git a/src/backend/libpq/pqcomm.c b/src/backend/libpq/pqcomm.c
index 2802efc63fc..9eeda9effb3 100644
--- a/src/backend/libpq/pqcomm.c
+++ b/src/backend/libpq/pqcomm.c
@@ -29,11 +29,11 @@
* INTERFACE ROUTINES
*
* setup/teardown:
- * StreamServerPort - Open postmaster's server port
- * StreamConnection - Create new connection with client
- * StreamClose - Close a client/backend connection
+ * ListenServerPort - Open postmaster's server port
+ * AcceptClientConnection - Create new connection with client
+ * InitClientConnection - Initialize a client connection
* TouchSocketFiles - Protect socket files against /tmp cleaners
- * pq_init - initialize libpq at backend startup
+ * pq_init - initialize libpq at backend startup
* socket_comm_reset - reset libpq during error recovery
* socket_close - shutdown libpq at backend exit
*
@@ -304,7 +304,7 @@ socket_close(int code, Datum arg)
/*
- * StreamServerPort -- open a "listening" port to accept connections.
+ * ListenServerPort -- open a "listening" port to accept connections.
*
* family should be AF_UNIX or AF_UNSPEC; portNumber is the port number.
* For AF_UNIX ports, hostName should be NULL and unixSocketDir must be
@@ -319,7 +319,7 @@ socket_close(int code, Datum arg)
* RETURNS: STATUS_OK or STATUS_ERROR
*/
int
-StreamServerPort(int family, const char *hostName, unsigned short portNumber,
+ListenServerPort(int family, const char *hostName, unsigned short portNumber,
const char *unixSocketDir,
pgsocket ListenSockets[], int *NumListenSockets, int MaxListen)
{
@@ -685,8 +685,9 @@ Setup_AF_UNIX(const char *sock_path)
/*
- * StreamConnection -- create a new connection with client using
- * server port. Set port->sock to the FD of the new connection.
+ * AcceptClientConnection -- accept a new connection with client using
+ * server port. Fills *client_sock with the FD and endpoint info
+ * of the new connection.
*
* ASSUME: that this doesn't need to be non-blocking because
* the Postmaster waits for the socket to be ready to accept().
@@ -694,13 +695,13 @@ Setup_AF_UNIX(const char *sock_path)
* RETURNS: STATUS_OK or STATUS_ERROR
*/
int
-StreamConnection(pgsocket server_fd, Port *port)
+AcceptClientConnection(pgsocket server_fd, ClientSocket *client_sock)
{
/* accept connection and fill in the client (remote) address */
- port->raddr.salen = sizeof(port->raddr.addr);
- if ((port->sock = accept(server_fd,
- (struct sockaddr *) &port->raddr.addr,
- &port->raddr.salen)) == PGINVALID_SOCKET)
+ client_sock->raddr.salen = sizeof(client_sock->raddr.addr);
+ if ((client_sock->sock = accept(server_fd,
+ (struct sockaddr *) &client_sock->raddr.addr,
+ &client_sock->raddr.salen)) == PGINVALID_SOCKET)
{
ereport(LOG,
(errcode_for_socket_access(),
@@ -718,10 +719,10 @@ StreamConnection(pgsocket server_fd, Port *port)
}
/* fill in the server (local) address */
- port->laddr.salen = sizeof(port->laddr.addr);
- if (getsockname(port->sock,
- (struct sockaddr *) &port->laddr.addr,
- &port->laddr.salen) < 0)
+ client_sock->laddr.salen = sizeof(client_sock->laddr.addr);
+ if (getsockname(client_sock->sock,
+ (struct sockaddr *) &client_sock->laddr.addr,
+ &client_sock->laddr.salen) < 0)
{
ereport(LOG,
(errmsg("%s() failed: %m", "getsockname")));
@@ -729,7 +730,7 @@ StreamConnection(pgsocket server_fd, Port *port)
}
/* select NODELAY and KEEPALIVE options if it's a TCP connection */
- if (port->laddr.addr.ss_family != AF_UNIX)
+ if (client_sock->laddr.addr.ss_family != AF_UNIX)
{
int on;
#ifdef WIN32
@@ -740,7 +741,7 @@ StreamConnection(pgsocket server_fd, Port *port)
#ifdef TCP_NODELAY
on = 1;
- if (setsockopt(port->sock, IPPROTO_TCP, TCP_NODELAY,
+ if (setsockopt(client_sock->sock, IPPROTO_TCP, TCP_NODELAY,
(char *) &on, sizeof(on)) < 0)
{
ereport(LOG,
@@ -749,7 +750,7 @@ StreamConnection(pgsocket server_fd, Port *port)
}
#endif
on = 1;
- if (setsockopt(port->sock, SOL_SOCKET, SO_KEEPALIVE,
+ if (setsockopt(client_sock->sock, SOL_SOCKET, SO_KEEPALIVE,
(char *) &on, sizeof(on)) < 0)
{
ereport(LOG,
@@ -781,7 +782,7 @@ StreamConnection(pgsocket server_fd, Port *port)
* https://msdn.microsoft.com/en-us/library/bb736549%28v=vs.85%29.aspx
*/
optlen = sizeof(oldopt);
- if (getsockopt(port->sock, SOL_SOCKET, SO_SNDBUF, (char *) &oldopt,
+ if (getsockopt(client_sock->sock, SOL_SOCKET, SO_SNDBUF, (char *) &oldopt,
&optlen) < 0)
{
ereport(LOG,
@@ -791,7 +792,7 @@ StreamConnection(pgsocket server_fd, Port *port)
newopt = PQ_SEND_BUFFER_SIZE * 4;
if (oldopt < newopt)
{
- if (setsockopt(port->sock, SOL_SOCKET, SO_SNDBUF, (char *) &newopt,
+ if (setsockopt(client_sock->sock, SOL_SOCKET, SO_SNDBUF, (char *) &newopt,
sizeof(newopt)) < 0)
{
ereport(LOG,
@@ -800,13 +801,34 @@ StreamConnection(pgsocket server_fd, Port *port)
}
}
#endif
+ }
+ return STATUS_OK;
+}
+
+/*
+ * InitClientConnection -- create a new connection from the given socket.
+ *
+ * This runs in the backend process.
+ */
+Port *
+InitClientConnection(ClientSocket *client_sock)
+{
+ Port *port;
+
+ port = palloc0(sizeof(Port));
+ port->sock = client_sock->sock;
+ port->laddr = client_sock->laddr;
+ port->raddr = client_sock->raddr;
+ /* Apply the current keepalive parameters if it's a TCP connection */
+ if (port->laddr.addr.ss_family != AF_UNIX)
+ {
/*
- * Also apply the current keepalive parameters. If we fail to set a
- * parameter, don't error out, because these aren't universally
- * supported. (Note: you might think we need to reset the GUC
- * variables to 0 in such a case, but it's not necessary because the
- * show hooks for these variables report the truth anyway.)
+ * If we fail to set a parameter, don't error out, because these
+ * aren't universally supported. (Note: you might think we need to
+ * reset the GUC variables to 0 in such a case, but it's not necessary
+ * because the show hooks for these variables report the truth
+ * anyway.)
*/
(void) pq_setkeepalivesidle(tcp_keepalives_idle, port);
(void) pq_setkeepalivesinterval(tcp_keepalives_interval, port);
@@ -814,24 +836,7 @@ StreamConnection(pgsocket server_fd, Port *port)
(void) pq_settcpusertimeout(tcp_user_timeout, port);
}
- return STATUS_OK;
-}
-
-/*
- * StreamClose -- close a client/backend connection
- *
- * NOTE: this is NOT used to terminate a session; it is just used to release
- * the file descriptor in a process that should no longer have the socket
- * open. (For example, the postmaster calls this after passing ownership
- * of the connection to a child process.) It is expected that someone else
- * still has the socket open. So, we only want to close the descriptor,
- * we do NOT want to send anything to the far end.
- */
-void
-StreamClose(pgsocket sock)
-{
- if (closesocket(sock) != 0)
- elog(LOG, "could not close client or listen socket: %m");
+ return port;
}
/*
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 8a12ca696de..257a1bee331 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -425,15 +425,15 @@ typedef enum CAC_state
CAC_TOOMANY,
} CAC_state;
-static void BackendInitialize(Port *port, CAC_state cac);
-static void BackendRun(Port *port) pg_attribute_noreturn();
+static void BackendInitialize(ClientSocket *client_sock, CAC_state cac);
+static void BackendRun(void) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
-static int BackendStartup(Port *port);
+static int BackendStartup(ClientSocket *client_sock);
static int ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done);
static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
static void processCancelRequest(Port *port, void *pkt);
-static void report_fork_failure_to_client(Port *port, int errnum);
+static void report_fork_failure_to_client(ClientSocket *client_sock, int errnum);
static CAC_state canAcceptConnections(int backend_type);
static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
@@ -482,8 +482,8 @@ typedef struct
} win32_deadchild_waitinfo;
#endif /* WIN32 */
-static pid_t backend_forkexec(Port *port, CAC_state cac);
-static pid_t internal_forkexec(int argc, char *argv[], Port *port, BackgroundWorker *worker);
+static pid_t backend_forkexec(ClientSocket *client_sock, CAC_state cac);
+static pid_t internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker);
/* Type for a socket that can be inherited to a client process */
#ifdef WIN32
@@ -502,9 +502,9 @@ typedef int InheritableSocket;
*/
typedef struct
{
- bool has_port;
- Port port;
- InheritableSocket portsocket;
+ bool has_client_sock;
+ ClientSocket client_sock;
+ InheritableSocket inh_sock;
bool has_bgworker;
BackgroundWorker bgworker;
@@ -553,13 +553,13 @@ typedef struct
char pkglib_path[MAXPGPATH];
} BackendParameters;
-static void read_backend_variables(char *id, Port **port, BackgroundWorker **worker);
-static void restore_backend_variables(BackendParameters *param, Port **port, BackgroundWorker **worker);
+static void read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **worker);
+static void restore_backend_variables(BackendParameters *param, ClientSocket **client_sock, BackgroundWorker **worker);
#ifndef WIN32
-static bool save_backend_variables(BackendParameters *param, Port *port, BackgroundWorker *worker);
+static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker);
#else
-static bool save_backend_variables(BackendParameters *param, Port *port, BackgroundWorker *worker,
+static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker,
HANDLE childProcess, pid_t childPid);
#endif
@@ -1219,14 +1219,14 @@ PostmasterMain(int argc, char *argv[])
char *curhost = (char *) lfirst(l);
if (strcmp(curhost, "*") == 0)
- status = StreamServerPort(AF_UNSPEC, NULL,
+ status = ListenServerPort(AF_UNSPEC, NULL,
(unsigned short) PostPortNumber,
NULL,
ListenSockets,
&NumListenSockets,
MAXLISTEN);
else
- status = StreamServerPort(AF_UNSPEC, curhost,
+ status = ListenServerPort(AF_UNSPEC, curhost,
(unsigned short) PostPortNumber,
NULL,
ListenSockets,
@@ -1320,7 +1320,7 @@ PostmasterMain(int argc, char *argv[])
{
char *socketdir = (char *) lfirst(l);
- status = StreamServerPort(AF_UNIX, NULL,
+ status = ListenServerPort(AF_UNIX, NULL,
(unsigned short) PostPortNumber,
socketdir,
ListenSockets,
@@ -1500,7 +1500,10 @@ CloseServerPorts(int status, Datum arg)
* condition if a new postmaster wants to re-use the TCP port number.
*/
for (i = 0; i < NumListenSockets; i++)
- StreamClose(ListenSockets[i]);
+ {
+ if (closesocket(ListenSockets[i]) != 0)
+ elog(LOG, "could not close listen socket: %m");
+ }
NumListenSockets = 0;
/*
@@ -1770,18 +1773,20 @@ ServerLoop(void)
if (events[i].events & WL_SOCKET_ACCEPT)
{
- Port port;
+ ClientSocket s;
- memset(&port, 0, sizeof(port));
- if (StreamConnection(events[i].fd, &port) == STATUS_OK)
- BackendStartup(&port);
+ if (AcceptClientConnection(events[i].fd, &s) == STATUS_OK)
+ BackendStartup(&s);
/*
* We no longer need the open socket or port structure in this
* process
*/
- if (port.sock != PGINVALID_SOCKET)
- StreamClose(port.sock);
+ if (s.sock != PGINVALID_SOCKET)
+ {
+ if (closesocket(s.sock) != 0)
+ elog(LOG, "could not close client socket: %m");
+ }
}
}
@@ -2146,11 +2151,7 @@ retry1:
/*
* Now fetch parameters out of startup packet and save them into the Port
- * structure. All data structures attached to the Port struct must be
- * allocated in TopMemoryContext so that they will remain available in a
- * running backend (even after PostmasterContext is destroyed). We need
- * not worry about leaking this storage on failure, since we aren't in the
- * postmaster process anymore.
+ * structure.
*/
oldcontext = MemoryContextSwitchTo(TopMemoryContext);
@@ -2286,7 +2287,7 @@ retry1:
port->database_name[0] = '\0';
/*
- * Done putting stuff in TopMemoryContext.
+ * Done filling the Port structure
*/
MemoryContextSwitchTo(oldcontext);
@@ -2490,7 +2491,10 @@ ClosePostmasterPorts(bool am_syslogger)
if (ListenSockets)
{
for (int i = 0; i < NumListenSockets; i++)
- StreamClose(ListenSockets[i]);
+ {
+ if (closesocket(ListenSockets[i]) != 0)
+ elog(LOG, "could not close listen socket: %m");
+ }
pfree(ListenSockets);
}
NumListenSockets = 0;
@@ -3996,7 +4000,7 @@ TerminateChildren(int signal)
* Note: if you change this code, also consider StartAutovacuumWorker.
*/
static int
-BackendStartup(Port *port)
+BackendStartup(ClientSocket *client_sock)
{
Backend *bn; /* for backend cleanup */
pid_t pid;
@@ -4047,7 +4051,7 @@ BackendStartup(Port *port)
bn->bgworker_notify = false;
#ifdef EXEC_BACKEND
- pid = backend_forkexec(port, cac);
+ pid = backend_forkexec(client_sock, cac);
#else /* !EXEC_BACKEND */
pid = fork_process();
if (pid == 0) /* child */
@@ -4059,10 +4063,10 @@ BackendStartup(Port *port)
ClosePostmasterPorts(false);
/* Perform additional initialization and collect startup packet */
- BackendInitialize(port, cac);
+ BackendInitialize(client_sock, cac);
/* And run the backend */
- BackendRun(port);
+ BackendRun();
}
#endif /* EXEC_BACKEND */
@@ -4077,14 +4081,14 @@ BackendStartup(Port *port)
errno = save_errno;
ereport(LOG,
(errmsg("could not fork new process for connection: %m")));
- report_fork_failure_to_client(port, save_errno);
+ report_fork_failure_to_client(client_sock, save_errno);
return STATUS_ERROR;
}
/* in parent, successful fork */
ereport(DEBUG2,
(errmsg_internal("forked new backend, pid=%d socket=%d",
- (int) pid, (int) port->sock)));
+ (int) pid, (int) client_sock->sock)));
/*
* Everything's been successful, it's safe to add this backend to our list
@@ -4111,7 +4115,7 @@ BackendStartup(Port *port)
* it's not up and running.
*/
static void
-report_fork_failure_to_client(Port *port, int errnum)
+report_fork_failure_to_client(ClientSocket *client_sock, int errnum)
{
char buffer[1000];
int rc;
@@ -4122,13 +4126,13 @@ report_fork_failure_to_client(Port *port, int errnum)
strerror(errnum));
/* Set port to non-blocking. Don't do send() if this fails */
- if (!pg_set_noblock(port->sock))
+ if (!pg_set_noblock(client_sock->sock))
return;
/* We'll retry after EINTR, but ignore all other failures */
do
{
- rc = send(port->sock, buffer, strlen(buffer) + 1, 0);
+ rc = send(client_sock->sock, buffer, strlen(buffer) + 1, 0);
} while (rc < 0 && errno == EINTR);
}
@@ -4146,18 +4150,30 @@ report_fork_failure_to_client(Port *port, int errnum)
* but have not yet set up most of our local pointers to shmem structures.
*/
static void
-BackendInitialize(Port *port, CAC_state cac)
+BackendInitialize(ClientSocket *client_sock, CAC_state cac)
{
int status;
int ret;
+ Port *port;
char remote_host[NI_MAXHOST];
char remote_port[NI_MAXSERV];
StringInfoData ps_data;
+ MemoryContext oldcontext;
- /* Save port etc. for ps status */
+ /*
+ * Create the Port structure.
+ *
+ * The Port structure and all data structures attached to it are allocated
+ * in TopMemoryContext, so that they survive into PostgresMain execution.
+ * We need not worry about leaking this storage on failure, since we
+ * aren't in the postmaster process anymore.
+ */
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
+ port = InitClientConnection(client_sock);
MyProcPort = port;
+ MemoryContextSwitchTo(oldcontext);
- /* Tell fd.c about the long-lived FD associated with the port */
+ /* Tell fd.c about the long-lived FD associated with the client_sock */
ReserveExternalFD();
/*
@@ -4216,8 +4232,9 @@ BackendInitialize(Port *port, CAC_state cac)
* Save remote_host and remote_port in port structure (after this, they
* will appear in log_line_prefix data for log messages).
*/
- port->remote_host = strdup(remote_host);
- port->remote_port = strdup(remote_port);
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
+ port->remote_host = pstrdup(remote_host);
+ port->remote_port = pstrdup(remote_port);
/* And now we can issue the Log_connections message, if wanted */
if (Log_connections)
@@ -4248,7 +4265,8 @@ BackendInitialize(Port *port, CAC_state cac)
ret == 0 &&
strspn(remote_host, "0123456789.") < strlen(remote_host) &&
strspn(remote_host, "0123456789ABCDEFabcdef:") < strlen(remote_host))
- port->remote_hostname = strdup(remote_host);
+ port->remote_hostname = pstrdup(remote_host);
+ MemoryContextSwitchTo(oldcontext);
/*
* Ready to begin client interaction. We will give up and _exit(1) after
@@ -4369,7 +4387,7 @@ BackendInitialize(Port *port, CAC_state cac)
* Doesn't return at all.
*/
static void
-BackendRun(Port *port)
+BackendRun(void)
{
/*
* Create a per-backend PGPROC struct in shared memory. We must do this
@@ -4383,7 +4401,7 @@ BackendRun(Port *port)
*/
MemoryContextSwitchTo(TopMemoryContext);
- PostgresMain(port->database_name, port->user_name);
+ PostgresMain(MyProcPort->database_name, MyProcPort->user_name);
}
@@ -4417,7 +4435,7 @@ postmaster_forkexec(int argc, char *argv[])
* returns the pid of the fork/exec'd process, or -1 on failure
*/
static pid_t
-backend_forkexec(Port *port, CAC_state cac)
+backend_forkexec(ClientSocket *client_sock, CAC_state cac)
{
char *av[5];
int ac = 0;
@@ -4433,7 +4451,7 @@ backend_forkexec(Port *port, CAC_state cac)
av[ac] = NULL;
Assert(ac < lengthof(av));
- return internal_forkexec(ac, av, port, NULL);
+ return internal_forkexec(ac, av, client_sock, NULL);
}
#ifndef WIN32
@@ -4445,7 +4463,7 @@ backend_forkexec(Port *port, CAC_state cac)
* - fork():s, and then exec():s the child process
*/
static pid_t
-internal_forkexec(int argc, char *argv[], Port *port, BackgroundWorker *worker)
+internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker)
{
static unsigned long tmpBackendFileNum = 0;
pid_t pid;
@@ -4461,7 +4479,7 @@ internal_forkexec(int argc, char *argv[], Port *port, BackgroundWorker *worker)
*/
memset(¶m, 0, sizeof(BackendParameters));
- if (!save_backend_variables(¶m, port, worker))
+ if (!save_backend_variables(¶m, client_sock, worker))
return -1; /* log made by save_backend_variables */
/* Calculate name for temp file */
@@ -4759,7 +4777,7 @@ retry:
void
SubPostmasterMain(int argc, char *argv[])
{
- Port *port;
+ ClientSocket *client_sock;
BackgroundWorker *worker;
/* In EXEC_BACKEND case we will not have inherited these settings */
@@ -4774,7 +4792,7 @@ SubPostmasterMain(int argc, char *argv[])
elog(FATAL, "invalid subpostmaster invocation");
/* Read in the variables file */
- read_backend_variables(argv[2], &port, &worker);
+ read_backend_variables(argv[2], &client_sock, &worker);
/* Close the postmaster's sockets (as soon as we know them) */
ClosePostmasterPorts(strcmp(argv[1], "--forklog") == 0);
@@ -4872,13 +4890,14 @@ SubPostmasterMain(int argc, char *argv[])
* PGPROC slots, we have already initialized libpq and are able to
* report the error to the client.
*/
- BackendInitialize(port, cac);
+ BackendInitialize(client_sock, cac);
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
/* And run the backend */
- BackendRun(port); /* does not return */
+ BackendRun(); /* does not return */
+
}
if (strcmp(argv[1], "--forkaux") == 0)
{
@@ -5959,24 +5978,24 @@ static void read_inheritable_socket(SOCKET *dest, InheritableSocket *src);
/* Save critical backend variables into the BackendParameters struct */
#ifndef WIN32
static bool
-save_backend_variables(BackendParameters *param, Port *port, BackgroundWorker *worker)
+save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker)
#else
static bool
-save_backend_variables(BackendParameters *param, Port *port, BackgroundWorker *worker,
+save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker,
HANDLE childProcess, pid_t childPid)
#endif
{
- if (port)
+ if (client_sock)
{
- memcpy(¶m->port, port, sizeof(Port));
- if (!write_inheritable_socket(¶m->portsocket, port->sock, childPid))
+ memcpy(¶m->client_sock, client_sock, sizeof(ClientSocket));
+ if (!write_inheritable_socket(¶m->inh_sock, client_sock->sock, childPid))
return false;
- param->has_port = true;
+ param->has_client_sock = true;
}
else
{
- memset(¶m->port, 0, sizeof(Port));
- param->has_port = false;
+ memset(¶m->client_sock, 0, sizeof(ClientSocket));
+ param->has_client_sock = false;
}
if (worker)
@@ -6143,7 +6162,7 @@ read_inheritable_socket(SOCKET *dest, InheritableSocket *src)
#endif
static void
-read_backend_variables(char *id, Port **port, BackgroundWorker **worker)
+read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **worker)
{
BackendParameters param;
@@ -6210,21 +6229,21 @@ read_backend_variables(char *id, Port **port, BackgroundWorker **worker)
}
#endif
- restore_backend_variables(¶m, port, worker);
+ restore_backend_variables(¶m, client_sock, worker);
}
/* Restore critical backend variables from the BackendParameters struct */
static void
-restore_backend_variables(BackendParameters *param, Port **port, BackgroundWorker **worker)
+restore_backend_variables(BackendParameters *param, ClientSocket **client_sock, BackgroundWorker **worker)
{
- if (param->has_port)
+ if (param->has_client_sock)
{
- *port = (Port *) MemoryContextAlloc(TopMemoryContext, sizeof(Port));
- memcpy(*port, ¶m->port, sizeof(Port));
- read_inheritable_socket(&(*port)->sock, ¶m->portsocket);
+ *client_sock = (ClientSocket *) MemoryContextAlloc(TopMemoryContext, sizeof(ClientSocket));
+ memcpy(*client_sock, ¶m->client_sock, sizeof(ClientSocket));
+ read_inheritable_socket(&(*client_sock)->sock, ¶m->inh_sock);
}
else
- *port = NULL;
+ *client_sock = NULL;
if (param->has_bgworker)
{
@@ -6290,7 +6309,7 @@ restore_backend_variables(BackendParameters *param, Port **port, BackgroundWorke
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
- * BackendInitialize will handle this for port->sock.)
+ * BackendInitialize will handle this for (*client_sock)->sock.)
*/
#ifndef WIN32
if (postmaster_alive_fds[0] >= 0)
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 0dc2132dac5..ffe554ef43f 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -4213,10 +4213,7 @@ PostgresMain(const char *dbname, const char *username)
/*
* If the PostmasterContext is still around, recycle the space; we don't
- * need it anymore after InitPostgres completes. Note this does not trash
- * *MyProcPort, because that space is allocated in stack ... else we'd
- * need to copy the Port data first. Also, subsidiary data such as the
- * username isn't lost either; see ProcessStartupPacket().
+ * need it anymore after InitPostgres completes.
*/
if (PostmasterContext)
{
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 335cb2de44a..3f8550eb25e 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -110,12 +110,9 @@ typedef struct ClientConnectionInfo
} ClientConnectionInfo;
/*
- * This is used by the postmaster in its communication with frontends. It
- * contains all state information needed during this communication before the
- * backend is run. The Port structure is kept in malloc'd memory and is
- * still available when a backend is running (see MyProcPort). The data
- * it points to must also be malloc'd, or else palloc'd in TopMemoryContext,
- * so that it survives into PostgresMain execution!
+ * The Port structure holds state information about a client connection in a
+ * backend process. It is available in the global variable MyProcPort. The
+ * struct and all the data it points are kept in TopMemoryContext.
*
* remote_hostname is set if we did a successful reverse lookup of the
* client's IP address during connection setup.
@@ -217,6 +214,17 @@ typedef struct Port
#endif
} Port;
+/*
+ * ClientSocket holds a socket for an accepted connection, along with the
+ * information about the endpoints.
+ */
+typedef struct ClientSocket
+{
+ pgsocket sock; /* File descriptor */
+ SockAddr laddr; /* local addr (postmaster) */
+ SockAddr raddr; /* remote addr (client) */
+} ClientSocket;
+
#ifdef USE_SSL
/*
* Hardcoded DH parameters, used in ephemeral DH keying. (See also
diff --git a/src/include/libpq/libpq.h b/src/include/libpq/libpq.h
index a6104d8cd02..16919499368 100644
--- a/src/include/libpq/libpq.h
+++ b/src/include/libpq/libpq.h
@@ -64,11 +64,11 @@ extern PGDLLIMPORT WaitEventSet *FeBeWaitSet;
#define FeBeWaitSetLatchPos 1
#define FeBeWaitSetNEvents 3
-extern int StreamServerPort(int family, const char *hostName,
+extern int ListenServerPort(int family, const char *hostName,
unsigned short portNumber, const char *unixSocketDir,
pgsocket ListenSocket[], int *NumListenSockets, int MaxListen);
-extern int StreamConnection(pgsocket server_fd, Port *port);
-extern void StreamClose(pgsocket sock);
+extern int AcceptClientConnection(pgsocket server_fd, ClientSocket *client_sock);
+extern Port *InitClientConnection(ClientSocket *client_sock);
extern void TouchSocketFiles(void);
extern void RemoveSocketFiles(void);
extern void pq_init(void);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 0a324aa4e7e..90217c4939a 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -382,6 +382,7 @@ ClientCertMode
ClientCertName
ClientConnectionInfo
ClientData
+ClientSocket
ClonePtrType
ClosePortalStmt
ClosePtrType
--
2.39.2
v5-0004-Extract-registration-of-Win32-deadchild-callback-.patchtext/x-patch; charset=UTF-8; name=v5-0004-Extract-registration-of-Win32-deadchild-callback-.patchDownload
From e67d56ca87f300d24cb1b56cf67a9063a496b70e Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Fri, 8 Dec 2023 10:31:49 +0200
Subject: [PATCH v5 4/8] Extract registration of Win32 deadchild callback to
separate function
The next commits will move the internal_forkexec() function to
different source file, but it makes sense to keep all the code related
to the win32 waitpid() emulation in postmaster.c. Split it off to a
separate function now, to make the commit that moves
internal_forkexec() more mechanical.
---
src/backend/postmaster/postmaster.c | 49 ++++++++++++++++++-----------
1 file changed, 30 insertions(+), 19 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 257a1bee331..4022c3c750a 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -471,6 +471,7 @@ static void InitPostmasterDeathWatchHandle(void);
static pid_t waitpid(pid_t pid, int *exitstatus, int options);
static void WINAPI pgwin32_deadchild_callback(PVOID lpParameter, BOOLEAN TimerOrWaitFired);
+static void pgwin32_register_deadchild_callback(HANDLE procHandle, DWORD procId);
static HANDLE win32ChildQueue;
@@ -4735,26 +4736,10 @@ retry:
return -1;
}
- /*
- * Queue a waiter to signal when this child dies. The wait will be handled
- * automatically by an operating system thread pool. The memory will be
- * freed by a later call to waitpid().
- */
- childinfo = palloc(sizeof(win32_deadchild_waitinfo));
- childinfo->procHandle = pi.hProcess;
- childinfo->procId = pi.dwProcessId;
-
- if (!RegisterWaitForSingleObject(&childinfo->waitHandle,
- pi.hProcess,
- pgwin32_deadchild_callback,
- childinfo,
- INFINITE,
- WT_EXECUTEONLYONCE | WT_EXECUTEINWAITTHREAD))
- ereport(FATAL,
- (errmsg_internal("could not register process for wait: error code %lu",
- GetLastError())));
+ /* Set up notification when the child process dies */
+ pgwin32_register_deadchild_callback(pi.hProcess, pi.dwProcessId);
- /* Don't close pi.hProcess here - waitpid() needs access to it */
+ /* Don't close pi.hProcess, it's owned by the deadchild callback now */
CloseHandle(pi.hThread);
@@ -6441,6 +6426,32 @@ pgwin32_deadchild_callback(PVOID lpParameter, BOOLEAN TimerOrWaitFired)
/* Queue SIGCHLD signal. */
pg_queue_signal(SIGCHLD);
}
+
+/*
+ * Queue a waiter to signal when this child dies. The wait will be handled
+ * automatically by an operating system thread pool. The memory and the
+ * process handle will be freed by a later call to waitpid().
+ */
+static void
+pgwin32_register_deadchild_callback(HANDLE procHandle, DWORD procId)
+{
+ win32_deadchild_waitinfo *childinfo;
+
+ childinfo = palloc(sizeof(win32_deadchild_waitinfo));
+ childinfo->procHandle = procHandle;
+ childinfo->procId = procId;
+
+ if (!RegisterWaitForSingleObject(&childinfo->waitHandle,
+ procHandle,
+ pgwin32_deadchild_callback,
+ childinfo,
+ INFINITE,
+ WT_EXECUTEONLYONCE | WT_EXECUTEINWAITTHREAD))
+ ereport(FATAL,
+ (errmsg_internal("could not register process for wait: error code %lu",
+ GetLastError())));
+}
+
#endif /* WIN32 */
/*
--
2.39.2
v5-0005-Move-some-functions-from-postmaster.c-to-new-sour.patchtext/x-patch; charset=UTF-8; name=v5-0005-Move-some-functions-from-postmaster.c-to-new-sour.patchDownload
From 8741662d8edccf962a6d51073fa18ac8d169381f Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Fri, 8 Dec 2023 10:31:57 +0200
Subject: [PATCH v5 5/8] Move some functions from postmaster.c to new source
file
This just moves the functions, with no other changes, to make the next
commits smaller and easier to review. The moved functions are related
to forking and execing a new backend in EXEC_BACKEND mode.
One noteworthy change is that the code to register win32 deadchild
callback is extracted to a separate function, so that it can be called
from the new file without exposing pgwin32_deadchild_callback().
---
src/backend/postmaster/Makefile | 1 +
src/backend/postmaster/launch_backend.c | 813 +++++++++++++++++++++
src/backend/postmaster/meson.build | 1 +
src/backend/postmaster/postmaster.c | 759 +------------------
src/backend/replication/logical/launcher.c | 1 -
src/include/postmaster/postmaster.h | 5 +
6 files changed, 830 insertions(+), 750 deletions(-)
create mode 100644 src/backend/postmaster/launch_backend.c
diff --git a/src/backend/postmaster/Makefile b/src/backend/postmaster/Makefile
index 047448b34eb..c75c540143a 100644
--- a/src/backend/postmaster/Makefile
+++ b/src/backend/postmaster/Makefile
@@ -20,6 +20,7 @@ OBJS = \
checkpointer.o \
fork_process.o \
interrupt.o \
+ launch_backend.o \
pgarch.o \
postmaster.o \
startup.o \
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
new file mode 100644
index 00000000000..65f141a9f5a
--- /dev/null
+++ b/src/backend/postmaster/launch_backend.c
@@ -0,0 +1,813 @@
+/*-------------------------------------------------------------------------
+ *
+ * launch_backend.c
+ * Functions for launching backends and other postmaster child
+ * processes.
+ *
+ * On Unix systems, a new child process is launched with fork(). It inherits
+ * all the global variables and data structures that had been initialized in
+ * the postmaster. After forking, the child process closes the file
+ * descriptors that are not needed in the child process, and sets up the
+ * mechanism to detect death of the parent postmaster process, etc. After
+ * that, it calls the right Main function depending on the kind of child
+ * process.
+ *
+ * In EXEC_BACKEND mode, which is used on Windows but can be enabled on other
+ * platforms for testing, the child process is launched by fork() + exec() (or
+ * CreateProcess() on Windows). It does not inherit the state from the
+ * postmaster, so it needs to re-attach to the shared memory, re-initialize
+ * global variables, re-load the config file etc.
+ *
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/postmaster/launch_backend.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include <unistd.h>
+
+#include "access/xlog.h"
+#include "common/file_utils.h"
+#include "libpq/libpq-be.h"
+#include "libpq/pqsignal.h"
+#include "miscadmin.h"
+#include "nodes/queryjumble.h"
+#include "port.h"
+#include "postmaster/autovacuum.h"
+#include "postmaster/auxprocess.h"
+#include "postmaster/bgworker_internals.h"
+#include "postmaster/bgwriter.h"
+#include "postmaster/fork_process.h"
+#include "postmaster/pgarch.h"
+#include "postmaster/postmaster.h"
+#include "postmaster/startup.h"
+#include "postmaster/syslogger.h"
+#include "postmaster/walwriter.h"
+#include "replication/walreceiver.h"
+#include "storage/fd.h"
+#include "storage/ipc.h"
+#include "storage/pg_shmem.h"
+#include "storage/pmsignal.h"
+#include "storage/proc.h"
+#include "tcop/tcopprot.h"
+#include "utils/builtins.h"
+#include "utils/datetime.h"
+#include "utils/guc.h"
+#include "utils/memutils.h"
+
+#include "utils/timestamp.h"
+
+#ifdef EXEC_BACKEND
+#include "storage/spin.h"
+#endif
+
+
+#ifdef EXEC_BACKEND
+
+/* Type for a socket that can be inherited to a client process */
+#ifdef WIN32
+typedef struct
+{
+ SOCKET origsocket; /* Original socket value, or PGINVALID_SOCKET
+ * if not a socket */
+ WSAPROTOCOL_INFO wsainfo;
+} InheritableSocket;
+#else
+typedef int InheritableSocket;
+#endif
+
+/*
+ * Structure contains all variables passed to exec:ed backends
+ */
+typedef struct
+{
+ bool has_client_sock;
+ ClientSocket client_sock;
+ InheritableSocket inh_sock;
+
+ bool has_bgworker;
+ BackgroundWorker bgworker;
+
+ char DataDir[MAXPGPATH];
+ int32 MyCancelKey;
+ int MyPMChildSlot;
+#ifndef WIN32
+ unsigned long UsedShmemSegID;
+#else
+ void *ShmemProtectiveRegion;
+ HANDLE UsedShmemSegID;
+#endif
+ void *UsedShmemSegAddr;
+ slock_t *ShmemLock;
+ struct bkend *ShmemBackendArray;
+#ifndef HAVE_SPINLOCKS
+ PGSemaphore *SpinlockSemaArray;
+#endif
+ int NamedLWLockTrancheRequests;
+ NamedLWLockTranche *NamedLWLockTrancheArray;
+ LWLockPadded *MainLWLockArray;
+ slock_t *ProcStructLock;
+ PROC_HDR *ProcGlobal;
+ PGPROC *AuxiliaryProcs;
+ PGPROC *PreparedXactProcs;
+ PMSignalData *PMSignalState;
+ pid_t PostmasterPid;
+ TimestampTz PgStartTime;
+ TimestampTz PgReloadTime;
+ pg_time_t first_syslogger_file_time;
+ bool redirection_done;
+ bool IsBinaryUpgrade;
+ bool query_id_enabled;
+ int max_safe_fds;
+ int MaxBackends;
+#ifdef WIN32
+ HANDLE PostmasterHandle;
+ HANDLE initial_signal_pipe;
+ HANDLE syslogPipe[2];
+#else
+ int postmaster_alive_fds[2];
+ int syslogPipe[2];
+#endif
+ char my_exec_path[MAXPGPATH];
+ char pkglib_path[MAXPGPATH];
+} BackendParameters;
+
+#define SizeOfBackendParameters(startup_data_len) (offsetof(BackendParameters, startup_data) + startup_data_len)
+
+void read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **worker);
+static void restore_backend_variables(BackendParameters *param, ClientSocket **client_sock, BackgroundWorker **worker);
+
+#ifndef WIN32
+static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker);
+#else
+static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker,
+ HANDLE childProcess, pid_t childPid);
+#endif
+
+pid_t internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker);
+
+#ifndef WIN32
+
+/*
+ * internal_forkexec non-win32 implementation
+ *
+ * - writes out backend variables to the parameter file
+ * - fork():s, and then exec():s the child process
+ */
+pid_t
+internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker)
+{
+ static unsigned long tmpBackendFileNum = 0;
+ pid_t pid;
+ char tmpfilename[MAXPGPATH];
+ BackendParameters param;
+ FILE *fp;
+
+ /*
+ * Make sure padding bytes are initialized, to prevent Valgrind from
+ * complaining about writing uninitialized bytes to the file. This isn't
+ * performance critical, and the win32 implementation initializes the
+ * padding bytes to zeros, so do it even when not using Valgrind.
+ */
+ memset(¶m, 0, sizeof(BackendParameters));
+
+ if (!save_backend_variables(¶m, client_sock, worker))
+ return -1; /* log made by save_backend_variables */
+
+ /* Calculate name for temp file */
+ snprintf(tmpfilename, MAXPGPATH, "%s/%s.backend_var.%d.%lu",
+ PG_TEMP_FILES_DIR, PG_TEMP_FILE_PREFIX,
+ MyProcPid, ++tmpBackendFileNum);
+
+ /* Open file */
+ fp = AllocateFile(tmpfilename, PG_BINARY_W);
+ if (!fp)
+ {
+ /*
+ * As in OpenTemporaryFileInTablespace, try to make the temp-file
+ * directory, ignoring errors.
+ */
+ (void) MakePGDirectory(PG_TEMP_FILES_DIR);
+
+ fp = AllocateFile(tmpfilename, PG_BINARY_W);
+ if (!fp)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ tmpfilename)));
+ return -1;
+ }
+ }
+
+ if (fwrite(¶m, sizeof(param), 1, fp) != 1)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not write to file \"%s\": %m", tmpfilename)));
+ FreeFile(fp);
+ return -1;
+ }
+
+ /* Release file */
+ if (FreeFile(fp))
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not write to file \"%s\": %m", tmpfilename)));
+ return -1;
+ }
+
+ /* Make sure caller set up argv properly */
+ Assert(argc >= 3);
+ Assert(argv[argc] == NULL);
+ Assert(strncmp(argv[1], "--fork", 6) == 0);
+ Assert(argv[2] == NULL);
+
+ /* Insert temp file name after --fork argument */
+ argv[2] = tmpfilename;
+
+ /* Fire off execv in child */
+ if ((pid = fork_process()) == 0)
+ {
+ if (execv(postgres_exec_path, argv) < 0)
+ {
+ ereport(LOG,
+ (errmsg("could not execute server process \"%s\": %m",
+ postgres_exec_path)));
+ /* We're already in the child process here, can't return */
+ exit(1);
+ }
+ }
+
+ return pid; /* Parent returns pid, or -1 on fork failure */
+}
+#else /* WIN32 */
+
+/*
+ * internal_forkexec win32 implementation
+ *
+ * - starts backend using CreateProcess(), in suspended state
+ * - writes out backend variables to the parameter file
+ * - during this, duplicates handles and sockets required for
+ * inheritance into the new process
+ * - resumes execution of the new process once the backend parameter
+ * file is complete.
+ */
+pid_t
+internal_forkexec(int argc, char *argv[], Port *port, BackgroundWorker *worker)
+{
+ int retry_count = 0;
+ STARTUPINFO si;
+ PROCESS_INFORMATION pi;
+ int i;
+ int j;
+ char cmdLine[MAXPGPATH * 2];
+ HANDLE paramHandle;
+ BackendParameters *param;
+ SECURITY_ATTRIBUTES sa;
+ char paramHandleStr[32];
+ win32_deadchild_waitinfo *childinfo;
+
+ /* Make sure caller set up argv properly */
+ Assert(argc >= 3);
+ Assert(argv[argc] == NULL);
+ Assert(strncmp(argv[1], "--fork", 6) == 0);
+ Assert(argv[2] == NULL);
+
+ /* Resume here if we need to retry */
+retry:
+
+ /* Set up shared memory for parameter passing */
+ ZeroMemory(&sa, sizeof(sa));
+ sa.nLength = sizeof(sa);
+ sa.bInheritHandle = TRUE;
+ paramHandle = CreateFileMapping(INVALID_HANDLE_VALUE,
+ &sa,
+ PAGE_READWRITE,
+ 0,
+ sizeof(BackendParameters),
+ NULL);
+ if (paramHandle == INVALID_HANDLE_VALUE)
+ {
+ ereport(LOG,
+ (errmsg("could not create backend parameter file mapping: error code %lu",
+ GetLastError())));
+ return -1;
+ }
+
+ param = MapViewOfFile(paramHandle, FILE_MAP_WRITE, 0, 0, sizeof(BackendParameters));
+ if (!param)
+ {
+ ereport(LOG,
+ (errmsg("could not map backend parameter memory: error code %lu",
+ GetLastError())));
+ CloseHandle(paramHandle);
+ return -1;
+ }
+
+ /* Insert temp file name after --fork argument */
+#ifdef _WIN64
+ sprintf(paramHandleStr, "%llu", (LONG_PTR) paramHandle);
+#else
+ sprintf(paramHandleStr, "%lu", (DWORD) paramHandle);
+#endif
+ argv[2] = paramHandleStr;
+
+ /* Format the cmd line */
+ cmdLine[sizeof(cmdLine) - 1] = '\0';
+ cmdLine[sizeof(cmdLine) - 2] = '\0';
+ snprintf(cmdLine, sizeof(cmdLine) - 1, "\"%s\"", postgres_exec_path);
+ i = 0;
+ while (argv[++i] != NULL)
+ {
+ j = strlen(cmdLine);
+ snprintf(cmdLine + j, sizeof(cmdLine) - 1 - j, " \"%s\"", argv[i]);
+ }
+ if (cmdLine[sizeof(cmdLine) - 2] != '\0')
+ {
+ ereport(LOG,
+ (errmsg("subprocess command line too long")));
+ UnmapViewOfFile(param);
+ CloseHandle(paramHandle);
+ return -1;
+ }
+
+ memset(&pi, 0, sizeof(pi));
+ memset(&si, 0, sizeof(si));
+ si.cb = sizeof(si);
+
+ /*
+ * Create the subprocess in a suspended state. This will be resumed later,
+ * once we have written out the parameter file.
+ */
+ if (!CreateProcess(NULL, cmdLine, NULL, NULL, TRUE, CREATE_SUSPENDED,
+ NULL, NULL, &si, &pi))
+ {
+ ereport(LOG,
+ (errmsg("CreateProcess() call failed: %m (error code %lu)",
+ GetLastError())));
+ UnmapViewOfFile(param);
+ CloseHandle(paramHandle);
+ return -1;
+ }
+
+ if (!save_backend_variables(param, port, worker, pi.hProcess, pi.dwProcessId))
+ {
+ /*
+ * log made by save_backend_variables, but we have to clean up the
+ * mess with the half-started process
+ */
+ if (!TerminateProcess(pi.hProcess, 255))
+ ereport(LOG,
+ (errmsg_internal("could not terminate unstarted process: error code %lu",
+ GetLastError())));
+ CloseHandle(pi.hProcess);
+ CloseHandle(pi.hThread);
+ UnmapViewOfFile(param);
+ CloseHandle(paramHandle);
+ return -1; /* log made by save_backend_variables */
+ }
+
+ /* Drop the parameter shared memory that is now inherited to the backend */
+ if (!UnmapViewOfFile(param))
+ ereport(LOG,
+ (errmsg("could not unmap view of backend parameter file: error code %lu",
+ GetLastError())));
+ if (!CloseHandle(paramHandle))
+ ereport(LOG,
+ (errmsg("could not close handle to backend parameter file: error code %lu",
+ GetLastError())));
+
+ /*
+ * Reserve the memory region used by our main shared memory segment before
+ * we resume the child process. Normally this should succeed, but if ASLR
+ * is active then it might sometimes fail due to the stack or heap having
+ * gotten mapped into that range. In that case, just terminate the
+ * process and retry.
+ */
+ if (!pgwin32_ReserveSharedMemoryRegion(pi.hProcess))
+ {
+ /* pgwin32_ReserveSharedMemoryRegion already made a log entry */
+ if (!TerminateProcess(pi.hProcess, 255))
+ ereport(LOG,
+ (errmsg_internal("could not terminate process that failed to reserve memory: error code %lu",
+ GetLastError())));
+ CloseHandle(pi.hProcess);
+ CloseHandle(pi.hThread);
+ if (++retry_count < 100)
+ goto retry;
+ ereport(LOG,
+ (errmsg("giving up after too many tries to reserve shared memory"),
+ errhint("This might be caused by ASLR or antivirus software.")));
+ return -1;
+ }
+
+ /*
+ * Now that the backend variables are written out, we start the child
+ * thread so it can start initializing while we set up the rest of the
+ * parent state.
+ */
+ if (ResumeThread(pi.hThread) == -1)
+ {
+ if (!TerminateProcess(pi.hProcess, 255))
+ {
+ ereport(LOG,
+ (errmsg_internal("could not terminate unstartable process: error code %lu",
+ GetLastError())));
+ CloseHandle(pi.hProcess);
+ CloseHandle(pi.hThread);
+ return -1;
+ }
+ CloseHandle(pi.hProcess);
+ CloseHandle(pi.hThread);
+ ereport(LOG,
+ (errmsg_internal("could not resume thread of unstarted process: error code %lu",
+ GetLastError())));
+ return -1;
+ }
+
+ /* Set up notification when the child process dies */
+ pgwin32_register_deadchild_callback(pi.hProcess, pi.dwProcessId);
+
+ /* Don't close pi.hProcess, it's owned by the deadchild callback now */
+
+ CloseHandle(pi.hThread);
+
+ return pi.dwProcessId;
+}
+#endif /* WIN32 */
+
+/*
+ * The following need to be available to the save/restore_backend_variables
+ * functions. They are marked NON_EXEC_STATIC in their home modules.
+ */
+extern slock_t *ShmemLock;
+extern slock_t *ProcStructLock;
+extern PGPROC *AuxiliaryProcs;
+extern PMSignalData *PMSignalState;
+extern pg_time_t first_syslogger_file_time;
+extern struct bkend *ShmemBackendArray;
+extern bool redirection_done;
+
+/* Save critical backend variables into the BackendParameters struct */
+#ifndef WIN32
+#define write_inheritable_socket(dest, src, childpid) ((*(dest) = (src)), true)
+#define read_inheritable_socket(dest, src) (*(dest) = *(src))
+#else
+static bool write_duplicated_handle(HANDLE *dest, HANDLE src, HANDLE child);
+static bool write_inheritable_socket(InheritableSocket *dest, SOCKET src,
+ pid_t childPid);
+static void read_inheritable_socket(SOCKET *dest, InheritableSocket *src);
+#endif
+
+#ifndef WIN32
+static bool
+save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker)
+#else
+static bool
+save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker,
+ HANDLE childProcess, pid_t childPid)
+#endif
+{
+ if (client_sock)
+ {
+ memcpy(¶m->client_sock, client_sock, sizeof(ClientSocket));
+ if (!write_inheritable_socket(¶m->inh_sock, client_sock->sock, childPid))
+ return false;
+ param->has_client_sock = true;
+ }
+ else
+ {
+ memset(¶m->client_sock, 0, sizeof(ClientSocket));
+ param->has_client_sock = false;
+ }
+
+ if (worker)
+ {
+ memcpy(¶m->bgworker, worker, sizeof(BackgroundWorker));
+ param->has_bgworker = true;
+ }
+ else
+ {
+ memset(¶m->bgworker, 0, sizeof(BackgroundWorker));
+ param->has_bgworker = false;
+ }
+
+ strlcpy(param->DataDir, DataDir, MAXPGPATH);
+
+ param->MyCancelKey = MyCancelKey;
+ param->MyPMChildSlot = MyPMChildSlot;
+
+#ifdef WIN32
+ param->ShmemProtectiveRegion = ShmemProtectiveRegion;
+#endif
+ param->UsedShmemSegID = UsedShmemSegID;
+ param->UsedShmemSegAddr = UsedShmemSegAddr;
+
+ param->ShmemLock = ShmemLock;
+ param->ShmemBackendArray = ShmemBackendArray;
+
+#ifndef HAVE_SPINLOCKS
+ param->SpinlockSemaArray = SpinlockSemaArray;
+#endif
+ param->NamedLWLockTrancheRequests = NamedLWLockTrancheRequests;
+ param->NamedLWLockTrancheArray = NamedLWLockTrancheArray;
+ param->MainLWLockArray = MainLWLockArray;
+ param->ProcStructLock = ProcStructLock;
+ param->ProcGlobal = ProcGlobal;
+ param->AuxiliaryProcs = AuxiliaryProcs;
+ param->PreparedXactProcs = PreparedXactProcs;
+ param->PMSignalState = PMSignalState;
+
+ param->PostmasterPid = PostmasterPid;
+ param->PgStartTime = PgStartTime;
+ param->PgReloadTime = PgReloadTime;
+ param->first_syslogger_file_time = first_syslogger_file_time;
+
+ param->redirection_done = redirection_done;
+ param->IsBinaryUpgrade = IsBinaryUpgrade;
+ param->query_id_enabled = query_id_enabled;
+ param->max_safe_fds = max_safe_fds;
+
+ param->MaxBackends = MaxBackends;
+
+#ifdef WIN32
+ param->PostmasterHandle = PostmasterHandle;
+ if (!write_duplicated_handle(¶m->initial_signal_pipe,
+ pgwin32_create_signal_listener(childPid),
+ childProcess))
+ return false;
+#else
+ memcpy(¶m->postmaster_alive_fds, &postmaster_alive_fds,
+ sizeof(postmaster_alive_fds));
+#endif
+
+ memcpy(¶m->syslogPipe, &syslogPipe, sizeof(syslogPipe));
+
+ strlcpy(param->my_exec_path, my_exec_path, MAXPGPATH);
+
+ strlcpy(param->pkglib_path, pkglib_path, MAXPGPATH);
+
+ return true;
+}
+
+#ifdef WIN32
+/*
+ * Duplicate a handle for usage in a child process, and write the child
+ * process instance of the handle to the parameter file.
+ */
+static bool
+write_duplicated_handle(HANDLE *dest, HANDLE src, HANDLE childProcess)
+{
+ HANDLE hChild = INVALID_HANDLE_VALUE;
+
+ if (!DuplicateHandle(GetCurrentProcess(),
+ src,
+ childProcess,
+ &hChild,
+ 0,
+ TRUE,
+ DUPLICATE_CLOSE_SOURCE | DUPLICATE_SAME_ACCESS))
+ {
+ ereport(LOG,
+ (errmsg_internal("could not duplicate handle to be written to backend parameter file: error code %lu",
+ GetLastError())));
+ return false;
+ }
+
+ *dest = hChild;
+ return true;
+}
+
+/*
+ * Duplicate a socket for usage in a child process, and write the resulting
+ * structure to the parameter file.
+ * This is required because a number of LSPs (Layered Service Providers) very
+ * common on Windows (antivirus, firewalls, download managers etc) break
+ * straight socket inheritance.
+ */
+static bool
+write_inheritable_socket(InheritableSocket *dest, SOCKET src, pid_t childpid)
+{
+ dest->origsocket = src;
+ if (src != 0 && src != PGINVALID_SOCKET)
+ {
+ /* Actual socket */
+ if (WSADuplicateSocket(src, childpid, &dest->wsainfo) != 0)
+ {
+ ereport(LOG,
+ (errmsg("could not duplicate socket %d for use in backend: error code %d",
+ (int) src, WSAGetLastError())));
+ return false;
+ }
+ }
+ return true;
+}
+
+/*
+ * Read a duplicate socket structure back, and get the socket descriptor.
+ */
+static void
+read_inheritable_socket(SOCKET *dest, InheritableSocket *src)
+{
+ SOCKET s;
+
+ if (src->origsocket == PGINVALID_SOCKET || src->origsocket == 0)
+ {
+ /* Not a real socket! */
+ *dest = src->origsocket;
+ }
+ else
+ {
+ /* Actual socket, so create from structure */
+ s = WSASocket(FROM_PROTOCOL_INFO,
+ FROM_PROTOCOL_INFO,
+ FROM_PROTOCOL_INFO,
+ &src->wsainfo,
+ 0,
+ 0);
+ if (s == INVALID_SOCKET)
+ {
+ write_stderr("could not create inherited socket: error code %d\n",
+ WSAGetLastError());
+ exit(1);
+ }
+ *dest = s;
+
+ /*
+ * To make sure we don't get two references to the same socket, close
+ * the original one. (This would happen when inheritance actually
+ * works..
+ */
+ closesocket(src->origsocket);
+ }
+}
+#endif
+
+void
+read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **worker)
+{
+ BackendParameters param;
+
+#ifndef WIN32
+ /* Non-win32 implementation reads from file */
+ FILE *fp;
+
+ /* Open file */
+ fp = AllocateFile(id, PG_BINARY_R);
+ if (!fp)
+ {
+ write_stderr("could not open backend variables file \"%s\": %s\n",
+ id, strerror(errno));
+ exit(1);
+ }
+
+ if (fread(¶m, sizeof(param), 1, fp) != 1)
+ {
+ write_stderr("could not read from backend variables file \"%s\": %s\n",
+ id, strerror(errno));
+ exit(1);
+ }
+
+ /* Release file */
+ FreeFile(fp);
+ if (unlink(id) != 0)
+ {
+ write_stderr("could not remove file \"%s\": %s\n",
+ id, strerror(errno));
+ exit(1);
+ }
+#else
+ /* Win32 version uses mapped file */
+ HANDLE paramHandle;
+ BackendParameters *paramp;
+
+#ifdef _WIN64
+ paramHandle = (HANDLE) _atoi64(id);
+#else
+ paramHandle = (HANDLE) atol(id);
+#endif
+ paramp = MapViewOfFile(paramHandle, FILE_MAP_READ, 0, 0, 0);
+ if (!paramp)
+ {
+ write_stderr("could not map view of backend variables: error code %lu\n",
+ GetLastError());
+ exit(1);
+ }
+
+ memcpy(¶m, paramp, sizeof(BackendParameters));
+
+ if (!UnmapViewOfFile(paramp))
+ {
+ write_stderr("could not unmap view of backend variables: error code %lu\n",
+ GetLastError());
+ exit(1);
+ }
+
+ if (!CloseHandle(paramHandle))
+ {
+ write_stderr("could not close handle to backend parameter variables: error code %lu\n",
+ GetLastError());
+ exit(1);
+ }
+#endif
+
+ restore_backend_variables(¶m, client_sock, worker);
+}
+
+/* Restore critical backend variables from the BackendParameters struct */
+static void
+restore_backend_variables(BackendParameters *param, ClientSocket **client_sock, BackgroundWorker **worker)
+{
+ if (param->has_client_sock)
+ {
+ *client_sock = (ClientSocket *) MemoryContextAlloc(TopMemoryContext, sizeof(ClientSocket));
+ memcpy(*client_sock, ¶m->client_sock, sizeof(ClientSocket));
+ read_inheritable_socket(&(*client_sock)->sock, ¶m->inh_sock);
+ }
+ else
+ *client_sock = NULL;
+
+ if (param->has_bgworker)
+ {
+ *worker = (BackgroundWorker *)
+ MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
+ memcpy(*worker, ¶m->bgworker, sizeof(BackgroundWorker));
+ }
+ else
+ *worker = NULL;
+
+ SetDataDir(param->DataDir);
+
+ MyCancelKey = param->MyCancelKey;
+ MyPMChildSlot = param->MyPMChildSlot;
+
+#ifdef WIN32
+ ShmemProtectiveRegion = param->ShmemProtectiveRegion;
+#endif
+ UsedShmemSegID = param->UsedShmemSegID;
+ UsedShmemSegAddr = param->UsedShmemSegAddr;
+
+ ShmemLock = param->ShmemLock;
+ ShmemBackendArray = param->ShmemBackendArray;
+
+#ifndef HAVE_SPINLOCKS
+ SpinlockSemaArray = param->SpinlockSemaArray;
+#endif
+ NamedLWLockTrancheRequests = param->NamedLWLockTrancheRequests;
+ NamedLWLockTrancheArray = param->NamedLWLockTrancheArray;
+ MainLWLockArray = param->MainLWLockArray;
+ ProcStructLock = param->ProcStructLock;
+ ProcGlobal = param->ProcGlobal;
+ AuxiliaryProcs = param->AuxiliaryProcs;
+ PreparedXactProcs = param->PreparedXactProcs;
+ PMSignalState = param->PMSignalState;
+
+ PostmasterPid = param->PostmasterPid;
+ PgStartTime = param->PgStartTime;
+ PgReloadTime = param->PgReloadTime;
+ first_syslogger_file_time = param->first_syslogger_file_time;
+
+ redirection_done = param->redirection_done;
+ IsBinaryUpgrade = param->IsBinaryUpgrade;
+ query_id_enabled = param->query_id_enabled;
+ max_safe_fds = param->max_safe_fds;
+
+ MaxBackends = param->MaxBackends;
+
+#ifdef WIN32
+ PostmasterHandle = param->PostmasterHandle;
+ pgwin32_initial_signal_pipe = param->initial_signal_pipe;
+#else
+ memcpy(&postmaster_alive_fds, ¶m->postmaster_alive_fds,
+ sizeof(postmaster_alive_fds));
+#endif
+
+ memcpy(&syslogPipe, ¶m->syslogPipe, sizeof(syslogPipe));
+
+ strlcpy(my_exec_path, param->my_exec_path, MAXPGPATH);
+
+ strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+
+ /*
+ * We need to restore fd.c's counts of externally-opened FDs; to avoid
+ * confusion, be sure to do this after restoring max_safe_fds. (Note:
+ * BackendInitialize will handle this for (*client_sock)->sock.)
+ */
+#ifndef WIN32
+ if (postmaster_alive_fds[0] >= 0)
+ ReserveExternalFD();
+ if (postmaster_alive_fds[1] >= 0)
+ ReserveExternalFD();
+#endif
+}
+
+#endif /* EXEC_BACKEND */
diff --git a/src/backend/postmaster/meson.build b/src/backend/postmaster/meson.build
index cda921fd10b..313f46d597b 100644
--- a/src/backend/postmaster/meson.build
+++ b/src/backend/postmaster/meson.build
@@ -8,6 +8,7 @@ backend_sources += files(
'checkpointer.c',
'fork_process.c',
'interrupt.c',
+ 'launch_backend.c',
'pgarch.c',
'postmaster.c',
'startup.c',
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 4022c3c750a..586aac78d53 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -102,7 +102,6 @@
#include "libpq/libpq.h"
#include "libpq/pqformat.h"
#include "libpq/pqsignal.h"
-#include "nodes/queryjumble.h"
#include "pg_getopt.h"
#include "pgstat.h"
#include "port/pg_bswap.h"
@@ -130,10 +129,6 @@
#include "utils/timestamp.h"
#include "utils/varlena.h"
-#ifdef EXEC_BACKEND
-#include "storage/spin.h"
-#endif
-
/*
* Possible types of a backend. Beyond being the possible bkend_type values in
@@ -186,7 +181,7 @@ typedef struct bkend
static dlist_head BackendList = DLIST_STATIC_INIT(BackendList);
#ifdef EXEC_BACKEND
-static Backend *ShmemBackendArray;
+Backend *ShmemBackendArray;
#endif
BackgroundWorker *MyBgworkerEntry = NULL;
@@ -471,7 +466,6 @@ static void InitPostmasterDeathWatchHandle(void);
static pid_t waitpid(pid_t pid, int *exitstatus, int options);
static void WINAPI pgwin32_deadchild_callback(PVOID lpParameter, BOOLEAN TimerOrWaitFired);
-static void pgwin32_register_deadchild_callback(HANDLE procHandle, DWORD procId);
static HANDLE win32ChildQueue;
@@ -484,85 +478,11 @@ typedef struct
#endif /* WIN32 */
static pid_t backend_forkexec(ClientSocket *client_sock, CAC_state cac);
-static pid_t internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker);
-
-/* Type for a socket that can be inherited to a client process */
-#ifdef WIN32
-typedef struct
-{
- SOCKET origsocket; /* Original socket value, or PGINVALID_SOCKET
- * if not a socket */
- WSAPROTOCOL_INFO wsainfo;
-} InheritableSocket;
-#else
-typedef int InheritableSocket;
-#endif
-
-/*
- * Structure contains all variables passed to exec:ed backends
- */
-typedef struct
-{
- bool has_client_sock;
- ClientSocket client_sock;
- InheritableSocket inh_sock;
-
- bool has_bgworker;
- BackgroundWorker bgworker;
-
- char DataDir[MAXPGPATH];
- int32 MyCancelKey;
- int MyPMChildSlot;
-#ifndef WIN32
- unsigned long UsedShmemSegID;
-#else
- void *ShmemProtectiveRegion;
- HANDLE UsedShmemSegID;
-#endif
- void *UsedShmemSegAddr;
- slock_t *ShmemLock;
- Backend *ShmemBackendArray;
-#ifndef HAVE_SPINLOCKS
- PGSemaphore *SpinlockSemaArray;
-#endif
- int NamedLWLockTrancheRequests;
- NamedLWLockTranche *NamedLWLockTrancheArray;
- LWLockPadded *MainLWLockArray;
- slock_t *ProcStructLock;
- PROC_HDR *ProcGlobal;
- PGPROC *AuxiliaryProcs;
- PGPROC *PreparedXactProcs;
- PMSignalData *PMSignalState;
- pid_t PostmasterPid;
- TimestampTz PgStartTime;
- TimestampTz PgReloadTime;
- pg_time_t first_syslogger_file_time;
- bool redirection_done;
- bool IsBinaryUpgrade;
- bool query_id_enabled;
- int max_safe_fds;
- int MaxBackends;
-#ifdef WIN32
- HANDLE PostmasterHandle;
- HANDLE initial_signal_pipe;
- HANDLE syslogPipe[2];
-#else
- int postmaster_alive_fds[2];
- int syslogPipe[2];
-#endif
- char my_exec_path[MAXPGPATH];
- char pkglib_path[MAXPGPATH];
-} BackendParameters;
-static void read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **worker);
-static void restore_backend_variables(BackendParameters *param, ClientSocket **client_sock, BackgroundWorker **worker);
-#ifndef WIN32
-static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker);
-#else
-static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker,
- HANDLE childProcess, pid_t childPid);
-#endif
+/* in launch_backend.c */
+extern pid_t internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker);
+extern void read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **worker);
static void ShmemBackendArrayAdd(Backend *bn);
static void ShmemBackendArrayRemove(Backend *bn);
@@ -1116,11 +1036,11 @@ PostmasterMain(int argc, char *argv[])
/*
* Clean out the temp directory used to transmit parameters to child
- * processes (see internal_forkexec, below). We must do this before
- * launching any child processes, else we have a race condition: we could
- * remove a parameter file before the child can read it. It should be
- * safe to do so now, because we verified earlier that there are no
- * conflicting Postgres processes in this data directory.
+ * processes (see internal_forkexec). We must do this before launching
+ * any child processes, else we have a race condition: we could remove a
+ * parameter file before the child can read it. It should be safe to do
+ * so now, because we verified earlier that there are no conflicting
+ * Postgres processes in this data directory.
*/
RemovePgTempFilesInDir(PG_TEMP_FILES_DIR, true, false);
#endif
@@ -4455,299 +4375,6 @@ backend_forkexec(ClientSocket *client_sock, CAC_state cac)
return internal_forkexec(ac, av, client_sock, NULL);
}
-#ifndef WIN32
-
-/*
- * internal_forkexec non-win32 implementation
- *
- * - writes out backend variables to the parameter file
- * - fork():s, and then exec():s the child process
- */
-static pid_t
-internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker)
-{
- static unsigned long tmpBackendFileNum = 0;
- pid_t pid;
- char tmpfilename[MAXPGPATH];
- BackendParameters param;
- FILE *fp;
-
- /*
- * Make sure padding bytes are initialized, to prevent Valgrind from
- * complaining about writing uninitialized bytes to the file. This isn't
- * performance critical, and the win32 implementation initializes the
- * padding bytes to zeros, so do it even when not using Valgrind.
- */
- memset(¶m, 0, sizeof(BackendParameters));
-
- if (!save_backend_variables(¶m, client_sock, worker))
- return -1; /* log made by save_backend_variables */
-
- /* Calculate name for temp file */
- snprintf(tmpfilename, MAXPGPATH, "%s/%s.backend_var.%d.%lu",
- PG_TEMP_FILES_DIR, PG_TEMP_FILE_PREFIX,
- MyProcPid, ++tmpBackendFileNum);
-
- /* Open file */
- fp = AllocateFile(tmpfilename, PG_BINARY_W);
- if (!fp)
- {
- /*
- * As in OpenTemporaryFileInTablespace, try to make the temp-file
- * directory, ignoring errors.
- */
- (void) MakePGDirectory(PG_TEMP_FILES_DIR);
-
- fp = AllocateFile(tmpfilename, PG_BINARY_W);
- if (!fp)
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not create file \"%s\": %m",
- tmpfilename)));
- return -1;
- }
- }
-
- if (fwrite(¶m, sizeof(param), 1, fp) != 1)
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not write to file \"%s\": %m", tmpfilename)));
- FreeFile(fp);
- return -1;
- }
-
- /* Release file */
- if (FreeFile(fp))
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not write to file \"%s\": %m", tmpfilename)));
- return -1;
- }
-
- /* Make sure caller set up argv properly */
- Assert(argc >= 3);
- Assert(argv[argc] == NULL);
- Assert(strncmp(argv[1], "--fork", 6) == 0);
- Assert(argv[2] == NULL);
-
- /* Insert temp file name after --fork argument */
- argv[2] = tmpfilename;
-
- /* Fire off execv in child */
- if ((pid = fork_process()) == 0)
- {
- if (execv(postgres_exec_path, argv) < 0)
- {
- ereport(LOG,
- (errmsg("could not execute server process \"%s\": %m",
- postgres_exec_path)));
- /* We're already in the child process here, can't return */
- exit(1);
- }
- }
-
- return pid; /* Parent returns pid, or -1 on fork failure */
-}
-#else /* WIN32 */
-
-/*
- * internal_forkexec win32 implementation
- *
- * - starts backend using CreateProcess(), in suspended state
- * - writes out backend variables to the parameter file
- * - during this, duplicates handles and sockets required for
- * inheritance into the new process
- * - resumes execution of the new process once the backend parameter
- * file is complete.
- */
-static pid_t
-internal_forkexec(int argc, char *argv[], Port *port, BackgroundWorker *worker)
-{
- int retry_count = 0;
- STARTUPINFO si;
- PROCESS_INFORMATION pi;
- int i;
- int j;
- char cmdLine[MAXPGPATH * 2];
- HANDLE paramHandle;
- BackendParameters *param;
- SECURITY_ATTRIBUTES sa;
- char paramHandleStr[32];
- win32_deadchild_waitinfo *childinfo;
-
- /* Make sure caller set up argv properly */
- Assert(argc >= 3);
- Assert(argv[argc] == NULL);
- Assert(strncmp(argv[1], "--fork", 6) == 0);
- Assert(argv[2] == NULL);
-
- /* Resume here if we need to retry */
-retry:
-
- /* Set up shared memory for parameter passing */
- ZeroMemory(&sa, sizeof(sa));
- sa.nLength = sizeof(sa);
- sa.bInheritHandle = TRUE;
- paramHandle = CreateFileMapping(INVALID_HANDLE_VALUE,
- &sa,
- PAGE_READWRITE,
- 0,
- sizeof(BackendParameters),
- NULL);
- if (paramHandle == INVALID_HANDLE_VALUE)
- {
- ereport(LOG,
- (errmsg("could not create backend parameter file mapping: error code %lu",
- GetLastError())));
- return -1;
- }
-
- param = MapViewOfFile(paramHandle, FILE_MAP_WRITE, 0, 0, sizeof(BackendParameters));
- if (!param)
- {
- ereport(LOG,
- (errmsg("could not map backend parameter memory: error code %lu",
- GetLastError())));
- CloseHandle(paramHandle);
- return -1;
- }
-
- /* Insert temp file name after --fork argument */
-#ifdef _WIN64
- sprintf(paramHandleStr, "%llu", (LONG_PTR) paramHandle);
-#else
- sprintf(paramHandleStr, "%lu", (DWORD) paramHandle);
-#endif
- argv[2] = paramHandleStr;
-
- /* Format the cmd line */
- cmdLine[sizeof(cmdLine) - 1] = '\0';
- cmdLine[sizeof(cmdLine) - 2] = '\0';
- snprintf(cmdLine, sizeof(cmdLine) - 1, "\"%s\"", postgres_exec_path);
- i = 0;
- while (argv[++i] != NULL)
- {
- j = strlen(cmdLine);
- snprintf(cmdLine + j, sizeof(cmdLine) - 1 - j, " \"%s\"", argv[i]);
- }
- if (cmdLine[sizeof(cmdLine) - 2] != '\0')
- {
- ereport(LOG,
- (errmsg("subprocess command line too long")));
- UnmapViewOfFile(param);
- CloseHandle(paramHandle);
- return -1;
- }
-
- memset(&pi, 0, sizeof(pi));
- memset(&si, 0, sizeof(si));
- si.cb = sizeof(si);
-
- /*
- * Create the subprocess in a suspended state. This will be resumed later,
- * once we have written out the parameter file.
- */
- if (!CreateProcess(NULL, cmdLine, NULL, NULL, TRUE, CREATE_SUSPENDED,
- NULL, NULL, &si, &pi))
- {
- ereport(LOG,
- (errmsg("CreateProcess() call failed: %m (error code %lu)",
- GetLastError())));
- UnmapViewOfFile(param);
- CloseHandle(paramHandle);
- return -1;
- }
-
- if (!save_backend_variables(param, port, worker, pi.hProcess, pi.dwProcessId))
- {
- /*
- * log made by save_backend_variables, but we have to clean up the
- * mess with the half-started process
- */
- if (!TerminateProcess(pi.hProcess, 255))
- ereport(LOG,
- (errmsg_internal("could not terminate unstarted process: error code %lu",
- GetLastError())));
- CloseHandle(pi.hProcess);
- CloseHandle(pi.hThread);
- UnmapViewOfFile(param);
- CloseHandle(paramHandle);
- return -1; /* log made by save_backend_variables */
- }
-
- /* Drop the parameter shared memory that is now inherited to the backend */
- if (!UnmapViewOfFile(param))
- ereport(LOG,
- (errmsg("could not unmap view of backend parameter file: error code %lu",
- GetLastError())));
- if (!CloseHandle(paramHandle))
- ereport(LOG,
- (errmsg("could not close handle to backend parameter file: error code %lu",
- GetLastError())));
-
- /*
- * Reserve the memory region used by our main shared memory segment before
- * we resume the child process. Normally this should succeed, but if ASLR
- * is active then it might sometimes fail due to the stack or heap having
- * gotten mapped into that range. In that case, just terminate the
- * process and retry.
- */
- if (!pgwin32_ReserveSharedMemoryRegion(pi.hProcess))
- {
- /* pgwin32_ReserveSharedMemoryRegion already made a log entry */
- if (!TerminateProcess(pi.hProcess, 255))
- ereport(LOG,
- (errmsg_internal("could not terminate process that failed to reserve memory: error code %lu",
- GetLastError())));
- CloseHandle(pi.hProcess);
- CloseHandle(pi.hThread);
- if (++retry_count < 100)
- goto retry;
- ereport(LOG,
- (errmsg("giving up after too many tries to reserve shared memory"),
- errhint("This might be caused by ASLR or antivirus software.")));
- return -1;
- }
-
- /*
- * Now that the backend variables are written out, we start the child
- * thread so it can start initializing while we set up the rest of the
- * parent state.
- */
- if (ResumeThread(pi.hThread) == -1)
- {
- if (!TerminateProcess(pi.hProcess, 255))
- {
- ereport(LOG,
- (errmsg_internal("could not terminate unstartable process: error code %lu",
- GetLastError())));
- CloseHandle(pi.hProcess);
- CloseHandle(pi.hThread);
- return -1;
- }
- CloseHandle(pi.hProcess);
- CloseHandle(pi.hThread);
- ereport(LOG,
- (errmsg_internal("could not resume thread of unstarted process: error code %lu",
- GetLastError())));
- return -1;
- }
-
- /* Set up notification when the child process dies */
- pgwin32_register_deadchild_callback(pi.hProcess, pi.dwProcessId);
-
- /* Don't close pi.hProcess, it's owned by the deadchild callback now */
-
- CloseHandle(pi.hThread);
-
- return pi.dwProcessId;
-}
-#endif /* WIN32 */
-
-
/*
* SubPostmasterMain -- Get the fork/exec'd process into a state equivalent
* to what it would be if we'd simply forked on Unix, and then
@@ -5939,372 +5566,6 @@ PostmasterMarkPIDForWorkerNotify(int pid)
#ifdef EXEC_BACKEND
-/*
- * The following need to be available to the save/restore_backend_variables
- * functions. They are marked NON_EXEC_STATIC in their home modules.
- */
-extern slock_t *ShmemLock;
-extern slock_t *ProcStructLock;
-extern PGPROC *AuxiliaryProcs;
-extern PMSignalData *PMSignalState;
-extern pg_time_t first_syslogger_file_time;
-
-#ifndef WIN32
-#define write_inheritable_socket(dest, src, childpid) ((*(dest) = (src)), true)
-#define read_inheritable_socket(dest, src) (*(dest) = *(src))
-#else
-static bool write_duplicated_handle(HANDLE *dest, HANDLE src, HANDLE child);
-static bool write_inheritable_socket(InheritableSocket *dest, SOCKET src,
- pid_t childPid);
-static void read_inheritable_socket(SOCKET *dest, InheritableSocket *src);
-#endif
-
-
-/* Save critical backend variables into the BackendParameters struct */
-#ifndef WIN32
-static bool
-save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker)
-#else
-static bool
-save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker,
- HANDLE childProcess, pid_t childPid)
-#endif
-{
- if (client_sock)
- {
- memcpy(¶m->client_sock, client_sock, sizeof(ClientSocket));
- if (!write_inheritable_socket(¶m->inh_sock, client_sock->sock, childPid))
- return false;
- param->has_client_sock = true;
- }
- else
- {
- memset(¶m->client_sock, 0, sizeof(ClientSocket));
- param->has_client_sock = false;
- }
-
- if (worker)
- {
- memcpy(¶m->bgworker, worker, sizeof(BackgroundWorker));
- param->has_bgworker = true;
- }
- else
- {
- memset(¶m->bgworker, 0, sizeof(BackgroundWorker));
- param->has_bgworker = false;
- }
-
- strlcpy(param->DataDir, DataDir, MAXPGPATH);
-
- param->MyCancelKey = MyCancelKey;
- param->MyPMChildSlot = MyPMChildSlot;
-
-#ifdef WIN32
- param->ShmemProtectiveRegion = ShmemProtectiveRegion;
-#endif
- param->UsedShmemSegID = UsedShmemSegID;
- param->UsedShmemSegAddr = UsedShmemSegAddr;
-
- param->ShmemLock = ShmemLock;
- param->ShmemBackendArray = ShmemBackendArray;
-
-#ifndef HAVE_SPINLOCKS
- param->SpinlockSemaArray = SpinlockSemaArray;
-#endif
- param->NamedLWLockTrancheRequests = NamedLWLockTrancheRequests;
- param->NamedLWLockTrancheArray = NamedLWLockTrancheArray;
- param->MainLWLockArray = MainLWLockArray;
- param->ProcStructLock = ProcStructLock;
- param->ProcGlobal = ProcGlobal;
- param->AuxiliaryProcs = AuxiliaryProcs;
- param->PreparedXactProcs = PreparedXactProcs;
- param->PMSignalState = PMSignalState;
-
- param->PostmasterPid = PostmasterPid;
- param->PgStartTime = PgStartTime;
- param->PgReloadTime = PgReloadTime;
- param->first_syslogger_file_time = first_syslogger_file_time;
-
- param->redirection_done = redirection_done;
- param->IsBinaryUpgrade = IsBinaryUpgrade;
- param->query_id_enabled = query_id_enabled;
- param->max_safe_fds = max_safe_fds;
-
- param->MaxBackends = MaxBackends;
-
-#ifdef WIN32
- param->PostmasterHandle = PostmasterHandle;
- if (!write_duplicated_handle(¶m->initial_signal_pipe,
- pgwin32_create_signal_listener(childPid),
- childProcess))
- return false;
-#else
- memcpy(¶m->postmaster_alive_fds, &postmaster_alive_fds,
- sizeof(postmaster_alive_fds));
-#endif
-
- memcpy(¶m->syslogPipe, &syslogPipe, sizeof(syslogPipe));
-
- strlcpy(param->my_exec_path, my_exec_path, MAXPGPATH);
-
- strlcpy(param->pkglib_path, pkglib_path, MAXPGPATH);
-
- return true;
-}
-
-
-#ifdef WIN32
-/*
- * Duplicate a handle for usage in a child process, and write the child
- * process instance of the handle to the parameter file.
- */
-static bool
-write_duplicated_handle(HANDLE *dest, HANDLE src, HANDLE childProcess)
-{
- HANDLE hChild = INVALID_HANDLE_VALUE;
-
- if (!DuplicateHandle(GetCurrentProcess(),
- src,
- childProcess,
- &hChild,
- 0,
- TRUE,
- DUPLICATE_CLOSE_SOURCE | DUPLICATE_SAME_ACCESS))
- {
- ereport(LOG,
- (errmsg_internal("could not duplicate handle to be written to backend parameter file: error code %lu",
- GetLastError())));
- return false;
- }
-
- *dest = hChild;
- return true;
-}
-
-/*
- * Duplicate a socket for usage in a child process, and write the resulting
- * structure to the parameter file.
- * This is required because a number of LSPs (Layered Service Providers) very
- * common on Windows (antivirus, firewalls, download managers etc) break
- * straight socket inheritance.
- */
-static bool
-write_inheritable_socket(InheritableSocket *dest, SOCKET src, pid_t childpid)
-{
- dest->origsocket = src;
- if (src != 0 && src != PGINVALID_SOCKET)
- {
- /* Actual socket */
- if (WSADuplicateSocket(src, childpid, &dest->wsainfo) != 0)
- {
- ereport(LOG,
- (errmsg("could not duplicate socket %d for use in backend: error code %d",
- (int) src, WSAGetLastError())));
- return false;
- }
- }
- return true;
-}
-
-/*
- * Read a duplicate socket structure back, and get the socket descriptor.
- */
-static void
-read_inheritable_socket(SOCKET *dest, InheritableSocket *src)
-{
- SOCKET s;
-
- if (src->origsocket == PGINVALID_SOCKET || src->origsocket == 0)
- {
- /* Not a real socket! */
- *dest = src->origsocket;
- }
- else
- {
- /* Actual socket, so create from structure */
- s = WSASocket(FROM_PROTOCOL_INFO,
- FROM_PROTOCOL_INFO,
- FROM_PROTOCOL_INFO,
- &src->wsainfo,
- 0,
- 0);
- if (s == INVALID_SOCKET)
- {
- write_stderr("could not create inherited socket: error code %d\n",
- WSAGetLastError());
- exit(1);
- }
- *dest = s;
-
- /*
- * To make sure we don't get two references to the same socket, close
- * the original one. (This would happen when inheritance actually
- * works..
- */
- closesocket(src->origsocket);
- }
-}
-#endif
-
-static void
-read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **worker)
-{
- BackendParameters param;
-
-#ifndef WIN32
- /* Non-win32 implementation reads from file */
- FILE *fp;
-
- /* Open file */
- fp = AllocateFile(id, PG_BINARY_R);
- if (!fp)
- {
- write_stderr("could not open backend variables file \"%s\": %s\n",
- id, strerror(errno));
- exit(1);
- }
-
- if (fread(¶m, sizeof(param), 1, fp) != 1)
- {
- write_stderr("could not read from backend variables file \"%s\": %s\n",
- id, strerror(errno));
- exit(1);
- }
-
- /* Release file */
- FreeFile(fp);
- if (unlink(id) != 0)
- {
- write_stderr("could not remove file \"%s\": %s\n",
- id, strerror(errno));
- exit(1);
- }
-#else
- /* Win32 version uses mapped file */
- HANDLE paramHandle;
- BackendParameters *paramp;
-
-#ifdef _WIN64
- paramHandle = (HANDLE) _atoi64(id);
-#else
- paramHandle = (HANDLE) atol(id);
-#endif
- paramp = MapViewOfFile(paramHandle, FILE_MAP_READ, 0, 0, 0);
- if (!paramp)
- {
- write_stderr("could not map view of backend variables: error code %lu\n",
- GetLastError());
- exit(1);
- }
-
- memcpy(¶m, paramp, sizeof(BackendParameters));
-
- if (!UnmapViewOfFile(paramp))
- {
- write_stderr("could not unmap view of backend variables: error code %lu\n",
- GetLastError());
- exit(1);
- }
-
- if (!CloseHandle(paramHandle))
- {
- write_stderr("could not close handle to backend parameter variables: error code %lu\n",
- GetLastError());
- exit(1);
- }
-#endif
-
- restore_backend_variables(¶m, client_sock, worker);
-}
-
-/* Restore critical backend variables from the BackendParameters struct */
-static void
-restore_backend_variables(BackendParameters *param, ClientSocket **client_sock, BackgroundWorker **worker)
-{
- if (param->has_client_sock)
- {
- *client_sock = (ClientSocket *) MemoryContextAlloc(TopMemoryContext, sizeof(ClientSocket));
- memcpy(*client_sock, ¶m->client_sock, sizeof(ClientSocket));
- read_inheritable_socket(&(*client_sock)->sock, ¶m->inh_sock);
- }
- else
- *client_sock = NULL;
-
- if (param->has_bgworker)
- {
- *worker = (BackgroundWorker *)
- MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
- memcpy(*worker, ¶m->bgworker, sizeof(BackgroundWorker));
- }
- else
- *worker = NULL;
-
- SetDataDir(param->DataDir);
-
- MyCancelKey = param->MyCancelKey;
- MyPMChildSlot = param->MyPMChildSlot;
-
-#ifdef WIN32
- ShmemProtectiveRegion = param->ShmemProtectiveRegion;
-#endif
- UsedShmemSegID = param->UsedShmemSegID;
- UsedShmemSegAddr = param->UsedShmemSegAddr;
-
- ShmemLock = param->ShmemLock;
- ShmemBackendArray = param->ShmemBackendArray;
-
-#ifndef HAVE_SPINLOCKS
- SpinlockSemaArray = param->SpinlockSemaArray;
-#endif
- NamedLWLockTrancheRequests = param->NamedLWLockTrancheRequests;
- NamedLWLockTrancheArray = param->NamedLWLockTrancheArray;
- MainLWLockArray = param->MainLWLockArray;
- ProcStructLock = param->ProcStructLock;
- ProcGlobal = param->ProcGlobal;
- AuxiliaryProcs = param->AuxiliaryProcs;
- PreparedXactProcs = param->PreparedXactProcs;
- PMSignalState = param->PMSignalState;
-
- PostmasterPid = param->PostmasterPid;
- PgStartTime = param->PgStartTime;
- PgReloadTime = param->PgReloadTime;
- first_syslogger_file_time = param->first_syslogger_file_time;
-
- redirection_done = param->redirection_done;
- IsBinaryUpgrade = param->IsBinaryUpgrade;
- query_id_enabled = param->query_id_enabled;
- max_safe_fds = param->max_safe_fds;
-
- MaxBackends = param->MaxBackends;
-
-#ifdef WIN32
- PostmasterHandle = param->PostmasterHandle;
- pgwin32_initial_signal_pipe = param->initial_signal_pipe;
-#else
- memcpy(&postmaster_alive_fds, ¶m->postmaster_alive_fds,
- sizeof(postmaster_alive_fds));
-#endif
-
- memcpy(&syslogPipe, ¶m->syslogPipe, sizeof(syslogPipe));
-
- strlcpy(my_exec_path, param->my_exec_path, MAXPGPATH);
-
- strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
-
- /*
- * We need to restore fd.c's counts of externally-opened FDs; to avoid
- * confusion, be sure to do this after restoring max_safe_fds. (Note:
- * BackendInitialize will handle this for (*client_sock)->sock.)
- */
-#ifndef WIN32
- if (postmaster_alive_fds[0] >= 0)
- ReserveExternalFD();
- if (postmaster_alive_fds[1] >= 0)
- ReserveExternalFD();
-#endif
-}
-
-
Size
ShmemBackendArraySize(void)
{
@@ -6432,7 +5693,7 @@ pgwin32_deadchild_callback(PVOID lpParameter, BOOLEAN TimerOrWaitFired)
* automatically by an operating system thread pool. The memory and the
* process handle will be freed by a later call to waitpid().
*/
-static void
+void
pgwin32_register_deadchild_callback(HANDLE procHandle, DWORD procId)
{
win32_deadchild_waitinfo *childinfo;
diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c
index 501910b4454..a4098c23b2b 100644
--- a/src/backend/replication/logical/launcher.c
+++ b/src/backend/replication/logical/launcher.c
@@ -30,7 +30,6 @@
#include "miscadmin.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "replication/logicallauncher.h"
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index 3b3889c58c0..45d4b78c3a6 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -59,11 +59,16 @@ extern int MaxLivePostmasterChildren(void);
extern bool PostmasterMarkPIDForWorkerNotify(int);
#ifdef EXEC_BACKEND
+
extern pid_t postmaster_forkexec(int argc, char *argv[]);
extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
extern Size ShmemBackendArraySize(void);
extern void ShmemBackendArrayAllocation(void);
+
+#ifdef WIN32
+extern void pgwin32_register_deadchild_callback(HANDLE procHandle, DWORD procId);
+#endif
#endif
/*
--
2.39.2
v5-0006-Refactor-AuxProcess-startup.patchtext/x-patch; charset=UTF-8; name=v5-0006-Refactor-AuxProcess-startup.patchDownload
From 3b6fc2b08260d437bc908378cfc050f538c6433e Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Mon, 4 Dec 2023 13:24:01 +0200
Subject: [PATCH v5 6/8] Refactor AuxProcess startup
Old:
AuxProcessMain()
Initialize a bunch of stuff
<aux process specific Main function>()
New:
AuxProcessMain()
<aux process specific Main function>()
AuxiliaryProcessInit()
Initialize a bunch of stuff
This isn't too useful as is, but the next commits will remove the
AuxProcessMain() function and dispatch directly to the aux-process
specific Main function, like this:
<aux process specific Main function>()
AuxiliaryProcessInit()
Initialize a bunch of stuff
This commit makes that next commit smaller.
XXX: We now have functions called AuxiliaryProcessInit() and
InitAuxiliaryProcess(). Confusing.
---
src/backend/postmaster/auxprocess.c | 86 ++++++++++++---------------
src/backend/postmaster/bgwriter.c | 5 ++
src/backend/postmaster/checkpointer.c | 5 ++
src/backend/postmaster/pgarch.c | 5 ++
src/backend/postmaster/startup.c | 5 ++
src/backend/postmaster/walwriter.c | 5 ++
src/backend/replication/walreceiver.c | 5 ++
src/include/postmaster/auxprocess.h | 3 +-
8 files changed, 69 insertions(+), 50 deletions(-)
diff --git a/src/backend/postmaster/auxprocess.c b/src/backend/postmaster/auxprocess.c
index bae6f68c402..d62adf4c993 100644
--- a/src/backend/postmaster/auxprocess.c
+++ b/src/backend/postmaster/auxprocess.c
@@ -44,7 +44,6 @@ static void ShutdownAuxiliaryProcess(int code, Datum arg);
AuxProcType MyAuxProcType = NotAnAuxProcess; /* declared in miscadmin.h */
-
/*
* AuxiliaryProcessMain
*
@@ -58,33 +57,56 @@ AuxiliaryProcessMain(AuxProcType auxtype)
{
Assert(IsUnderPostmaster);
- MyAuxProcType = auxtype;
-
switch (MyAuxProcType)
{
case StartupProcess:
- MyBackendType = B_STARTUP;
- break;
+ StartupProcessMain();
+ proc_exit(1);
+
case ArchiverProcess:
- MyBackendType = B_ARCHIVER;
- break;
+ PgArchiverMain();
+ proc_exit(1);
+
case BgWriterProcess:
- MyBackendType = B_BG_WRITER;
- break;
+ BackgroundWriterMain();
+ proc_exit(1);
+
case CheckpointerProcess:
- MyBackendType = B_CHECKPOINTER;
- break;
+ CheckpointerMain();
+ proc_exit(1);
+
case WalWriterProcess:
- MyBackendType = B_WAL_WRITER;
- break;
+ WalWriterMain();
+ proc_exit(1);
+
case WalReceiverProcess:
- MyBackendType = B_WAL_RECEIVER;
- break;
+ WalReceiverMain();
+ proc_exit(1);
+
default:
elog(PANIC, "unrecognized process type: %d", (int) MyAuxProcType);
- MyBackendType = B_INVALID;
+ proc_exit(1);
+ }
+}
+
+/*
+ * AuxiliaryProcessInit
+ *
+ * Common initialization code for auxiliary processes, such as the bgwriter,
+ * walwriter, walreceiver, bootstrapper and the shared memory checker code.
+ */
+void
+AuxiliaryProcessInit(void)
+{
+ /* Release postmaster's working memory context */
+ if (PostmasterContext)
+ {
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
}
+ Assert(IsUnderPostmaster);
+
init_ps_display(NULL);
SetProcessingMode(BootstrapProcessing);
@@ -122,7 +144,6 @@ AuxiliaryProcessMain(AuxProcType auxtype)
*/
CreateAuxProcessResourceOwner();
-
/* Initialize backend status information */
pgstat_beinit();
pgstat_bestart();
@@ -131,37 +152,6 @@ AuxiliaryProcessMain(AuxProcType auxtype)
before_shmem_exit(ShutdownAuxiliaryProcess, 0);
SetProcessingMode(NormalProcessing);
-
- switch (MyAuxProcType)
- {
- case StartupProcess:
- StartupProcessMain();
- proc_exit(1);
-
- case ArchiverProcess:
- PgArchiverMain();
- proc_exit(1);
-
- case BgWriterProcess:
- BackgroundWriterMain();
- proc_exit(1);
-
- case CheckpointerProcess:
- CheckpointerMain();
- proc_exit(1);
-
- case WalWriterProcess:
- WalWriterMain();
- proc_exit(1);
-
- case WalReceiverProcess:
- WalReceiverMain();
- proc_exit(1);
-
- default:
- elog(PANIC, "unrecognized process type: %d", (int) MyAuxProcType);
- proc_exit(1);
- }
}
/*
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index d02dc17b9c1..95abdd7fa6d 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -36,6 +36,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/bgwriter.h"
#include "postmaster/interrupt.h"
#include "storage/buf_internals.h"
@@ -95,6 +96,10 @@ BackgroundWriterMain(void)
bool prev_hibernate;
WritebackContext wb_context;
+ MyAuxProcType = BgWriterProcess;
+ MyBackendType = B_BG_WRITER;
+ AuxiliaryProcessInit();
+
/*
* Properly accept or ignore signals that might be sent to us.
*/
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index dc2da5a2cd8..1871ac52921 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -42,6 +42,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/bgwriter.h"
#include "postmaster/interrupt.h"
#include "replication/syncrep.h"
@@ -174,6 +175,10 @@ CheckpointerMain(void)
sigjmp_buf local_sigjmp_buf;
MemoryContext checkpointer_context;
+ MyAuxProcType = CheckpointerProcess;
+ MyBackendType = B_CHECKPOINTER;
+ AuxiliaryProcessInit();
+
CheckpointerShmem->checkpointer_pid = MyProcPid;
/*
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index a2555e8578c..5d5c5733340 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -36,6 +36,7 @@
#include "lib/binaryheap.h"
#include "libpq/pqsignal.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/interrupt.h"
#include "postmaster/pgarch.h"
#include "storage/fd.h"
@@ -213,6 +214,10 @@ PgArchCanRestart(void)
void
PgArchiverMain(void)
{
+ MyAuxProcType = ArchiverProcess;
+ MyBackendType = B_ARCHIVER;
+ AuxiliaryProcessInit();
+
/*
* Ignore all signals usually bound to some action in the postmaster,
* except for SIGHUP, SIGTERM, SIGUSR1, SIGUSR2, and SIGQUIT.
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index 082c870e03a..0fdfa1822db 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -27,6 +27,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/startup.h"
#include "storage/ipc.h"
#include "storage/latch.h"
@@ -243,6 +244,10 @@ StartupProcExit(int code, Datum arg)
void
StartupProcessMain(void)
{
+ MyAuxProcType = StartupProcess;
+ MyBackendType = B_STARTUP;
+ AuxiliaryProcessInit();
+
/* Arrange to clean up at startup process exit */
on_shmem_exit(StartupProcExit, 0);
diff --git a/src/backend/postmaster/walwriter.c b/src/backend/postmaster/walwriter.c
index 48bc92205b5..0575d2c967d 100644
--- a/src/backend/postmaster/walwriter.c
+++ b/src/backend/postmaster/walwriter.c
@@ -48,6 +48,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/interrupt.h"
#include "postmaster/walwriter.h"
#include "storage/bufmgr.h"
@@ -95,6 +96,10 @@ WalWriterMain(void)
int left_till_hibernate;
bool hibernating;
+ MyAuxProcType = WalWriterProcess;
+ MyBackendType = B_WAL_WRITER;
+ AuxiliaryProcessInit();
+
/*
* Properly accept or ignore signals the postmaster might send us
*
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index 26ded928a71..51fd1de9c8b 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -65,6 +65,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/interrupt.h"
#include "replication/walreceiver.h"
#include "replication/walsender.h"
@@ -199,6 +200,10 @@ WalReceiverMain(void)
char *sender_host = NULL;
int sender_port = 0;
+ MyAuxProcType = WalReceiverProcess;
+ MyBackendType = B_WAL_RECEIVER;
+ AuxiliaryProcessInit();
+
/*
* WalRcv should be set up already (if we are a backend, we inherit this
* by fork() or EXEC_BACKEND mechanism from the postmaster).
diff --git a/src/include/postmaster/auxprocess.h b/src/include/postmaster/auxprocess.h
index 5c2d6527ff6..cc75f246818 100644
--- a/src/include/postmaster/auxprocess.h
+++ b/src/include/postmaster/auxprocess.h
@@ -13,8 +13,7 @@
#ifndef AUXPROCESS_H
#define AUXPROCESS_H
-#include "miscadmin.h"
-
extern void AuxiliaryProcessMain(AuxProcType auxtype) pg_attribute_noreturn();
+extern void AuxiliaryProcessInit(void);
#endif /* AUXPROCESS_H */
--
2.39.2
v5-0007-Refactor-postmaster-child-process-launching.patchtext/x-patch; charset=UTF-8; name=v5-0007-Refactor-postmaster-child-process-launching.patchDownload
From 596bd4540d0dccef35c7cb4d44cb08910636769c Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Fri, 8 Dec 2023 13:31:35 +0200
Subject: [PATCH v5 7/8] Refactor postmaster child process launching
- Introduce new postmaster_child_launch() function that deals with the
differences between EXEC_BACKEND and fork mode.
- Refactor the mechanism of passing information from the parent to
child process. Instead of using different command-line arguments when
launching the child process in EXEC_BACKEND mode, pass a
variable-length blob of data along with all the global variables. The
contents of that blob depend on the kind of child process being
launched. In !EXEC_BACKEND mode, we use the same blob, but it's simply
inherited from the parent to child process.
Reviewed-by: Tristan Partin, Andres Freund
Discussion: https://www.postgresql.org/message-id/7a59b073-5b5b-151e-7ed3-8b01ff7ce9ef@iki.fi
---
src/backend/postmaster/autovacuum.c | 153 +-----
src/backend/postmaster/auxprocess.c | 45 --
src/backend/postmaster/bgworker.c | 14 +-
src/backend/postmaster/bgwriter.c | 4 +-
src/backend/postmaster/checkpointer.c | 4 +-
src/backend/postmaster/launch_backend.c | 411 ++++++++++++----
src/backend/postmaster/pgarch.c | 4 +-
src/backend/postmaster/postmaster.c | 518 ++++----------------
src/backend/postmaster/startup.c | 4 +-
src/backend/postmaster/syslogger.c | 275 +++++------
src/backend/postmaster/walwriter.c | 4 +-
src/backend/replication/walreceiver.c | 4 +-
src/backend/utils/init/globals.c | 1 +
src/include/postmaster/autovacuum.h | 10 +-
src/include/postmaster/auxprocess.h | 1 -
src/include/postmaster/bgworker_internals.h | 2 +-
src/include/postmaster/bgwriter.h | 4 +-
src/include/postmaster/pgarch.h | 2 +-
src/include/postmaster/postmaster.h | 45 +-
src/include/postmaster/startup.h | 2 +-
src/include/postmaster/syslogger.h | 4 +-
src/include/postmaster/walwriter.h | 2 +-
src/include/replication/walreceiver.h | 2 +-
src/tools/pgindent/typedefs.list | 3 +
24 files changed, 602 insertions(+), 916 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index b04fcfc8c8d..6849072d3c2 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -84,7 +84,6 @@
#include "nodes/makefuncs.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "storage/bufmgr.h"
@@ -315,13 +314,6 @@ static WorkerInfo MyWorkerInfo = NULL;
/* PID of launcher, valid only in worker while shutting down */
int AutovacuumLauncherPid = 0;
-#ifdef EXEC_BACKEND
-static pid_t avlauncher_forkexec(void);
-static pid_t avworker_forkexec(void);
-#endif
-NON_EXEC_STATIC void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
-NON_EXEC_STATIC void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
-
static Oid do_start_worker(void);
static void HandleAutoVacLauncherInterrupts(void);
static void AutoVacLauncherShutdown(void) pg_attribute_noreturn();
@@ -365,76 +357,21 @@ static void avl_sigusr2_handler(SIGNAL_ARGS);
* AUTOVACUUM LAUNCHER CODE
********************************************************************/
-#ifdef EXEC_BACKEND
/*
- * forkexec routine for the autovacuum launcher process.
- *
- * Format up the arglist, then fork and exec.
- */
-static pid_t
-avlauncher_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkavlauncher";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif
-
-/*
- * Main entry point for autovacuum launcher process, to be called from the
- * postmaster.
+ * Main loop for the autovacuum launcher process.
*/
-int
-StartAutoVacLauncher(void)
+void
+AutoVacLauncherMain(char *startup_data, size_t startup_data_len)
{
- pid_t AutoVacPID;
+ sigjmp_buf local_sigjmp_buf;
-#ifdef EXEC_BACKEND
- switch ((AutoVacPID = avlauncher_forkexec()))
-#else
- switch ((AutoVacPID = fork_process()))
-#endif
+ /* Release postmaster's working memory context */
+ if (PostmasterContext)
{
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum launcher process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacLauncherMain(0, NULL);
- break;
-#endif
- default:
- return (int) AutoVacPID;
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
}
- /* shouldn't get here */
- return 0;
-}
-
-/*
- * Main loop for the autovacuum launcher process.
- */
-NON_EXEC_STATIC void
-AutoVacLauncherMain(int argc, char *argv[])
-{
- sigjmp_buf local_sigjmp_buf;
-
am_autovacuum_launcher = true;
MyBackendType = B_AUTOVAC_LAUNCHER;
@@ -1423,78 +1360,22 @@ avl_sigusr2_handler(SIGNAL_ARGS)
* AUTOVACUUM WORKER CODE
********************************************************************/
-#ifdef EXEC_BACKEND
-/*
- * forkexec routines for the autovacuum worker.
- *
- * Format up the arglist, then fork and exec.
- */
-static pid_t
-avworker_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkavworker";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif
-
-/*
- * Main entry point for autovacuum worker process.
- *
- * This code is heavily based on pgarch.c, q.v.
- */
-int
-StartAutoVacWorker(void)
-{
- pid_t worker_pid;
-
-#ifdef EXEC_BACKEND
- switch ((worker_pid = avworker_forkexec()))
-#else
- switch ((worker_pid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum worker process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacWorkerMain(0, NULL);
- break;
-#endif
- default:
- return (int) worker_pid;
- }
-
- /* shouldn't get here */
- return 0;
-}
-
/*
* AutoVacWorkerMain
*/
-NON_EXEC_STATIC void
-AutoVacWorkerMain(int argc, char *argv[])
+void
+AutoVacWorkerMain(char *startup_data, size_t startup_data_len)
{
sigjmp_buf local_sigjmp_buf;
Oid dbid;
+ /* Release postmaster's working memory context */
+ if (PostmasterContext)
+ {
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
+ }
+
am_autovacuum_worker = true;
MyBackendType = B_AUTOVAC_WORKER;
diff --git a/src/backend/postmaster/auxprocess.c b/src/backend/postmaster/auxprocess.c
index d62adf4c993..dad57c52a15 100644
--- a/src/backend/postmaster/auxprocess.c
+++ b/src/backend/postmaster/auxprocess.c
@@ -44,51 +44,6 @@ static void ShutdownAuxiliaryProcess(int code, Datum arg);
AuxProcType MyAuxProcType = NotAnAuxProcess; /* declared in miscadmin.h */
-/*
- * AuxiliaryProcessMain
- *
- * The main entry point for auxiliary processes, such as the bgwriter,
- * walwriter, walreceiver, bootstrapper and the shared memory checker code.
- *
- * This code is here just because of historical reasons.
- */
-void
-AuxiliaryProcessMain(AuxProcType auxtype)
-{
- Assert(IsUnderPostmaster);
-
- switch (MyAuxProcType)
- {
- case StartupProcess:
- StartupProcessMain();
- proc_exit(1);
-
- case ArchiverProcess:
- PgArchiverMain();
- proc_exit(1);
-
- case BgWriterProcess:
- BackgroundWriterMain();
- proc_exit(1);
-
- case CheckpointerProcess:
- CheckpointerMain();
- proc_exit(1);
-
- case WalWriterProcess:
- WalWriterMain();
- proc_exit(1);
-
- case WalReceiverProcess:
- WalReceiverMain();
- proc_exit(1);
-
- default:
- elog(PANIC, "unrecognized process type: %d", (int) MyAuxProcType);
- proc_exit(1);
- }
-}
-
/*
* AuxiliaryProcessInit
*
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index 3c99cf6047b..610c2a5b947 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -722,17 +722,27 @@ bgworker_die(SIGNAL_ARGS)
* Main entry point for background worker processes.
*/
void
-BackgroundWorkerMain(void)
+BackgroundWorkerMain(char *startup_data, size_t startup_data_len)
{
sigjmp_buf local_sigjmp_buf;
- BackgroundWorker *worker = MyBgworkerEntry;
+ BackgroundWorker *worker;
bgworker_main_type entrypt;
+ /* Release postmaster's working memory context */
+ if (PostmasterContext)
+ {
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
+ }
+
+ Assert(startup_data_len == sizeof(BackgroundWorker));
+ worker = (BackgroundWorker *) startup_data;
if (worker == NULL)
elog(FATAL, "unable to find bgworker entry");
IsBackgroundWorker = true;
+ MyBgworkerEntry = worker;
MyBackendType = B_BG_WORKER;
init_ps_display(worker->bgw_name);
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index 95abdd7fa6d..027590f824b 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -89,13 +89,15 @@ static XLogRecPtr last_snapshot_lsn = InvalidXLogRecPtr;
* basic execution environment, but not enabled signals yet.
*/
void
-BackgroundWriterMain(void)
+BackgroundWriterMain(char *startup_data, size_t startup_data_len)
{
sigjmp_buf local_sigjmp_buf;
MemoryContext bgwriter_context;
bool prev_hibernate;
WritebackContext wb_context;
+ Assert(startup_data_len == 0);
+
MyAuxProcType = BgWriterProcess;
MyBackendType = B_BG_WRITER;
AuxiliaryProcessInit();
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 1871ac52921..5201481732e 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -170,11 +170,13 @@ static void ReqCheckpointHandler(SIGNAL_ARGS);
* basic execution environment, but not enabled signals yet.
*/
void
-CheckpointerMain(void)
+CheckpointerMain(char *startup_data, size_t startup_data_len)
{
sigjmp_buf local_sigjmp_buf;
MemoryContext checkpointer_context;
+ Assert(startup_data_len == 0);
+
MyAuxProcType = CheckpointerProcess;
MyBackendType = B_CHECKPOINTER;
AuxiliaryProcessInit();
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 65f141a9f5a..50069cbf359 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -84,17 +84,10 @@ typedef int InheritableSocket;
#endif
/*
- * Structure contains all variables passed to exec:ed backends
+ * Structure contains all global variables passed to exec:ed backends
*/
typedef struct
{
- bool has_client_sock;
- ClientSocket client_sock;
- InheritableSocket inh_sock;
-
- bool has_bgworker;
- BackgroundWorker bgworker;
-
char DataDir[MAXPGPATH];
int32 MyCancelKey;
int MyPMChildSlot;
@@ -137,22 +130,129 @@ typedef struct
#endif
char my_exec_path[MAXPGPATH];
char pkglib_path[MAXPGPATH];
+
+ /*
+ * These are only used by backend processes, but it's here because passing
+ * a socket needs some special handling on Windows. 'client_sock' is an
+ * explicit argument to postmaster_child_launch, but is stored in
+ * MyClientSocket in the child process.
+ */
+ ClientSocket client_sock;
+ InheritableSocket inh_sock;
+
+ size_t startup_data_len;
+ /* startup data follows */
+ char startup_data[FLEXIBLE_ARRAY_MEMBER];
} BackendParameters;
#define SizeOfBackendParameters(startup_data_len) (offsetof(BackendParameters, startup_data) + startup_data_len)
-void read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **worker);
-static void restore_backend_variables(BackendParameters *param, ClientSocket **client_sock, BackgroundWorker **worker);
+static void read_backend_variables(char *id, char **startup_data, size_t *startup_data_len);
+static void restore_backend_variables(BackendParameters *param);
-#ifndef WIN32
-static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker);
-#else
-static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker,
- HANDLE childProcess, pid_t childPid);
+static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock,
+#ifdef WIN32
+ HANDLE childProcess, pid_t childPid,
#endif
+ char *startup_data, size_t startup_data_len);
+
+static pid_t internal_forkexec(const char *child_kind, char *startup_data, size_t startup_data_len, ClientSocket *client_sock);
+
+#endif /* EXEC_BACKEND */
+
+/*
+ * Information needed to launch different kinds of child processes.
+ */
+typedef struct
+{
+ const char *name;
+ void (*main_fn) (char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
+ bool shmem_attach;
+} child_process_kind;
+
+child_process_kind child_process_kinds[] = {
+ [PMC_BACKEND] = {"backend", BackendMain, true},
+
+ [PMC_AV_LAUNCHER] = {"autovacuum launcher", AutoVacLauncherMain, true},
+ [PMC_AV_WORKER] = {"autovacuum worker", AutoVacWorkerMain, true},
+ [PMC_BGWORKER] = {"bgworker", BackgroundWorkerMain, true},
+ [PMC_SYSLOGGER] = {"syslogger", SysLoggerMain, false},
+
+ [PMC_STARTUP] = {"startup", StartupProcessMain, true},
+ [PMC_BGWRITER] = {"bgwriter", BackgroundWriterMain, true},
+ [PMC_ARCHIVER] = {"archiver", PgArchiverMain, true},
+ [PMC_CHECKPOINTER] = {"checkpointer", CheckpointerMain, true},
+ [PMC_WAL_WRITER] = {"wal_writer", WalWriterMain, true},
+ [PMC_WAL_RECEIVER] = {"wal_receiver", WalReceiverMain, true},
+};
+
+const char *
+PostmasterChildName(PostmasterChildType child_type)
+{
+ Assert(child_type >= 0 && child_type < lengthof(child_process_kinds));
+ return child_process_kinds[child_type].name;
+}
+
+/*
+ * Start a new postmaster child process.
+ *
+ * The child process will be restored to roughly the same state, whether
+ * EXEC_BACKEND is used or not: it will be attached to shared memory, and fds
+ * and other resources that we've inherited from postmaster that are not
+ * needed in a child process have been closed.
+ *
+ * 'startup_data' is an optional contiguous chunk of data that is passed to
+ * the child process.
+ */
+pid_t
+postmaster_child_launch(PostmasterChildType child_type, char *startup_data, size_t startup_data_len, ClientSocket *client_sock)
+{
+ pid_t pid;
+
+ Assert(child_type >= 0 && child_type < lengthof(child_process_kinds));
+ Assert(IsPostmasterEnvironment && !IsUnderPostmaster);
+
+#ifdef EXEC_BACKEND
+ pid = internal_forkexec(child_process_kinds[child_type].name,
+ startup_data, startup_data_len, client_sock);
+ /* the child process will arrive in SubPostmasterMain */
+#else /* !EXEC_BACKEND */
+ pid = fork_process();
+ if (pid == 0) /* child */
+ {
+ /* Close the postmaster's sockets */
+ ClosePostmasterPorts(child_type == PMC_SYSLOGGER);
-pid_t internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker);
+ /* Detangle from postmaster */
+ InitPostmasterChild();
+ /*
+ * Before blowing away PostmasterContext (in the Main function), save
+ * the startup data.
+ */
+ MemoryContextSwitchTo(TopMemoryContext);
+ if (startup_data != NULL)
+ {
+ char *cp = palloc(startup_data_len);
+
+ memcpy(cp, startup_data, startup_data_len);
+ startup_data = cp;
+ }
+
+ if (client_sock)
+ {
+ MyClientSocket = palloc(sizeof(ClientSocket));
+ memcpy(MyClientSocket, client_sock, sizeof(ClientSocket));
+ }
+
+ child_process_kinds[child_type].main_fn(startup_data, startup_data_len);
+ pg_unreachable(); /* main_fn never returns */
+ }
+#endif /* EXEC_BACKEND */
+ return pid;
+}
+
+#ifdef EXEC_BACKEND
#ifndef WIN32
/*
@@ -161,25 +261,25 @@ pid_t internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, Back
* - writes out backend variables to the parameter file
* - fork():s, and then exec():s the child process
*/
-pid_t
-internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker)
+static pid_t
+internal_forkexec(const char *child_kind, char *startup_data, size_t startup_data_len, ClientSocket *client_sock)
{
static unsigned long tmpBackendFileNum = 0;
pid_t pid;
char tmpfilename[MAXPGPATH];
- BackendParameters param;
+ size_t paramsz;
+ BackendParameters *param;
FILE *fp;
+ char *argv[4];
+ char forkav[MAXPGPATH];
- /*
- * Make sure padding bytes are initialized, to prevent Valgrind from
- * complaining about writing uninitialized bytes to the file. This isn't
- * performance critical, and the win32 implementation initializes the
- * padding bytes to zeros, so do it even when not using Valgrind.
- */
- memset(¶m, 0, sizeof(BackendParameters));
-
- if (!save_backend_variables(¶m, client_sock, worker))
+ paramsz = SizeOfBackendParameters(startup_data_len);
+ param = palloc(paramsz);
+ if (!save_backend_variables(param, client_sock, startup_data, startup_data_len))
+ {
+ pfree(param);
return -1; /* log made by save_backend_variables */
+ }
/* Calculate name for temp file */
snprintf(tmpfilename, MAXPGPATH, "%s/%s.backend_var.%d.%lu",
@@ -207,7 +307,7 @@ internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundW
}
}
- if (fwrite(¶m, sizeof(param), 1, fp) != 1)
+ if (fwrite(param, paramsz, 1, fp) != 1)
{
ereport(LOG,
(errcode_for_file_access(),
@@ -225,14 +325,13 @@ internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundW
return -1;
}
- /* Make sure caller set up argv properly */
- Assert(argc >= 3);
- Assert(argv[argc] == NULL);
- Assert(strncmp(argv[1], "--fork", 6) == 0);
- Assert(argv[2] == NULL);
-
- /* Insert temp file name after --fork argument */
+ /* set up argv properly */
+ argv[0] = "postgres";
+ snprintf(forkav, MAXPGPATH, "--forkchild=%s", child_kind);
+ argv[1] = forkav;
+ /* Insert temp file name after --forkchild argument */
argv[2] = tmpfilename;
+ argv[3] = NULL;
/* Fire off execv in child */
if ((pid = fork_process()) == 0)
@@ -261,26 +360,21 @@ internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundW
* - resumes execution of the new process once the backend parameter
* file is complete.
*/
-pid_t
-internal_forkexec(int argc, char *argv[], Port *port, BackgroundWorker *worker)
+static pid_t
+internal_forkexec(const char *child_kind, char *startup_data, size_t startup_data_len, ClientSocket *client_sock)
{
int retry_count = 0;
STARTUPINFO si;
PROCESS_INFORMATION pi;
- int i;
- int j;
char cmdLine[MAXPGPATH * 2];
HANDLE paramHandle;
BackendParameters *param;
SECURITY_ATTRIBUTES sa;
+ size_t paramsz;
char paramHandleStr[32];
- win32_deadchild_waitinfo *childinfo;
+ int l;
- /* Make sure caller set up argv properly */
- Assert(argc >= 3);
- Assert(argv[argc] == NULL);
- Assert(strncmp(argv[1], "--fork", 6) == 0);
- Assert(argv[2] == NULL);
+ paramsz = SizeOfBackendParameters(startup_data_len);
/* Resume here if we need to retry */
retry:
@@ -293,7 +387,7 @@ retry:
&sa,
PAGE_READWRITE,
0,
- sizeof(BackendParameters),
+ paramsz,
NULL);
if (paramHandle == INVALID_HANDLE_VALUE)
{
@@ -302,8 +396,7 @@ retry:
GetLastError())));
return -1;
}
-
- param = MapViewOfFile(paramHandle, FILE_MAP_WRITE, 0, 0, sizeof(BackendParameters));
+ param = MapViewOfFile(paramHandle, FILE_MAP_WRITE, 0, 0, paramsz);
if (!param)
{
ereport(LOG,
@@ -313,25 +406,15 @@ retry:
return -1;
}
- /* Insert temp file name after --fork argument */
+ /* Format the cmd line */
#ifdef _WIN64
sprintf(paramHandleStr, "%llu", (LONG_PTR) paramHandle);
#else
sprintf(paramHandleStr, "%lu", (DWORD) paramHandle);
#endif
- argv[2] = paramHandleStr;
-
- /* Format the cmd line */
- cmdLine[sizeof(cmdLine) - 1] = '\0';
- cmdLine[sizeof(cmdLine) - 2] = '\0';
- snprintf(cmdLine, sizeof(cmdLine) - 1, "\"%s\"", postgres_exec_path);
- i = 0;
- while (argv[++i] != NULL)
- {
- j = strlen(cmdLine);
- snprintf(cmdLine + j, sizeof(cmdLine) - 1 - j, " \"%s\"", argv[i]);
- }
- if (cmdLine[sizeof(cmdLine) - 2] != '\0')
+ l = snprintf(cmdLine, sizeof(cmdLine) - 1, "\"%s\" --forkchild=\"%s\" %s",
+ postgres_exec_path, child_kind, paramHandleStr);
+ if (l >= sizeof(cmdLine))
{
ereport(LOG,
(errmsg("subprocess command line too long")));
@@ -359,7 +442,7 @@ retry:
return -1;
}
- if (!save_backend_variables(param, port, worker, pi.hProcess, pi.dwProcessId))
+ if (!save_backend_variables(param, client_sock, pi.hProcess, pi.dwProcessId, startup_data, startup_data_len))
{
/*
* log made by save_backend_variables, but we have to clean up the
@@ -445,6 +528,117 @@ retry:
}
#endif /* WIN32 */
+/*
+ * SubPostmasterMain -- Get the fork/exec'd process into a state equivalent
+ * to what it would be if we'd simply forked on Unix, and then
+ * dispatch to the appropriate place.
+ *
+ * The first two command line arguments are expected to be "--forkchild=<name>",
+ * where <name> indicates which postmaster child we are to become, and
+ * the name of a variables file that we can read to load data that would
+ * have been inherited by fork() on Unix.
+ */
+void
+SubPostmasterMain(int argc, char *argv[])
+{
+ char *startup_data;
+ size_t startup_data_len;
+ char *child_kind;
+ PostmasterChildType child_type;
+ bool found = false;
+
+ /* In EXEC_BACKEND case we will not have inherited these settings */
+ IsPostmasterEnvironment = true;
+ whereToSendOutput = DestNone;
+
+ /* Setup essential subsystems (to ensure elog() behaves sanely) */
+ InitializeGUCOptions();
+
+ /* Check we got appropriate args */
+ if (argc != 3)
+ elog(FATAL, "invalid subpostmaster invocation");
+
+ /* Find the entry in child_process_kinds */
+ if (strncmp(argv[1], "--forkchild=", 12) != 0)
+ elog(FATAL, "invalid subpostmaster invocation (--forkchild argument missing)");
+ child_kind = argv[1] + 12;
+ found = false;
+ for (int idx = 0; idx < lengthof(child_process_kinds); idx++)
+ {
+ if (strcmp(child_process_kinds[idx].name, child_kind) == 0)
+ {
+ child_type = idx;
+ found = true;
+ break;
+ }
+ }
+ if (!found)
+ elog(ERROR, "unknown child kind %s", child_kind);
+
+ /* Read in the variables file */
+ read_backend_variables(argv[2], &startup_data, &startup_data_len);
+
+ /* Close the postmaster's sockets (as soon as we know them) */
+ ClosePostmasterPorts(child_type == PMC_SYSLOGGER);
+
+ /* Setup as postmaster child */
+ InitPostmasterChild();
+
+ /*
+ * If appropriate, physically re-attach to shared memory segment. We want
+ * to do this before going any further to ensure that we can attach at the
+ * same address the postmaster used. On the other hand, if we choose not
+ * to re-attach, we may have other cleanup to do.
+ *
+ * If testing EXEC_BACKEND on Linux, you should run this as root before
+ * starting the postmaster:
+ *
+ * sysctl -w kernel.randomize_va_space=0
+ *
+ * This prevents using randomized stack and code addresses that cause the
+ * child process's memory map to be different from the parent's, making it
+ * sometimes impossible to attach to shared memory at the desired address.
+ * Return the setting to its old value (usually '1' or '2') when finished.
+ */
+ if (child_process_kinds[child_type].shmem_attach)
+ PGSharedMemoryReAttach();
+ else
+ PGSharedMemoryNoReAttach();
+
+ /* Read in remaining GUC variables */
+ read_nondefault_variables();
+
+ /*
+ * Check that the data directory looks valid, which will also check the
+ * privileges on the data directory and update our umask and file/group
+ * variables for creating files later. Note: this should really be done
+ * before we create any files or directories.
+ */
+ checkDataDir();
+
+ /*
+ * (re-)read control file, as it contains config. The postmaster will
+ * already have read this, but this process doesn't know about that.
+ */
+ LocalProcessControlFile(false);
+
+ /*
+ * Reload any libraries that were preloaded by the postmaster. Since we
+ * exec'd this process, those libraries didn't come along with us; but we
+ * should load them into all child processes to be consistent with the
+ * non-EXEC_BACKEND behavior.
+ */
+ process_shared_preload_libraries();
+
+ /* Restore basic shared memory pointers */
+ if (UsedShmemSegAddr != NULL)
+ InitShmemAccess(UsedShmemSegAddr);
+
+ /* Run backend or appropriate child */
+ child_process_kinds[child_type].main_fn(startup_data, startup_data_len);
+ pg_unreachable(); /* main_fn never returns */
+}
+
/*
* The following need to be available to the save/restore_backend_variables
* functions. They are marked NON_EXEC_STATIC in their home modules.
@@ -468,38 +662,22 @@ static bool write_inheritable_socket(InheritableSocket *dest, SOCKET src,
static void read_inheritable_socket(SOCKET *dest, InheritableSocket *src);
#endif
-#ifndef WIN32
-static bool
-save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker)
-#else
+
static bool
-save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker,
- HANDLE childProcess, pid_t childPid)
+save_backend_variables(BackendParameters *param, ClientSocket *client_sock,
+#ifdef WIN32
+ HANDLE childProcess, pid_t childPid,
#endif
+ char *startup_data, size_t startup_data_len)
{
if (client_sock)
- {
memcpy(¶m->client_sock, client_sock, sizeof(ClientSocket));
- if (!write_inheritable_socket(¶m->inh_sock, client_sock->sock, childPid))
- return false;
- param->has_client_sock = true;
- }
else
- {
memset(¶m->client_sock, 0, sizeof(ClientSocket));
- param->has_client_sock = false;
- }
-
- if (worker)
- {
- memcpy(¶m->bgworker, worker, sizeof(BackgroundWorker));
- param->has_bgworker = true;
- }
- else
- {
- memset(¶m->bgworker, 0, sizeof(BackgroundWorker));
- param->has_bgworker = false;
- }
+ if (!write_inheritable_socket(¶m->inh_sock,
+ client_sock ? client_sock->sock : PGINVALID_SOCKET,
+ childPid))
+ return false;
strlcpy(param->DataDir, DataDir, MAXPGPATH);
@@ -556,6 +734,9 @@ save_backend_variables(BackendParameters *param, ClientSocket *client_sock, Back
strlcpy(param->pkglib_path, pkglib_path, MAXPGPATH);
+ param->startup_data_len = startup_data_len;
+ memcpy(param->startup_data, startup_data, startup_data_len);
+
return true;
}
@@ -652,8 +833,8 @@ read_inheritable_socket(SOCKET *dest, InheritableSocket *src)
}
#endif
-void
-read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **worker)
+static void
+read_backend_variables(char *id, char **startup_data, size_t *startup_data_len)
{
BackendParameters param;
@@ -677,6 +858,21 @@ read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **
exit(1);
}
+ /* read startup data */
+ *startup_data_len = param.startup_data_len;
+ if (param.startup_data_len > 0)
+ {
+ *startup_data = palloc(*startup_data_len);
+ if (fread(*startup_data, *startup_data_len, 1, fp) != 1)
+ {
+ write_stderr("could not read startup data from backend variables file \"%s\": %s\n",
+ id, strerror(errno));
+ exit(1);
+ }
+ }
+ else
+ *startup_data = NULL;
+
/* Release file */
FreeFile(fp);
if (unlink(id) != 0)
@@ -705,6 +901,16 @@ read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **
memcpy(¶m, paramp, sizeof(BackendParameters));
+ /* read startup data */
+ *startup_data_len = param.startup_data_len;
+ if (param.startup_data_len > 0)
+ {
+ *startup_data = palloc(paramp->startup_data_len);
+ memcpy(*startup_data, paramp->startup_data, param.startup_data_len);
+ }
+ else
+ *startup_data = NULL;
+
if (!UnmapViewOfFile(paramp))
{
write_stderr("could not unmap view of backend variables: error code %lu\n",
@@ -720,30 +926,19 @@ read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **
}
#endif
- restore_backend_variables(¶m, client_sock, worker);
+ restore_backend_variables(¶m);
}
/* Restore critical backend variables from the BackendParameters struct */
static void
-restore_backend_variables(BackendParameters *param, ClientSocket **client_sock, BackgroundWorker **worker)
+restore_backend_variables(BackendParameters *param)
{
- if (param->has_client_sock)
- {
- *client_sock = (ClientSocket *) MemoryContextAlloc(TopMemoryContext, sizeof(ClientSocket));
- memcpy(*client_sock, ¶m->client_sock, sizeof(ClientSocket));
- read_inheritable_socket(&(*client_sock)->sock, ¶m->inh_sock);
- }
- else
- *client_sock = NULL;
-
- if (param->has_bgworker)
+ if (param->client_sock.sock != PGINVALID_SOCKET)
{
- *worker = (BackgroundWorker *)
- MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
- memcpy(*worker, ¶m->bgworker, sizeof(BackgroundWorker));
+ MyClientSocket = MemoryContextAlloc(TopMemoryContext, sizeof(ClientSocket));
+ memcpy(MyClientSocket, ¶m->client_sock, sizeof(ClientSocket));
+ read_inheritable_socket(&MyClientSocket->sock, ¶m->inh_sock);
}
- else
- *worker = NULL;
SetDataDir(param->DataDir);
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 5d5c5733340..c7b0a3f1064 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -212,8 +212,10 @@ PgArchCanRestart(void)
/* Main entry point for archiver process */
void
-PgArchiverMain(void)
+PgArchiverMain(char *startup_data, size_t startup_data_len)
{
+ Assert(startup_data_len == 0);
+
MyAuxProcType = ArchiverProcess;
MyBackendType = B_ARCHIVER;
AuxiliaryProcessInit();
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 586aac78d53..4842cb1bcfd 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -2,9 +2,9 @@
*
* postmaster.c
* This program acts as a clearing house for requests to the
- * POSTGRES system. Frontend programs send a startup message
- * to the Postmaster and the postmaster uses the info in the
- * message to setup a backend process.
+ * POSTGRES system. Frontend programs connect to the Postmaster,
+ * and postmaster forks a new backend process to handle the
+ * connection.
*
* The postmaster also manages system-wide operations such as
* startup and shutdown. The postmaster itself doesn't do those
@@ -108,7 +108,6 @@
#include "postmaster/autovacuum.h"
#include "postmaster/auxprocess.h"
#include "postmaster/bgworker_internals.h"
-#include "postmaster/fork_process.h"
#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
#include "postmaster/syslogger.h"
@@ -116,7 +115,6 @@
#include "replication/walsender.h"
#include "storage/fd.h"
#include "storage/ipc.h"
-#include "storage/pg_shmem.h"
#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "tcop/tcopprot.h"
@@ -421,7 +419,6 @@ typedef enum CAC_state
} CAC_state;
static void BackendInitialize(ClientSocket *client_sock, CAC_state cac);
-static void BackendRun(void) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
static int BackendStartup(ClientSocket *client_sock);
@@ -442,7 +439,7 @@ static int CountChildren(int target);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
-static pid_t StartChildProcess(AuxProcType type);
+static pid_t StartChildProcess(PostmasterChildType type);
static void StartAutovacuumWorker(void);
static void MaybeStartWalReceiver(void);
static void InitPostmasterDeathWatchHandle(void);
@@ -477,23 +474,18 @@ typedef struct
} win32_deadchild_waitinfo;
#endif /* WIN32 */
-static pid_t backend_forkexec(ClientSocket *client_sock, CAC_state cac);
-
-
-/* in launch_backend.c */
-extern pid_t internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker);
-extern void read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **worker);
-
static void ShmemBackendArrayAdd(Backend *bn);
static void ShmemBackendArrayRemove(Backend *bn);
#endif /* EXEC_BACKEND */
-#define StartupDataBase() StartChildProcess(StartupProcess)
-#define StartArchiver() StartChildProcess(ArchiverProcess)
-#define StartBackgroundWriter() StartChildProcess(BgWriterProcess)
-#define StartCheckpointer() StartChildProcess(CheckpointerProcess)
-#define StartWalWriter() StartChildProcess(WalWriterProcess)
-#define StartWalReceiver() StartChildProcess(WalReceiverProcess)
+#define StartupDataBase() StartChildProcess(PMC_STARTUP)
+#define StartArchiver() StartChildProcess(PMC_ARCHIVER)
+#define StartBackgroundWriter() StartChildProcess(PMC_BGWRITER)
+#define StartCheckpointer() StartChildProcess(PMC_CHECKPOINTER)
+#define StartWalWriter() StartChildProcess(PMC_WAL_WRITER)
+#define StartWalReceiver() StartChildProcess(PMC_WAL_RECEIVER)
+#define StartAutoVacLauncher() StartChildProcess(PMC_AV_LAUNCHER);
+#define StartAutoVacWorker() StartChildProcess(PMC_AV_WORKER);
/* Macros to check exit status of a child process */
#define EXIT_STATUS_0(st) ((st) == 0)
@@ -3913,6 +3905,12 @@ TerminateChildren(int signal)
signal_child(PgArchPID, signal);
}
+/* Information passed from postmaster to backend process */
+typedef struct BackendStartupInfo
+{
+ CAC_state canAcceptConnections;
+} BackendStartupInfo;
+
/*
* BackendStartup -- start backend process
*
@@ -3925,7 +3923,7 @@ BackendStartup(ClientSocket *client_sock)
{
Backend *bn; /* for backend cleanup */
pid_t pid;
- CAC_state cac;
+ BackendStartupInfo info;
/*
* Create backend data structure. Better before the fork() so we can
@@ -3954,11 +3952,10 @@ BackendStartup(ClientSocket *client_sock)
return STATUS_ERROR;
}
- bn->cancel_key = MyCancelKey;
-
/* Pass down canAcceptConnections state */
- cac = canAcceptConnections(BACKEND_TYPE_NORMAL);
- bn->dead_end = (cac != CAC_OK);
+ info.canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
+ bn->dead_end = (info.canAcceptConnections != CAC_OK);
+ bn->cancel_key = MyCancelKey;
/*
* Unless it's a dead_end child, assign it a child slot number
@@ -3971,26 +3968,7 @@ BackendStartup(ClientSocket *client_sock)
/* Hasn't asked to be notified about any bgworkers yet */
bn->bgworker_notify = false;
-#ifdef EXEC_BACKEND
- pid = backend_forkexec(client_sock, cac);
-#else /* !EXEC_BACKEND */
- pid = fork_process();
- if (pid == 0) /* child */
- {
- /* Detangle from postmaster */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Perform additional initialization and collect startup packet */
- BackendInitialize(client_sock, cac);
-
- /* And run the backend */
- BackendRun();
- }
-#endif /* EXEC_BACKEND */
-
+ pid = postmaster_child_launch(PMC_BACKEND, (char *) &info, sizeof(info), client_sock);
if (pid < 0)
{
/* in parent, fork failed */
@@ -4300,264 +4278,57 @@ BackendInitialize(ClientSocket *client_sock, CAC_state cac)
set_ps_display("initializing");
}
-
-/*
- * BackendRun -- set up the backend's argument list and invoke PostgresMain()
- *
- * returns:
- * Doesn't return at all.
- */
-static void
-BackendRun(void)
-{
- /*
- * Create a per-backend PGPROC struct in shared memory. We must do this
- * before we can use LWLocks or access any shared memory.
- */
- InitProcess();
-
- /*
- * Make sure we aren't in PostmasterContext anymore. (We can't delete it
- * just yet, though, because InitPostgres will need the HBA data.)
- */
- MemoryContextSwitchTo(TopMemoryContext);
-
- PostgresMain(MyProcPort->database_name, MyProcPort->user_name);
-}
-
-
-#ifdef EXEC_BACKEND
-
-/*
- * postmaster_forkexec -- fork and exec a postmaster subprocess
- *
- * The caller must have set up the argv array already, except for argv[2]
- * which will be filled with the name of the temp variable file.
- *
- * Returns the child process PID, or -1 on fork failure (a suitable error
- * message has been logged on failure).
- *
- * All uses of this routine will dispatch to SubPostmasterMain in the
- * child process.
- */
-pid_t
-postmaster_forkexec(int argc, char *argv[])
-{
- return internal_forkexec(argc, argv, NULL, NULL);
-}
-
-/*
- * backend_forkexec -- fork/exec off a backend process
- *
- * Some operating systems (WIN32) don't have fork() so we have to simulate
- * it by storing parameters that need to be passed to the child and
- * then create a new child process.
- *
- * returns the pid of the fork/exec'd process, or -1 on failure
- */
-static pid_t
-backend_forkexec(ClientSocket *client_sock, CAC_state cac)
-{
- char *av[5];
- int ac = 0;
- char cacbuf[10];
-
- av[ac++] = "postgres";
- av[ac++] = "--forkbackend";
- av[ac++] = NULL; /* filled in by internal_forkexec */
-
- snprintf(cacbuf, sizeof(cacbuf), "%d", (int) cac);
- av[ac++] = cacbuf;
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return internal_forkexec(ac, av, client_sock, NULL);
-}
-
-/*
- * SubPostmasterMain -- Get the fork/exec'd process into a state equivalent
- * to what it would be if we'd simply forked on Unix, and then
- * dispatch to the appropriate place.
- *
- * The first two command line arguments are expected to be "--forkFOO"
- * (where FOO indicates which postmaster child we are to become), and
- * the name of a variables file that we can read to load data that would
- * have been inherited by fork() on Unix. Remaining arguments go to the
- * subprocess FooMain() routine.
- */
void
-SubPostmasterMain(int argc, char *argv[])
+BackendMain(char *startup_data, size_t startup_data_len)
{
- ClientSocket *client_sock;
- BackgroundWorker *worker;
-
- /* In EXEC_BACKEND case we will not have inherited these settings */
- IsPostmasterEnvironment = true;
- whereToSendOutput = DestNone;
-
- /* Setup essential subsystems (to ensure elog() behaves sanely) */
- InitializeGUCOptions();
-
- /* Check we got appropriate args */
- if (argc < 3)
- elog(FATAL, "invalid subpostmaster invocation");
+ BackendStartupInfo *info = (BackendStartupInfo *) startup_data;
- /* Read in the variables file */
- read_backend_variables(argv[2], &client_sock, &worker);
+ Assert(startup_data_len == sizeof(BackendStartupInfo));
+ Assert(MyClientSocket != NULL);
- /* Close the postmaster's sockets (as soon as we know them) */
- ClosePostmasterPorts(strcmp(argv[1], "--forklog") == 0);
-
- /* Setup as postmaster child */
- InitPostmasterChild();
+#ifdef EXEC_BACKEND
/*
- * If appropriate, physically re-attach to shared memory segment. We want
- * to do this before going any further to ensure that we can attach at the
- * same address the postmaster used. On the other hand, if we choose not
- * to re-attach, we may have other cleanup to do.
+ * Need to reinitialize the SSL library in the backend, since the context
+ * structures contain function pointers and cannot be passed through the
+ * parameter file.
*
- * If testing EXEC_BACKEND on Linux, you should run this as root before
- * starting the postmaster:
+ * If for some reason reload fails (maybe the user installed broken key
+ * files), soldier on without SSL; that's better than all connections
+ * becoming impossible.
*
- * sysctl -w kernel.randomize_va_space=0
- *
- * This prevents using randomized stack and code addresses that cause the
- * child process's memory map to be different from the parent's, making it
- * sometimes impossible to attach to shared memory at the desired address.
- * Return the setting to its old value (usually '1' or '2') when finished.
+ * XXX should we do this in all child processes? For the moment it's
+ * enough to do it in backend children.
*/
- if (strcmp(argv[1], "--forkbackend") == 0 ||
- strcmp(argv[1], "--forkavlauncher") == 0 ||
- strcmp(argv[1], "--forkavworker") == 0 ||
- strcmp(argv[1], "--forkaux") == 0 ||
- strcmp(argv[1], "--forkbgworker") == 0)
- PGSharedMemoryReAttach();
- else
- PGSharedMemoryNoReAttach();
-
- /* Read in remaining GUC variables */
- read_nondefault_variables();
+#ifdef USE_SSL
+ if (EnableSSL)
+ {
+ if (secure_initialize(false) == 0)
+ LoadedSSL = true;
+ else
+ ereport(LOG,
+ (errmsg("SSL configuration could not be loaded in child process")));
+ }
+#endif
+#endif
- /*
- * Check that the data directory looks valid, which will also check the
- * privileges on the data directory and update our umask and file/group
- * variables for creating files later. Note: this should really be done
- * before we create any files or directories.
- */
- checkDataDir();
+ /* Perform additional initialization and collect startup packet */
+ BackendInitialize(MyClientSocket, info->canAcceptConnections);
/*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
+ * Create a per-backend PGPROC struct in shared memory. We must do this
+ * before we can use LWLocks or access any shared memory.
*/
- LocalProcessControlFile(false);
+ InitProcess();
/*
- * Reload any libraries that were preloaded by the postmaster. Since we
- * exec'd this process, those libraries didn't come along with us; but we
- * should load them into all child processes to be consistent with the
- * non-EXEC_BACKEND behavior.
+ * Make sure we aren't in PostmasterContext anymore. (We can't delete it
+ * just yet, though, because InitPostgres will need the HBA data.)
*/
- process_shared_preload_libraries();
-
- /* Run backend or appropriate child */
- if (strcmp(argv[1], "--forkbackend") == 0)
- {
- CAC_state cac;
-
- Assert(argc == 4);
- cac = (CAC_state) atoi(argv[3]);
-
- /*
- * Need to reinitialize the SSL library in the backend, since the
- * context structures contain function pointers and cannot be passed
- * through the parameter file.
- *
- * If for some reason reload fails (maybe the user installed broken
- * key files), soldier on without SSL; that's better than all
- * connections becoming impossible.
- *
- * XXX should we do this in all child processes? For the moment it's
- * enough to do it in backend children.
- */
-#ifdef USE_SSL
- if (EnableSSL)
- {
- if (secure_initialize(false) == 0)
- LoadedSSL = true;
- else
- ereport(LOG,
- (errmsg("SSL configuration could not be loaded in child process")));
- }
-#endif
-
- /*
- * Perform additional initialization and collect startup packet.
- *
- * We want to do this before InitProcess() for a couple of reasons: 1.
- * so that we aren't eating up a PGPROC slot while waiting on the
- * client. 2. so that if InitProcess() fails due to being out of
- * PGPROC slots, we have already initialized libpq and are able to
- * report the error to the client.
- */
- BackendInitialize(client_sock, cac);
-
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* And run the backend */
- BackendRun(); /* does not return */
-
- }
- if (strcmp(argv[1], "--forkaux") == 0)
- {
- AuxProcType auxtype;
-
- Assert(argc == 4);
-
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- auxtype = atoi(argv[3]);
- AuxiliaryProcessMain(auxtype); /* does not return */
- }
- if (strcmp(argv[1], "--forkavlauncher") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- AutoVacLauncherMain(argc - 2, argv + 2); /* does not return */
- }
- if (strcmp(argv[1], "--forkavworker") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
- }
- if (strcmp(argv[1], "--forkbgworker") == 0)
- {
- /* do this as early as possible; in particular, before InitProcess() */
- IsBackgroundWorker = true;
-
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- MyBgworkerEntry = worker;
- BackgroundWorkerMain();
- }
- if (strcmp(argv[1], "--forklog") == 0)
- {
- /* Do not want to attach to shared memory */
-
- SysLoggerMain(argc, argv); /* does not return */
- }
+ MemoryContextSwitchTo(TopMemoryContext);
- abort(); /* shouldn't get here */
+ PostgresMain(MyProcPort->database_name, MyProcPort->user_name);
}
-#endif /* EXEC_BACKEND */
/*
@@ -4852,93 +4623,23 @@ CountChildren(int target)
* to start subprocess.
*/
static pid_t
-StartChildProcess(AuxProcType type)
+StartChildProcess(PostmasterChildType type)
{
pid_t pid;
-#ifdef EXEC_BACKEND
- {
- char *av[10];
- int ac = 0;
- char typebuf[32];
-
- /*
- * Set up command-line arguments for subprocess
- */
- av[ac++] = "postgres";
- av[ac++] = "--forkaux";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- snprintf(typebuf, sizeof(typebuf), "%d", type);
- av[ac++] = typebuf;
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- pid = postmaster_forkexec(ac, av);
- }
-#else /* !EXEC_BACKEND */
- pid = fork_process();
-
- if (pid == 0) /* child */
- {
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Release postmaster's working memory context */
- MemoryContextSwitchTo(TopMemoryContext);
- MemoryContextDelete(PostmasterContext);
- PostmasterContext = NULL;
-
- AuxiliaryProcessMain(type); /* does not return */
- }
-#endif /* EXEC_BACKEND */
-
+ pid = postmaster_child_launch(type, NULL, 0, NULL);
if (pid < 0)
{
/* in parent, fork failed */
- int save_errno = errno;
-
- errno = save_errno;
- switch (type)
- {
- case StartupProcess:
- ereport(LOG,
- (errmsg("could not fork startup process: %m")));
- break;
- case ArchiverProcess:
- ereport(LOG,
- (errmsg("could not fork archiver process: %m")));
- break;
- case BgWriterProcess:
- ereport(LOG,
- (errmsg("could not fork background writer process: %m")));
- break;
- case CheckpointerProcess:
- ereport(LOG,
- (errmsg("could not fork checkpointer process: %m")));
- break;
- case WalWriterProcess:
- ereport(LOG,
- (errmsg("could not fork WAL writer process: %m")));
- break;
- case WalReceiverProcess:
- ereport(LOG,
- (errmsg("could not fork WAL receiver process: %m")));
- break;
- default:
- ereport(LOG,
- (errmsg("could not fork process: %m")));
- break;
- }
+ /* XXX: translation? */
+ ereport(LOG,
+ (errmsg("could not fork %s process: %m", PostmasterChildName(type))));
/*
* fork failure is fatal during startup, but there's no need to choke
* immediately if starting other child types fails.
*/
- if (type == StartupProcess)
+ if (type == PMC_STARTUP)
ExitPostmaster(1);
return 0;
}
@@ -5202,24 +4903,6 @@ BackgroundWorkerUnblockSignals(void)
sigprocmask(SIG_SETMASK, &UnBlockSig, NULL);
}
-#ifdef EXEC_BACKEND
-static pid_t
-bgworker_forkexec(BackgroundWorker *worker)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkbgworker";
- av[ac++] = NULL; /* filled in by internal_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return internal_forkexec(ac, av, NULL, worker);
-}
-#endif
-
/*
* Start a new bgworker.
* Starting time conditions must have been checked already.
@@ -5256,65 +4939,32 @@ do_start_bgworker(RegisteredBgWorker *rw)
(errmsg_internal("starting background worker process \"%s\"",
rw->rw_worker.bgw_name)));
-#ifdef EXEC_BACKEND
- switch ((worker_pid = bgworker_forkexec(&rw->rw_worker)))
-#else
- switch ((worker_pid = fork_process()))
-#endif
+ worker_pid = postmaster_child_launch(PMC_BGWORKER, (char *) &rw->rw_worker, sizeof(BackgroundWorker), NULL);
+ if (worker_pid == -1)
{
- case -1:
- /* in postmaster, fork failed ... */
- ereport(LOG,
- (errmsg("could not fork worker process: %m")));
- /* undo what assign_backendlist_entry did */
- ReleasePostmasterChildSlot(rw->rw_child_slot);
- rw->rw_child_slot = 0;
- pfree(rw->rw_backend);
- rw->rw_backend = NULL;
- /* mark entry as crashed, so we'll try again later */
- rw->rw_crashed_at = GetCurrentTimestamp();
- break;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /*
- * Before blowing away PostmasterContext, save this bgworker's
- * data where it can find it.
- */
- MyBgworkerEntry = (BackgroundWorker *)
- MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
- memcpy(MyBgworkerEntry, &rw->rw_worker, sizeof(BackgroundWorker));
-
- /* Release postmaster's working memory context */
- MemoryContextSwitchTo(TopMemoryContext);
- MemoryContextDelete(PostmasterContext);
- PostmasterContext = NULL;
-
- BackgroundWorkerMain();
+ /* in postmaster, fork failed ... */
+ ereport(LOG,
+ (errmsg("could not fork worker process: %m")));
+ /* undo what assign_backendlist_entry did */
+ ReleasePostmasterChildSlot(rw->rw_child_slot);
+ rw->rw_child_slot = 0;
+ pfree(rw->rw_backend);
+ rw->rw_backend = NULL;
+ /* mark entry as crashed, so we'll try again later */
+ rw->rw_crashed_at = GetCurrentTimestamp();
+ return false;
+ }
- exit(1); /* should not get here */
- break;
-#endif
- default:
- /* in postmaster, fork successful ... */
- rw->rw_pid = worker_pid;
- rw->rw_backend->pid = rw->rw_pid;
- ReportBackgroundWorkerPID(rw);
- /* add new worker to lists of backends */
- dlist_push_head(&BackendList, &rw->rw_backend->elem);
+ /* in postmaster, fork successful ... */
+ rw->rw_pid = worker_pid;
+ rw->rw_backend->pid = rw->rw_pid;
+ ReportBackgroundWorkerPID(rw);
+ /* add new worker to lists of backends */
+ dlist_push_head(&BackendList, &rw->rw_backend->elem);
#ifdef EXEC_BACKEND
- ShmemBackendArrayAdd(rw->rw_backend);
+ ShmemBackendArrayAdd(rw->rw_backend);
#endif
- return true;
- }
-
- return false;
+ return true;
}
/*
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index 0fdfa1822db..00dc2b90354 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -242,8 +242,10 @@ StartupProcExit(int code, Datum arg)
* ----------------------------------
*/
void
-StartupProcessMain(void)
+StartupProcessMain(char *startup_data, size_t startup_data_len)
{
+ Assert(startup_data_len == 0);
+
MyAuxProcType = StartupProcess;
MyBackendType = B_STARTUP;
AuxiliaryProcessInit();
diff --git a/src/backend/postmaster/syslogger.c b/src/backend/postmaster/syslogger.c
index 96dd03d9e06..c923843532f 100644
--- a/src/backend/postmaster/syslogger.c
+++ b/src/backend/postmaster/syslogger.c
@@ -39,7 +39,6 @@
#include "pgstat.h"
#include "pgtime.h"
#include "port/pg_bitutils.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "postmaster/syslogger.h"
@@ -50,6 +49,7 @@
#include "storage/pg_shmem.h"
#include "tcop/tcopprot.h"
#include "utils/guc.h"
+#include "utils/memutils.h"
#include "utils/ps_status.h"
#include "utils/timestamp.h"
@@ -134,10 +134,7 @@ static volatile sig_atomic_t rotation_requested = false;
#ifdef EXEC_BACKEND
static int syslogger_fdget(FILE *file);
static FILE *syslogger_fdopen(int fd);
-static pid_t syslogger_forkexec(void);
-static void syslogger_parseArgs(int argc, char *argv[]);
#endif
-NON_EXEC_STATIC void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
static void process_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static void flush_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static FILE *logfile_open(const char *filename, const char *mode,
@@ -156,13 +153,19 @@ static void set_next_rotation_time(void);
static void sigUsr1Handler(SIGNAL_ARGS);
static void update_metainfo_datafile(void);
+typedef struct
+{
+ int syslogFile;
+ int csvlogFile;
+ int jsonlogFile;
+} syslogger_startup_data;
/*
* Main entry point for syslogger process
* argc/argv parameters are valid only in EXEC_BACKEND case.
*/
-NON_EXEC_STATIC void
-SysLoggerMain(int argc, char *argv[])
+void
+SysLoggerMain(char *startup_data, size_t startup_data_len)
{
#ifndef WIN32
char logbuffer[READ_BUF_SIZE];
@@ -174,11 +177,34 @@ SysLoggerMain(int argc, char *argv[])
pg_time_t now;
WaitEventSet *wes;
- now = MyStartTime;
+ /* Release postmaster's working memory context */
+ if (PostmasterContext)
+ {
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
+ }
+ /*
+ * Re-open the error output files that were opened by SysLogger_Start().
+ *
+ * We expect this will always succeed, which is too optimistic, but if it
+ * fails there's not a lot we can do to report the problem anyway. As
+ * coded, we'll just crash on a null pointer dereference after failure...
+ */
#ifdef EXEC_BACKEND
- syslogger_parseArgs(argc, argv);
-#endif /* EXEC_BACKEND */
+ {
+ syslogger_startup_data *info = (syslogger_startup_data *) startup_data;
+
+ Assert(startup_data_len == sizeof(*info));
+ syslogFile = syslogger_fdopen(info->syslogFile);
+ csvlogFile = syslogger_fdopen(info->csvlogFile);
+ jsonlogFile = syslogger_fdopen(info->jsonlogFile);
+ }
+#else
+ Assert(startup_data_len == 0);
+#endif
+
+ now = MyStartTime;
MyBackendType = B_LOGGER;
init_ps_display(NULL);
@@ -568,6 +594,9 @@ SysLogger_Start(void)
{
pid_t sysloggerPid;
char *filename;
+#ifdef EXEC_BACKEND
+ syslogger_startup_data startup_data;
+#endif /* EXEC_BACKEND */
if (!Logging_collector)
return 0;
@@ -667,112 +696,95 @@ SysLogger_Start(void)
}
#ifdef EXEC_BACKEND
- switch ((sysloggerPid = syslogger_forkexec()))
+ startup_data.syslogFile = syslogger_fdget(syslogFile);
+ startup_data.csvlogFile = syslogger_fdget(csvlogFile);
+ startup_data.jsonlogFile = syslogger_fdget(jsonlogFile);
+ sysloggerPid = postmaster_child_launch(PMC_SYSLOGGER, (char *) &startup_data, sizeof(startup_data), NULL);
#else
- switch ((sysloggerPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork system logger: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(true);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
+ sysloggerPid = postmaster_child_launch(PMC_SYSLOGGER, NULL, 0, NULL);
+#endif /* EXEC_BACKEND */
- /* do the work */
- SysLoggerMain(0, NULL);
- break;
-#endif
+ if (sysloggerPid == -1)
+ {
+ ereport(LOG,
+ (errmsg("could not fork system logger: %m")));
+ return 0;
+ }
- default:
- /* success, in postmaster */
+ /* success, in postmaster */
- /* now we redirect stderr, if not done already */
- if (!redirection_done)
- {
+ /* now we redirect stderr, if not done already */
+ if (!redirection_done)
+ {
#ifdef WIN32
- int fd;
+ int fd;
#endif
- /*
- * Leave a breadcrumb trail when redirecting, in case the user
- * forgets that redirection is active and looks only at the
- * original stderr target file.
- */
- ereport(LOG,
- (errmsg("redirecting log output to logging collector process"),
- errhint("Future log output will appear in directory \"%s\".",
- Log_directory)));
+ /*
+ * Leave a breadcrumb trail when redirecting, in case the user forgets
+ * that redirection is active and looks only at the original stderr
+ * target file.
+ */
+ ereport(LOG,
+ (errmsg("redirecting log output to logging collector process"),
+ errhint("Future log output will appear in directory \"%s\".",
+ Log_directory)));
#ifndef WIN32
- fflush(stdout);
- if (dup2(syslogPipe[1], STDOUT_FILENO) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stdout: %m")));
- fflush(stderr);
- if (dup2(syslogPipe[1], STDERR_FILENO) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stderr: %m")));
- /* Now we are done with the write end of the pipe. */
- close(syslogPipe[1]);
- syslogPipe[1] = -1;
+ fflush(stdout);
+ if (dup2(syslogPipe[1], STDOUT_FILENO) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stdout: %m")));
+ fflush(stderr);
+ if (dup2(syslogPipe[1], STDERR_FILENO) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stderr: %m")));
+ /* Now we are done with the write end of the pipe. */
+ close(syslogPipe[1]);
+ syslogPipe[1] = -1;
#else
- /*
- * open the pipe in binary mode and make sure stderr is binary
- * after it's been dup'ed into, to avoid disturbing the pipe
- * chunking protocol.
- */
- fflush(stderr);
- fd = _open_osfhandle((intptr_t) syslogPipe[1],
- _O_APPEND | _O_BINARY);
- if (dup2(fd, STDERR_FILENO) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stderr: %m")));
- close(fd);
- _setmode(STDERR_FILENO, _O_BINARY);
+ /*
+ * open the pipe in binary mode and make sure stderr is binary after
+ * it's been dup'ed into, to avoid disturbing the pipe chunking
+ * protocol.
+ */
+ fflush(stderr);
+ fd = _open_osfhandle((intptr_t) syslogPipe[1],
+ _O_APPEND | _O_BINARY);
+ if (dup2(fd, STDERR_FILENO) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stderr: %m")));
+ close(fd);
+ _setmode(STDERR_FILENO, _O_BINARY);
- /*
- * Now we are done with the write end of the pipe.
- * CloseHandle() must not be called because the preceding
- * close() closes the underlying handle.
- */
- syslogPipe[1] = 0;
+ /*
+ * Now we are done with the write end of the pipe. CloseHandle() must
+ * not be called because the preceding close() closes the underlying
+ * handle.
+ */
+ syslogPipe[1] = 0;
#endif
- redirection_done = true;
- }
-
- /* postmaster will never write the file(s); close 'em */
- fclose(syslogFile);
- syslogFile = NULL;
- if (csvlogFile != NULL)
- {
- fclose(csvlogFile);
- csvlogFile = NULL;
- }
- if (jsonlogFile != NULL)
- {
- fclose(jsonlogFile);
- jsonlogFile = NULL;
- }
- return (int) sysloggerPid;
+ redirection_done = true;
}
- /* we should never reach here */
- return 0;
+ /* postmaster will never write the file(s); close 'em */
+ fclose(syslogFile);
+ syslogFile = NULL;
+ if (csvlogFile != NULL)
+ {
+ fclose(csvlogFile);
+ csvlogFile = NULL;
+ }
+ if (jsonlogFile != NULL)
+ {
+ fclose(jsonlogFile);
+ jsonlogFile = NULL;
+ }
+ return (int) sysloggerPid;
}
@@ -831,69 +843,6 @@ syslogger_fdopen(int fd)
return file;
}
-
-/*
- * syslogger_forkexec() -
- *
- * Format up the arglist for, then fork and exec, a syslogger process
- */
-static pid_t
-syslogger_forkexec(void)
-{
- char *av[10];
- int ac = 0;
- char filenobuf[32];
- char csvfilenobuf[32];
- char jsonfilenobuf[32];
-
- av[ac++] = "postgres";
- av[ac++] = "--forklog";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- /* static variables (those not passed by write_backend_variables) */
- snprintf(filenobuf, sizeof(filenobuf), "%d",
- syslogger_fdget(syslogFile));
- av[ac++] = filenobuf;
- snprintf(csvfilenobuf, sizeof(csvfilenobuf), "%d",
- syslogger_fdget(csvlogFile));
- av[ac++] = csvfilenobuf;
- snprintf(jsonfilenobuf, sizeof(jsonfilenobuf), "%d",
- syslogger_fdget(jsonlogFile));
- av[ac++] = jsonfilenobuf;
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-
-/*
- * syslogger_parseArgs() -
- *
- * Extract data from the arglist for exec'ed syslogger process
- */
-static void
-syslogger_parseArgs(int argc, char *argv[])
-{
- int fd;
-
- Assert(argc == 6);
- argv += 3;
-
- /*
- * Re-open the error output files that were opened by SysLogger_Start().
- *
- * We expect this will always succeed, which is too optimistic, but if it
- * fails there's not a lot we can do to report the problem anyway. As
- * coded, we'll just crash on a null pointer dereference after failure...
- */
- fd = atoi(*argv++);
- syslogFile = syslogger_fdopen(fd);
- fd = atoi(*argv++);
- csvlogFile = syslogger_fdopen(fd);
- fd = atoi(*argv++);
- jsonlogFile = syslogger_fdopen(fd);
-}
#endif /* EXEC_BACKEND */
diff --git a/src/backend/postmaster/walwriter.c b/src/backend/postmaster/walwriter.c
index 0575d2c967d..89950350aee 100644
--- a/src/backend/postmaster/walwriter.c
+++ b/src/backend/postmaster/walwriter.c
@@ -89,13 +89,15 @@ static void HandleWalWriterInterrupts(void);
* basic execution environment, but not enabled signals yet.
*/
void
-WalWriterMain(void)
+WalWriterMain(char *startup_data, size_t startup_data_len)
{
sigjmp_buf local_sigjmp_buf;
MemoryContext walwriter_context;
int left_till_hibernate;
bool hibernating;
+ Assert(startup_data_len == 0);
+
MyAuxProcType = WalWriterProcess;
MyBackendType = B_WAL_WRITER;
AuxiliaryProcessInit();
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index 51fd1de9c8b..9b01132704b 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -184,7 +184,7 @@ ProcessWalRcvInterrupts(void)
/* Main entry point for walreceiver process */
void
-WalReceiverMain(void)
+WalReceiverMain(char *startup_data, size_t startup_data_len)
{
char conninfo[MAXCONNINFO];
char *tmp_conninfo;
@@ -200,6 +200,8 @@ WalReceiverMain(void)
char *sender_host = NULL;
int sender_port = 0;
+ Assert(startup_data_len == 0);
+
MyAuxProcType = WalReceiverProcess;
MyBackendType = B_WAL_RECEIVER;
AuxiliaryProcessInit();
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 60bc1217fb4..b6c3055027e 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -44,6 +44,7 @@ volatile uint32 CritSectionCount = 0;
int MyProcPid;
pg_time_t MyStartTime;
TimestampTz MyStartTimestamp;
+struct ClientSocket *MyClientSocket;
struct Port *MyProcPort;
int32 MyCancelKey;
int MyPMChildSlot;
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index c9ef31ae66a..ee12e477f11 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -55,18 +55,14 @@ extern bool IsAutoVacuumWorkerProcess(void);
#define IsAnyAutoVacuumProcess() \
(IsAutoVacuumLauncherProcess() || IsAutoVacuumWorkerProcess())
-/* Functions to start autovacuum process, called from postmaster */
+/* called from postmaster at server startup */
extern void autovac_init(void);
-extern int StartAutoVacLauncher(void);
-extern int StartAutoVacWorker(void);
/* called from postmaster when a worker could not be forked */
extern void AutoVacWorkerFailed(void);
-#ifdef EXEC_BACKEND
-extern void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
-extern void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
+extern void AutoVacLauncherMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
+extern void AutoVacWorkerMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern bool AutoVacuumRequestWork(AutoVacuumWorkItemType type,
Oid relationId, BlockNumber blkno);
diff --git a/src/include/postmaster/auxprocess.h b/src/include/postmaster/auxprocess.h
index cc75f246818..75394ca0155 100644
--- a/src/include/postmaster/auxprocess.h
+++ b/src/include/postmaster/auxprocess.h
@@ -13,7 +13,6 @@
#ifndef AUXPROCESS_H
#define AUXPROCESS_H
-extern void AuxiliaryProcessMain(AuxProcType auxtype) pg_attribute_noreturn();
extern void AuxiliaryProcessInit(void);
#endif /* AUXPROCESS_H */
diff --git a/src/include/postmaster/bgworker_internals.h b/src/include/postmaster/bgworker_internals.h
index 323f1e07291..4055d2f5626 100644
--- a/src/include/postmaster/bgworker_internals.h
+++ b/src/include/postmaster/bgworker_internals.h
@@ -55,6 +55,6 @@ extern void ForgetUnstartedBackgroundWorkers(void);
extern void ResetBackgroundWorkerCrashTimes(void);
/* Entry point for background worker processes */
-extern void BackgroundWorkerMain(void) pg_attribute_noreturn();
+extern void BackgroundWorkerMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
#endif /* BGWORKER_INTERNALS_H */
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index a66722873f4..ee54fc401ef 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -27,8 +27,8 @@ extern PGDLLIMPORT int CheckPointTimeout;
extern PGDLLIMPORT int CheckPointWarning;
extern PGDLLIMPORT double CheckPointCompletionTarget;
-extern void BackgroundWriterMain(void) pg_attribute_noreturn();
-extern void CheckpointerMain(void) pg_attribute_noreturn();
+extern void BackgroundWriterMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
+extern void CheckpointerMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern void RequestCheckpoint(int flags);
extern void CheckpointWriteDelay(int flags, double progress);
diff --git a/src/include/postmaster/pgarch.h b/src/include/postmaster/pgarch.h
index 3bd4fac71e5..577fc14e1d0 100644
--- a/src/include/postmaster/pgarch.h
+++ b/src/include/postmaster/pgarch.h
@@ -29,7 +29,7 @@
extern Size PgArchShmemSize(void);
extern void PgArchShmemInit(void);
extern bool PgArchCanRestart(void);
-extern void PgArchiverMain(void) pg_attribute_noreturn();
+extern void PgArchiverMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern void PgArchWakeup(void);
extern void PgArchForceDirScan(void);
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index 45d4b78c3a6..031a2ff1521 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -58,11 +58,9 @@ extern int MaxLivePostmasterChildren(void);
extern bool PostmasterMarkPIDForWorkerNotify(int);
-#ifdef EXEC_BACKEND
-
-extern pid_t postmaster_forkexec(int argc, char *argv[]);
-extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
+extern void BackendMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
+#ifdef EXEC_BACKEND
extern Size ShmemBackendArraySize(void);
extern void ShmemBackendArrayAllocation(void);
@@ -71,6 +69,45 @@ extern void pgwin32_register_deadchild_callback(HANDLE procHandle, DWORD procId)
#endif
#endif
+/* in launch_backend.c */
+
+/* this better match the list in launch_backend.c */
+typedef enum PostmasterChildType
+{
+ PMC_BACKEND = 0,
+ PMC_AV_LAUNCHER,
+ PMC_AV_WORKER,
+ PMC_BGWORKER,
+ PMC_SYSLOGGER,
+
+ /*
+ * so-called "aux processes". These access shared memory, but are not
+ * attached to any particular database. Only one of each of these can be
+ * running at a time.
+ */
+ PMC_STARTUP,
+ PMC_BGWRITER,
+ PMC_ARCHIVER,
+ PMC_CHECKPOINTER,
+ PMC_WAL_WRITER,
+ PMC_WAL_RECEIVER,
+} PostmasterChildType;
+
+/* defined in libpq-be.h */
+extern struct ClientSocket *MyClientSocket;
+
+extern pid_t postmaster_child_launch(PostmasterChildType child_type, char *startup_data, size_t startup_data_len, struct ClientSocket *sock);
+
+#ifdef EXEC_BACKEND
+extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
+#endif
+
+#ifdef WIN32
+extern void pgwin32_register_deadchild_callback(HANDLE procHandle, DWORD procId);
+#endif
+
+const char *PostmasterChildName(PostmasterChildType child_type);
+
/*
* Note: MAX_BACKENDS is limited to 2^18-1 because that's the width reserved
* for buffer references in buf_internals.h. This limitation could be lifted
diff --git a/src/include/postmaster/startup.h b/src/include/postmaster/startup.h
index 6a2e4c4526b..ec885063aab 100644
--- a/src/include/postmaster/startup.h
+++ b/src/include/postmaster/startup.h
@@ -26,7 +26,7 @@
extern PGDLLIMPORT int log_startup_progress_interval;
extern void HandleStartupProcInterrupts(void);
-extern void StartupProcessMain(void) pg_attribute_noreturn();
+extern void StartupProcessMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern void PreRestoreCommand(void);
extern void PostRestoreCommand(void);
extern bool IsPromoteSignaled(void);
diff --git a/src/include/postmaster/syslogger.h b/src/include/postmaster/syslogger.h
index 34da778f1ef..7dc41b30e7c 100644
--- a/src/include/postmaster/syslogger.h
+++ b/src/include/postmaster/syslogger.h
@@ -86,9 +86,7 @@ extern int SysLogger_Start(void);
extern void write_syslogger_file(const char *buffer, int count, int destination);
-#ifdef EXEC_BACKEND
-extern void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
+extern void SysLoggerMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern bool CheckLogrotateSignal(void);
extern void RemoveLogrotateSignalFiles(void);
diff --git a/src/include/postmaster/walwriter.h b/src/include/postmaster/walwriter.h
index 6eba7ad79cf..99b7cc07fb2 100644
--- a/src/include/postmaster/walwriter.h
+++ b/src/include/postmaster/walwriter.h
@@ -18,6 +18,6 @@
extern PGDLLIMPORT int WalWriterDelay;
extern PGDLLIMPORT int WalWriterFlushAfter;
-extern void WalWriterMain(void) pg_attribute_noreturn();
+extern void WalWriterMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
#endif /* _WALWRITER_H */
diff --git a/src/include/replication/walreceiver.h b/src/include/replication/walreceiver.h
index 949e874f219..a3a5ab6814d 100644
--- a/src/include/replication/walreceiver.h
+++ b/src/include/replication/walreceiver.h
@@ -456,7 +456,7 @@ walrcv_clear_result(WalRcvExecResult *walres)
}
/* prototypes for functions in walreceiver.c */
-extern void WalReceiverMain(void) pg_attribute_noreturn();
+extern void WalReceiverMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern void ProcessWalRcvInterrupts(void);
extern void WalRcvForceReply(void);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 90217c4939a..0e433100f0e 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -230,6 +230,7 @@ BY_HANDLE_FILE_INFORMATION
Backend
BackendId
BackendParameters
+BackendStartupInfo
BackendState
BackendType
BackgroundWorker
@@ -2121,6 +2122,7 @@ PortalStrategy
PostParseColumnRefHook
PostgresPollingStatusType
PostingItem
+PostmasterChildType
PreParseColumnRefHook
PredClass
PredIterInfo
@@ -3792,6 +3794,7 @@ substitute_actual_parameters_context
substitute_actual_srf_parameters_context
substitute_phv_relids_context
symbol
+syslogger_startup_data
tablespaceinfo
teSection
temp_tablespaces_extra
--
2.39.2
v5-0008-Move-code-for-backend-startup-to-separate-file.patchtext/x-patch; charset=UTF-8; name=v5-0008-Move-code-for-backend-startup-to-separate-file.patchDownload
From b5b07eb257cd8f856a317fe4499cc835129b2438 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Fri, 8 Dec 2023 14:23:28 +0200
Subject: [PATCH v5 8/8] Move code for backend startup to separate file
This is code that runs in the backend process after forking.
---
src/backend/postmaster/postmaster.c | 763 +--------------------------
src/backend/tcop/Makefile | 1 +
src/backend/tcop/backend_startup.c | 777 ++++++++++++++++++++++++++++
src/backend/tcop/meson.build | 1 +
src/include/postmaster/postmaster.h | 4 +
src/include/tcop/backend_startup.h | 41 ++
6 files changed, 829 insertions(+), 758 deletions(-)
create mode 100644 src/backend/tcop/backend_startup.c
create mode 100644 src/include/tcop/backend_startup.h
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 4842cb1bcfd..2afb016c767 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -96,11 +96,9 @@
#include "common/file_utils.h"
#include "common/ip.h"
#include "common/pg_prng.h"
-#include "common/string.h"
#include "lib/ilist.h"
#include "libpq/auth.h"
#include "libpq/libpq.h"
-#include "libpq/pqformat.h"
#include "libpq/pqsignal.h"
#include "pg_getopt.h"
#include "pgstat.h"
@@ -117,13 +115,11 @@
#include "storage/ipc.h"
#include "storage/pmsignal.h"
#include "storage/proc.h"
+#include "tcop/backend_startup.h"
#include "tcop/tcopprot.h"
-#include "utils/builtins.h"
#include "utils/datetime.h"
#include "utils/memutils.h"
#include "utils/pidfile.h"
-#include "utils/ps_status.h"
-#include "utils/timeout.h"
#include "utils/timestamp.h"
#include "utils/varlena.h"
@@ -376,7 +372,7 @@ static WaitEventSet *pm_wait_set;
#ifdef USE_SSL
/* Set when and if SSL has been initialized properly */
-static bool LoadedSSL = false;
+bool LoadedSSL = false;
#endif
#ifdef USE_BONJOUR
@@ -398,9 +394,7 @@ static void process_pm_pmsignal(void);
static void process_pm_child_exit(void);
static void process_pm_reload_request(void);
static void process_pm_shutdown_request(void);
-static void process_startup_packet_die(SIGNAL_ARGS);
static void dummy_handler(SIGNAL_ARGS);
-static void StartupPacketTimeoutHandler(void);
static void CleanupBackend(int pid, int exitstatus);
static bool CleanupBackgroundWorker(int pid, int exitstatus);
static void HandleChildCrash(int pid, int exitstatus, const char *procname);
@@ -408,23 +402,9 @@ static void LogChildExit(int lev, const char *procname,
int pid, int exitstatus);
static void PostmasterStateMachine(void);
-typedef enum CAC_state
-{
- CAC_OK,
- CAC_STARTUP,
- CAC_SHUTDOWN,
- CAC_RECOVERY,
- CAC_NOTCONSISTENT,
- CAC_TOOMANY,
-} CAC_state;
-
-static void BackendInitialize(ClientSocket *client_sock, CAC_state cac);
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
-static int BackendStartup(ClientSocket *client_sock);
-static int ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done);
-static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
-static void processCancelRequest(Port *port, void *pkt);
+static int BackendStartup(ClientSocket *port);
static void report_fork_failure_to_client(ClientSocket *client_sock, int errnum);
static CAC_state canAcceptConnections(int backend_type);
static bool RandomCancelKey(int32 *cancel_key);
@@ -1841,412 +1821,14 @@ ServerLoop(void)
}
}
-/*
- * Read a client's startup packet and do something according to it.
- *
- * Returns STATUS_OK or STATUS_ERROR, or might call ereport(FATAL) and
- * not return at all.
- *
- * (Note that ereport(FATAL) stuff is sent to the client, so only use it
- * if that's what you want. Return STATUS_ERROR if you don't want to
- * send anything to the client, which would typically be appropriate
- * if we detect a communications failure.)
- *
- * Set ssl_done and/or gss_done when negotiation of an encrypted layer
- * (currently, TLS or GSSAPI) is completed. A successful negotiation of either
- * encryption layer sets both flags, but a rejected negotiation sets only the
- * flag for that layer, since the client may wish to try the other one. We
- * should make no assumption here about the order in which the client may make
- * requests.
- */
-static int
-ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
-{
- int32 len;
- char *buf;
- ProtocolVersion proto;
- MemoryContext oldcontext;
-
- pq_startmsgread();
-
- /*
- * Grab the first byte of the length word separately, so that we can tell
- * whether we have no data at all or an incomplete packet. (This might
- * sound inefficient, but it's not really, because of buffering in
- * pqcomm.c.)
- */
- if (pq_getbytes((char *) &len, 1) == EOF)
- {
- /*
- * If we get no data at all, don't clutter the log with a complaint;
- * such cases often occur for legitimate reasons. An example is that
- * we might be here after responding to NEGOTIATE_SSL_CODE, and if the
- * client didn't like our response, it'll probably just drop the
- * connection. Service-monitoring software also often just opens and
- * closes a connection without sending anything. (So do port
- * scanners, which may be less benign, but it's not really our job to
- * notice those.)
- */
- return STATUS_ERROR;
- }
-
- if (pq_getbytes(((char *) &len) + 1, 3) == EOF)
- {
- /* Got a partial length word, so bleat about that */
- if (!ssl_done && !gss_done)
- ereport(COMMERROR,
- (errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("incomplete startup packet")));
- return STATUS_ERROR;
- }
-
- len = pg_ntoh32(len);
- len -= 4;
-
- if (len < (int32) sizeof(ProtocolVersion) ||
- len > MAX_STARTUP_PACKET_LENGTH)
- {
- ereport(COMMERROR,
- (errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("invalid length of startup packet")));
- return STATUS_ERROR;
- }
-
- /*
- * Allocate space to hold the startup packet, plus one extra byte that's
- * initialized to be zero. This ensures we will have null termination of
- * all strings inside the packet.
- */
- buf = palloc(len + 1);
- buf[len] = '\0';
-
- if (pq_getbytes(buf, len) == EOF)
- {
- ereport(COMMERROR,
- (errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("incomplete startup packet")));
- return STATUS_ERROR;
- }
- pq_endmsgread();
-
- /*
- * The first field is either a protocol version number or a special
- * request code.
- */
- port->proto = proto = pg_ntoh32(*((ProtocolVersion *) buf));
-
- if (proto == CANCEL_REQUEST_CODE)
- {
- if (len != sizeof(CancelRequestPacket))
- {
- ereport(COMMERROR,
- (errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("invalid length of startup packet")));
- return STATUS_ERROR;
- }
- processCancelRequest(port, buf);
- /* Not really an error, but we don't want to proceed further */
- return STATUS_ERROR;
- }
-
- if (proto == NEGOTIATE_SSL_CODE && !ssl_done)
- {
- char SSLok;
-
-#ifdef USE_SSL
- /* No SSL when disabled or on Unix sockets */
- if (!LoadedSSL || port->laddr.addr.ss_family == AF_UNIX)
- SSLok = 'N';
- else
- SSLok = 'S'; /* Support for SSL */
-#else
- SSLok = 'N'; /* No support for SSL */
-#endif
-
-retry1:
- if (send(port->sock, &SSLok, 1, 0) != 1)
- {
- if (errno == EINTR)
- goto retry1; /* if interrupted, just retry */
- ereport(COMMERROR,
- (errcode_for_socket_access(),
- errmsg("failed to send SSL negotiation response: %m")));
- return STATUS_ERROR; /* close the connection */
- }
-
-#ifdef USE_SSL
- if (SSLok == 'S' && secure_open_server(port) == -1)
- return STATUS_ERROR;
-#endif
-
- /*
- * At this point we should have no data already buffered. If we do,
- * it was received before we performed the SSL handshake, so it wasn't
- * encrypted and indeed may have been injected by a man-in-the-middle.
- * We report this case to the client.
- */
- if (pq_buffer_has_data())
- ereport(FATAL,
- (errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("received unencrypted data after SSL request"),
- errdetail("This could be either a client-software bug or evidence of an attempted man-in-the-middle attack.")));
-
- /*
- * regular startup packet, cancel, etc packet should follow, but not
- * another SSL negotiation request, and a GSS request should only
- * follow if SSL was rejected (client may negotiate in either order)
- */
- return ProcessStartupPacket(port, true, SSLok == 'S');
- }
- else if (proto == NEGOTIATE_GSS_CODE && !gss_done)
- {
- char GSSok = 'N';
-
-#ifdef ENABLE_GSS
- /* No GSSAPI encryption when on Unix socket */
- if (port->laddr.addr.ss_family != AF_UNIX)
- GSSok = 'G';
-#endif
-
- while (send(port->sock, &GSSok, 1, 0) != 1)
- {
- if (errno == EINTR)
- continue;
- ereport(COMMERROR,
- (errcode_for_socket_access(),
- errmsg("failed to send GSSAPI negotiation response: %m")));
- return STATUS_ERROR; /* close the connection */
- }
-
-#ifdef ENABLE_GSS
- if (GSSok == 'G' && secure_open_gssapi(port) == -1)
- return STATUS_ERROR;
-#endif
-
- /*
- * At this point we should have no data already buffered. If we do,
- * it was received before we performed the GSS handshake, so it wasn't
- * encrypted and indeed may have been injected by a man-in-the-middle.
- * We report this case to the client.
- */
- if (pq_buffer_has_data())
- ereport(FATAL,
- (errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("received unencrypted data after GSSAPI encryption request"),
- errdetail("This could be either a client-software bug or evidence of an attempted man-in-the-middle attack.")));
-
- /*
- * regular startup packet, cancel, etc packet should follow, but not
- * another GSS negotiation request, and an SSL request should only
- * follow if GSS was rejected (client may negotiate in either order)
- */
- return ProcessStartupPacket(port, GSSok == 'G', true);
- }
-
- /* Could add additional special packet types here */
-
- /*
- * Set FrontendProtocol now so that ereport() knows what format to send if
- * we fail during startup.
- */
- FrontendProtocol = proto;
-
- /* Check that the major protocol version is in range. */
- if (PG_PROTOCOL_MAJOR(proto) < PG_PROTOCOL_MAJOR(PG_PROTOCOL_EARLIEST) ||
- PG_PROTOCOL_MAJOR(proto) > PG_PROTOCOL_MAJOR(PG_PROTOCOL_LATEST))
- ereport(FATAL,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("unsupported frontend protocol %u.%u: server supports %u.0 to %u.%u",
- PG_PROTOCOL_MAJOR(proto), PG_PROTOCOL_MINOR(proto),
- PG_PROTOCOL_MAJOR(PG_PROTOCOL_EARLIEST),
- PG_PROTOCOL_MAJOR(PG_PROTOCOL_LATEST),
- PG_PROTOCOL_MINOR(PG_PROTOCOL_LATEST))));
-
- /*
- * Now fetch parameters out of startup packet and save them into the Port
- * structure.
- */
- oldcontext = MemoryContextSwitchTo(TopMemoryContext);
-
- /* Handle protocol version 3 startup packet */
- {
- int32 offset = sizeof(ProtocolVersion);
- List *unrecognized_protocol_options = NIL;
-
- /*
- * Scan packet body for name/option pairs. We can assume any string
- * beginning within the packet body is null-terminated, thanks to
- * zeroing extra byte above.
- */
- port->guc_options = NIL;
-
- while (offset < len)
- {
- char *nameptr = buf + offset;
- int32 valoffset;
- char *valptr;
-
- if (*nameptr == '\0')
- break; /* found packet terminator */
- valoffset = offset + strlen(nameptr) + 1;
- if (valoffset >= len)
- break; /* missing value, will complain below */
- valptr = buf + valoffset;
-
- if (strcmp(nameptr, "database") == 0)
- port->database_name = pstrdup(valptr);
- else if (strcmp(nameptr, "user") == 0)
- port->user_name = pstrdup(valptr);
- else if (strcmp(nameptr, "options") == 0)
- port->cmdline_options = pstrdup(valptr);
- else if (strcmp(nameptr, "replication") == 0)
- {
- /*
- * Due to backward compatibility concerns the replication
- * parameter is a hybrid beast which allows the value to be
- * either boolean or the string 'database'. The latter
- * connects to a specific database which is e.g. required for
- * logical decoding while.
- */
- if (strcmp(valptr, "database") == 0)
- {
- am_walsender = true;
- am_db_walsender = true;
- }
- else if (!parse_bool(valptr, &am_walsender))
- ereport(FATAL,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("invalid value for parameter \"%s\": \"%s\"",
- "replication",
- valptr),
- errhint("Valid values are: \"false\", 0, \"true\", 1, \"database\".")));
- }
- else if (strncmp(nameptr, "_pq_.", 5) == 0)
- {
- /*
- * Any option beginning with _pq_. is reserved for use as a
- * protocol-level option, but at present no such options are
- * defined.
- */
- unrecognized_protocol_options =
- lappend(unrecognized_protocol_options, pstrdup(nameptr));
- }
- else
- {
- /* Assume it's a generic GUC option */
- port->guc_options = lappend(port->guc_options,
- pstrdup(nameptr));
- port->guc_options = lappend(port->guc_options,
- pstrdup(valptr));
-
- /*
- * Copy application_name to port if we come across it. This
- * is done so we can log the application_name in the
- * connection authorization message. Note that the GUC would
- * be used but we haven't gone through GUC setup yet.
- */
- if (strcmp(nameptr, "application_name") == 0)
- {
- port->application_name = pg_clean_ascii(valptr, 0);
- }
- }
- offset = valoffset + strlen(valptr) + 1;
- }
-
- /*
- * If we didn't find a packet terminator exactly at the end of the
- * given packet length, complain.
- */
- if (offset != len - 1)
- ereport(FATAL,
- (errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("invalid startup packet layout: expected terminator as last byte")));
-
- /*
- * If the client requested a newer protocol version or if the client
- * requested any protocol options we didn't recognize, let them know
- * the newest minor protocol version we do support and the names of
- * any unrecognized options.
- */
- if (PG_PROTOCOL_MINOR(proto) > PG_PROTOCOL_MINOR(PG_PROTOCOL_LATEST) ||
- unrecognized_protocol_options != NIL)
- SendNegotiateProtocolVersion(unrecognized_protocol_options);
- }
-
- /* Check a user name was given. */
- if (port->user_name == NULL || port->user_name[0] == '\0')
- ereport(FATAL,
- (errcode(ERRCODE_INVALID_AUTHORIZATION_SPECIFICATION),
- errmsg("no PostgreSQL user name specified in startup packet")));
-
- /* The database defaults to the user name. */
- if (port->database_name == NULL || port->database_name[0] == '\0')
- port->database_name = pstrdup(port->user_name);
-
- if (am_walsender)
- MyBackendType = B_WAL_SENDER;
- else
- MyBackendType = B_BACKEND;
-
- /*
- * Normal walsender backends, e.g. for streaming replication, are not
- * connected to a particular database. But walsenders used for logical
- * replication need to connect to a specific database. We allow streaming
- * replication commands to be issued even if connected to a database as it
- * can make sense to first make a basebackup and then stream changes
- * starting from that.
- */
- if (am_walsender && !am_db_walsender)
- port->database_name[0] = '\0';
-
- /*
- * Done filling the Port structure
- */
- MemoryContextSwitchTo(oldcontext);
-
- return STATUS_OK;
-}
-
-/*
- * Send a NegotiateProtocolVersion to the client. This lets the client know
- * that they have requested a newer minor protocol version than we are able
- * to speak. We'll speak the highest version we know about; the client can,
- * of course, abandon the connection if that's a problem.
- *
- * We also include in the response a list of protocol options we didn't
- * understand. This allows clients to include optional parameters that might
- * be present either in newer protocol versions or third-party protocol
- * extensions without fear of having to reconnect if those options are not
- * understood, while at the same time making certain that the client is aware
- * of which options were actually accepted.
- */
-static void
-SendNegotiateProtocolVersion(List *unrecognized_protocol_options)
-{
- StringInfoData buf;
- ListCell *lc;
-
- pq_beginmessage(&buf, PqMsg_NegotiateProtocolVersion);
- pq_sendint32(&buf, PG_PROTOCOL_LATEST);
- pq_sendint32(&buf, list_length(unrecognized_protocol_options));
- foreach(lc, unrecognized_protocol_options)
- pq_sendstring(&buf, lfirst(lc));
- pq_endmessage(&buf);
-
- /* no need to flush, some other message will follow */
-}
-
/*
* The client has sent a cancel request packet, not a normal
* start-a-new-connection packet. Perform the necessary processing.
* Nothing is sent back to the client.
*/
-static void
-processCancelRequest(Port *port, void *pkt)
+void
+processCancelRequest(int backendPID, int32 cancelAuthCode)
{
- CancelRequestPacket *canc = (CancelRequestPacket *) pkt;
- int backendPID;
- int32 cancelAuthCode;
Backend *bp;
#ifndef EXEC_BACKEND
@@ -2255,9 +1837,6 @@ processCancelRequest(Port *port, void *pkt)
int i;
#endif
- backendPID = (int) pg_ntoh32(canc->backendPID);
- cancelAuthCode = (int32) pg_ntoh32(canc->cancelAuthCode);
-
/*
* See if we have a matching backend. In the EXEC_BACKEND case, we can no
* longer access the postmaster's own backend list, and must rely on the
@@ -3905,12 +3484,6 @@ TerminateChildren(int signal)
signal_child(PgArchPID, signal);
}
-/* Information passed from postmaster to backend process */
-typedef struct BackendStartupInfo
-{
- CAC_state canAcceptConnections;
-} BackendStartupInfo;
-
/*
* BackendStartup -- start backend process
*
@@ -4035,302 +3608,6 @@ report_fork_failure_to_client(ClientSocket *client_sock, int errnum)
} while (rc < 0 && errno == EINTR);
}
-
-/*
- * BackendInitialize -- initialize an interactive (postmaster-child)
- * backend process, and collect the client's startup packet.
- *
- * returns: nothing. Will not return at all if there's any failure.
- *
- * Note: this code does not depend on having any access to shared memory.
- * Indeed, our approach to SIGTERM/timeout handling *requires* that
- * shared memory not have been touched yet; see comments within.
- * In the EXEC_BACKEND case, we are physically attached to shared memory
- * but have not yet set up most of our local pointers to shmem structures.
- */
-static void
-BackendInitialize(ClientSocket *client_sock, CAC_state cac)
-{
- int status;
- int ret;
- Port *port;
- char remote_host[NI_MAXHOST];
- char remote_port[NI_MAXSERV];
- StringInfoData ps_data;
- MemoryContext oldcontext;
-
- /*
- * Create the Port structure.
- *
- * The Port structure and all data structures attached to it are allocated
- * in TopMemoryContext, so that they survive into PostgresMain execution.
- * We need not worry about leaking this storage on failure, since we
- * aren't in the postmaster process anymore.
- */
- oldcontext = MemoryContextSwitchTo(TopMemoryContext);
- port = InitClientConnection(client_sock);
- MyProcPort = port;
- MemoryContextSwitchTo(oldcontext);
-
- /* Tell fd.c about the long-lived FD associated with the client_sock */
- ReserveExternalFD();
-
- /*
- * PreAuthDelay is a debugging aid for investigating problems in the
- * authentication cycle: it can be set in postgresql.conf to allow time to
- * attach to the newly-forked backend with a debugger. (See also
- * PostAuthDelay, which we allow clients to pass through PGOPTIONS, but it
- * is not honored until after authentication.)
- */
- if (PreAuthDelay > 0)
- pg_usleep(PreAuthDelay * 1000000L);
-
- /* This flag will remain set until InitPostgres finishes authentication */
- ClientAuthInProgress = true; /* limit visibility of log messages */
-
- /* set these to empty in case they are needed before we set them up */
- port->remote_host = "";
- port->remote_port = "";
-
- /*
- * Initialize libpq and enable reporting of ereport errors to the client.
- * Must do this now because authentication uses libpq to send messages.
- */
- pq_init(); /* initialize libpq to talk to client */
- whereToSendOutput = DestRemote; /* now safe to ereport to client */
-
- /*
- * We arrange to do _exit(1) if we receive SIGTERM or timeout while trying
- * to collect the startup packet; while SIGQUIT results in _exit(2).
- * Otherwise the postmaster cannot shutdown the database FAST or IMMED
- * cleanly if a buggy client fails to send the packet promptly.
- *
- * Exiting with _exit(1) is only possible because we have not yet touched
- * shared memory; therefore no outside-the-process state needs to get
- * cleaned up.
- */
- pqsignal(SIGTERM, process_startup_packet_die);
- /* SIGQUIT handler was already set up by InitPostmasterChild */
- InitializeTimeouts(); /* establishes SIGALRM handler */
- sigprocmask(SIG_SETMASK, &StartupBlockSig, NULL);
-
- /*
- * Get the remote host name and port for logging and status display.
- */
- remote_host[0] = '\0';
- remote_port[0] = '\0';
- if ((ret = pg_getnameinfo_all(&port->raddr.addr, port->raddr.salen,
- remote_host, sizeof(remote_host),
- remote_port, sizeof(remote_port),
- (log_hostname ? 0 : NI_NUMERICHOST) | NI_NUMERICSERV)) != 0)
- ereport(WARNING,
- (errmsg_internal("pg_getnameinfo_all() failed: %s",
- gai_strerror(ret))));
-
- /*
- * Save remote_host and remote_port in port structure (after this, they
- * will appear in log_line_prefix data for log messages).
- */
- oldcontext = MemoryContextSwitchTo(TopMemoryContext);
- port->remote_host = pstrdup(remote_host);
- port->remote_port = pstrdup(remote_port);
-
- /* And now we can issue the Log_connections message, if wanted */
- if (Log_connections)
- {
- if (remote_port[0])
- ereport(LOG,
- (errmsg("connection received: host=%s port=%s",
- remote_host,
- remote_port)));
- else
- ereport(LOG,
- (errmsg("connection received: host=%s",
- remote_host)));
- }
-
- /*
- * If we did a reverse lookup to name, we might as well save the results
- * rather than possibly repeating the lookup during authentication.
- *
- * Note that we don't want to specify NI_NAMEREQD above, because then we'd
- * get nothing useful for a client without an rDNS entry. Therefore, we
- * must check whether we got a numeric IPv4 or IPv6 address, and not save
- * it into remote_hostname if so. (This test is conservative and might
- * sometimes classify a hostname as numeric, but an error in that
- * direction is safe; it only results in a possible extra lookup.)
- */
- if (log_hostname &&
- ret == 0 &&
- strspn(remote_host, "0123456789.") < strlen(remote_host) &&
- strspn(remote_host, "0123456789ABCDEFabcdef:") < strlen(remote_host))
- port->remote_hostname = pstrdup(remote_host);
- MemoryContextSwitchTo(oldcontext);
-
- /*
- * Ready to begin client interaction. We will give up and _exit(1) after
- * a time delay, so that a broken client can't hog a connection
- * indefinitely. PreAuthDelay and any DNS interactions above don't count
- * against the time limit.
- *
- * Note: AuthenticationTimeout is applied here while waiting for the
- * startup packet, and then again in InitPostgres for the duration of any
- * authentication operations. So a hostile client could tie up the
- * process for nearly twice AuthenticationTimeout before we kick him off.
- *
- * Note: because PostgresMain will call InitializeTimeouts again, the
- * registration of STARTUP_PACKET_TIMEOUT will be lost. This is okay
- * since we never use it again after this function.
- */
- RegisterTimeout(STARTUP_PACKET_TIMEOUT, StartupPacketTimeoutHandler);
- enable_timeout_after(STARTUP_PACKET_TIMEOUT, AuthenticationTimeout * 1000);
-
- /*
- * Receive the startup packet (which might turn out to be a cancel request
- * packet).
- */
- status = ProcessStartupPacket(port, false, false);
-
- /*
- * If we're going to reject the connection due to database state, say so
- * now instead of wasting cycles on an authentication exchange. (This also
- * allows a pg_ping utility to be written.)
- */
- switch (cac)
- {
- case CAC_STARTUP:
- ereport(FATAL,
- (errcode(ERRCODE_CANNOT_CONNECT_NOW),
- errmsg("the database system is starting up")));
- break;
- case CAC_NOTCONSISTENT:
- if (EnableHotStandby)
- ereport(FATAL,
- (errcode(ERRCODE_CANNOT_CONNECT_NOW),
- errmsg("the database system is not yet accepting connections"),
- errdetail("Consistent recovery state has not been yet reached.")));
- else
- ereport(FATAL,
- (errcode(ERRCODE_CANNOT_CONNECT_NOW),
- errmsg("the database system is not accepting connections"),
- errdetail("Hot standby mode is disabled.")));
- break;
- case CAC_SHUTDOWN:
- ereport(FATAL,
- (errcode(ERRCODE_CANNOT_CONNECT_NOW),
- errmsg("the database system is shutting down")));
- break;
- case CAC_RECOVERY:
- ereport(FATAL,
- (errcode(ERRCODE_CANNOT_CONNECT_NOW),
- errmsg("the database system is in recovery mode")));
- break;
- case CAC_TOOMANY:
- ereport(FATAL,
- (errcode(ERRCODE_TOO_MANY_CONNECTIONS),
- errmsg("sorry, too many clients already")));
- break;
- case CAC_OK:
- break;
- }
-
- /*
- * Disable the timeout, and prevent SIGTERM again.
- */
- disable_timeout(STARTUP_PACKET_TIMEOUT, false);
- sigprocmask(SIG_SETMASK, &BlockSig, NULL);
-
- /*
- * As a safety check that nothing in startup has yet performed
- * shared-memory modifications that would need to be undone if we had
- * exited through SIGTERM or timeout above, check that no on_shmem_exit
- * handlers have been registered yet. (This isn't terribly bulletproof,
- * since someone might misuse an on_proc_exit handler for shmem cleanup,
- * but it's a cheap and helpful check. We cannot disallow on_proc_exit
- * handlers unfortunately, since pq_init() already registered one.)
- */
- check_on_shmem_exit_lists_are_empty();
-
- /*
- * Stop here if it was bad or a cancel packet. ProcessStartupPacket
- * already did any appropriate error reporting.
- */
- if (status != STATUS_OK)
- proc_exit(0);
-
- /*
- * Now that we have the user and database name, we can set the process
- * title for ps. It's good to do this as early as possible in startup.
- */
- initStringInfo(&ps_data);
- if (am_walsender)
- appendStringInfo(&ps_data, "%s ", GetBackendTypeDesc(B_WAL_SENDER));
- appendStringInfo(&ps_data, "%s ", port->user_name);
- if (port->database_name[0] != '\0')
- appendStringInfo(&ps_data, "%s ", port->database_name);
- appendStringInfoString(&ps_data, port->remote_host);
- if (port->remote_port[0] != '\0')
- appendStringInfo(&ps_data, "(%s)", port->remote_port);
-
- init_ps_display(ps_data.data);
- pfree(ps_data.data);
-
- set_ps_display("initializing");
-}
-
-void
-BackendMain(char *startup_data, size_t startup_data_len)
-{
- BackendStartupInfo *info = (BackendStartupInfo *) startup_data;
-
- Assert(startup_data_len == sizeof(BackendStartupInfo));
- Assert(MyClientSocket != NULL);
-
-#ifdef EXEC_BACKEND
-
- /*
- * Need to reinitialize the SSL library in the backend, since the context
- * structures contain function pointers and cannot be passed through the
- * parameter file.
- *
- * If for some reason reload fails (maybe the user installed broken key
- * files), soldier on without SSL; that's better than all connections
- * becoming impossible.
- *
- * XXX should we do this in all child processes? For the moment it's
- * enough to do it in backend children.
- */
-#ifdef USE_SSL
- if (EnableSSL)
- {
- if (secure_initialize(false) == 0)
- LoadedSSL = true;
- else
- ereport(LOG,
- (errmsg("SSL configuration could not be loaded in child process")));
- }
-#endif
-#endif
-
- /* Perform additional initialization and collect startup packet */
- BackendInitialize(MyClientSocket, info->canAcceptConnections);
-
- /*
- * Create a per-backend PGPROC struct in shared memory. We must do this
- * before we can use LWLocks or access any shared memory.
- */
- InitProcess();
-
- /*
- * Make sure we aren't in PostmasterContext anymore. (We can't delete it
- * just yet, though, because InitPostgres will need the HBA data.)
- */
- MemoryContextSwitchTo(TopMemoryContext);
-
- PostgresMain(MyProcPort->database_name, MyProcPort->user_name);
-}
-
-
/*
* ExitPostmaster -- cleanup
*
@@ -4519,25 +3796,6 @@ process_pm_pmsignal(void)
}
}
-/*
- * SIGTERM while processing startup packet.
- *
- * Running proc_exit() from a signal handler would be quite unsafe.
- * However, since we have not yet touched shared memory, we can just
- * pull the plug and exit without running any atexit handlers.
- *
- * One might be tempted to try to send a message, or log one, indicating
- * why we are disconnecting. However, that would be quite unsafe in itself.
- * Also, it seems undesirable to provide clues about the database's state
- * to a client that has not yet completed authentication, or even sent us
- * a startup packet.
- */
-static void
-process_startup_packet_die(SIGNAL_ARGS)
-{
- _exit(1);
-}
-
/*
* Dummy signal handler
*
@@ -4552,17 +3810,6 @@ dummy_handler(SIGNAL_ARGS)
{
}
-/*
- * Timeout while processing startup packet.
- * As for process_startup_packet_die(), we exit via _exit(1).
- */
-static void
-StartupPacketTimeoutHandler(void)
-{
- _exit(1);
-}
-
-
/*
* Generate a random cancel key.
*/
diff --git a/src/backend/tcop/Makefile b/src/backend/tcop/Makefile
index f662a7dd1cf..9119667345a 100644
--- a/src/backend/tcop/Makefile
+++ b/src/backend/tcop/Makefile
@@ -13,6 +13,7 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
OBJS = \
+ backend_startup.o \
cmdtag.o \
dest.o \
fastpath.o \
diff --git a/src/backend/tcop/backend_startup.c b/src/backend/tcop/backend_startup.c
new file mode 100644
index 00000000000..c588c24c294
--- /dev/null
+++ b/src/backend/tcop/backend_startup.c
@@ -0,0 +1,777 @@
+/*-------------------------------------------------------------------------
+ *
+ * backend_startup.c
+ * Backend startup code
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/tcop/backend_startup.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include <unistd.h>
+
+#include "access/xlog.h"
+#include "common/ip.h"
+#include "common/string.h"
+#include "libpq/libpq.h"
+#include "libpq/libpq-be.h"
+#include "libpq/pqformat.h"
+#include "libpq/pqsignal.h"
+#include "miscadmin.h"
+#include "postmaster/postmaster.h"
+#include "replication/walsender.h"
+#include "storage/fd.h"
+#include "storage/ipc.h"
+#include "storage/proc.h"
+#include "tcop/backend_startup.h"
+#include "tcop/tcopprot.h"
+#include "utils/builtins.h"
+#include "utils/memutils.h"
+#include "utils/ps_status.h"
+#include "utils/timeout.h"
+
+static void BackendInitialize(ClientSocket *client_sock, CAC_state cac);
+static int ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done);
+static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
+static void process_startup_packet_die(SIGNAL_ARGS);
+static void StartupPacketTimeoutHandler(void);
+
+/*
+ * Entry point for a new backend process.
+ *
+ * Initialize the connection, read the startup packet.
+ */
+void
+BackendMain(char *startup_data, size_t startup_data_len)
+{
+ BackendStartupInfo *info = (BackendStartupInfo *) startup_data;
+
+ Assert(startup_data_len == sizeof(BackendStartupInfo));
+ Assert(MyClientSocket != NULL);
+
+#ifdef EXEC_BACKEND
+
+ /*
+ * Need to reinitialize the SSL library in the backend, since the context
+ * structures contain function pointers and cannot be passed through the
+ * parameter file.
+ *
+ * If for some reason reload fails (maybe the user installed broken key
+ * files), soldier on without SSL; that's better than all connections
+ * becoming impossible.
+ *
+ * XXX should we do this in all child processes? For the moment it's
+ * enough to do it in backend children. XXX good question indeed
+ */
+#ifdef USE_SSL
+ if (EnableSSL)
+ {
+ if (secure_initialize(false) == 0)
+ LoadedSSL = true;
+ else
+ ereport(LOG,
+ (errmsg("SSL configuration could not be loaded in child process")));
+ }
+#endif
+#endif
+
+ /* Perform additional initialization and collect startup packet */
+ BackendInitialize(MyClientSocket, info->canAcceptConnections);
+
+ /*
+ * Create a per-backend PGPROC struct in shared memory. We must do this
+ * before we can use LWLocks or access any shared memory.
+ */
+ InitProcess();
+
+ /*
+ * Make sure we aren't in PostmasterContext anymore. (We can't delete it
+ * just yet, though, because InitPostgres will need the HBA data.)
+ */
+ MemoryContextSwitchTo(TopMemoryContext);
+
+ PostgresMain(MyProcPort->database_name, MyProcPort->user_name);
+}
+
+
+/*
+ * BackendInitialize -- initialize an interactive (postmaster-child)
+ * backend process, and collect the client's startup packet.
+ *
+ * returns: nothing. Will not return at all if there's any failure.
+ *
+ * Note: this code does not depend on having any access to shared memory.
+ * Indeed, our approach to SIGTERM/timeout handling *requires* that
+ * shared memory not have been touched yet; see comments within.
+ * In the EXEC_BACKEND case, we are physically attached to shared memory
+ * but have not yet set up most of our local pointers to shmem structures.
+ */
+static void
+BackendInitialize(ClientSocket *client_sock, CAC_state cac)
+{
+ int status;
+ int ret;
+ Port *port;
+ char remote_host[NI_MAXHOST];
+ char remote_port[NI_MAXSERV];
+ StringInfoData ps_data;
+ MemoryContext oldcontext;
+
+ /*
+ * Create the Port structure.
+ *
+ * The Port structure and all data structures attached to it are allocated
+ * in TopMemoryContext, so that they survive into PostgresMain execution.
+ * We need not worry about leaking this storage on failure, since we
+ * aren't in the postmaster process anymore.
+ */
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
+ port = InitClientConnection(client_sock);
+ MyProcPort = port;
+ MemoryContextSwitchTo(oldcontext);
+
+ /* Tell fd.c about the long-lived FD associated with the client_sock */
+ ReserveExternalFD();
+
+ /*
+ * PreAuthDelay is a debugging aid for investigating problems in the
+ * authentication cycle: it can be set in postgresql.conf to allow time to
+ * attach to the newly-forked backend with a debugger. (See also
+ * PostAuthDelay, which we allow clients to pass through PGOPTIONS, but it
+ * is not honored until after authentication.)
+ */
+ if (PreAuthDelay > 0)
+ pg_usleep(PreAuthDelay * 1000000L);
+
+ /* This flag will remain set until InitPostgres finishes authentication */
+ ClientAuthInProgress = true; /* limit visibility of log messages */
+
+ /* set these to empty in case they are needed before we set them up */
+ port->remote_host = "";
+ port->remote_port = "";
+
+ /*
+ * Initialize libpq and enable reporting of ereport errors to the client.
+ * Must do this now because authentication uses libpq to send messages.
+ */
+ pq_init(); /* initialize libpq to talk to client */
+ whereToSendOutput = DestRemote; /* now safe to ereport to client */
+
+ /*
+ * We arrange to do _exit(1) if we receive SIGTERM or timeout while trying
+ * to collect the startup packet; while SIGQUIT results in _exit(2).
+ * Otherwise the postmaster cannot shutdown the database FAST or IMMED
+ * cleanly if a buggy client fails to send the packet promptly.
+ *
+ * Exiting with _exit(1) is only possible because we have not yet touched
+ * shared memory; therefore no outside-the-process state needs to get
+ * cleaned up.
+ */
+ pqsignal(SIGTERM, process_startup_packet_die);
+ /* SIGQUIT handler was already set up by InitPostmasterChild */
+ InitializeTimeouts(); /* establishes SIGALRM handler */
+ sigprocmask(SIG_SETMASK, &StartupBlockSig, NULL);
+
+ /*
+ * Get the remote host name and port for logging and status display.
+ */
+ remote_host[0] = '\0';
+ remote_port[0] = '\0';
+ if ((ret = pg_getnameinfo_all(&port->raddr.addr, port->raddr.salen,
+ remote_host, sizeof(remote_host),
+ remote_port, sizeof(remote_port),
+ (log_hostname ? 0 : NI_NUMERICHOST) | NI_NUMERICSERV)) != 0)
+ ereport(WARNING,
+ (errmsg_internal("pg_getnameinfo_all() failed: %s",
+ gai_strerror(ret))));
+
+ /*
+ * Save remote_host and remote_port in port structure (after this, they
+ * will appear in log_line_prefix data for log messages).
+ */
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
+ port->remote_host = pstrdup(remote_host);
+ port->remote_port = pstrdup(remote_port);
+
+ /* And now we can issue the Log_connections message, if wanted */
+ if (Log_connections)
+ {
+ if (remote_port[0])
+ ereport(LOG,
+ (errmsg("connection received: host=%s port=%s",
+ remote_host,
+ remote_port)));
+ else
+ ereport(LOG,
+ (errmsg("connection received: host=%s",
+ remote_host)));
+ }
+
+ /*
+ * If we did a reverse lookup to name, we might as well save the results
+ * rather than possibly repeating the lookup during authentication.
+ *
+ * Note that we don't want to specify NI_NAMEREQD above, because then we'd
+ * get nothing useful for a client without an rDNS entry. Therefore, we
+ * must check whether we got a numeric IPv4 or IPv6 address, and not save
+ * it into remote_hostname if so. (This test is conservative and might
+ * sometimes classify a hostname as numeric, but an error in that
+ * direction is safe; it only results in a possible extra lookup.)
+ */
+ if (log_hostname &&
+ ret == 0 &&
+ strspn(remote_host, "0123456789.") < strlen(remote_host) &&
+ strspn(remote_host, "0123456789ABCDEFabcdef:") < strlen(remote_host))
+ port->remote_hostname = pstrdup(remote_host);
+ MemoryContextSwitchTo(oldcontext);
+
+ /*
+ * Ready to begin client interaction. We will give up and _exit(1) after
+ * a time delay, so that a broken client can't hog a connection
+ * indefinitely. PreAuthDelay and any DNS interactions above don't count
+ * against the time limit.
+ *
+ * Note: AuthenticationTimeout is applied here while waiting for the
+ * startup packet, and then again in InitPostgres for the duration of any
+ * authentication operations. So a hostile client could tie up the
+ * process for nearly twice AuthenticationTimeout before we kick him off.
+ *
+ * Note: because PostgresMain will call InitializeTimeouts again, the
+ * registration of STARTUP_PACKET_TIMEOUT will be lost. This is okay
+ * since we never use it again after this function.
+ */
+ RegisterTimeout(STARTUP_PACKET_TIMEOUT, StartupPacketTimeoutHandler);
+ enable_timeout_after(STARTUP_PACKET_TIMEOUT, AuthenticationTimeout * 1000);
+
+ /*
+ * Receive the startup packet (which might turn out to be a cancel request
+ * packet).
+ */
+ status = ProcessStartupPacket(port, false, false);
+
+ /*
+ * If we're going to reject the connection due to database state, say so
+ * now instead of wasting cycles on an authentication exchange. (This also
+ * allows a pg_ping utility to be written.)
+ */
+ switch (cac)
+ {
+ case CAC_STARTUP:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is starting up")));
+ break;
+ case CAC_NOTCONSISTENT:
+ if (EnableHotStandby)
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is not yet accepting connections"),
+ errdetail("Consistent recovery state has not been yet reached.")));
+ else
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is not accepting connections"),
+ errdetail("Hot standby mode is disabled.")));
+ break;
+ case CAC_SHUTDOWN:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is shutting down")));
+ break;
+ case CAC_RECOVERY:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is in recovery mode")));
+ break;
+ case CAC_TOOMANY:
+ ereport(FATAL,
+ (errcode(ERRCODE_TOO_MANY_CONNECTIONS),
+ errmsg("sorry, too many clients already")));
+ break;
+ case CAC_OK:
+ break;
+ }
+
+ /*
+ * Disable the timeout, and prevent SIGTERM again.
+ */
+ disable_timeout(STARTUP_PACKET_TIMEOUT, false);
+ sigprocmask(SIG_SETMASK, &BlockSig, NULL);
+
+ /*
+ * As a safety check that nothing in startup has yet performed
+ * shared-memory modifications that would need to be undone if we had
+ * exited through SIGTERM or timeout above, check that no on_shmem_exit
+ * handlers have been registered yet. (This isn't terribly bulletproof,
+ * since someone might misuse an on_proc_exit handler for shmem cleanup,
+ * but it's a cheap and helpful check. We cannot disallow on_proc_exit
+ * handlers unfortunately, since pq_init() already registered one.)
+ */
+ check_on_shmem_exit_lists_are_empty();
+
+ /*
+ * Stop here if it was bad or a cancel packet. ProcessStartupPacket
+ * already did any appropriate error reporting.
+ */
+ if (status != STATUS_OK)
+ proc_exit(0);
+
+ /*
+ * Now that we have the user and database name, we can set the process
+ * title for ps. It's good to do this as early as possible in startup.
+ */
+ initStringInfo(&ps_data);
+ if (am_walsender)
+ appendStringInfo(&ps_data, "%s ", GetBackendTypeDesc(B_WAL_SENDER));
+ appendStringInfo(&ps_data, "%s ", port->user_name);
+ if (port->database_name[0] != '\0')
+ appendStringInfo(&ps_data, "%s ", port->database_name);
+ appendStringInfoString(&ps_data, port->remote_host);
+ if (port->remote_port[0] != '\0')
+ appendStringInfo(&ps_data, "(%s)", port->remote_port);
+
+ init_ps_display(ps_data.data);
+ pfree(ps_data.data);
+
+ set_ps_display("initializing");
+}
+
+/*
+ * Read a client's startup packet and do something according to it.
+ *
+ * Returns STATUS_OK or STATUS_ERROR, or might call ereport(FATAL) and
+ * not return at all.
+ *
+ * (Note that ereport(FATAL) stuff is sent to the client, so only use it
+ * if that's what you want. Return STATUS_ERROR if you don't want to
+ * send anything to the client, which would typically be appropriate
+ * if we detect a communications failure.)
+ *
+ * Set ssl_done and/or gss_done when negotiation of an encrypted layer
+ * (currently, TLS or GSSAPI) is completed. A successful negotiation of either
+ * encryption layer sets both flags, but a rejected negotiation sets only the
+ * flag for that layer, since the client may wish to try the other one. We
+ * should make no assumption here about the order in which the client may make
+ * requests.
+ */
+static int
+ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
+{
+ int32 len;
+ char *buf;
+ ProtocolVersion proto;
+ MemoryContext oldcontext;
+
+ pq_startmsgread();
+
+ /*
+ * Grab the first byte of the length word separately, so that we can tell
+ * whether we have no data at all or an incomplete packet. (This might
+ * sound inefficient, but it's not really, because of buffering in
+ * pqcomm.c.)
+ */
+ if (pq_getbytes((char *) &len, 1) == EOF)
+ {
+ /*
+ * If we get no data at all, don't clutter the log with a complaint;
+ * such cases often occur for legitimate reasons. An example is that
+ * we might be here after responding to NEGOTIATE_SSL_CODE, and if the
+ * client didn't like our response, it'll probably just drop the
+ * connection. Service-monitoring software also often just opens and
+ * closes a connection without sending anything. (So do port
+ * scanners, which may be less benign, but it's not really our job to
+ * notice those.)
+ */
+ return STATUS_ERROR;
+ }
+
+ if (pq_getbytes(((char *) &len) + 1, 3) == EOF)
+ {
+ /* Got a partial length word, so bleat about that */
+ if (!ssl_done && !gss_done)
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("incomplete startup packet")));
+ return STATUS_ERROR;
+ }
+
+ len = pg_ntoh32(len);
+ len -= 4;
+
+ if (len < (int32) sizeof(ProtocolVersion) ||
+ len > MAX_STARTUP_PACKET_LENGTH)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("invalid length of startup packet")));
+ return STATUS_ERROR;
+ }
+
+ /*
+ * Allocate space to hold the startup packet, plus one extra byte that's
+ * initialized to be zero. This ensures we will have null termination of
+ * all strings inside the packet.
+ */
+ buf = palloc(len + 1);
+ buf[len] = '\0';
+
+ if (pq_getbytes(buf, len) == EOF)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("incomplete startup packet")));
+ return STATUS_ERROR;
+ }
+ pq_endmsgread();
+
+ /*
+ * The first field is either a protocol version number or a special
+ * request code.
+ */
+ port->proto = proto = pg_ntoh32(*((ProtocolVersion *) buf));
+
+ if (proto == CANCEL_REQUEST_CODE)
+ {
+ CancelRequestPacket *canc;
+ int backendPID;
+ int32 cancelAuthCode;
+
+ if (len != sizeof(CancelRequestPacket))
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("invalid length of startup packet")));
+ return STATUS_ERROR;
+ }
+ canc = (CancelRequestPacket *) buf;
+ backendPID = (int) pg_ntoh32(canc->backendPID);
+ cancelAuthCode = (int32) pg_ntoh32(canc->cancelAuthCode);
+
+ processCancelRequest(backendPID, cancelAuthCode);
+ /* Not really an error, but we don't want to proceed further */
+ return STATUS_ERROR;
+ }
+
+ if (proto == NEGOTIATE_SSL_CODE && !ssl_done)
+ {
+ char SSLok;
+
+#ifdef USE_SSL
+ /* No SSL when disabled or on Unix sockets */
+ if (!LoadedSSL || port->laddr.addr.ss_family == AF_UNIX)
+ SSLok = 'N';
+ else
+ SSLok = 'S'; /* Support for SSL */
+#else
+ SSLok = 'N'; /* No support for SSL */
+#endif
+
+retry1:
+ if (send(port->sock, &SSLok, 1, 0) != 1)
+ {
+ if (errno == EINTR)
+ goto retry1; /* if interrupted, just retry */
+ ereport(COMMERROR,
+ (errcode_for_socket_access(),
+ errmsg("failed to send SSL negotiation response: %m")));
+ return STATUS_ERROR; /* close the connection */
+ }
+
+#ifdef USE_SSL
+ if (SSLok == 'S' && secure_open_server(port) == -1)
+ return STATUS_ERROR;
+#endif
+
+ /*
+ * At this point we should have no data already buffered. If we do,
+ * it was received before we performed the SSL handshake, so it wasn't
+ * encrypted and indeed may have been injected by a man-in-the-middle.
+ * We report this case to the client.
+ */
+ if (pq_buffer_has_data())
+ ereport(FATAL,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("received unencrypted data after SSL request"),
+ errdetail("This could be either a client-software bug or evidence of an attempted man-in-the-middle attack.")));
+
+ /*
+ * regular startup packet, cancel, etc packet should follow, but not
+ * another SSL negotiation request, and a GSS request should only
+ * follow if SSL was rejected (client may negotiate in either order)
+ */
+ return ProcessStartupPacket(port, true, SSLok == 'S');
+ }
+ else if (proto == NEGOTIATE_GSS_CODE && !gss_done)
+ {
+ char GSSok = 'N';
+
+#ifdef ENABLE_GSS
+ /* No GSSAPI encryption when on Unix socket */
+ if (port->laddr.addr.ss_family != AF_UNIX)
+ GSSok = 'G';
+#endif
+
+ while (send(port->sock, &GSSok, 1, 0) != 1)
+ {
+ if (errno == EINTR)
+ continue;
+ ereport(COMMERROR,
+ (errcode_for_socket_access(),
+ errmsg("failed to send GSSAPI negotiation response: %m")));
+ return STATUS_ERROR; /* close the connection */
+ }
+
+#ifdef ENABLE_GSS
+ if (GSSok == 'G' && secure_open_gssapi(port) == -1)
+ return STATUS_ERROR;
+#endif
+
+ /*
+ * At this point we should have no data already buffered. If we do,
+ * it was received before we performed the GSS handshake, so it wasn't
+ * encrypted and indeed may have been injected by a man-in-the-middle.
+ * We report this case to the client.
+ */
+ if (pq_buffer_has_data())
+ ereport(FATAL,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("received unencrypted data after GSSAPI encryption request"),
+ errdetail("This could be either a client-software bug or evidence of an attempted man-in-the-middle attack.")));
+
+ /*
+ * regular startup packet, cancel, etc packet should follow, but not
+ * another GSS negotiation request, and an SSL request should only
+ * follow if GSS was rejected (client may negotiate in either order)
+ */
+ return ProcessStartupPacket(port, GSSok == 'G', true);
+ }
+
+ /* Could add additional special packet types here */
+
+ /*
+ * Set FrontendProtocol now so that ereport() knows what format to send if
+ * we fail during startup.
+ */
+ FrontendProtocol = proto;
+
+ /* Check that the major protocol version is in range. */
+ if (PG_PROTOCOL_MAJOR(proto) < PG_PROTOCOL_MAJOR(PG_PROTOCOL_EARLIEST) ||
+ PG_PROTOCOL_MAJOR(proto) > PG_PROTOCOL_MAJOR(PG_PROTOCOL_LATEST))
+ ereport(FATAL,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("unsupported frontend protocol %u.%u: server supports %u.0 to %u.%u",
+ PG_PROTOCOL_MAJOR(proto), PG_PROTOCOL_MINOR(proto),
+ PG_PROTOCOL_MAJOR(PG_PROTOCOL_EARLIEST),
+ PG_PROTOCOL_MAJOR(PG_PROTOCOL_LATEST),
+ PG_PROTOCOL_MINOR(PG_PROTOCOL_LATEST))));
+
+ /*
+ * Now fetch parameters out of startup packet and save them into the Port
+ * structure.
+ */
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
+
+ /* Handle protocol version 3 startup packet */
+ {
+ int32 offset = sizeof(ProtocolVersion);
+ List *unrecognized_protocol_options = NIL;
+
+ /*
+ * Scan packet body for name/option pairs. We can assume any string
+ * beginning within the packet body is null-terminated, thanks to
+ * zeroing extra byte above.
+ */
+ port->guc_options = NIL;
+
+ while (offset < len)
+ {
+ char *nameptr = buf + offset;
+ int32 valoffset;
+ char *valptr;
+
+ if (*nameptr == '\0')
+ break; /* found packet terminator */
+ valoffset = offset + strlen(nameptr) + 1;
+ if (valoffset >= len)
+ break; /* missing value, will complain below */
+ valptr = buf + valoffset;
+
+ if (strcmp(nameptr, "database") == 0)
+ port->database_name = pstrdup(valptr);
+ else if (strcmp(nameptr, "user") == 0)
+ port->user_name = pstrdup(valptr);
+ else if (strcmp(nameptr, "options") == 0)
+ port->cmdline_options = pstrdup(valptr);
+ else if (strcmp(nameptr, "replication") == 0)
+ {
+ /*
+ * Due to backward compatibility concerns the replication
+ * parameter is a hybrid beast which allows the value to be
+ * either boolean or the string 'database'. The latter
+ * connects to a specific database which is e.g. required for
+ * logical decoding while.
+ */
+ if (strcmp(valptr, "database") == 0)
+ {
+ am_walsender = true;
+ am_db_walsender = true;
+ }
+ else if (!parse_bool(valptr, &am_walsender))
+ ereport(FATAL,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid value for parameter \"%s\": \"%s\"",
+ "replication",
+ valptr),
+ errhint("Valid values are: \"false\", 0, \"true\", 1, \"database\".")));
+ }
+ else if (strncmp(nameptr, "_pq_.", 5) == 0)
+ {
+ /*
+ * Any option beginning with _pq_. is reserved for use as a
+ * protocol-level option, but at present no such options are
+ * defined.
+ */
+ unrecognized_protocol_options =
+ lappend(unrecognized_protocol_options, pstrdup(nameptr));
+ }
+ else
+ {
+ /* Assume it's a generic GUC option */
+ port->guc_options = lappend(port->guc_options,
+ pstrdup(nameptr));
+ port->guc_options = lappend(port->guc_options,
+ pstrdup(valptr));
+
+ /*
+ * Copy application_name to port if we come across it. This
+ * is done so we can log the application_name in the
+ * connection authorization message. Note that the GUC would
+ * be used but we haven't gone through GUC setup yet.
+ */
+ if (strcmp(nameptr, "application_name") == 0)
+ {
+ port->application_name = pg_clean_ascii(valptr, 0);
+ }
+ }
+ offset = valoffset + strlen(valptr) + 1;
+ }
+
+ /*
+ * If we didn't find a packet terminator exactly at the end of the
+ * given packet length, complain.
+ */
+ if (offset != len - 1)
+ ereport(FATAL,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("invalid startup packet layout: expected terminator as last byte")));
+
+ /*
+ * If the client requested a newer protocol version or if the client
+ * requested any protocol options we didn't recognize, let them know
+ * the newest minor protocol version we do support and the names of
+ * any unrecognized options.
+ */
+ if (PG_PROTOCOL_MINOR(proto) > PG_PROTOCOL_MINOR(PG_PROTOCOL_LATEST) ||
+ unrecognized_protocol_options != NIL)
+ SendNegotiateProtocolVersion(unrecognized_protocol_options);
+ }
+
+ /* Check a user name was given. */
+ if (port->user_name == NULL || port->user_name[0] == '\0')
+ ereport(FATAL,
+ (errcode(ERRCODE_INVALID_AUTHORIZATION_SPECIFICATION),
+ errmsg("no PostgreSQL user name specified in startup packet")));
+
+ /* The database defaults to the user name. */
+ if (port->database_name == NULL || port->database_name[0] == '\0')
+ port->database_name = pstrdup(port->user_name);
+
+ if (am_walsender)
+ MyBackendType = B_WAL_SENDER;
+ else
+ MyBackendType = B_BACKEND;
+
+ /*
+ * Normal walsender backends, e.g. for streaming replication, are not
+ * connected to a particular database. But walsenders used for logical
+ * replication need to connect to a specific database. We allow streaming
+ * replication commands to be issued even if connected to a database as it
+ * can make sense to first make a basebackup and then stream changes
+ * starting from that.
+ */
+ if (am_walsender && !am_db_walsender)
+ port->database_name[0] = '\0';
+
+ /*
+ * Done filling the Port structure
+ */
+ MemoryContextSwitchTo(oldcontext);
+
+ return STATUS_OK;
+}
+
+/*
+ * Send a NegotiateProtocolVersion to the client. This lets the client know
+ * that they have requested a newer minor protocol version than we are able
+ * to speak. We'll speak the highest version we know about; the client can,
+ * of course, abandon the connection if that's a problem.
+ *
+ * We also include in the response a list of protocol options we didn't
+ * understand. This allows clients to include optional parameters that might
+ * be present either in newer protocol versions or third-party protocol
+ * extensions without fear of having to reconnect if those options are not
+ * understood, while at the same time making certain that the client is aware
+ * of which options were actually accepted.
+ */
+static void
+SendNegotiateProtocolVersion(List *unrecognized_protocol_options)
+{
+ StringInfoData buf;
+ ListCell *lc;
+
+ pq_beginmessage(&buf, PqMsg_NegotiateProtocolVersion);
+ pq_sendint32(&buf, PG_PROTOCOL_LATEST);
+ pq_sendint32(&buf, list_length(unrecognized_protocol_options));
+ foreach(lc, unrecognized_protocol_options)
+ pq_sendstring(&buf, lfirst(lc));
+ pq_endmessage(&buf);
+
+ /* no need to flush, some other message will follow */
+}
+
+
+/*
+ * SIGTERM while processing startup packet.
+ *
+ * Running proc_exit() from a signal handler would be quite unsafe.
+ * However, since we have not yet touched shared memory, we can just
+ * pull the plug and exit without running any atexit handlers.
+ *
+ * One might be tempted to try to send a message, or log one, indicating
+ * why we are disconnecting. However, that would be quite unsafe in itself.
+ * Also, it seems undesirable to provide clues about the database's state
+ * to a client that has not yet completed authentication, or even sent us
+ * a startup packet.
+ */
+static void
+process_startup_packet_die(SIGNAL_ARGS)
+{
+ _exit(1);
+}
+
+/*
+ * Timeout while processing startup packet.
+ * As for process_startup_packet_die(), we exit via _exit(1).
+ */
+static void
+StartupPacketTimeoutHandler(void)
+{
+ _exit(1);
+}
diff --git a/src/backend/tcop/meson.build b/src/backend/tcop/meson.build
index 55c49809fe8..5ce40033383 100644
--- a/src/backend/tcop/meson.build
+++ b/src/backend/tcop/meson.build
@@ -1,6 +1,7 @@
# Copyright (c) 2022-2023, PostgreSQL Global Development Group
backend_sources += files(
+ 'backend_startup.c',
'cmdtag.c',
'dest.c',
'fastpath.c',
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index 031a2ff1521..98670202070 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -50,6 +50,9 @@ extern PGDLLIMPORT int postmaster_alive_fds[2];
extern PGDLLIMPORT const char *progname;
+/* XXX: where does this belong? */
+extern bool LoadedSSL;
+
extern void PostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
extern void ClosePostmasterPorts(bool am_syslogger);
extern void InitProcessGlobals(void);
@@ -59,6 +62,7 @@ extern int MaxLivePostmasterChildren(void);
extern bool PostmasterMarkPIDForWorkerNotify(int);
extern void BackendMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
+extern void processCancelRequest(int backendPID, int32 cancelAuthCode);
#ifdef EXEC_BACKEND
extern Size ShmemBackendArraySize(void);
diff --git a/src/include/tcop/backend_startup.h b/src/include/tcop/backend_startup.h
new file mode 100644
index 00000000000..643aa7150b7
--- /dev/null
+++ b/src/include/tcop/backend_startup.h
@@ -0,0 +1,41 @@
+/*-------------------------------------------------------------------------
+ *
+ * backend_startup.h
+ * prototypes for backend_startup.c.
+ *
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/tcop/backend_startup.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef BACKEND_STARTUP_H
+#define BACKEND_STARTUP_H
+
+/*
+ * CAC_state is passed from postmaster to the backend process, to indicate
+ * whether the connection should be accepted, or if the process should just
+ * send an error to the client and close the cnnection. Note that the
+ * connection can fail for various reasons even if postmaster passed CAC_OK.
+ */
+typedef enum CAC_state
+{
+ CAC_OK,
+ CAC_STARTUP,
+ CAC_SHUTDOWN,
+ CAC_RECOVERY,
+ CAC_NOTCONSISTENT,
+ CAC_TOOMANY,
+} CAC_state;
+
+/* Information passed from postmaster to backend process in 'startup_data' */
+typedef struct BackendStartupInfo
+{
+ CAC_state canAcceptConnections;
+} BackendStartupInfo;
+
+extern void BackendMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
+
+#endif /* BACKEND_STARTUP_H */
--
2.39.2
Re: Refactoring backend fork+exec code
On 08/12/2023 14:33, Heikki Linnakangas wrote:
+ [PMC_AV_LAUNCHER] = {"autovacuum launcher", AutoVacLauncherMain, true}, + [PMC_AV_WORKER] = {"autovacuum worker", AutoVacWorkerMain, true}, + [PMC_BGWORKER] = {"bgworker", BackgroundWorkerMain, true}, + [PMC_SYSLOGGER] = {"syslogger", SysLoggerMain, false}, + + [PMC_STARTUP] = {"startup", StartupProcessMain, true}, + [PMC_BGWRITER] = {"bgwriter", BackgroundWriterMain, true}, + [PMC_ARCHIVER] = {"archiver", PgArchiverMain, true}, + [PMC_CHECKPOINTER] = {"checkpointer", CheckpointerMain, true}, + [PMC_WAL_WRITER] = {"wal_writer", WalWriterMain, true}, + [PMC_WAL_RECEIVER] = {"wal_receiver", WalReceiverMain, true}, +};It feels like we have too many different ways of documenting the type of a
process. This new PMC_ stuff, enum AuxProcType, enum BackendType.Agreed. And "am_walsender" and such variables.
Here's a patch that gets rid of AuxProcType. It's independent of the
other patches in this thread; if this is committed, I'll rebase the rest
of the patches over this and get rid of the new PMC_* enum.
Three patches, actually. The first one fixes an existing comment that I
noticed to be incorrect while working on this. I'll push that soon,
barring objections. The second one gets rid of AuxProcType, and the
third one replaces IsBackgroundWorker, IsAutoVacuumLauncherProcess() and
IsAutoVacuumWorkerProcess() with checks on MyBackendType. So
MyBackendType is now the primary way to check what kind of a process the
current process is.
'am_walsender' would also be fairly straightforward to remove and
replace with AmWalSenderProcess(). I didn't do that because we also have
am_db_walsender and am_cascading_walsender which cannot be directly
replaced with MyBackendType. Given that, it might be best to keep
am_walsender for symmetry.
--
Heikki Linnakangas
Neon (https://neon.tech)
Attachments:
v6-0001-Fix-incorrect-comment-on-how-BackendStatusArray-i.patchtext/x-patch; charset=UTF-8; name=v6-0001-Fix-incorrect-comment-on-how-BackendStatusArray-i.patchDownload
From a0ecc54763252de74907fd0e4aed30fdb753ff86 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 10 Jan 2024 13:46:23 +0200
Subject: [PATCH v6 1/3] Fix incorrect comment on how BackendStatusArray is
indexed
The comment was copy-pasted from the call to ProcSignalInit() in
AuxiliaryProcessMain(), which uses a similar scheme of having reserved
slots for aux processes after MaxBackends slots for backends. However,
ProcSignalInit() indexing starts from 1, whereas BackendStatusArray
starts from 0. The code is correct, but the comment was wrong.
Reviewed-by: XXX
Discussion: XXX
---
src/backend/utils/activity/backend_status.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 3dac79c30eb..1a1050c8da1 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -263,9 +263,9 @@ pgstat_beinit(void)
* Assign the MyBEEntry for an auxiliary process. Since it doesn't
* have a BackendId, the slot is statically allocated based on the
* auxiliary process type (MyAuxProcType). Backends use slots indexed
- * in the range from 1 to MaxBackends (inclusive), so we use
- * MaxBackends + AuxBackendType + 1 as the index of the slot for an
- * auxiliary process.
+ * in the range from 0 to MaxBackends (exclusive), so we use
+ * MaxBackends + AuxProcType as the index of the slot for an auxiliary
+ * process.
*/
MyBEEntry = &BackendStatusArray[MaxBackends + MyAuxProcType];
}
--
2.39.2
v6-0002-Remove-MyAuxProcType-use-MyBackendType-instead.patchtext/x-patch; charset=UTF-8; name=v6-0002-Remove-MyAuxProcType-use-MyBackendType-instead.patchDownload
From 68b8c662af229f93244b45282f3112d9a5d809b9 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 10 Jan 2024 14:00:09 +0200
Subject: [PATCH v6 2/3] Remove MyAuxProcType, use MyBackendType instead
MyAuxProcType was redundant with MyBackendType.
Reviewed-by: XXX
Discussion: XXX
---
src/backend/postmaster/auxprocess.c | 68 +++++---------------
src/backend/postmaster/postmaster.c | 36 +++++------
src/backend/utils/activity/backend_status.c | 11 ++--
src/include/miscadmin.h | 71 ++++++++++-----------
src/include/postmaster/auxprocess.h | 2 +-
src/tools/pgindent/typedefs.list | 1 -
6 files changed, 74 insertions(+), 115 deletions(-)
diff --git a/src/backend/postmaster/auxprocess.c b/src/backend/postmaster/auxprocess.c
index ab86e802f21..bd3815b63d0 100644
--- a/src/backend/postmaster/auxprocess.c
+++ b/src/backend/postmaster/auxprocess.c
@@ -38,14 +38,6 @@
static void ShutdownAuxiliaryProcess(int code, Datum arg);
-/* ----------------
- * global variables
- * ----------------
- */
-
-AuxProcType MyAuxProcType = NotAnAuxProcess; /* declared in miscadmin.h */
-
-
/*
* AuxiliaryProcessMain
*
@@ -55,39 +47,13 @@ AuxProcType MyAuxProcType = NotAnAuxProcess; /* declared in miscadmin.h */
* This code is here just because of historical reasons.
*/
void
-AuxiliaryProcessMain(AuxProcType auxtype)
+AuxiliaryProcessMain(BackendType auxtype)
{
Assert(IsUnderPostmaster);
- MyAuxProcType = auxtype;
-
- switch (MyAuxProcType)
- {
- case StartupProcess:
- MyBackendType = B_STARTUP;
- break;
- case ArchiverProcess:
- MyBackendType = B_ARCHIVER;
- break;
- case BgWriterProcess:
- MyBackendType = B_BG_WRITER;
- break;
- case CheckpointerProcess:
- MyBackendType = B_CHECKPOINTER;
- break;
- case WalWriterProcess:
- MyBackendType = B_WAL_WRITER;
- break;
- case WalReceiverProcess:
- MyBackendType = B_WAL_RECEIVER;
- break;
- case WalSummarizerProcess:
- MyBackendType = B_WAL_SUMMARIZER;
- break;
- default:
- elog(PANIC, "unrecognized process type: %d", (int) MyAuxProcType);
- MyBackendType = B_INVALID;
- }
+ MyBackendType = auxtype;
+ if (!IsAuxProcess(MyBackendType))
+ elog(PANIC, "unrecognized process type: %d", (int) MyBackendType);
init_ps_display(NULL);
@@ -110,14 +76,14 @@ AuxiliaryProcessMain(AuxProcType auxtype)
/*
* Assign the ProcSignalSlot for an auxiliary process. Since it doesn't
* have a BackendId, the slot is statically allocated based on the
- * auxiliary process type (MyAuxProcType). Backends use slots indexed in
- * the range from 1 to MaxBackends (inclusive), so we use MaxBackends +
- * AuxProcType + 1 as the index of the slot for an auxiliary process.
+ * auxiliary process type. Backends use slots indexed in the range from 1
+ * to MaxBackends (inclusive), and aux processes use the slots after that,
+ * one reserved slot for each different kind of aux process.
*
* This will need rethinking if we ever want more than one of a particular
* auxiliary process type.
*/
- ProcSignalInit(MaxBackends + MyAuxProcType + 1);
+ ProcSignalInit(MaxBackends + MyBackendType - FIRST_AUX_PROC + 1);
/*
* Auxiliary processes don't run transactions, but they may need a
@@ -136,38 +102,38 @@ AuxiliaryProcessMain(AuxProcType auxtype)
SetProcessingMode(NormalProcessing);
- switch (MyAuxProcType)
+ switch (MyBackendType)
{
- case StartupProcess:
+ case B_STARTUP:
StartupProcessMain();
proc_exit(1);
- case ArchiverProcess:
+ case B_ARCHIVER:
PgArchiverMain();
proc_exit(1);
- case BgWriterProcess:
+ case B_BG_WRITER:
BackgroundWriterMain();
proc_exit(1);
- case CheckpointerProcess:
+ case B_CHECKPOINTER:
CheckpointerMain();
proc_exit(1);
- case WalWriterProcess:
+ case B_WAL_WRITER:
WalWriterMain();
proc_exit(1);
- case WalReceiverProcess:
+ case B_WAL_RECEIVER:
WalReceiverMain();
proc_exit(1);
- case WalSummarizerProcess:
+ case B_WAL_SUMMARIZER:
WalSummarizerMain();
proc_exit(1);
default:
- elog(PANIC, "unrecognized process type: %d", (int) MyAuxProcType);
+ elog(PANIC, "unrecognized process type: %d", (int) MyBackendType);
proc_exit(1);
}
}
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index feb471dd1df..c51b6a1376a 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -440,7 +440,7 @@ static int CountChildren(int target);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
-static pid_t StartChildProcess(AuxProcType type);
+static pid_t StartChildProcess(BackendType type);
static void StartAutovacuumWorker(void);
static void MaybeStartWalReceiver(void);
static void MaybeStartWalSummarizer(void);
@@ -561,13 +561,13 @@ static void ShmemBackendArrayAdd(Backend *bn);
static void ShmemBackendArrayRemove(Backend *bn);
#endif /* EXEC_BACKEND */
-#define StartupDataBase() StartChildProcess(StartupProcess)
-#define StartArchiver() StartChildProcess(ArchiverProcess)
-#define StartBackgroundWriter() StartChildProcess(BgWriterProcess)
-#define StartCheckpointer() StartChildProcess(CheckpointerProcess)
-#define StartWalWriter() StartChildProcess(WalWriterProcess)
-#define StartWalReceiver() StartChildProcess(WalReceiverProcess)
-#define StartWalSummarizer() StartChildProcess(WalSummarizerProcess)
+#define StartupDataBase() StartChildProcess(B_STARTUP)
+#define StartArchiver() StartChildProcess(B_ARCHIVER)
+#define StartBackgroundWriter() StartChildProcess(B_BG_WRITER)
+#define StartCheckpointer() StartChildProcess(B_CHECKPOINTER)
+#define StartWalWriter() StartChildProcess(B_WAL_WRITER)
+#define StartWalReceiver() StartChildProcess(B_WAL_RECEIVER)
+#define StartWalSummarizer() StartChildProcess(B_WAL_SUMMARIZER)
/* Macros to check exit status of a child process */
#define EXIT_STATUS_0(st) ((st) == 0)
@@ -4953,7 +4953,7 @@ SubPostmasterMain(int argc, char *argv[])
}
if (strcmp(argv[1], "--forkaux") == 0)
{
- AuxProcType auxtype;
+ BackendType auxtype;
Assert(argc == 4);
@@ -5292,7 +5292,7 @@ CountChildren(int target)
* to start subprocess.
*/
static pid_t
-StartChildProcess(AuxProcType type)
+StartChildProcess(BackendType type)
{
pid_t pid;
@@ -5344,31 +5344,31 @@ StartChildProcess(AuxProcType type)
errno = save_errno;
switch (type)
{
- case StartupProcess:
+ case B_STARTUP:
ereport(LOG,
(errmsg("could not fork startup process: %m")));
break;
- case ArchiverProcess:
+ case B_ARCHIVER:
ereport(LOG,
(errmsg("could not fork archiver process: %m")));
break;
- case BgWriterProcess:
+ case B_BG_WRITER:
ereport(LOG,
(errmsg("could not fork background writer process: %m")));
break;
- case CheckpointerProcess:
+ case B_CHECKPOINTER:
ereport(LOG,
(errmsg("could not fork checkpointer process: %m")));
break;
- case WalWriterProcess:
+ case B_WAL_WRITER:
ereport(LOG,
(errmsg("could not fork WAL writer process: %m")));
break;
- case WalReceiverProcess:
+ case B_WAL_RECEIVER:
ereport(LOG,
(errmsg("could not fork WAL receiver process: %m")));
break;
- case WalSummarizerProcess:
+ case B_WAL_SUMMARIZER:
ereport(LOG,
(errmsg("could not fork WAL summarizer process: %m")));
break;
@@ -5382,7 +5382,7 @@ StartChildProcess(AuxProcType type)
* fork failure is fatal during startup, but there's no need to choke
* immediately if starting other child types fails.
*/
- if (type == StartupProcess)
+ if (type == B_STARTUP)
ExitPostmaster(1);
return 0;
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 1a1050c8da1..92f24db4e18 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -257,17 +257,16 @@ pgstat_beinit(void)
else
{
/* Must be an auxiliary process */
- Assert(MyAuxProcType != NotAnAuxProcess);
+ Assert(IsAuxProcess(MyBackendType));
/*
* Assign the MyBEEntry for an auxiliary process. Since it doesn't
* have a BackendId, the slot is statically allocated based on the
- * auxiliary process type (MyAuxProcType). Backends use slots indexed
- * in the range from 0 to MaxBackends (exclusive), so we use
- * MaxBackends + AuxProcType as the index of the slot for an auxiliary
- * process.
+ * auxiliary process type. Backends use slots indexed in the range
+ * from 0 to MaxBackends (exclusive), and aux processes use the slots
+ * after that.
*/
- MyBEEntry = &BackendStatusArray[MaxBackends + MyAuxProcType];
+ MyBEEntry = &BackendStatusArray[MaxBackends + MyBackendType - FIRST_AUX_PROC];
}
/* Set up a process-exit hook to clean up */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 0b01c1f0935..70da52a7876 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -321,21 +321,36 @@ extern void InitProcessLocalLatch(void);
extern void SwitchToSharedLatch(void);
extern void SwitchBackToLocalLatch(void);
+/*
+ * MyBackendType indicates what kind of a backend this is.
+ */
typedef enum BackendType
{
B_INVALID = 0,
- B_ARCHIVER,
+
+ /* Backends and other backend-like processes */
+ B_BACKEND,
B_AUTOVAC_LAUNCHER,
B_AUTOVAC_WORKER,
- B_BACKEND,
B_BG_WORKER,
+ B_WAL_SENDER,
+
+ B_STANDALONE_BACKEND,
+ B_LOGGER,
+
+ /*
+ * Auxiliary processes. These have PGPROC entries, but they are not
+ * attached to any particular database. There can be only one of each of
+ * these running at a time.
+ *
+ * If you modify these, make sure to update NUM_AUXILIARY_PROCS and the
+ * glossary in the docs.
+ */
+ B_ARCHIVER,
B_BG_WRITER,
B_CHECKPOINTER,
- B_LOGGER,
- B_STANDALONE_BACKEND,
B_STARTUP,
B_WAL_RECEIVER,
- B_WAL_SENDER,
B_WAL_SUMMARIZER,
B_WAL_WRITER,
} BackendType;
@@ -344,6 +359,19 @@ typedef enum BackendType
extern PGDLLIMPORT BackendType MyBackendType;
+#define FIRST_AUX_PROC B_ARCHIVER
+#define NUM_AUXPROCTYPES (BACKEND_NUM_TYPES - FIRST_AUX_PROC)
+
+#define AmStartupProcess() (MyBackendType == B_STARTUP)
+#define AmBackgroundWriterProcess() (MyBackendType == B_BG_WRITER)
+#define AmArchiverProcess() (MyBackendType == B_ARCHIVER)
+#define AmCheckpointerProcess() (MyBackendType == B_CHECKPOINTER)
+#define AmWalWriterProcess() (MyBackendType == B_WAL_WRITER)
+#define AmWalReceiverProcess() (MyBackendType == B_WAL_RECEIVER)
+#define AmWalSummarizerProcess() (MyBackendType == B_WAL_SUMMARIZER)
+
+#define IsAuxProcess(type) (MyBackendType >= FIRST_AUX_PROC)
+
extern const char *GetBackendTypeDesc(BackendType backendType);
extern void SetDatabasePath(const char *path);
@@ -426,39 +454,6 @@ extern PGDLLIMPORT ProcessingMode Mode;
} while(0)
-/*
- * Auxiliary-process type identifiers. These used to be in bootstrap.h
- * but it seems saner to have them here, with the ProcessingMode stuff.
- * The MyAuxProcType global is defined and set in auxprocess.c.
- *
- * Make sure to list in the glossary any items you add here.
- */
-
-typedef enum
-{
- NotAnAuxProcess = -1,
- StartupProcess = 0,
- BgWriterProcess,
- ArchiverProcess,
- CheckpointerProcess,
- WalWriterProcess,
- WalReceiverProcess,
- WalSummarizerProcess,
-
- NUM_AUXPROCTYPES /* Must be last! */
-} AuxProcType;
-
-extern PGDLLIMPORT AuxProcType MyAuxProcType;
-
-#define AmStartupProcess() (MyAuxProcType == StartupProcess)
-#define AmBackgroundWriterProcess() (MyAuxProcType == BgWriterProcess)
-#define AmArchiverProcess() (MyAuxProcType == ArchiverProcess)
-#define AmCheckpointerProcess() (MyAuxProcType == CheckpointerProcess)
-#define AmWalWriterProcess() (MyAuxProcType == WalWriterProcess)
-#define AmWalReceiverProcess() (MyAuxProcType == WalReceiverProcess)
-#define AmWalSummarizerProcess() (MyAuxProcType == WalSummarizerProcess)
-
-
/*****************************************************************************
* pinit.h -- *
* POSTGRES initialization and cleanup definitions. *
diff --git a/src/include/postmaster/auxprocess.h b/src/include/postmaster/auxprocess.h
index 1fdde3bb77b..3e443edde70 100644
--- a/src/include/postmaster/auxprocess.h
+++ b/src/include/postmaster/auxprocess.h
@@ -15,6 +15,6 @@
#include "miscadmin.h"
-extern void AuxiliaryProcessMain(AuxProcType auxtype) pg_attribute_noreturn();
+extern void AuxiliaryProcessMain(BackendType auxtype) pg_attribute_noreturn();
#endif /* AUXPROCESS_H */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 5fd46b7bd1f..2abe43c57f6 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -172,7 +172,6 @@ AutoVacOpts
AutoVacuumShmemStruct
AutoVacuumWorkItem
AutoVacuumWorkItemType
-AuxProcType
BF_ctx
BF_key
BF_word
--
2.39.2
v6-0003-Use-MyBackendType-in-more-places-to-check-what-pr.patchtext/x-patch; charset=UTF-8; name=v6-0003-Use-MyBackendType-in-more-places-to-check-what-pr.patchDownload
From 795929a5f5a5d6ea4fa8a46bb15c68d2ff46ad3d Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 10 Jan 2024 12:59:48 +0200
Subject: [PATCH v6 3/3] Use MyBackendType in more places to check what process
this is
Remove IsBackgroundWorker, IsAutoVacuumLauncherProcess() and
IsAutoVacuumWorkerProcess() in favor of new Am*Process() macros that
use MyBackendType. For consistency with the existing Am*Process()
macros.
Reviewed-by: XXX
Discussion: XXX
---
src/backend/access/gin/ginfast.c | 2 +-
src/backend/access/gin/ginvacuum.c | 6 ++---
src/backend/access/heap/vacuumlazy.c | 4 +--
src/backend/commands/analyze.c | 4 +--
src/backend/commands/vacuum.c | 8 +++---
src/backend/postmaster/autovacuum.c | 26 --------------------
src/backend/postmaster/bgworker.c | 2 --
src/backend/postmaster/postmaster.c | 3 ---
src/backend/statistics/extended_stats.c | 2 +-
src/backend/storage/ipc/ipc.c | 2 +-
src/backend/storage/lmgr/proc.c | 19 +++++++-------
src/backend/tcop/postgres.c | 6 ++---
src/backend/utils/activity/pgstat_relation.c | 4 +--
src/backend/utils/init/globals.c | 1 -
src/backend/utils/init/miscinit.c | 2 +-
src/backend/utils/init/postinit.c | 8 +++---
src/include/miscadmin.h | 8 +++---
src/include/postmaster/autovacuum.h | 5 ----
18 files changed, 38 insertions(+), 74 deletions(-)
diff --git a/src/backend/access/gin/ginfast.c b/src/backend/access/gin/ginfast.c
index cff6850ef86..e118cecb9a4 100644
--- a/src/backend/access/gin/ginfast.c
+++ b/src/backend/access/gin/ginfast.c
@@ -812,7 +812,7 @@ ginInsertCleanup(GinState *ginstate, bool full_clean,
*/
LockPage(index, GIN_METAPAGE_BLKNO, ExclusiveLock);
workMemory =
- (IsAutoVacuumWorkerProcess() && autovacuum_work_mem != -1) ?
+ (AmAutoVacuumWorkerProcess() && autovacuum_work_mem != -1) ?
autovacuum_work_mem : maintenance_work_mem;
}
else
diff --git a/src/backend/access/gin/ginvacuum.c b/src/backend/access/gin/ginvacuum.c
index aee6aab44ae..b3f415e2849 100644
--- a/src/backend/access/gin/ginvacuum.c
+++ b/src/backend/access/gin/ginvacuum.c
@@ -590,7 +590,7 @@ ginbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
/*
* and cleanup any pending inserts
*/
- ginInsertCleanup(&gvs.ginstate, !IsAutoVacuumWorkerProcess(),
+ ginInsertCleanup(&gvs.ginstate, !AmAutoVacuumWorkerProcess(),
false, true, stats);
}
@@ -701,7 +701,7 @@ ginvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats)
*/
if (info->analyze_only)
{
- if (IsAutoVacuumWorkerProcess())
+ if (AmAutoVacuumWorkerProcess())
{
initGinState(&ginstate, index);
ginInsertCleanup(&ginstate, false, true, true, stats);
@@ -717,7 +717,7 @@ ginvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats)
{
stats = (IndexBulkDeleteResult *) palloc0(sizeof(IndexBulkDeleteResult));
initGinState(&ginstate, index);
- ginInsertCleanup(&ginstate, !IsAutoVacuumWorkerProcess(),
+ ginInsertCleanup(&ginstate, !AmAutoVacuumWorkerProcess(),
false, true, stats);
}
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index abbba8947fa..2194d4fced3 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -324,7 +324,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
char **indnames = NULL;
verbose = (params->options & VACOPT_VERBOSE) != 0;
- instrument = (verbose || (IsAutoVacuumWorkerProcess() &&
+ instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
params->log_min_duration >= 0));
if (instrument)
{
@@ -3148,7 +3148,7 @@ static int
dead_items_max_items(LVRelState *vacrel)
{
int64 max_items;
- int vac_work_mem = IsAutoVacuumWorkerProcess() &&
+ int vac_work_mem = AmAutoVacuumWorkerProcess() &&
autovacuum_work_mem != -1 ?
autovacuum_work_mem : maintenance_work_mem;
diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c
index deef865ce6d..655dc09444b 100644
--- a/src/backend/commands/analyze.c
+++ b/src/backend/commands/analyze.c
@@ -351,7 +351,7 @@ do_analyze_rel(Relation onerel, VacuumParams *params,
save_nestlevel = NewGUCNestLevel();
/* measure elapsed time iff autovacuum logging requires it */
- if (IsAutoVacuumWorkerProcess() && params->log_min_duration >= 0)
+ if (AmAutoVacuumWorkerProcess() && params->log_min_duration >= 0)
{
if (track_io_timing)
{
@@ -729,7 +729,7 @@ do_analyze_rel(Relation onerel, VacuumParams *params,
vac_close_indexes(nindexes, Irel, NoLock);
/* Log the action if appropriate */
- if (IsAutoVacuumWorkerProcess() && params->log_min_duration >= 0)
+ if (AmAutoVacuumWorkerProcess() && params->log_min_duration >= 0)
{
TimestampTz endtime = GetCurrentTimestamp();
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 64da8486276..e40cef6566e 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -564,7 +564,7 @@ vacuum(List *relations, VacuumParams *params, BufferAccessStrategy bstrategy,
else
{
Assert(params->options & VACOPT_ANALYZE);
- if (IsAutoVacuumWorkerProcess())
+ if (AmAutoVacuumWorkerProcess())
use_own_xacts = true;
else if (in_outer_xact)
use_own_xacts = false;
@@ -809,7 +809,7 @@ vacuum_open_relation(Oid relid, RangeVar *relation, bits32 options,
* statements in the permission checks; otherwise, only log if the caller
* so requested.
*/
- if (!IsAutoVacuumWorkerProcess())
+ if (!AmAutoVacuumWorkerProcess())
elevel = WARNING;
else if (verbose)
elevel = LOG;
@@ -896,7 +896,7 @@ expand_vacuum_rel(VacuumRelation *vrel, MemoryContext vac_context,
* Since autovacuum workers supply OIDs when calling vacuum(), no
* autovacuum worker should reach this code.
*/
- Assert(!IsAutoVacuumWorkerProcess());
+ Assert(!AmAutoVacuumWorkerProcess());
/*
* We transiently take AccessShareLock to protect the syscache lookup
@@ -2336,7 +2336,7 @@ vacuum_delay_point(void)
* [autovacuum_]vacuum_cost_delay to take effect while a table is being
* vacuumed or analyzed.
*/
- if (ConfigReloadPending && IsAutoVacuumWorkerProcess())
+ if (ConfigReloadPending && AmAutoVacuumWorkerProcess())
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 2c3099f76f1..cecbaaa5b91 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -135,10 +135,6 @@ int Log_autovacuum_min_duration = 600000;
#define MIN_AUTOVAC_SLEEPTIME 100.0 /* milliseconds */
#define MAX_AUTOVAC_SLEEPTIME 300 /* seconds */
-/* Flags to tell if we are in an autovacuum process */
-static bool am_autovacuum_launcher = false;
-static bool am_autovacuum_worker = false;
-
/*
* Variables to save the cost-related storage parameters for the current
* relation being vacuumed by this autovacuum worker. Using these, we can
@@ -435,8 +431,6 @@ AutoVacLauncherMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
- am_autovacuum_launcher = true;
-
MyBackendType = B_AUTOVAC_LAUNCHER;
init_ps_display(NULL);
@@ -1495,8 +1489,6 @@ AutoVacWorkerMain(int argc, char *argv[])
sigjmp_buf local_sigjmp_buf;
Oid dbid;
- am_autovacuum_worker = true;
-
MyBackendType = B_AUTOVAC_WORKER;
init_ps_display(NULL);
@@ -3355,24 +3347,6 @@ autovac_init(void)
errhint("Enable the \"track_counts\" option.")));
}
-/*
- * IsAutoVacuum functions
- * Return whether this is either a launcher autovacuum process or a worker
- * process.
- */
-bool
-IsAutoVacuumLauncherProcess(void)
-{
- return am_autovacuum_launcher;
-}
-
-bool
-IsAutoVacuumWorkerProcess(void)
-{
- return am_autovacuum_worker;
-}
-
-
/*
* AutoVacuumShmemSize
* Compute space needed for autovacuum-related shared memory
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index 67f92c24db1..d8e89de7494 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -731,8 +731,6 @@ BackgroundWorkerMain(void)
if (worker == NULL)
elog(FATAL, "unable to find bgworker entry");
- IsBackgroundWorker = true;
-
MyBackendType = B_BG_WORKER;
init_ps_display(worker->bgw_name);
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index c51b6a1376a..8aa2ddf0c46 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -4979,9 +4979,6 @@ SubPostmasterMain(int argc, char *argv[])
}
if (strcmp(argv[1], "--forkbgworker") == 0)
{
- /* do this as early as possible; in particular, before InitProcess() */
- IsBackgroundWorker = true;
-
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
diff --git a/src/backend/statistics/extended_stats.c b/src/backend/statistics/extended_stats.c
index c5461514d8f..135151a2723 100644
--- a/src/backend/statistics/extended_stats.c
+++ b/src/backend/statistics/extended_stats.c
@@ -173,7 +173,7 @@ BuildRelationExtStatistics(Relation onerel, bool inh, double totalrows,
natts, vacattrstats);
if (!stats)
{
- if (!IsAutoVacuumWorkerProcess())
+ if (!AmAutoVacuumWorkerProcess())
ereport(WARNING,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("statistics object \"%s.%s\" could not be computed for relation \"%s.%s\"",
diff --git a/src/backend/storage/ipc/ipc.c b/src/backend/storage/ipc/ipc.c
index 2681e4cdff5..b06e4b84528 100644
--- a/src/backend/storage/ipc/ipc.c
+++ b/src/backend/storage/ipc/ipc.c
@@ -136,7 +136,7 @@ proc_exit(int code)
*/
char gprofDirName[32];
- if (IsAutoVacuumWorkerProcess())
+ if (AmAutoVacuumWorkerProcess())
snprintf(gprofDirName, 32, "gprof/avworker");
else
snprintf(gprofDirName, 32, "gprof/%d", (int) getpid());
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index 4ad96beb87a..b904fab5de5 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -41,7 +41,6 @@
#include "postmaster/autovacuum.h"
#include "replication/slot.h"
#include "replication/syncrep.h"
-#include "replication/walsender.h"
#include "storage/condition_variable.h"
#include "storage/ipc.h"
#include "storage/lmgr.h"
@@ -309,11 +308,11 @@ InitProcess(void)
elog(ERROR, "you already exist");
/* Decide which list should supply our PGPROC. */
- if (IsAnyAutoVacuumProcess())
+ if (AmAutoVacuumLauncherProcess() || AmAutoVacuumWorkerProcess())
procgloballist = &ProcGlobal->autovacFreeProcs;
- else if (IsBackgroundWorker)
+ else if (AmBackgroundWorkerProcess())
procgloballist = &ProcGlobal->bgworkerFreeProcs;
- else if (am_walsender)
+ else if (AmWalSenderProcess())
procgloballist = &ProcGlobal->walsenderFreeProcs;
else
procgloballist = &ProcGlobal->freeProcs;
@@ -343,7 +342,7 @@ InitProcess(void)
* in the autovacuum case?
*/
SpinLockRelease(ProcStructLock);
- if (am_walsender)
+ if (AmWalSenderProcess())
ereport(FATAL,
(errcode(ERRCODE_TOO_MANY_CONNECTIONS),
errmsg("number of requested standby connections exceeds max_wal_senders (currently %d)",
@@ -365,7 +364,7 @@ InitProcess(void)
* cleaning up. (XXX autovac launcher currently doesn't participate in
* this; it probably should.)
*/
- if (IsUnderPostmaster && !IsAutoVacuumLauncherProcess())
+ if (IsUnderPostmaster && !AmAutoVacuumLauncherProcess())
MarkPostmasterChildActive();
/*
@@ -385,11 +384,11 @@ InitProcess(void)
MyProc->databaseId = InvalidOid;
MyProc->roleId = InvalidOid;
MyProc->tempNamespaceId = InvalidOid;
- MyProc->isBackgroundWorker = IsBackgroundWorker;
+ MyProc->isBackgroundWorker = AmBackgroundWorkerProcess();
MyProc->delayChkptFlags = 0;
MyProc->statusFlags = 0;
/* NB -- autovac launcher intentionally does not set IS_AUTOVACUUM */
- if (IsAutoVacuumWorkerProcess())
+ if (AmAutoVacuumWorkerProcess())
MyProc->statusFlags |= PROC_IS_AUTOVACUUM;
MyProc->lwWaiting = LW_WS_NOT_WAITING;
MyProc->lwWaitMode = 0;
@@ -580,7 +579,7 @@ InitAuxiliaryProcess(void)
MyProc->databaseId = InvalidOid;
MyProc->roleId = InvalidOid;
MyProc->tempNamespaceId = InvalidOid;
- MyProc->isBackgroundWorker = IsBackgroundWorker;
+ MyProc->isBackgroundWorker = AmBackgroundWorkerProcess();
MyProc->delayChkptFlags = 0;
MyProc->statusFlags = 0;
MyProc->lwWaiting = LW_WS_NOT_WAITING;
@@ -935,7 +934,7 @@ ProcKill(int code, Datum arg)
* way, so tell the postmaster we've cleaned up acceptably well. (XXX
* autovac launcher should be included here someday)
*/
- if (IsUnderPostmaster && !IsAutoVacuumLauncherProcess())
+ if (IsUnderPostmaster && !AmAutoVacuumLauncherProcess())
MarkPostmasterChildInactive();
/* wake autovac launcher if needed -- see comments in FreeWorkerInfo */
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 1eaaf3c6c58..1f74de17edf 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -3267,7 +3267,7 @@ ProcessInterrupts(void)
ereport(FATAL,
(errcode(ERRCODE_QUERY_CANCELED),
errmsg("canceling authentication due to timeout")));
- else if (IsAutoVacuumWorkerProcess())
+ else if (AmAutoVacuumWorkerProcess())
ereport(FATAL,
(errcode(ERRCODE_ADMIN_SHUTDOWN),
errmsg("terminating autovacuum process due to administrator command")));
@@ -3286,7 +3286,7 @@ ProcessInterrupts(void)
*/
proc_exit(1);
}
- else if (IsBackgroundWorker)
+ else if (AmBackgroundWorkerProcess())
ereport(FATAL,
(errcode(ERRCODE_ADMIN_SHUTDOWN),
errmsg("terminating background worker \"%s\" due to administrator command",
@@ -3386,7 +3386,7 @@ ProcessInterrupts(void)
(errcode(ERRCODE_QUERY_CANCELED),
errmsg("canceling statement due to statement timeout")));
}
- if (IsAutoVacuumWorkerProcess())
+ if (AmAutoVacuumWorkerProcess())
{
LockErrorCleanup();
ereport(ERROR,
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 111050725a6..b20a60b5a87 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -246,7 +246,7 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
*/
tabentry->ins_since_vacuum = 0;
- if (IsAutoVacuumWorkerProcess())
+ if (AmAutoVacuumWorkerProcess())
{
tabentry->last_autovacuum_time = ts;
tabentry->autovacuum_count++;
@@ -337,7 +337,7 @@ pgstat_report_analyze(Relation rel,
if (resetcounter)
tabentry->mod_since_analyze = 0;
- if (IsAutoVacuumWorkerProcess())
+ if (AmAutoVacuumWorkerProcess())
{
tabentry->last_autoanalyze_time = GetCurrentTimestamp();
tabentry->autoanalyze_count++;
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 88b03e8fa3c..ce1fa1092f2 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -114,7 +114,6 @@ pid_t PostmasterPid = 0;
bool IsPostmasterEnvironment = false;
bool IsUnderPostmaster = false;
bool IsBinaryUpgrade = false;
-bool IsBackgroundWorker = false;
bool ExitOnAnyError = false;
diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c
index 23f77a59e58..9c91e9a1324 100644
--- a/src/backend/utils/init/miscinit.c
+++ b/src/backend/utils/init/miscinit.c
@@ -837,7 +837,7 @@ InitializeSessionUserIdStandalone(void)
* This function should only be called in single-user mode, in autovacuum
* workers, and in background workers.
*/
- Assert(!IsUnderPostmaster || IsAutoVacuumWorkerProcess() || IsBackgroundWorker);
+ Assert(!IsUnderPostmaster || AmAutoVacuumWorkerProcess() || AmBackgroundWorkerProcess());
/* call only once */
Assert(!OidIsValid(AuthenticatedUserId));
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index 1ad33671598..8323c02acf2 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -344,7 +344,7 @@ CheckMyDatabase(const char *name, bool am_superuser, bool override_allow_connect
*
* We do not enforce them for autovacuum worker processes either.
*/
- if (IsUnderPostmaster && !IsAutoVacuumWorkerProcess())
+ if (IsUnderPostmaster && !AmAutoVacuumWorkerProcess())
{
/*
* Check that the database is currently allowing connections.
@@ -833,7 +833,7 @@ InitPostgres(const char *in_dbname, Oid dboid,
before_shmem_exit(ShutdownPostgres, 0);
/* The autovacuum launcher is done here */
- if (IsAutoVacuumLauncherProcess())
+ if (AmAutoVacuumLauncherProcess())
{
/* report this backend in the PgBackendStatus array */
pgstat_bestart();
@@ -877,7 +877,7 @@ InitPostgres(const char *in_dbname, Oid dboid,
* In standalone mode and in autovacuum worker processes, we use a fixed
* ID, otherwise we figure it out from the authenticated user name.
*/
- if (bootstrap || IsAutoVacuumWorkerProcess())
+ if (bootstrap || AmAutoVacuumWorkerProcess())
{
InitializeSessionUserIdStandalone();
am_superuser = true;
@@ -893,7 +893,7 @@ InitPostgres(const char *in_dbname, Oid dboid,
errhint("You should immediately run CREATE USER \"%s\" SUPERUSER;.",
username != NULL ? username : "postgres")));
}
- else if (IsBackgroundWorker)
+ else if (AmBackgroundWorkerProcess())
{
if (username == NULL && !OidIsValid(useroid))
{
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 70da52a7876..f7d08b1dafd 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -164,7 +164,6 @@ do { \
extern PGDLLIMPORT pid_t PostmasterPid;
extern PGDLLIMPORT bool IsPostmasterEnvironment;
extern PGDLLIMPORT bool IsUnderPostmaster;
-extern PGDLLIMPORT bool IsBackgroundWorker;
extern PGDLLIMPORT bool IsBinaryUpgrade;
extern PGDLLIMPORT bool ExitOnAnyError;
@@ -361,7 +360,12 @@ extern PGDLLIMPORT BackendType MyBackendType;
#define FIRST_AUX_PROC B_ARCHIVER
#define NUM_AUXPROCTYPES (BACKEND_NUM_TYPES - FIRST_AUX_PROC)
+#define IsAuxProcess(type) (MyBackendType >= FIRST_AUX_PROC)
+#define AmAutoVacuumLauncherProcess() (MyBackendType == B_AUTOVAC_LAUNCHER)
+#define AmAutoVacuumWorkerProcess() (MyBackendType == B_AUTOVAC_WORKER)
+#define AmBackgroundWorkerProcess() (MyBackendType == B_BG_WORKER)
+#define AmWalSenderProcess() (MyBackendType == B_WAL_SENDER)
#define AmStartupProcess() (MyBackendType == B_STARTUP)
#define AmBackgroundWriterProcess() (MyBackendType == B_BG_WRITER)
#define AmArchiverProcess() (MyBackendType == B_ARCHIVER)
@@ -370,8 +374,6 @@ extern PGDLLIMPORT BackendType MyBackendType;
#define AmWalReceiverProcess() (MyBackendType == B_WAL_RECEIVER)
#define AmWalSummarizerProcess() (MyBackendType == B_WAL_SUMMARIZER)
-#define IsAuxProcess(type) (MyBackendType >= FIRST_AUX_PROC)
-
extern const char *GetBackendTypeDesc(BackendType backendType);
extern void SetDatabasePath(const char *path);
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 1994aedef03..80cf4cdd969 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -49,11 +49,6 @@ extern PGDLLIMPORT int Log_autovacuum_min_duration;
/* Status inquiry functions */
extern bool AutoVacuumingActive(void);
-extern bool IsAutoVacuumLauncherProcess(void);
-extern bool IsAutoVacuumWorkerProcess(void);
-
-#define IsAnyAutoVacuumProcess() \
- (IsAutoVacuumLauncherProcess() || IsAutoVacuumWorkerProcess())
/* Functions to start autovacuum process, called from postmaster */
extern void autovac_init(void);
--
2.39.2
Re: Refactoring backend fork+exec code
Hi,
On 2024-01-10 14:35:52 +0200, Heikki Linnakangas wrote:
Here's a patch that gets rid of AuxProcType. It's independent of the other
patches in this thread; if this is committed, I'll rebase the rest of the
patches over this and get rid of the new PMC_* enum.Three patches, actually. The first one fixes an existing comment that I
noticed to be incorrect while working on this. I'll push that soon, barring
objections. The second one gets rid of AuxProcType, and the third one
replaces IsBackgroundWorker, IsAutoVacuumLauncherProcess() and
IsAutoVacuumWorkerProcess() with checks on MyBackendType. So MyBackendType
is now the primary way to check what kind of a process the current process
is.'am_walsender' would also be fairly straightforward to remove and replace
with AmWalSenderProcess(). I didn't do that because we also have
am_db_walsender and am_cascading_walsender which cannot be directly replaced
with MyBackendType. Given that, it might be best to keep am_walsender for
symmetry.
@@ -561,13 +561,13 @@ static void ShmemBackendArrayAdd(Backend *bn);
static void ShmemBackendArrayRemove(Backend *bn);
#endif /* EXEC_BACKEND */-#define StartupDataBase() StartChildProcess(StartupProcess) -#define StartArchiver() StartChildProcess(ArchiverProcess) -#define StartBackgroundWriter() StartChildProcess(BgWriterProcess) -#define StartCheckpointer() StartChildProcess(CheckpointerProcess) -#define StartWalWriter() StartChildProcess(WalWriterProcess) -#define StartWalReceiver() StartChildProcess(WalReceiverProcess) -#define StartWalSummarizer() StartChildProcess(WalSummarizerProcess) +#define StartupDataBase() StartChildProcess(B_STARTUP) +#define StartArchiver() StartChildProcess(B_ARCHIVER) +#define StartBackgroundWriter() StartChildProcess(B_BG_WRITER) +#define StartCheckpointer() StartChildProcess(B_CHECKPOINTER) +#define StartWalWriter() StartChildProcess(B_WAL_WRITER) +#define StartWalReceiver() StartChildProcess(B_WAL_RECEIVER) +#define StartWalSummarizer() StartChildProcess(B_WAL_SUMMARIZER)
Not for this commit, but we ought to rip out these macros - all they do is to
make it harder to understand the code.
@@ -5344,31 +5344,31 @@ StartChildProcess(AuxProcType type)
errno = save_errno;
switch (type)
{
- case StartupProcess:
+ case B_STARTUP:
ereport(LOG,
(errmsg("could not fork startup process: %m")));
break;
- case ArchiverProcess:
+ case B_ARCHIVER:
ereport(LOG,
(errmsg("could not fork archiver process: %m")));
break;
- case BgWriterProcess:
+ case B_BG_WRITER:
ereport(LOG,
(errmsg("could not fork background writer process: %m")));
break;
- case CheckpointerProcess:
+ case B_CHECKPOINTER:
ereport(LOG,
(errmsg("could not fork checkpointer process: %m")));
break;
- case WalWriterProcess:
+ case B_WAL_WRITER:
ereport(LOG,
(errmsg("could not fork WAL writer process: %m")));
break;
- case WalReceiverProcess:
+ case B_WAL_RECEIVER:
ereport(LOG,
(errmsg("could not fork WAL receiver process: %m")));
break;
- case WalSummarizerProcess:
+ case B_WAL_SUMMARIZER:
ereport(LOG,
(errmsg("could not fork WAL summarizer process: %m")));
break;
Seems we should replace this with something slightly more generic one of these
days...
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c index 1a1050c8da1..92f24db4e18 100644 --- a/src/backend/utils/activity/backend_status.c +++ b/src/backend/utils/activity/backend_status.c @@ -257,17 +257,16 @@ pgstat_beinit(void) else { /* Must be an auxiliary process */ - Assert(MyAuxProcType != NotAnAuxProcess); + Assert(IsAuxProcess(MyBackendType));/* * Assign the MyBEEntry for an auxiliary process. Since it doesn't * have a BackendId, the slot is statically allocated based on the - * auxiliary process type (MyAuxProcType). Backends use slots indexed - * in the range from 0 to MaxBackends (exclusive), so we use - * MaxBackends + AuxProcType as the index of the slot for an auxiliary - * process. + * auxiliary process type. Backends use slots indexed in the range + * from 0 to MaxBackends (exclusive), and aux processes use the slots + * after that. */ - MyBEEntry = &BackendStatusArray[MaxBackends + MyAuxProcType]; + MyBEEntry = &BackendStatusArray[MaxBackends + MyBackendType - FIRST_AUX_PROC]; }
Hm, this seems less than pretty. It's probably ok for now, but it seems like a
better fix might be to just start assigning backend ids to aux procs or switch
to indexing by pgprocno?
From 795929a5f5a5d6ea4fa8a46bb15c68d2ff46ad3d Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 10 Jan 2024 12:59:48 +0200
Subject: [PATCH v6 3/3] Use MyBackendType in more places to check what process
this isRemove IsBackgroundWorker, IsAutoVacuumLauncherProcess() and
IsAutoVacuumWorkerProcess() in favor of new Am*Process() macros that
use MyBackendType. For consistency with the existing Am*Process()
macros.
The Am*Process() macros aren't realy new, they are just implemented
differently, right? I guess there are a few more of them now.
Given that we are probably going to have more process types in the future, it
seems like a better direction would be a AmProcessType(proctype) style
macro/inline function. That we we don't have to mirror the list of process
types in the enum and a set of macros.
Greetings,
Andres Freund
Re: Refactoring backend fork+exec code
On 22/01/2024 23:07, Andres Freund wrote:
On 2024-01-10 14:35:52 +0200, Heikki Linnakangas wrote:
@@ -5344,31 +5344,31 @@ StartChildProcess(AuxProcType type)
errno = save_errno;
switch (type)
{
- case StartupProcess:
+ case B_STARTUP:
ereport(LOG,
(errmsg("could not fork startup process: %m")));
break;
- case ArchiverProcess:
+ case B_ARCHIVER:
ereport(LOG,
(errmsg("could not fork archiver process: %m")));
break;
- case BgWriterProcess:
+ case B_BG_WRITER:
ereport(LOG,
(errmsg("could not fork background writer process: %m")));
break;
- case CheckpointerProcess:
+ case B_CHECKPOINTER:
ereport(LOG,
(errmsg("could not fork checkpointer process: %m")));
break;
- case WalWriterProcess:
+ case B_WAL_WRITER:
ereport(LOG,
(errmsg("could not fork WAL writer process: %m")));
break;
- case WalReceiverProcess:
+ case B_WAL_RECEIVER:
ereport(LOG,
(errmsg("could not fork WAL receiver process: %m")));
break;
- case WalSummarizerProcess:
+ case B_WAL_SUMMARIZER:
ereport(LOG,
(errmsg("could not fork WAL summarizer process: %m")));
break;Seems we should replace this with something slightly more generic one of these
days...
The later patches in this thread will turn these into
ereport(LOG,
(errmsg("could not fork %s process: %m",
PostmasterChildName(type))));
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c index 1a1050c8da1..92f24db4e18 100644 --- a/src/backend/utils/activity/backend_status.c +++ b/src/backend/utils/activity/backend_status.c @@ -257,17 +257,16 @@ pgstat_beinit(void) else { /* Must be an auxiliary process */ - Assert(MyAuxProcType != NotAnAuxProcess); + Assert(IsAuxProcess(MyBackendType));/* * Assign the MyBEEntry for an auxiliary process. Since it doesn't * have a BackendId, the slot is statically allocated based on the - * auxiliary process type (MyAuxProcType). Backends use slots indexed - * in the range from 0 to MaxBackends (exclusive), so we use - * MaxBackends + AuxProcType as the index of the slot for an auxiliary - * process. + * auxiliary process type. Backends use slots indexed in the range + * from 0 to MaxBackends (exclusive), and aux processes use the slots + * after that. */ - MyBEEntry = &BackendStatusArray[MaxBackends + MyAuxProcType]; + MyBEEntry = &BackendStatusArray[MaxBackends + MyBackendType - FIRST_AUX_PROC]; }Hm, this seems less than pretty. It's probably ok for now, but it seems like a
better fix might be to just start assigning backend ids to aux procs or switch
to indexing by pgprocno?
Using pgprocno is a good idea. Come to think of it, why do we even have
a concept of backend ID that's separate from pgprocno? backend ID is
used to index the ProcState array, but AFAICS we could use pgprocno as
the index to that, too.
--
Heikki Linnakangas
Neon (https://neon.tech)
Re: Refactoring backend fork+exec code
Hi,
On 2024-01-23 21:07:08 +0200, Heikki Linnakangas wrote:
On 22/01/2024 23:07, Andres Freund wrote:
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c index 1a1050c8da1..92f24db4e18 100644 --- a/src/backend/utils/activity/backend_status.c +++ b/src/backend/utils/activity/backend_status.c @@ -257,17 +257,16 @@ pgstat_beinit(void) else { /* Must be an auxiliary process */ - Assert(MyAuxProcType != NotAnAuxProcess); + Assert(IsAuxProcess(MyBackendType)); /* * Assign the MyBEEntry for an auxiliary process. Since it doesn't * have a BackendId, the slot is statically allocated based on the - * auxiliary process type (MyAuxProcType). Backends use slots indexed - * in the range from 0 to MaxBackends (exclusive), so we use - * MaxBackends + AuxProcType as the index of the slot for an auxiliary - * process. + * auxiliary process type. Backends use slots indexed in the range + * from 0 to MaxBackends (exclusive), and aux processes use the slots + * after that. */ - MyBEEntry = &BackendStatusArray[MaxBackends + MyAuxProcType]; + MyBEEntry = &BackendStatusArray[MaxBackends + MyBackendType - FIRST_AUX_PROC]; }Hm, this seems less than pretty. It's probably ok for now, but it seems like a
better fix might be to just start assigning backend ids to aux procs or switch
to indexing by pgprocno?Using pgprocno is a good idea. Come to think of it, why do we even have a
concept of backend ID that's separate from pgprocno? backend ID is used to
index the ProcState array, but AFAICS we could use pgprocno as the index to
that, too.
I think we should do that. There are a few processes not participating in
sinval, but it doesn't make enough of a difference to make sinval slower. And
I think there'd be far bigger efficiency improvements to sinvaladt than not
having a handful more entries.
Greetings,
Andres Freund
Re: Refactoring backend fork+exec code
On 23/01/2024 21:50, Andres Freund wrote:
On 2024-01-23 21:07:08 +0200, Heikki Linnakangas wrote:
On 22/01/2024 23:07, Andres Freund wrote:
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c index 1a1050c8da1..92f24db4e18 100644 --- a/src/backend/utils/activity/backend_status.c +++ b/src/backend/utils/activity/backend_status.c @@ -257,17 +257,16 @@ pgstat_beinit(void) else { /* Must be an auxiliary process */ - Assert(MyAuxProcType != NotAnAuxProcess); + Assert(IsAuxProcess(MyBackendType)); /* * Assign the MyBEEntry for an auxiliary process. Since it doesn't * have a BackendId, the slot is statically allocated based on the - * auxiliary process type (MyAuxProcType). Backends use slots indexed - * in the range from 0 to MaxBackends (exclusive), so we use - * MaxBackends + AuxProcType as the index of the slot for an auxiliary - * process. + * auxiliary process type. Backends use slots indexed in the range + * from 0 to MaxBackends (exclusive), and aux processes use the slots + * after that. */ - MyBEEntry = &BackendStatusArray[MaxBackends + MyAuxProcType]; + MyBEEntry = &BackendStatusArray[MaxBackends + MyBackendType - FIRST_AUX_PROC]; }Hm, this seems less than pretty. It's probably ok for now, but it seems like a
better fix might be to just start assigning backend ids to aux procs or switch
to indexing by pgprocno?Using pgprocno is a good idea. Come to think of it, why do we even have a
concept of backend ID that's separate from pgprocno? backend ID is used to
index the ProcState array, but AFAICS we could use pgprocno as the index to
that, too.I think we should do that. There are a few processes not participating in
sinval, but it doesn't make enough of a difference to make sinval slower. And
I think there'd be far bigger efficiency improvements to sinvaladt than not
having a handful more entries.
And here we go. BackendID is now a 1-based index directly into the
PGPROC array.
--
Heikki Linnakangas
Neon (https://neon.tech)
Attachments:
v7-0001-Remove-superfluous-pgprocno-field-from-PGPROC.patchtext/x-patch; charset=UTF-8; name=v7-0001-Remove-superfluous-pgprocno-field-from-PGPROC.patchDownload
From 87db35c877f4492b98848dfcd0baa49cc8100c1b Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 24 Jan 2024 23:15:55 +0200
Subject: [PATCH v7 1/4] Remove superfluous 'pgprocno' field from PGPROC
It was always just the index of the PGPROC entry from the beginning of
the proc array. Introduce a macro to compute it from the pointer
instead.
Reviewed-by: XXX
Discussion: https://www.postgresql.org/message-id/8171f1aa-496f-46a6-afc3-c46fe7a9b407%40iki.fi
---
src/backend/access/transam/clog.c | 4 ++--
src/backend/access/transam/twophase.c | 11 +++++------
src/backend/access/transam/xlog.c | 2 +-
src/backend/postmaster/bgwriter.c | 2 +-
src/backend/postmaster/pgarch.c | 2 +-
src/backend/postmaster/walsummarizer.c | 2 +-
src/backend/storage/buffer/bufmgr.c | 6 +++---
src/backend/storage/ipc/procarray.c | 6 +++---
src/backend/storage/lmgr/condition_variable.c | 12 ++++++------
src/backend/storage/lmgr/lwlock.c | 6 +++---
src/backend/storage/lmgr/predicate.c | 2 +-
src/backend/storage/lmgr/proc.c | 1 -
src/include/storage/lock.h | 2 +-
src/include/storage/proc.h | 6 +-----
14 files changed, 29 insertions(+), 35 deletions(-)
diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c
index f6e7da7ffc9..7550309c25a 100644
--- a/src/backend/access/transam/clog.c
+++ b/src/backend/access/transam/clog.c
@@ -458,7 +458,7 @@ TransactionGroupUpdateXidStatus(TransactionId xid, XidStatus status,
* less efficiently.
*/
if (nextidx != INVALID_PGPROCNO &&
- ProcGlobal->allProcs[nextidx].clogGroupMemberPage != proc->clogGroupMemberPage)
+ GetPGProcByNumber(nextidx)->clogGroupMemberPage != proc->clogGroupMemberPage)
{
/*
* Ensure that this proc is not a member of any clog group that
@@ -473,7 +473,7 @@ TransactionGroupUpdateXidStatus(TransactionId xid, XidStatus status,
if (pg_atomic_compare_exchange_u32(&procglobal->clogGroupFirst,
&nextidx,
- (uint32) proc->pgprocno))
+ (uint32) GetNumberFromPGProc(proc)))
break;
}
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 8426458f7f5..234c8d08ebc 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -284,7 +284,7 @@ TwoPhaseShmemInit(void)
TwoPhaseState->freeGXacts = &gxacts[i];
/* associate it with a PGPROC assigned by InitProcGlobal */
- gxacts[i].pgprocno = PreparedXactProcs[i].pgprocno;
+ gxacts[i].pgprocno = GetNumberFromPGProc(&PreparedXactProcs[i]);
/*
* Assign a unique ID for each dummy proc, so that the range of
@@ -461,7 +461,6 @@ MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid, const char *gid,
/* Initialize the PGPROC entry */
MemSet(proc, 0, sizeof(PGPROC));
- proc->pgprocno = gxact->pgprocno;
dlist_node_init(&proc->links);
proc->waitStatus = PROC_WAIT_STATUS_OK;
if (LocalTransactionIdIsValid(MyProc->lxid))
@@ -780,7 +779,7 @@ pg_prepared_xact(PG_FUNCTION_ARGS)
while (status->array != NULL && status->currIdx < status->ngxacts)
{
GlobalTransaction gxact = &status->array[status->currIdx++];
- PGPROC *proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ PGPROC *proc = GetPGProcByNumber(gxact->pgprocno);
Datum values[5] = {0};
bool nulls[5] = {0};
HeapTuple tuple;
@@ -935,7 +934,7 @@ TwoPhaseGetDummyProc(TransactionId xid, bool lock_held)
{
GlobalTransaction gxact = TwoPhaseGetGXact(xid, lock_held);
- return &ProcGlobal->allProcs[gxact->pgprocno];
+ return GetPGProcByNumber(gxact->pgprocno);
}
/************************************************************************/
@@ -1080,7 +1079,7 @@ save_state_data(const void *data, uint32 len)
void
StartPrepare(GlobalTransaction gxact)
{
- PGPROC *proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ PGPROC *proc = GetPGProcByNumber(gxact->pgprocno);
TransactionId xid = gxact->xid;
TwoPhaseFileHeader hdr;
TransactionId *children;
@@ -1539,7 +1538,7 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
* try to commit the same GID at once.
*/
gxact = LockGXact(gid, GetUserId());
- proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ proc = GetPGProcByNumber(gxact->pgprocno);
xid = gxact->xid;
/*
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 478377c4a23..be30a2388f5 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -1379,7 +1379,7 @@ WALInsertLockAcquire(void)
static int lockToTry = -1;
if (lockToTry == -1)
- lockToTry = MyProc->pgprocno % NUM_XLOGINSERT_LOCKS;
+ lockToTry = GetNumberFromPGProc(MyProc) % NUM_XLOGINSERT_LOCKS;
MyLockNo = lockToTry;
/*
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index d7d6cc0cd7b..0677e061216 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -331,7 +331,7 @@ BackgroundWriterMain(void)
if (rc == WL_TIMEOUT && can_hibernate && prev_hibernate)
{
/* Ask for notification at next buffer allocation */
- StrategyNotifyBgWriter(MyProc->pgprocno);
+ StrategyNotifyBgWriter(GetNumberFromPGProc(MyProc));
/* Sleep ... */
(void) WaitLatch(MyLatch,
WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH,
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 67693b05806..9bfe4fae046 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -242,7 +242,7 @@ PgArchiverMain(void)
* Advertise our pgprocno so that backends can use our latch to wake us up
* while we're sleeping.
*/
- PgArch->pgprocno = MyProc->pgprocno;
+ PgArch->pgprocno = GetNumberFromPGProc(MyProc);
/* Create workspace for pgarch_readyXlog() */
arch_files = palloc(sizeof(struct arch_files_state));
diff --git a/src/backend/postmaster/walsummarizer.c b/src/backend/postmaster/walsummarizer.c
index 9b883c21ca4..165bdd93fa3 100644
--- a/src/backend/postmaster/walsummarizer.c
+++ b/src/backend/postmaster/walsummarizer.c
@@ -248,7 +248,7 @@ WalSummarizerMain(void)
/* Advertise ourselves. */
on_shmem_exit(WalSummarizerShutdown, (Datum) 0);
LWLockAcquire(WALSummarizerLock, LW_EXCLUSIVE);
- WalSummarizerCtl->summarizer_pgprocno = MyProc->pgprocno;
+ WalSummarizerCtl->summarizer_pgprocno = GetNumberFromPGProc(MyProc);
LWLockRelease(WALSummarizerLock);
/* Create and switch to a memory context that we can reset on error. */
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 7d601bef6dd..e19d433a8d3 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -4792,7 +4792,7 @@ UnlockBuffers(void)
* got a cancel/die interrupt before getting the signal.
*/
if ((buf_state & BM_PIN_COUNT_WAITER) != 0 &&
- buf->wait_backend_pgprocno == MyProc->pgprocno)
+ buf->wait_backend_pgprocno == GetNumberFromPGProc(MyProc))
buf_state &= ~BM_PIN_COUNT_WAITER;
UnlockBufHdr(buf, buf_state);
@@ -4942,7 +4942,7 @@ LockBufferForCleanup(Buffer buffer)
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
elog(ERROR, "multiple backends attempting to wait for pincount 1");
}
- bufHdr->wait_backend_pgprocno = MyProc->pgprocno;
+ bufHdr->wait_backend_pgprocno = GetNumberFromPGProc(MyProc);
PinCountWaitBuf = bufHdr;
buf_state |= BM_PIN_COUNT_WAITER;
UnlockBufHdr(bufHdr, buf_state);
@@ -5006,7 +5006,7 @@ LockBufferForCleanup(Buffer buffer)
*/
buf_state = LockBufHdr(bufHdr);
if ((buf_state & BM_PIN_COUNT_WAITER) != 0 &&
- bufHdr->wait_backend_pgprocno == MyProc->pgprocno)
+ bufHdr->wait_backend_pgprocno == GetNumberFromPGProc(MyProc))
buf_state &= ~BM_PIN_COUNT_WAITER;
UnlockBufHdr(bufHdr, buf_state);
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index ee2d7f8585a..5a33eb7f7f7 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -505,7 +505,7 @@ ProcArrayAdd(PGPROC *proc)
Assert(allProcs[procno].pgxactoff == index);
/* If we have found our right position in the array, break */
- if (arrayP->pgprocnos[index] > proc->pgprocno)
+ if (arrayP->pgprocnos[index] > GetNumberFromPGProc(proc))
break;
}
@@ -523,7 +523,7 @@ ProcArrayAdd(PGPROC *proc)
&ProcGlobal->statusFlags[index],
movecount * sizeof(*ProcGlobal->statusFlags));
- arrayP->pgprocnos[index] = proc->pgprocno;
+ arrayP->pgprocnos[index] = GetNumberFromPGProc(proc);
proc->pgxactoff = index;
ProcGlobal->xids[index] = proc->xid;
ProcGlobal->subxidStates[index] = proc->subxidStatus;
@@ -808,7 +808,7 @@ ProcArrayGroupClearXid(PGPROC *proc, TransactionId latestXid)
if (pg_atomic_compare_exchange_u32(&procglobal->procArrayGroupFirst,
&nextidx,
- (uint32) proc->pgprocno))
+ (uint32) GetNumberFromPGProc(proc)))
break;
}
diff --git a/src/backend/storage/lmgr/condition_variable.c b/src/backend/storage/lmgr/condition_variable.c
index e2d6d685220..d4a50c70ea4 100644
--- a/src/backend/storage/lmgr/condition_variable.c
+++ b/src/backend/storage/lmgr/condition_variable.c
@@ -57,7 +57,7 @@ ConditionVariableInit(ConditionVariable *cv)
void
ConditionVariablePrepareToSleep(ConditionVariable *cv)
{
- int pgprocno = MyProc->pgprocno;
+ int pgprocno = GetNumberFromPGProc(MyProc);
/*
* If some other sleep is already prepared, cancel it; this is necessary
@@ -181,10 +181,10 @@ ConditionVariableTimedSleep(ConditionVariable *cv, long timeout,
* guarantee not to return spuriously, we'll avoid this obvious case.
*/
SpinLockAcquire(&cv->mutex);
- if (!proclist_contains(&cv->wakeup, MyProc->pgprocno, cvWaitLink))
+ if (!proclist_contains(&cv->wakeup, GetNumberFromPGProc(MyProc), cvWaitLink))
{
done = true;
- proclist_push_tail(&cv->wakeup, MyProc->pgprocno, cvWaitLink);
+ proclist_push_tail(&cv->wakeup, GetNumberFromPGProc(MyProc), cvWaitLink);
}
SpinLockRelease(&cv->mutex);
@@ -236,8 +236,8 @@ ConditionVariableCancelSleep(void)
return false;
SpinLockAcquire(&cv->mutex);
- if (proclist_contains(&cv->wakeup, MyProc->pgprocno, cvWaitLink))
- proclist_delete(&cv->wakeup, MyProc->pgprocno, cvWaitLink);
+ if (proclist_contains(&cv->wakeup, GetNumberFromPGProc(MyProc), cvWaitLink))
+ proclist_delete(&cv->wakeup, GetNumberFromPGProc(MyProc), cvWaitLink);
else
signaled = true;
SpinLockRelease(&cv->mutex);
@@ -281,7 +281,7 @@ ConditionVariableSignal(ConditionVariable *cv)
void
ConditionVariableBroadcast(ConditionVariable *cv)
{
- int pgprocno = MyProc->pgprocno;
+ int pgprocno = GetNumberFromPGProc(MyProc);
PGPROC *proc = NULL;
bool have_sentinel = false;
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index 98fa6035cc5..9b69e9f1e32 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -1056,9 +1056,9 @@ LWLockQueueSelf(LWLock *lock, LWLockMode mode)
/* LW_WAIT_UNTIL_FREE waiters are always at the front of the queue */
if (mode == LW_WAIT_UNTIL_FREE)
- proclist_push_head(&lock->waiters, MyProc->pgprocno, lwWaitLink);
+ proclist_push_head(&lock->waiters, GetNumberFromPGProc(MyProc), lwWaitLink);
else
- proclist_push_tail(&lock->waiters, MyProc->pgprocno, lwWaitLink);
+ proclist_push_tail(&lock->waiters, GetNumberFromPGProc(MyProc), lwWaitLink);
/* Can release the mutex now */
LWLockWaitListUnlock(lock);
@@ -1097,7 +1097,7 @@ LWLockDequeueSelf(LWLock *lock)
*/
on_waitlist = MyProc->lwWaiting == LW_WS_WAITING;
if (on_waitlist)
- proclist_delete(&lock->waiters, MyProc->pgprocno, lwWaitLink);
+ proclist_delete(&lock->waiters, GetNumberFromPGProc(MyProc), lwWaitLink);
if (proclist_is_empty(&lock->waiters) &&
(pg_atomic_read_u32(&lock->state) & LW_FLAG_HAS_WAITERS) != 0)
diff --git a/src/backend/storage/lmgr/predicate.c b/src/backend/storage/lmgr/predicate.c
index ee5ea1175c7..93e40ae22f0 100644
--- a/src/backend/storage/lmgr/predicate.c
+++ b/src/backend/storage/lmgr/predicate.c
@@ -1811,7 +1811,7 @@ GetSerializableTransactionSnapshotInt(Snapshot snapshot,
sxact->finishedBefore = InvalidTransactionId;
sxact->xmin = snapshot->xmin;
sxact->pid = MyProcPid;
- sxact->pgprocno = MyProc->pgprocno;
+ sxact->pgprocno = GetNumberFromPGProc(MyProc);
dlist_init(&sxact->predicateLocks);
dlist_node_init(&sxact->finishedLink);
sxact->flags = 0;
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index 4ad96beb87a..d9deef3a9bc 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -227,7 +227,6 @@ InitProcGlobal(void)
InitSharedLatch(&(proc->procLatch));
LWLockInitialize(&(proc->fpInfoLock), LWTRANCHE_LOCK_FASTPATH);
}
- proc->pgprocno = i;
/*
* Newly created PGPROCs for normal backends, autovacuum and bgworkers
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index 00679624f7d..ed6071f3286 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -540,7 +540,7 @@ typedef enum
* used for a given lock group is determined by the group leader's pgprocno.
*/
#define LockHashPartitionLockByProc(leader_pgproc) \
- LockHashPartitionLock((leader_pgproc)->pgprocno)
+ LockHashPartitionLock(GetNumberFromPGProc(leader_pgproc))
/*
* function prototypes
diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h
index 4bc226e36cd..37cf8b4067d 100644
--- a/src/include/storage/proc.h
+++ b/src/include/storage/proc.h
@@ -194,11 +194,6 @@ struct PGPROC
int pgxactoff; /* offset into various ProcGlobal->arrays with
* data mirrored from this PGPROC */
- int pgprocno; /* Number of this PGPROC in
- * ProcGlobal->allProcs array. This is set
- * once by InitProcGlobal().
- * ProcGlobal->allProcs[n].pgprocno == n */
-
/* These fields are zero while a backend is still starting up: */
BackendId backendId; /* This backend's backend ID (if assigned) */
Oid databaseId; /* OID of database this backend is using */
@@ -412,6 +407,7 @@ extern PGDLLIMPORT PGPROC *PreparedXactProcs;
/* Accessor for PGPROC given a pgprocno. */
#define GetPGProcByNumber(n) (&ProcGlobal->allProcs[(n)])
+#define GetNumberFromPGProc(proc) ((proc) - &ProcGlobal->allProcs[0])
/*
* We set aside some extra PGPROC structures for auxiliary processes,
--
2.39.2
v7-0002-Redefine-backend-ID-to-be-an-index-into-the-proc-.patchtext/x-patch; charset=UTF-8; name=v7-0002-Redefine-backend-ID-to-be-an-index-into-the-proc-.patchDownload
From f6b11c24bf28b3573eaab773531bf7c0a705b0c6 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Thu, 25 Jan 2024 00:49:17 +0200
Subject: [PATCH v7 2/4] Redefine backend ID to be an index into the proc
array.
Previously, backend ID was an index into the ProcState array, in the
shared cache invalidation manager (sinvaladt.c). The entry in the
ProcState array was reserved at backend startup by scanning the array
for a free entry, and that was also when the backend got its backend
ID. Things becomes slightly simpler if we redefine backend ID to be
the index into the PGPROC array, and directly use it also as an index
to the ProcState array. This uses a little more memory, as we reserve
a few extra slots in the ProcState array for aux processes that don't
need them, but the simplicity is worth it.
Aux processes now also have a backend ID. This simplifies the
reservation of BackendStatusArray and ProcSignal slots.
You can now convert a backend ID into an index into the PGPROC array
simply by subtracting 1. We still use 0-based "pgprocnos" in various
places, for indexes into the PGPROC array, but the only difference now
is that backend IDs start at 1 while pgprocnos start at 0.
One potential downside of this patch is that the ProcState array might
get less densely packed, as we we don't try so hard to assign
low-numbered backend ID anymore. If it's less densely packed,
lastBackend will stay at a higher value, and SIInsertDataEntries() and
SICleanupQueue() need to scan over more unused entries. I think that's
fine. They are performance critical enough to matter, and there was no
guarantee on dense packing before either: If you launched a lot of
backends concurrently, and kept the last one open, lastBackend would
also stay at a high value.
Another potential downside of not assigning low backend IDs as
aggressively is that you might end up with more pg_temp namespaces, if
more distinct backend IDs are used. I think that's fine, and there
hasn't been any guarantees on that either. But if those things matter,
we could work on the management of free PGPROC entries in proc.c, to
assign PGPROC entries and hence backend IDs differently.
Reviewed-by: XXX
Discussion: https://www.postgresql.org/message-id/8171f1aa-496f-46a6-afc3-c46fe7a9b407@iki.fi
---
src/backend/access/transam/twophase.c | 29 +---
src/backend/access/transam/xact.c | 3 +-
src/backend/catalog/namespace.c | 2 +-
src/backend/postmaster/auxprocess.c | 12 +-
src/backend/storage/ipc/procarray.c | 60 +++++++-
src/backend/storage/ipc/procsignal.c | 27 ++--
src/backend/storage/ipc/sinvaladt.c | 145 ++++----------------
src/backend/storage/lmgr/lock.c | 16 +--
src/backend/storage/lmgr/proc.c | 14 +-
src/backend/utils/activity/backend_status.c | 52 +++----
src/backend/utils/adt/mcxtfuncs.c | 2 +-
src/backend/utils/error/csvlog.c | 4 +-
src/backend/utils/error/elog.c | 6 +-
src/backend/utils/error/jsonlog.c | 4 +-
src/backend/utils/init/postinit.c | 10 +-
src/backend/utils/time/snapmgr.c | 4 +-
src/include/miscadmin.h | 2 -
src/include/storage/backendid.h | 12 +-
src/include/storage/lock.h | 2 +-
src/include/storage/proc.h | 10 +-
src/include/storage/procarray.h | 4 +
src/include/storage/procsignal.h | 2 +-
src/include/storage/sinvaladt.h | 4 -
23 files changed, 170 insertions(+), 256 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 234c8d08ebc..0b6dd3b3cf0 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -151,7 +151,6 @@ typedef struct GlobalTransactionData
{
GlobalTransaction next; /* list link for free list */
int pgprocno; /* ID of associated dummy PGPROC */
- BackendId dummyBackendId; /* similar to backend id for backends */
TimestampTz prepared_at; /* time of preparation */
/*
@@ -285,20 +284,6 @@ TwoPhaseShmemInit(void)
/* associate it with a PGPROC assigned by InitProcGlobal */
gxacts[i].pgprocno = GetNumberFromPGProc(&PreparedXactProcs[i]);
-
- /*
- * Assign a unique ID for each dummy proc, so that the range of
- * dummy backend IDs immediately follows the range of normal
- * backend IDs. We don't dare to assign a real backend ID to dummy
- * procs, because prepared transactions don't take part in cache
- * invalidation like a real backend ID would imply, but having a
- * unique ID for them is nevertheless handy. This arrangement
- * allows you to allocate an array of size (MaxBackends +
- * max_prepared_xacts + 1), and have a slot for every backend and
- * prepared transaction. Currently multixact.c uses that
- * technique.
- */
- gxacts[i].dummyBackendId = MaxBackends + 1 + i;
}
}
else
@@ -457,7 +442,7 @@ MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid, const char *gid,
Assert(LWLockHeldByMeInMode(TwoPhaseStateLock, LW_EXCLUSIVE));
Assert(gxact != NULL);
- proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ proc = GetPGProcByNumber(gxact->pgprocno);
/* Initialize the PGPROC entry */
MemSet(proc, 0, sizeof(PGPROC));
@@ -467,14 +452,12 @@ MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid, const char *gid,
{
/* clone VXID, for TwoPhaseGetXidByVirtualXID() to find */
proc->lxid = MyProc->lxid;
- proc->backendId = MyBackendId;
}
else
{
Assert(AmStartupProcess() || !IsPostmasterEnvironment);
/* GetLockConflicts() uses this to specify a wait on the XID */
proc->lxid = xid;
- proc->backendId = InvalidBackendId;
}
proc->xid = xid;
Assert(proc->xmin == InvalidTransactionId);
@@ -522,7 +505,7 @@ static void
GXactLoadSubxactData(GlobalTransaction gxact, int nsubxacts,
TransactionId *children)
{
- PGPROC *proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ PGPROC *proc = GetPGProcByNumber(gxact->pgprocno);
/* We need no extra lock since the GXACT isn't valid yet */
if (nsubxacts > PGPROC_MAX_CACHED_SUBXIDS)
@@ -559,7 +542,7 @@ MarkAsPrepared(GlobalTransaction gxact, bool lock_held)
* Put it into the global ProcArray so TransactionIdIsInProgress considers
* the XID as still running.
*/
- ProcArrayAdd(&ProcGlobal->allProcs[gxact->pgprocno]);
+ ProcArrayAdd(GetPGProcByNumber(gxact->pgprocno));
}
/*
@@ -583,7 +566,7 @@ LockGXact(const char *gid, Oid user)
for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
{
GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
- PGPROC *proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ PGPROC *proc = GetPGProcByNumber(gxact->pgprocno);
/* Ignore not-yet-valid GIDs */
if (!gxact->valid)
@@ -884,7 +867,7 @@ TwoPhaseGetXidByVirtualXID(VirtualTransactionId vxid,
if (!gxact->valid)
continue;
- proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ proc = GetPGProcByNumber(gxact->pgprocno);
GET_VXID_FROM_PGPROC(proc_vxid, *proc);
if (VirtualTransactionIdEquals(vxid, proc_vxid))
{
@@ -919,7 +902,7 @@ TwoPhaseGetDummyBackendId(TransactionId xid, bool lock_held)
{
GlobalTransaction gxact = TwoPhaseGetGXact(xid, lock_held);
- return gxact->dummyBackendId;
+ return gxact->pgprocno + 1;
}
/*
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 464858117e0..6c5c6147f13 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -2097,9 +2097,8 @@ StartTransaction(void)
/*
* Advertise it in the proc array. We assume assignment of
- * localTransactionId is atomic, and the backendId should be set already.
+ * localTransactionId is atomic.
*/
- Assert(MyProc->backendId == vxid.backendId);
MyProc->lxid = vxid.localTransactionId;
TRACE_POSTGRESQL_TRANSACTION_START(vxid.localTransactionId);
diff --git a/src/backend/catalog/namespace.c b/src/backend/catalog/namespace.c
index b610aa62423..e6eea92abd6 100644
--- a/src/backend/catalog/namespace.c
+++ b/src/backend/catalog/namespace.c
@@ -49,7 +49,7 @@
#include "parser/parse_func.h"
#include "storage/ipc.h"
#include "storage/lmgr.h"
-#include "storage/sinvaladt.h"
+#include "storage/procarray.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
diff --git a/src/backend/postmaster/auxprocess.c b/src/backend/postmaster/auxprocess.c
index ab86e802f21..39171fea06b 100644
--- a/src/backend/postmaster/auxprocess.c
+++ b/src/backend/postmaster/auxprocess.c
@@ -107,17 +107,7 @@ AuxiliaryProcessMain(AuxProcType auxtype)
BaseInit();
- /*
- * Assign the ProcSignalSlot for an auxiliary process. Since it doesn't
- * have a BackendId, the slot is statically allocated based on the
- * auxiliary process type (MyAuxProcType). Backends use slots indexed in
- * the range from 1 to MaxBackends (inclusive), so we use MaxBackends +
- * AuxProcType + 1 as the index of the slot for an auxiliary process.
- *
- * This will need rethinking if we ever want more than one of a particular
- * auxiliary process type.
- */
- ProcSignalInit(MaxBackends + MyAuxProcType + 1);
+ ProcSignalInit();
/*
* Auxiliary processes don't run transactions, but they may need a
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 5a33eb7f7f7..cd127a5357b 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -2546,7 +2546,7 @@ ProcArrayInstallImportedXmin(TransactionId xmin,
continue;
/* We are only interested in the specific virtual transaction. */
- if (proc->backendId != sourcevxid->backendId)
+ if (GetBackendIdFromPGProc(proc) != sourcevxid->backendId)
continue;
if (proc->lxid != sourcevxid->localTransactionId)
continue;
@@ -3097,6 +3097,64 @@ HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids, int type)
return result;
}
+/*
+ * BackendIdGetProc -- get a backend's PGPROC given its backend ID
+ *
+ * The result may be out of date arbitrarily quickly, so the caller
+ * must be careful about how this information is used. NULL is
+ * returned if the backend is not active.
+ */
+PGPROC *
+BackendIdGetProc(int backendID)
+{
+ PGPROC *result;
+
+ if (backendID < 1 || backendID > ProcGlobal->allProcCount)
+ return NULL;
+ result = GetPGProcByBackendId(backendID);
+
+ if (result->pid == 0)
+ return NULL;
+
+ return result;
+}
+
+/*
+ * BackendIdGetTransactionIds -- get a backend's transaction status
+ *
+ * Get the xid, xmin, nsubxid and overflow status of the backend. The
+ * result may be out of date arbitrarily quickly, so the caller must be
+ * careful about how this information is used.
+ */
+void
+BackendIdGetTransactionIds(int backendID, TransactionId *xid,
+ TransactionId *xmin, int *nsubxid, bool *overflowed)
+{
+ PGPROC *proc;
+
+ *xid = InvalidTransactionId;
+ *xmin = InvalidTransactionId;
+ *nsubxid = 0;
+ *overflowed = false;
+
+ if (backendID < 1 || backendID > ProcGlobal->allProcCount)
+ return;
+ proc = GetPGProcByBackendId(backendID);
+
+ /* Need to lock out additions/removals of backends */
+ LWLockAcquire(ProcArrayLock, LW_SHARED);
+
+ if (proc->pid != 0)
+ {
+ *xid = proc->xid;
+ *xmin = proc->xmin;
+ *nsubxid = proc->subxidStatus.count;
+ *overflowed = proc->subxidStatus.overflowed;
+ }
+
+ LWLockRelease(ProcArrayLock);
+}
+
/*
* BackendPidGetProc -- get a backend's PGPROC given its PID
*
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index e84619e5a58..d1d5bf0c152 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -87,7 +87,7 @@ typedef struct
* possible auxiliary process type. (This scheme assumes there is not
* more than one of any auxiliary process type at a time.)
*/
-#define NumProcSignalSlots (MaxBackends + NUM_AUXPROCTYPES)
+#define NumProcSignalSlots (MaxBackends + NUM_AUXILIARY_PROCS)
/* Check whether the relevant type bit is set in the flags. */
#define BARRIER_SHOULD_CHECK(flags, type) \
@@ -154,24 +154,23 @@ ProcSignalShmemInit(void)
/*
* ProcSignalInit
* Register the current process in the ProcSignal array
- *
- * The passed index should be my BackendId if the process has one,
- * or MaxBackends + aux process type if not.
*/
void
-ProcSignalInit(int pss_idx)
+ProcSignalInit(void)
{
ProcSignalSlot *slot;
uint64 barrier_generation;
- Assert(pss_idx >= 1 && pss_idx <= NumProcSignalSlots);
-
- slot = &ProcSignal->psh_slot[pss_idx - 1];
+ if (MyBackendId <= 0)
+ elog(ERROR, "MyBackendId not set");
+ if (MyBackendId > NumProcSignalSlots)
+ elog(ERROR, "unexpected MyBackendId %d in ProcSignalInit (max %d)", MyBackendId, NumProcSignalSlots);
+ slot = &ProcSignal->psh_slot[MyBackendId - 1];
/* sanity check */
if (slot->pss_pid != 0)
elog(LOG, "process %d taking over ProcSignal slot %d, but it's not empty",
- MyProcPid, pss_idx);
+ MyProcPid, (int) (slot - ProcSignal->psh_slot));
/* Clear out any leftover signal reasons */
MemSet(slot->pss_signalFlags, 0, NUM_PROCSIGNALS * sizeof(sig_atomic_t));
@@ -200,7 +199,7 @@ ProcSignalInit(int pss_idx)
MyProcSignalSlot = slot;
/* Set up to release the slot on process exit */
- on_shmem_exit(CleanupProcSignalState, Int32GetDatum(pss_idx));
+ on_shmem_exit(CleanupProcSignalState, (Datum) 0);
}
/*
@@ -212,11 +211,7 @@ ProcSignalInit(int pss_idx)
static void
CleanupProcSignalState(int status, Datum arg)
{
- int pss_idx = DatumGetInt32(arg);
- ProcSignalSlot *slot;
-
- slot = &ProcSignal->psh_slot[pss_idx - 1];
- Assert(slot == MyProcSignalSlot);
+ ProcSignalSlot *slot = MyProcSignalSlot;
/*
* Clear MyProcSignalSlot, so that a SIGUSR1 received after this point
@@ -233,7 +228,7 @@ CleanupProcSignalState(int status, Datum arg)
* infinite loop trying to exit
*/
elog(LOG, "process %d releasing ProcSignal slot %d, but it contains %d",
- MyProcPid, pss_idx, (int) slot->pss_pid);
+ MyProcPid, (int) (slot - ProcSignal->psh_slot), (int) slot->pss_pid);
return; /* XXX better to zero the slot anyway? */
}
diff --git a/src/backend/storage/ipc/sinvaladt.c b/src/backend/storage/ipc/sinvaladt.c
index 748a792a854..8105717c578 100644
--- a/src/backend/storage/ipc/sinvaladt.c
+++ b/src/backend/storage/ipc/sinvaladt.c
@@ -139,7 +139,6 @@ typedef struct ProcState
{
/* procPid is zero in an inactive ProcState array entry. */
pid_t procPid; /* PID of backend, for signaling */
- PGPROC *proc; /* PGPROC of backend */
/* nextMsgNum is meaningless if procPid == 0 or resetState is true. */
int nextMsgNum; /* next message number to read */
bool resetState; /* backend needs to reset its state */
@@ -173,7 +172,6 @@ typedef struct SISeg
int maxMsgNum; /* next message number to be assigned */
int nextThreshold; /* # of messages to call SICleanupQueue */
int lastBackend; /* index of last active procState entry, +1 */
- int maxBackends; /* size of procState array */
slock_t msgnumLock; /* spinlock protecting maxMsgNum */
@@ -183,11 +181,18 @@ typedef struct SISeg
SharedInvalidationMessage buffer[MAXNUMMESSAGES];
/*
- * Per-backend invalidation state info (has MaxBackends entries).
+ * Per-backend invalidation state info (has NumProcStateSlots entries).
*/
ProcState procState[FLEXIBLE_ARRAY_MEMBER];
} SISeg;
+/*
+ * We reserve a slot for each possible BackendId, plus one for each
+ * possible auxiliary process type. (This scheme assumes there is not
+ * more than one of any auxiliary process type at a time.)
+ */
+#define NumProcStateSlots (MaxBackends + NUM_AUXILIARY_PROCS)
+
static SISeg *shmInvalBuffer; /* pointer to the shared inval buffer */
@@ -205,16 +210,7 @@ SInvalShmemSize(void)
Size size;
size = offsetof(SISeg, procState);
-
- /*
- * In Hot Standby mode, the startup process requests a procState array
- * slot using InitRecoveryTransactionEnvironment(). Even though
- * MaxBackends doesn't account for the startup process, it is guaranteed
- * to get a free slot. This is because the autovacuum launcher and worker
- * processes, which are included in MaxBackends, are not started in Hot
- * Standby mode.
- */
- size = add_size(size, mul_size(sizeof(ProcState), MaxBackends));
+ size = add_size(size, mul_size(sizeof(ProcState), NumProcStateSlots));
return size;
}
@@ -240,16 +236,14 @@ CreateSharedInvalidationState(void)
shmInvalBuffer->maxMsgNum = 0;
shmInvalBuffer->nextThreshold = CLEANUP_MIN;
shmInvalBuffer->lastBackend = 0;
- shmInvalBuffer->maxBackends = MaxBackends;
SpinLockInit(&shmInvalBuffer->msgnumLock);
/* The buffer[] array is initially all unused, so we need not fill it */
/* Mark all backends inactive, and initialize nextLXID */
- for (i = 0; i < shmInvalBuffer->maxBackends; i++)
+ for (i = 0; i < NumProcStateSlots; i++)
{
shmInvalBuffer->procState[i].procPid = 0; /* inactive */
- shmInvalBuffer->procState[i].proc = NULL;
shmInvalBuffer->procState[i].nextMsgNum = 0; /* meaningless */
shmInvalBuffer->procState[i].resetState = false;
shmInvalBuffer->procState[i].signaled = false;
@@ -265,10 +259,17 @@ CreateSharedInvalidationState(void)
void
SharedInvalBackendInit(bool sendOnly)
{
- int index;
- ProcState *stateP = NULL;
+ ProcState *stateP;
+ pid_t oldPid;
SISeg *segP = shmInvalBuffer;
+ if (MyBackendId <= 0)
+ elog(ERROR, "MyBackendId not set");
+ if (MyBackendId > NumProcStateSlots)
+ elog(PANIC, "unexpected MyBackendId %d in SharedInvalBackendInit (max %d)",
+ MyBackendId, NumProcStateSlots);
+ stateP = &segP->procState[MyBackendId - 1];
+
/*
* This can run in parallel with read operations, but not with write
* operations, since SIInsertDataEntries relies on lastBackend to set
@@ -276,48 +277,22 @@ SharedInvalBackendInit(bool sendOnly)
*/
LWLockAcquire(SInvalWriteLock, LW_EXCLUSIVE);
- /* Look for a free entry in the procState array */
- for (index = 0; index < segP->lastBackend; index++)
- {
- if (segP->procState[index].procPid == 0) /* inactive slot? */
- {
- stateP = &segP->procState[index];
- break;
- }
- }
-
- if (stateP == NULL)
+ oldPid = stateP->procPid;
+ if (oldPid != 0)
{
- if (segP->lastBackend < segP->maxBackends)
- {
- stateP = &segP->procState[segP->lastBackend];
- Assert(stateP->procPid == 0);
- segP->lastBackend++;
- }
- else
- {
- /*
- * out of procState slots: MaxBackends exceeded -- report normally
- */
- MyBackendId = InvalidBackendId;
- LWLockRelease(SInvalWriteLock);
- ereport(FATAL,
- (errcode(ERRCODE_TOO_MANY_CONNECTIONS),
- errmsg("sorry, too many clients already")));
- }
+ LWLockRelease(SInvalWriteLock);
+ elog(ERROR, "sinval slot for backend %d is already in use by process %d",
+ MyBackendId, oldPid);
}
- MyBackendId = (stateP - &segP->procState[0]) + 1;
-
- /* Advertise assigned backend ID in MyProc */
- MyProc->backendId = MyBackendId;
+ if (MyBackendId > segP->lastBackend)
+ segP->lastBackend = MyBackendId;
/* Fetch next local transaction ID into local memory */
nextLocalTransactionId = stateP->nextLXID;
/* mark myself active, with all extant messages already read */
stateP->procPid = MyProcPid;
- stateP->proc = MyProc;
stateP->nextMsgNum = segP->maxMsgNum;
stateP->resetState = false;
stateP->signaled = false;
@@ -328,8 +303,6 @@ SharedInvalBackendInit(bool sendOnly)
/* register exit routine to mark my entry inactive at exit */
on_shmem_exit(CleanupInvalidationState, PointerGetDatum(segP));
-
- elog(DEBUG4, "my backend ID is %d", MyBackendId);
}
/*
@@ -358,7 +331,6 @@ CleanupInvalidationState(int status, Datum arg)
/* Mark myself inactive */
stateP->procPid = 0;
- stateP->proc = NULL;
stateP->nextMsgNum = 0;
stateP->resetState = false;
stateP->signaled = false;
@@ -374,71 +346,6 @@ CleanupInvalidationState(int status, Datum arg)
LWLockRelease(SInvalWriteLock);
}
-/*
- * BackendIdGetProc
- * Get the PGPROC structure for a backend, given the backend ID.
- * The result may be out of date arbitrarily quickly, so the caller
- * must be careful about how this information is used. NULL is
- * returned if the backend is not active.
- */
-PGPROC *
-BackendIdGetProc(int backendID)
-{
- PGPROC *result = NULL;
- SISeg *segP = shmInvalBuffer;
-
- /* Need to lock out additions/removals of backends */
- LWLockAcquire(SInvalWriteLock, LW_SHARED);
-
- if (backendID > 0 && backendID <= segP->lastBackend)
- {
- ProcState *stateP = &segP->procState[backendID - 1];
-
- result = stateP->proc;
- }
-
- LWLockRelease(SInvalWriteLock);
-
- return result;
-}
-
-/*
- * BackendIdGetTransactionIds
- * Get the xid, xmin, nsubxid and overflow status of the backend. The
- * result may be out of date arbitrarily quickly, so the caller must be
- * careful about how this information is used.
- */
-void
-BackendIdGetTransactionIds(int backendID, TransactionId *xid,
- TransactionId *xmin, int *nsubxid, bool *overflowed)
-{
- SISeg *segP = shmInvalBuffer;
-
- *xid = InvalidTransactionId;
- *xmin = InvalidTransactionId;
- *nsubxid = 0;
- *overflowed = false;
-
- /* Need to lock out additions/removals of backends */
- LWLockAcquire(SInvalWriteLock, LW_SHARED);
-
- if (backendID > 0 && backendID <= segP->lastBackend)
- {
- ProcState *stateP = &segP->procState[backendID - 1];
- PGPROC *proc = stateP->proc;
-
- if (proc != NULL)
- {
- *xid = proc->xid;
- *xmin = proc->xmin;
- *nsubxid = proc->subxidStatus.count;
- *overflowed = proc->subxidStatus.overflowed;
- }
- }
-
- LWLockRelease(SInvalWriteLock);
-}
-
/*
* SIInsertDataEntries
* Add new invalidation message(s) to the buffer.
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index c70a1adb9ad..0252cacf0a7 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -3625,7 +3625,7 @@ GetLockStatusData(void)
proc->fpRelId[f]);
instance->holdMask = lockbits << FAST_PATH_LOCKNUMBER_OFFSET;
instance->waitLockMode = NoLock;
- instance->backend = proc->backendId;
+ instance->backend = GetBackendIdFromPGProc(proc);
instance->lxid = proc->lxid;
instance->pid = proc->pid;
instance->leaderPid = proc->pid;
@@ -3652,14 +3652,14 @@ GetLockStatusData(void)
repalloc(data->locks, sizeof(LockInstanceData) * els);
}
- vxid.backendId = proc->backendId;
+ vxid.backendId = GetBackendIdFromPGProc(proc);
vxid.localTransactionId = proc->fpLocalTransactionId;
instance = &data->locks[el];
SET_LOCKTAG_VIRTUALTRANSACTION(instance->locktag, vxid);
instance->holdMask = LOCKBIT_ON(ExclusiveLock);
instance->waitLockMode = NoLock;
- instance->backend = proc->backendId;
+ instance->backend = GetBackendIdFromPGProc(proc);
instance->lxid = proc->lxid;
instance->pid = proc->pid;
instance->leaderPid = proc->pid;
@@ -3712,7 +3712,7 @@ GetLockStatusData(void)
instance->waitLockMode = proc->waitLockMode;
else
instance->waitLockMode = NoLock;
- instance->backend = proc->backendId;
+ instance->backend = GetBackendIdFromPGProc(proc);
instance->lxid = proc->lxid;
instance->pid = proc->pid;
instance->leaderPid = proclock->groupLeader->pid;
@@ -3888,7 +3888,7 @@ GetSingleProcBlockerStatusData(PGPROC *blocked_proc, BlockedProcsData *data)
instance->waitLockMode = proc->waitLockMode;
else
instance->waitLockMode = NoLock;
- instance->backend = proc->backendId;
+ instance->backend = GetBackendIdFromPGProc(proc);
instance->lxid = proc->lxid;
instance->pid = proc->pid;
instance->leaderPid = proclock->groupLeader->pid;
@@ -4391,7 +4391,7 @@ VirtualXactLockTableInsert(VirtualTransactionId vxid)
LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE);
- Assert(MyProc->backendId == vxid.backendId);
+ Assert(MyBackendId == vxid.backendId);
Assert(MyProc->fpLocalTransactionId == InvalidLocalTransactionId);
Assert(MyProc->fpVXIDLock == false);
@@ -4413,8 +4413,6 @@ VirtualXactLockTableCleanup(void)
bool fastpath;
LocalTransactionId lxid;
- Assert(MyProc->backendId != InvalidBackendId);
-
/*
* Clean up shared memory state.
*/
@@ -4541,7 +4539,7 @@ VirtualXactLock(VirtualTransactionId vxid, bool wait)
*/
LWLockAcquire(&proc->fpInfoLock, LW_EXCLUSIVE);
- if (proc->backendId != vxid.backendId
+ if (GetBackendIdFromPGProc(proc) != vxid.backendId
|| proc->fpLocalTransactionId != vxid.localTransactionId)
{
/* VXID ended */
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index d9deef3a9bc..ac5096f45aa 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -239,25 +239,25 @@ InitProcGlobal(void)
if (i < MaxConnections)
{
/* PGPROC for normal backend, add to freeProcs list */
- dlist_push_head(&ProcGlobal->freeProcs, &proc->links);
+ dlist_push_tail(&ProcGlobal->freeProcs, &proc->links);
proc->procgloballist = &ProcGlobal->freeProcs;
}
else if (i < MaxConnections + autovacuum_max_workers + 1)
{
/* PGPROC for AV launcher/worker, add to autovacFreeProcs list */
- dlist_push_head(&ProcGlobal->autovacFreeProcs, &proc->links);
+ dlist_push_tail(&ProcGlobal->autovacFreeProcs, &proc->links);
proc->procgloballist = &ProcGlobal->autovacFreeProcs;
}
else if (i < MaxConnections + autovacuum_max_workers + 1 + max_worker_processes)
{
/* PGPROC for bgworker, add to bgworkerFreeProcs list */
- dlist_push_head(&ProcGlobal->bgworkerFreeProcs, &proc->links);
+ dlist_push_tail(&ProcGlobal->bgworkerFreeProcs, &proc->links);
proc->procgloballist = &ProcGlobal->bgworkerFreeProcs;
}
else if (i < MaxBackends)
{
/* PGPROC for walsender, add to walsenderFreeProcs list */
- dlist_push_head(&ProcGlobal->walsenderFreeProcs, &proc->links);
+ dlist_push_tail(&ProcGlobal->walsenderFreeProcs, &proc->links);
proc->procgloballist = &ProcGlobal->walsenderFreeProcs;
}
@@ -351,6 +351,7 @@ InitProcess(void)
(errcode(ERRCODE_TOO_MANY_CONNECTIONS),
errmsg("sorry, too many clients already")));
}
+ MyBackendId = GetBackendIdFromPGProc(MyProc);
/*
* Cross-check that the PGPROC is of the type we expect; if this were not
@@ -380,7 +381,6 @@ InitProcess(void)
MyProc->xmin = InvalidTransactionId;
MyProc->pid = MyProcPid;
/* backendId, databaseId and roleId will be filled in later */
- MyProc->backendId = InvalidBackendId;
MyProc->databaseId = InvalidOid;
MyProc->roleId = InvalidOid;
MyProc->tempNamespaceId = InvalidOid;
@@ -561,6 +561,7 @@ InitAuxiliaryProcess(void)
((volatile PGPROC *) auxproc)->pid = MyProcPid;
MyProc = auxproc;
+ MyBackendId = GetBackendIdFromPGProc(MyProc);
SpinLockRelease(ProcStructLock);
@@ -575,7 +576,6 @@ InitAuxiliaryProcess(void)
MyProc->fpLocalTransactionId = InvalidLocalTransactionId;
MyProc->xid = InvalidTransactionId;
MyProc->xmin = InvalidTransactionId;
- MyProc->backendId = InvalidBackendId;
MyProc->databaseId = InvalidOid;
MyProc->roleId = InvalidOid;
MyProc->tempNamespaceId = InvalidOid;
@@ -905,6 +905,7 @@ ProcKill(int code, Datum arg)
proc = MyProc;
MyProc = NULL;
+ MyBackendId = InvalidBackendId;
DisownLatch(&proc->procLatch);
procgloballist = proc->procgloballist;
@@ -976,6 +977,7 @@ AuxiliaryProcKill(int code, Datum arg)
proc = MyProc;
MyProc = NULL;
+ MyBackendId = InvalidBackendId;
DisownLatch(&proc->procLatch);
SpinLockAcquire(ProcStructLock);
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 1a1050c8da1..3d3f7b06723 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -19,6 +19,7 @@
#include "port/atomics.h" /* for memory barriers */
#include "storage/ipc.h"
#include "storage/proc.h" /* for MyProc */
+#include "storage/procarray.h"
#include "storage/sinvaladt.h"
#include "utils/ascii.h"
#include "utils/backend_status.h"
@@ -29,13 +30,12 @@
/* ----------
* Total number of backends including auxiliary
*
- * We reserve a slot for each possible BackendId, plus one for each
- * possible auxiliary process type. (This scheme assumes there is not
- * more than one of any auxiliary process type at a time.) MaxBackends
- * includes autovacuum workers and background workers as well.
+ * We reserve a slot for each possible PGPROC entry, including aux processes.
+ * (But not including PGPROC entries reserved for prepared xacts; they are not
+ * real processes.)
* ----------
*/
-#define NumBackendStatSlots (MaxBackends + NUM_AUXPROCTYPES)
+#define NumBackendStatSlots (MaxBackends + NUM_AUXILIARY_PROCS)
/* ----------
@@ -238,10 +238,9 @@ CreateSharedBackendStatus(void)
/*
* Initialize pgstats backend activity state, and set up our on-proc-exit
- * hook. Called from InitPostgres and AuxiliaryProcessMain. For auxiliary
- * process, MyBackendId is invalid. Otherwise, MyBackendId must be set, but we
- * must not have started any transaction yet (since the exit hook must run
- * after the last transaction exit).
+ * hook. Called from InitPostgres and AuxiliaryProcessMain. MyBackendId must
+ * be set, but we must not have started any transaction yet (since the exit
+ * hook must run after the last transaction exit).
*
* NOTE: MyDatabaseId isn't set yet; so the shutdown hook has to be careful.
*/
@@ -249,26 +248,9 @@ void
pgstat_beinit(void)
{
/* Initialize MyBEEntry */
- if (MyBackendId != InvalidBackendId)
- {
- Assert(MyBackendId >= 1 && MyBackendId <= MaxBackends);
- MyBEEntry = &BackendStatusArray[MyBackendId - 1];
- }
- else
- {
- /* Must be an auxiliary process */
- Assert(MyAuxProcType != NotAnAuxProcess);
-
- /*
- * Assign the MyBEEntry for an auxiliary process. Since it doesn't
- * have a BackendId, the slot is statically allocated based on the
- * auxiliary process type (MyAuxProcType). Backends use slots indexed
- * in the range from 0 to MaxBackends (exclusive), so we use
- * MaxBackends + AuxProcType as the index of the slot for an auxiliary
- * process.
- */
- MyBEEntry = &BackendStatusArray[MaxBackends + MyAuxProcType];
- }
+ Assert(MyBackendId != InvalidBackendId);
+ Assert(MyBackendId >= 1 && MyBackendId <= NumBackendStatSlots);
+ MyBEEntry = &BackendStatusArray[MyBackendId - 1];
/* Set up a process-exit hook to clean up */
on_shmem_exit(pgstat_beshutdown_hook, 0);
@@ -281,12 +263,12 @@ pgstat_beinit(void)
* Initialize this backend's entry in the PgBackendStatus array.
* Called from InitPostgres.
*
- * Apart from auxiliary processes, MyBackendId, MyDatabaseId,
- * session userid, and application_name must be set for a
- * backend (hence, this cannot be combined with pgstat_beinit).
- * Note also that we must be inside a transaction if this isn't an aux
- * process, as we may need to do encoding conversion on some strings.
- * ----------
+ * Apart from auxiliary processes, MyDatabaseId, session userid, and
+ * application_name must already be set (hence, this cannot be combined
+ * with pgstat_beinit). Note also that we must be inside a transaction
+ * if this isn't an aux process, as we may need to do encoding conversion
+ * on some strings.
+ *----------
*/
void
pgstat_bestart(void)
diff --git a/src/backend/utils/adt/mcxtfuncs.c b/src/backend/utils/adt/mcxtfuncs.c
index 4708d73f5fa..c8c3b8dbaf1 100644
--- a/src/backend/utils/adt/mcxtfuncs.c
+++ b/src/backend/utils/adt/mcxtfuncs.c
@@ -159,7 +159,7 @@ pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
* have a valid backend id.
*/
if (proc != NULL)
- backendId = proc->backendId;
+ backendId = GetBackendIdFromPGProc(proc);
else
proc = AuxiliaryPidGetProc(pid);
diff --git a/src/backend/utils/error/csvlog.c b/src/backend/utils/error/csvlog.c
index 1b62b07f231..692ee3c1491 100644
--- a/src/backend/utils/error/csvlog.c
+++ b/src/backend/utils/error/csvlog.c
@@ -152,8 +152,8 @@ write_csvlog(ErrorData *edata)
/* Virtual transaction id */
/* keep VXID format in sync with lockfuncs.c */
- if (MyProc != NULL && MyProc->backendId != InvalidBackendId)
- appendStringInfo(&buf, "%d/%u", MyProc->backendId, MyProc->lxid);
+ if (MyProc != NULL)
+ appendStringInfo(&buf, "%d/%u", MyBackendId, MyProc->lxid);
appendStringInfoChar(&buf, ',');
/* Transaction id */
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 2c7a20e3d31..10249f140fe 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -3074,18 +3074,18 @@ log_status_format(StringInfo buf, const char *format, ErrorData *edata)
break;
case 'v':
/* keep VXID format in sync with lockfuncs.c */
- if (MyProc != NULL && MyProc->backendId != InvalidBackendId)
+ if (MyProc != NULL)
{
if (padding != 0)
{
char strfbuf[128];
snprintf(strfbuf, sizeof(strfbuf) - 1, "%d/%u",
- MyProc->backendId, MyProc->lxid);
+ MyBackendId, MyProc->lxid);
appendStringInfo(buf, "%*s", padding, strfbuf);
}
else
- appendStringInfo(buf, "%d/%u", MyProc->backendId, MyProc->lxid);
+ appendStringInfo(buf, "%d/%u", MyBackendId, MyProc->lxid);
}
else if (padding != 0)
appendStringInfoSpaces(buf,
diff --git a/src/backend/utils/error/jsonlog.c b/src/backend/utils/error/jsonlog.c
index 2903561f1c4..3ebe0b7b13c 100644
--- a/src/backend/utils/error/jsonlog.c
+++ b/src/backend/utils/error/jsonlog.c
@@ -197,8 +197,8 @@ write_jsonlog(ErrorData *edata)
/* Virtual transaction id */
/* keep VXID format in sync with lockfuncs.c */
- if (MyProc != NULL && MyProc->backendId != InvalidBackendId)
- appendJSONKeyValueFmt(&buf, "vxid", true, "%d/%u", MyProc->backendId,
+ if (MyProc != NULL)
+ appendJSONKeyValueFmt(&buf, "vxid", true, "%d/%u", MyBackendId,
MyProc->lxid);
/* Transaction id */
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index 1ad33671598..e822cd61d5f 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -740,18 +740,10 @@ InitPostgres(const char *in_dbname, Oid dboid,
/*
* Initialize my entry in the shared-invalidation manager's array of
* per-backend data.
- *
- * Sets up MyBackendId, a unique backend identifier.
*/
- MyBackendId = InvalidBackendId;
-
SharedInvalBackendInit(false);
- if (MyBackendId > MaxBackends || MyBackendId <= 0)
- elog(FATAL, "bad backend ID: %d", MyBackendId);
-
- /* Now that we have a BackendId, we can participate in ProcSignal */
- ProcSignalInit(MyBackendId);
+ ProcSignalInit();
/*
* Also set up timeout handlers needed for backend operation. We need
diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
index 675e81d82d7..94ddf68fa52 100644
--- a/src/backend/utils/time/snapmgr.c
+++ b/src/backend/utils/time/snapmgr.c
@@ -1154,7 +1154,7 @@ ExportSnapshot(Snapshot snapshot)
* inside the transaction from 1.
*/
snprintf(path, sizeof(path), SNAPSHOT_EXPORT_DIR "/%08X-%08X-%d",
- MyProc->backendId, MyProc->lxid, list_length(exportedSnapshots) + 1);
+ MyBackendId, MyProc->lxid, list_length(exportedSnapshots) + 1);
/*
* Copy the snapshot into TopTransactionContext, add it to the
@@ -1181,7 +1181,7 @@ ExportSnapshot(Snapshot snapshot)
*/
initStringInfo(&buf);
- appendStringInfo(&buf, "vxid:%d/%u\n", MyProc->backendId, MyProc->lxid);
+ appendStringInfo(&buf, "vxid:%d/%u\n", MyBackendId, MyProc->lxid);
appendStringInfo(&buf, "pid:%d\n", MyProcPid);
appendStringInfo(&buf, "dbid:%u\n", MyDatabaseId);
appendStringInfo(&buf, "iso:%d\n", XactIsoLevel);
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 0b01c1f0935..cbdc61b8576 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -444,8 +444,6 @@ typedef enum
WalWriterProcess,
WalReceiverProcess,
WalSummarizerProcess,
-
- NUM_AUXPROCTYPES /* Must be last! */
} AuxProcType;
extern PGDLLIMPORT AuxProcType MyAuxProcType;
diff --git a/src/include/storage/backendid.h b/src/include/storage/backendid.h
index 50ac982da19..01387723f79 100644
--- a/src/include/storage/backendid.h
+++ b/src/include/storage/backendid.h
@@ -14,11 +14,15 @@
#ifndef BACKENDID_H
#define BACKENDID_H
-/* ----------------
- * -cim 8/17/90
- * ----------------
+/*
+ * BackendId uniquely identifies an active backend or auxiliary process. It's
+ * assigned at backend startup after authentication. Note that a backend ID
+ * can be reused for a different backend immediately after a backend exits.
+ *
+ * Backend IDs are assigned starting from 1. For historical reasons, BackendId
+ * 0 is unused, but InvalidBackendId is defined as -1.
*/
-typedef int BackendId; /* unique currently active backend identifier */
+typedef int BackendId;
#define InvalidBackendId (-1)
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index ed6071f3286..03a920b2254 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -75,7 +75,7 @@ typedef struct
((vxid).backendId = InvalidBackendId, \
(vxid).localTransactionId = InvalidLocalTransactionId)
#define GET_VXID_FROM_PGPROC(vxid, proc) \
- ((vxid).backendId = (proc).backendId, \
+ ((vxid).backendId = GetBackendIdFromPGProc(&(proc)), \
(vxid).localTransactionId = (proc).lxid)
/* MAX_LOCKMODES cannot be larger than the # of bits in LOCKMASK */
diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h
index 37cf8b4067d..0d424a3334d 100644
--- a/src/include/storage/proc.h
+++ b/src/include/storage/proc.h
@@ -195,7 +195,6 @@ struct PGPROC
* data mirrored from this PGPROC */
/* These fields are zero while a backend is still starting up: */
- BackendId backendId; /* This backend's backend ID (if assigned) */
Oid databaseId; /* OID of database this backend is using */
Oid roleId; /* OID of role using this backend */
@@ -405,9 +404,16 @@ extern PGDLLIMPORT PROC_HDR *ProcGlobal;
extern PGDLLIMPORT PGPROC *PreparedXactProcs;
-/* Accessor for PGPROC given a pgprocno. */
+/*
+ * Accessors for getting PGPROC given a pgprocno or BackendId, and vice versa.
+ *
+ * For historical reasons, some code uses 0-based "proc numbers", while other
+ * code uses 1-based backend IDs.
+ */
#define GetPGProcByNumber(n) (&ProcGlobal->allProcs[(n)])
#define GetNumberFromPGProc(proc) ((proc) - &ProcGlobal->allProcs[0])
+#define GetPGProcByBackendId(n) (&ProcGlobal->allProcs[(n) - 1])
+#define GetBackendIdFromPGProc(proc) (GetNumberFromPGProc(proc) + 1)
/*
* We set aside some extra PGPROC structures for auxiliary processes,
diff --git a/src/include/storage/procarray.h b/src/include/storage/procarray.h
index f3eba9b7640..3af7577e8c6 100644
--- a/src/include/storage/procarray.h
+++ b/src/include/storage/procarray.h
@@ -64,6 +64,10 @@ extern VirtualTransactionId *GetVirtualXIDsDelayingChkpt(int *nvxids, int type);
extern bool HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids,
int nvxids, int type);
+extern PGPROC *BackendIdGetProc(int backendID);
+extern void BackendIdGetTransactionIds(int backendID, TransactionId *xid,
+ TransactionId *xmin, int *nsubxid,
+ bool *overflowed);
extern PGPROC *BackendPidGetProc(int pid);
extern PGPROC *BackendPidGetProcWithLock(int pid);
extern int BackendXidGetPid(TransactionId xid);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 52dcb4c2adf..febdda3611c 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -62,7 +62,7 @@ typedef enum
extern Size ProcSignalShmemSize(void);
extern void ProcSignalShmemInit(void);
-extern void ProcSignalInit(int pss_idx);
+extern void ProcSignalInit(void);
extern int SendProcSignal(pid_t pid, ProcSignalReason reason,
BackendId backendId);
diff --git a/src/include/storage/sinvaladt.h b/src/include/storage/sinvaladt.h
index aa3d203efca..c3c97b3f8b7 100644
--- a/src/include/storage/sinvaladt.h
+++ b/src/include/storage/sinvaladt.h
@@ -31,10 +31,6 @@
extern Size SInvalShmemSize(void);
extern void CreateSharedInvalidationState(void);
extern void SharedInvalBackendInit(bool sendOnly);
-extern PGPROC *BackendIdGetProc(int backendID);
-extern void BackendIdGetTransactionIds(int backendID, TransactionId *xid,
- TransactionId *xmin, int *nsubxid,
- bool *overflowed);
extern void SIInsertDataEntries(const SharedInvalidationMessage *data, int n);
extern int SIGetDataEntries(SharedInvalidationMessage *data, int datasize);
--
2.39.2
v7-0003-Remove-MyAuxProcType-use-MyBackendType-instead.patchtext/x-patch; charset=UTF-8; name=v7-0003-Remove-MyAuxProcType-use-MyBackendType-instead.patchDownload
From c087a94abc9ec9a795f8499d5f6fb524f59824a5 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 24 Jan 2024 22:52:50 +0200
Subject: [PATCH v7 3/4] Remove MyAuxProcType, use MyBackendType instead
MyAuxProcType was redundant with MyBackendType.
Reviewed-by: XXX
Discussion: XXX
---
src/backend/postmaster/auxprocess.c | 60 ++++++-------------------
src/backend/postmaster/postmaster.c | 36 +++++++--------
src/include/miscadmin.h | 68 ++++++++++++++---------------
src/include/postmaster/auxprocess.h | 2 +-
src/tools/pgindent/typedefs.list | 1 -
5 files changed, 64 insertions(+), 103 deletions(-)
diff --git a/src/backend/postmaster/auxprocess.c b/src/backend/postmaster/auxprocess.c
index 39171fea06b..5493a5f1ef1 100644
--- a/src/backend/postmaster/auxprocess.c
+++ b/src/backend/postmaster/auxprocess.c
@@ -38,14 +38,6 @@
static void ShutdownAuxiliaryProcess(int code, Datum arg);
-/* ----------------
- * global variables
- * ----------------
- */
-
-AuxProcType MyAuxProcType = NotAnAuxProcess; /* declared in miscadmin.h */
-
-
/*
* AuxiliaryProcessMain
*
@@ -55,39 +47,13 @@ AuxProcType MyAuxProcType = NotAnAuxProcess; /* declared in miscadmin.h */
* This code is here just because of historical reasons.
*/
void
-AuxiliaryProcessMain(AuxProcType auxtype)
+AuxiliaryProcessMain(BackendType auxtype)
{
Assert(IsUnderPostmaster);
- MyAuxProcType = auxtype;
-
- switch (MyAuxProcType)
- {
- case StartupProcess:
- MyBackendType = B_STARTUP;
- break;
- case ArchiverProcess:
- MyBackendType = B_ARCHIVER;
- break;
- case BgWriterProcess:
- MyBackendType = B_BG_WRITER;
- break;
- case CheckpointerProcess:
- MyBackendType = B_CHECKPOINTER;
- break;
- case WalWriterProcess:
- MyBackendType = B_WAL_WRITER;
- break;
- case WalReceiverProcess:
- MyBackendType = B_WAL_RECEIVER;
- break;
- case WalSummarizerProcess:
- MyBackendType = B_WAL_SUMMARIZER;
- break;
- default:
- elog(PANIC, "unrecognized process type: %d", (int) MyAuxProcType);
- MyBackendType = B_INVALID;
- }
+ MyBackendType = auxtype;
+ if (!IsAuxProcess(MyBackendType))
+ elog(PANIC, "unrecognized process type: %d", (int) MyBackendType);
init_ps_display(NULL);
@@ -126,38 +92,38 @@ AuxiliaryProcessMain(AuxProcType auxtype)
SetProcessingMode(NormalProcessing);
- switch (MyAuxProcType)
+ switch (MyBackendType)
{
- case StartupProcess:
+ case B_STARTUP:
StartupProcessMain();
proc_exit(1);
- case ArchiverProcess:
+ case B_ARCHIVER:
PgArchiverMain();
proc_exit(1);
- case BgWriterProcess:
+ case B_BG_WRITER:
BackgroundWriterMain();
proc_exit(1);
- case CheckpointerProcess:
+ case B_CHECKPOINTER:
CheckpointerMain();
proc_exit(1);
- case WalWriterProcess:
+ case B_WAL_WRITER:
WalWriterMain();
proc_exit(1);
- case WalReceiverProcess:
+ case B_WAL_RECEIVER:
WalReceiverMain();
proc_exit(1);
- case WalSummarizerProcess:
+ case B_WAL_SUMMARIZER:
WalSummarizerMain();
proc_exit(1);
default:
- elog(PANIC, "unrecognized process type: %d", (int) MyAuxProcType);
+ elog(PANIC, "unrecognized process type: %d", (int) MyBackendType);
proc_exit(1);
}
}
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index feb471dd1df..c51b6a1376a 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -440,7 +440,7 @@ static int CountChildren(int target);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
-static pid_t StartChildProcess(AuxProcType type);
+static pid_t StartChildProcess(BackendType type);
static void StartAutovacuumWorker(void);
static void MaybeStartWalReceiver(void);
static void MaybeStartWalSummarizer(void);
@@ -561,13 +561,13 @@ static void ShmemBackendArrayAdd(Backend *bn);
static void ShmemBackendArrayRemove(Backend *bn);
#endif /* EXEC_BACKEND */
-#define StartupDataBase() StartChildProcess(StartupProcess)
-#define StartArchiver() StartChildProcess(ArchiverProcess)
-#define StartBackgroundWriter() StartChildProcess(BgWriterProcess)
-#define StartCheckpointer() StartChildProcess(CheckpointerProcess)
-#define StartWalWriter() StartChildProcess(WalWriterProcess)
-#define StartWalReceiver() StartChildProcess(WalReceiverProcess)
-#define StartWalSummarizer() StartChildProcess(WalSummarizerProcess)
+#define StartupDataBase() StartChildProcess(B_STARTUP)
+#define StartArchiver() StartChildProcess(B_ARCHIVER)
+#define StartBackgroundWriter() StartChildProcess(B_BG_WRITER)
+#define StartCheckpointer() StartChildProcess(B_CHECKPOINTER)
+#define StartWalWriter() StartChildProcess(B_WAL_WRITER)
+#define StartWalReceiver() StartChildProcess(B_WAL_RECEIVER)
+#define StartWalSummarizer() StartChildProcess(B_WAL_SUMMARIZER)
/* Macros to check exit status of a child process */
#define EXIT_STATUS_0(st) ((st) == 0)
@@ -4953,7 +4953,7 @@ SubPostmasterMain(int argc, char *argv[])
}
if (strcmp(argv[1], "--forkaux") == 0)
{
- AuxProcType auxtype;
+ BackendType auxtype;
Assert(argc == 4);
@@ -5292,7 +5292,7 @@ CountChildren(int target)
* to start subprocess.
*/
static pid_t
-StartChildProcess(AuxProcType type)
+StartChildProcess(BackendType type)
{
pid_t pid;
@@ -5344,31 +5344,31 @@ StartChildProcess(AuxProcType type)
errno = save_errno;
switch (type)
{
- case StartupProcess:
+ case B_STARTUP:
ereport(LOG,
(errmsg("could not fork startup process: %m")));
break;
- case ArchiverProcess:
+ case B_ARCHIVER:
ereport(LOG,
(errmsg("could not fork archiver process: %m")));
break;
- case BgWriterProcess:
+ case B_BG_WRITER:
ereport(LOG,
(errmsg("could not fork background writer process: %m")));
break;
- case CheckpointerProcess:
+ case B_CHECKPOINTER:
ereport(LOG,
(errmsg("could not fork checkpointer process: %m")));
break;
- case WalWriterProcess:
+ case B_WAL_WRITER:
ereport(LOG,
(errmsg("could not fork WAL writer process: %m")));
break;
- case WalReceiverProcess:
+ case B_WAL_RECEIVER:
ereport(LOG,
(errmsg("could not fork WAL receiver process: %m")));
break;
- case WalSummarizerProcess:
+ case B_WAL_SUMMARIZER:
ereport(LOG,
(errmsg("could not fork WAL summarizer process: %m")));
break;
@@ -5382,7 +5382,7 @@ StartChildProcess(AuxProcType type)
* fork failure is fatal during startup, but there's no need to choke
* immediately if starting other child types fails.
*/
- if (type == StartupProcess)
+ if (type == B_STARTUP)
ExitPostmaster(1);
return 0;
}
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index cbdc61b8576..dc0f6765e82 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -321,21 +321,36 @@ extern void InitProcessLocalLatch(void);
extern void SwitchToSharedLatch(void);
extern void SwitchBackToLocalLatch(void);
+/*
+ * MyBackendType indicates what kind of a backend this is.
+ */
typedef enum BackendType
{
B_INVALID = 0,
- B_ARCHIVER,
+
+ /* Backends and other backend-like processes */
+ B_BACKEND,
B_AUTOVAC_LAUNCHER,
B_AUTOVAC_WORKER,
- B_BACKEND,
B_BG_WORKER,
+ B_WAL_SENDER,
+
+ B_STANDALONE_BACKEND,
+ B_LOGGER,
+
+ /*
+ * Auxiliary processes. These have PGPROC entries, but they are not
+ * attached to any particular database. There can be only one of each of
+ * these running at a time.
+ *
+ * If you modify these, make sure to update NUM_AUXILIARY_PROCS and the
+ * glossary in the docs.
+ */
+ B_ARCHIVER,
B_BG_WRITER,
B_CHECKPOINTER,
- B_LOGGER,
- B_STANDALONE_BACKEND,
B_STARTUP,
B_WAL_RECEIVER,
- B_WAL_SENDER,
B_WAL_SUMMARIZER,
B_WAL_WRITER,
} BackendType;
@@ -344,6 +359,18 @@ typedef enum BackendType
extern PGDLLIMPORT BackendType MyBackendType;
+#define FIRST_AUX_PROC B_ARCHIVER
+
+#define AmStartupProcess() (MyBackendType == B_STARTUP)
+#define AmBackgroundWriterProcess() (MyBackendType == B_BG_WRITER)
+#define AmArchiverProcess() (MyBackendType == B_ARCHIVER)
+#define AmCheckpointerProcess() (MyBackendType == B_CHECKPOINTER)
+#define AmWalWriterProcess() (MyBackendType == B_WAL_WRITER)
+#define AmWalReceiverProcess() (MyBackendType == B_WAL_RECEIVER)
+#define AmWalSummarizerProcess() (MyBackendType == B_WAL_SUMMARIZER)
+
+#define IsAuxProcess(type) (MyBackendType >= FIRST_AUX_PROC)
+
extern const char *GetBackendTypeDesc(BackendType backendType);
extern void SetDatabasePath(const char *path);
@@ -426,37 +453,6 @@ extern PGDLLIMPORT ProcessingMode Mode;
} while(0)
-/*
- * Auxiliary-process type identifiers. These used to be in bootstrap.h
- * but it seems saner to have them here, with the ProcessingMode stuff.
- * The MyAuxProcType global is defined and set in auxprocess.c.
- *
- * Make sure to list in the glossary any items you add here.
- */
-
-typedef enum
-{
- NotAnAuxProcess = -1,
- StartupProcess = 0,
- BgWriterProcess,
- ArchiverProcess,
- CheckpointerProcess,
- WalWriterProcess,
- WalReceiverProcess,
- WalSummarizerProcess,
-} AuxProcType;
-
-extern PGDLLIMPORT AuxProcType MyAuxProcType;
-
-#define AmStartupProcess() (MyAuxProcType == StartupProcess)
-#define AmBackgroundWriterProcess() (MyAuxProcType == BgWriterProcess)
-#define AmArchiverProcess() (MyAuxProcType == ArchiverProcess)
-#define AmCheckpointerProcess() (MyAuxProcType == CheckpointerProcess)
-#define AmWalWriterProcess() (MyAuxProcType == WalWriterProcess)
-#define AmWalReceiverProcess() (MyAuxProcType == WalReceiverProcess)
-#define AmWalSummarizerProcess() (MyAuxProcType == WalSummarizerProcess)
-
-
/*****************************************************************************
* pinit.h -- *
* POSTGRES initialization and cleanup definitions. *
diff --git a/src/include/postmaster/auxprocess.h b/src/include/postmaster/auxprocess.h
index 1fdde3bb77b..3e443edde70 100644
--- a/src/include/postmaster/auxprocess.h
+++ b/src/include/postmaster/auxprocess.h
@@ -15,6 +15,6 @@
#include "miscadmin.h"
-extern void AuxiliaryProcessMain(AuxProcType auxtype) pg_attribute_noreturn();
+extern void AuxiliaryProcessMain(BackendType auxtype) pg_attribute_noreturn();
#endif /* AUXPROCESS_H */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 7e866e3c3d0..caff3fc1158 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -172,7 +172,6 @@ AutoVacOpts
AutoVacuumShmemStruct
AutoVacuumWorkItem
AutoVacuumWorkItemType
-AuxProcType
BF_ctx
BF_key
BF_word
--
2.39.2
v7-0004-Use-MyBackendType-in-more-places-to-check-what-pr.patchtext/x-patch; charset=UTF-8; name=v7-0004-Use-MyBackendType-in-more-places-to-check-what-pr.patchDownload
From 510905b2e40623a19964f5eff9ebf11dbf63371d Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 10 Jan 2024 12:59:48 +0200
Subject: [PATCH v7 4/4] Use MyBackendType in more places to check what process
this is
Remove IsBackgroundWorker, IsAutoVacuumLauncherProcess() and
IsAutoVacuumWorkerProcess() in favor of new Am*Process() macros that
use MyBackendType. For consistency with the existing Am*Process()
macros.
Reviewed-by: XXX
Discussion: XXX
---
src/backend/access/gin/ginfast.c | 2 +-
src/backend/access/gin/ginvacuum.c | 6 ++---
src/backend/access/heap/vacuumlazy.c | 4 +--
src/backend/commands/analyze.c | 4 +--
src/backend/commands/vacuum.c | 8 +++---
src/backend/postmaster/autovacuum.c | 26 --------------------
src/backend/postmaster/bgworker.c | 2 --
src/backend/postmaster/postmaster.c | 3 ---
src/backend/statistics/extended_stats.c | 2 +-
src/backend/storage/ipc/ipc.c | 2 +-
src/backend/storage/lmgr/proc.c | 19 +++++++-------
src/backend/tcop/postgres.c | 6 ++---
src/backend/utils/activity/pgstat_relation.c | 4 +--
src/backend/utils/init/globals.c | 1 -
src/backend/utils/init/miscinit.c | 2 +-
src/backend/utils/init/postinit.c | 8 +++---
src/include/miscadmin.h | 8 +++---
src/include/postmaster/autovacuum.h | 5 ----
18 files changed, 38 insertions(+), 74 deletions(-)
diff --git a/src/backend/access/gin/ginfast.c b/src/backend/access/gin/ginfast.c
index cff6850ef86..e118cecb9a4 100644
--- a/src/backend/access/gin/ginfast.c
+++ b/src/backend/access/gin/ginfast.c
@@ -812,7 +812,7 @@ ginInsertCleanup(GinState *ginstate, bool full_clean,
*/
LockPage(index, GIN_METAPAGE_BLKNO, ExclusiveLock);
workMemory =
- (IsAutoVacuumWorkerProcess() && autovacuum_work_mem != -1) ?
+ (AmAutoVacuumWorkerProcess() && autovacuum_work_mem != -1) ?
autovacuum_work_mem : maintenance_work_mem;
}
else
diff --git a/src/backend/access/gin/ginvacuum.c b/src/backend/access/gin/ginvacuum.c
index aee6aab44ae..b3f415e2849 100644
--- a/src/backend/access/gin/ginvacuum.c
+++ b/src/backend/access/gin/ginvacuum.c
@@ -590,7 +590,7 @@ ginbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
/*
* and cleanup any pending inserts
*/
- ginInsertCleanup(&gvs.ginstate, !IsAutoVacuumWorkerProcess(),
+ ginInsertCleanup(&gvs.ginstate, !AmAutoVacuumWorkerProcess(),
false, true, stats);
}
@@ -701,7 +701,7 @@ ginvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats)
*/
if (info->analyze_only)
{
- if (IsAutoVacuumWorkerProcess())
+ if (AmAutoVacuumWorkerProcess())
{
initGinState(&ginstate, index);
ginInsertCleanup(&ginstate, false, true, true, stats);
@@ -717,7 +717,7 @@ ginvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats)
{
stats = (IndexBulkDeleteResult *) palloc0(sizeof(IndexBulkDeleteResult));
initGinState(&ginstate, index);
- ginInsertCleanup(&ginstate, !IsAutoVacuumWorkerProcess(),
+ ginInsertCleanup(&ginstate, !AmAutoVacuumWorkerProcess(),
false, true, stats);
}
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 0fb3953513d..eb36958b62b 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -307,7 +307,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
char **indnames = NULL;
verbose = (params->options & VACOPT_VERBOSE) != 0;
- instrument = (verbose || (IsAutoVacuumWorkerProcess() &&
+ instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
params->log_min_duration >= 0));
if (instrument)
{
@@ -3108,7 +3108,7 @@ static int
dead_items_max_items(LVRelState *vacrel)
{
int64 max_items;
- int vac_work_mem = IsAutoVacuumWorkerProcess() &&
+ int vac_work_mem = AmAutoVacuumWorkerProcess() &&
autovacuum_work_mem != -1 ?
autovacuum_work_mem : maintenance_work_mem;
diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c
index a03495d6c95..d105d2fad7b 100644
--- a/src/backend/commands/analyze.c
+++ b/src/backend/commands/analyze.c
@@ -351,7 +351,7 @@ do_analyze_rel(Relation onerel, VacuumParams *params,
save_nestlevel = NewGUCNestLevel();
/* measure elapsed time iff autovacuum logging requires it */
- if (IsAutoVacuumWorkerProcess() && params->log_min_duration >= 0)
+ if (AmAutoVacuumWorkerProcess() && params->log_min_duration >= 0)
{
if (track_io_timing)
{
@@ -729,7 +729,7 @@ do_analyze_rel(Relation onerel, VacuumParams *params,
vac_close_indexes(nindexes, Irel, NoLock);
/* Log the action if appropriate */
- if (IsAutoVacuumWorkerProcess() && params->log_min_duration >= 0)
+ if (AmAutoVacuumWorkerProcess() && params->log_min_duration >= 0)
{
TimestampTz endtime = GetCurrentTimestamp();
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 64da8486276..e40cef6566e 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -564,7 +564,7 @@ vacuum(List *relations, VacuumParams *params, BufferAccessStrategy bstrategy,
else
{
Assert(params->options & VACOPT_ANALYZE);
- if (IsAutoVacuumWorkerProcess())
+ if (AmAutoVacuumWorkerProcess())
use_own_xacts = true;
else if (in_outer_xact)
use_own_xacts = false;
@@ -809,7 +809,7 @@ vacuum_open_relation(Oid relid, RangeVar *relation, bits32 options,
* statements in the permission checks; otherwise, only log if the caller
* so requested.
*/
- if (!IsAutoVacuumWorkerProcess())
+ if (!AmAutoVacuumWorkerProcess())
elevel = WARNING;
else if (verbose)
elevel = LOG;
@@ -896,7 +896,7 @@ expand_vacuum_rel(VacuumRelation *vrel, MemoryContext vac_context,
* Since autovacuum workers supply OIDs when calling vacuum(), no
* autovacuum worker should reach this code.
*/
- Assert(!IsAutoVacuumWorkerProcess());
+ Assert(!AmAutoVacuumWorkerProcess());
/*
* We transiently take AccessShareLock to protect the syscache lookup
@@ -2336,7 +2336,7 @@ vacuum_delay_point(void)
* [autovacuum_]vacuum_cost_delay to take effect while a table is being
* vacuumed or analyzed.
*/
- if (ConfigReloadPending && IsAutoVacuumWorkerProcess())
+ if (ConfigReloadPending && AmAutoVacuumWorkerProcess())
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 2c3099f76f1..cecbaaa5b91 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -135,10 +135,6 @@ int Log_autovacuum_min_duration = 600000;
#define MIN_AUTOVAC_SLEEPTIME 100.0 /* milliseconds */
#define MAX_AUTOVAC_SLEEPTIME 300 /* seconds */
-/* Flags to tell if we are in an autovacuum process */
-static bool am_autovacuum_launcher = false;
-static bool am_autovacuum_worker = false;
-
/*
* Variables to save the cost-related storage parameters for the current
* relation being vacuumed by this autovacuum worker. Using these, we can
@@ -435,8 +431,6 @@ AutoVacLauncherMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
- am_autovacuum_launcher = true;
-
MyBackendType = B_AUTOVAC_LAUNCHER;
init_ps_display(NULL);
@@ -1495,8 +1489,6 @@ AutoVacWorkerMain(int argc, char *argv[])
sigjmp_buf local_sigjmp_buf;
Oid dbid;
- am_autovacuum_worker = true;
-
MyBackendType = B_AUTOVAC_WORKER;
init_ps_display(NULL);
@@ -3355,24 +3347,6 @@ autovac_init(void)
errhint("Enable the \"track_counts\" option.")));
}
-/*
- * IsAutoVacuum functions
- * Return whether this is either a launcher autovacuum process or a worker
- * process.
- */
-bool
-IsAutoVacuumLauncherProcess(void)
-{
- return am_autovacuum_launcher;
-}
-
-bool
-IsAutoVacuumWorkerProcess(void)
-{
- return am_autovacuum_worker;
-}
-
-
/*
* AutoVacuumShmemSize
* Compute space needed for autovacuum-related shared memory
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index 67f92c24db1..d8e89de7494 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -731,8 +731,6 @@ BackgroundWorkerMain(void)
if (worker == NULL)
elog(FATAL, "unable to find bgworker entry");
- IsBackgroundWorker = true;
-
MyBackendType = B_BG_WORKER;
init_ps_display(worker->bgw_name);
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index c51b6a1376a..8aa2ddf0c46 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -4979,9 +4979,6 @@ SubPostmasterMain(int argc, char *argv[])
}
if (strcmp(argv[1], "--forkbgworker") == 0)
{
- /* do this as early as possible; in particular, before InitProcess() */
- IsBackgroundWorker = true;
-
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
diff --git a/src/backend/statistics/extended_stats.c b/src/backend/statistics/extended_stats.c
index c5461514d8f..135151a2723 100644
--- a/src/backend/statistics/extended_stats.c
+++ b/src/backend/statistics/extended_stats.c
@@ -173,7 +173,7 @@ BuildRelationExtStatistics(Relation onerel, bool inh, double totalrows,
natts, vacattrstats);
if (!stats)
{
- if (!IsAutoVacuumWorkerProcess())
+ if (!AmAutoVacuumWorkerProcess())
ereport(WARNING,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("statistics object \"%s.%s\" could not be computed for relation \"%s.%s\"",
diff --git a/src/backend/storage/ipc/ipc.c b/src/backend/storage/ipc/ipc.c
index 2681e4cdff5..b06e4b84528 100644
--- a/src/backend/storage/ipc/ipc.c
+++ b/src/backend/storage/ipc/ipc.c
@@ -136,7 +136,7 @@ proc_exit(int code)
*/
char gprofDirName[32];
- if (IsAutoVacuumWorkerProcess())
+ if (AmAutoVacuumWorkerProcess())
snprintf(gprofDirName, 32, "gprof/avworker");
else
snprintf(gprofDirName, 32, "gprof/%d", (int) getpid());
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index ac5096f45aa..2ad195a0874 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -41,7 +41,6 @@
#include "postmaster/autovacuum.h"
#include "replication/slot.h"
#include "replication/syncrep.h"
-#include "replication/walsender.h"
#include "storage/condition_variable.h"
#include "storage/ipc.h"
#include "storage/lmgr.h"
@@ -308,11 +307,11 @@ InitProcess(void)
elog(ERROR, "you already exist");
/* Decide which list should supply our PGPROC. */
- if (IsAnyAutoVacuumProcess())
+ if (AmAutoVacuumLauncherProcess() || AmAutoVacuumWorkerProcess())
procgloballist = &ProcGlobal->autovacFreeProcs;
- else if (IsBackgroundWorker)
+ else if (AmBackgroundWorkerProcess())
procgloballist = &ProcGlobal->bgworkerFreeProcs;
- else if (am_walsender)
+ else if (AmWalSenderProcess())
procgloballist = &ProcGlobal->walsenderFreeProcs;
else
procgloballist = &ProcGlobal->freeProcs;
@@ -342,7 +341,7 @@ InitProcess(void)
* in the autovacuum case?
*/
SpinLockRelease(ProcStructLock);
- if (am_walsender)
+ if (AmWalSenderProcess())
ereport(FATAL,
(errcode(ERRCODE_TOO_MANY_CONNECTIONS),
errmsg("number of requested standby connections exceeds max_wal_senders (currently %d)",
@@ -365,7 +364,7 @@ InitProcess(void)
* cleaning up. (XXX autovac launcher currently doesn't participate in
* this; it probably should.)
*/
- if (IsUnderPostmaster && !IsAutoVacuumLauncherProcess())
+ if (IsUnderPostmaster && !AmAutoVacuumLauncherProcess())
MarkPostmasterChildActive();
/*
@@ -384,11 +383,11 @@ InitProcess(void)
MyProc->databaseId = InvalidOid;
MyProc->roleId = InvalidOid;
MyProc->tempNamespaceId = InvalidOid;
- MyProc->isBackgroundWorker = IsBackgroundWorker;
+ MyProc->isBackgroundWorker = AmBackgroundWorkerProcess();
MyProc->delayChkptFlags = 0;
MyProc->statusFlags = 0;
/* NB -- autovac launcher intentionally does not set IS_AUTOVACUUM */
- if (IsAutoVacuumWorkerProcess())
+ if (AmAutoVacuumWorkerProcess())
MyProc->statusFlags |= PROC_IS_AUTOVACUUM;
MyProc->lwWaiting = LW_WS_NOT_WAITING;
MyProc->lwWaitMode = 0;
@@ -579,7 +578,7 @@ InitAuxiliaryProcess(void)
MyProc->databaseId = InvalidOid;
MyProc->roleId = InvalidOid;
MyProc->tempNamespaceId = InvalidOid;
- MyProc->isBackgroundWorker = IsBackgroundWorker;
+ MyProc->isBackgroundWorker = AmBackgroundWorkerProcess();
MyProc->delayChkptFlags = 0;
MyProc->statusFlags = 0;
MyProc->lwWaiting = LW_WS_NOT_WAITING;
@@ -935,7 +934,7 @@ ProcKill(int code, Datum arg)
* way, so tell the postmaster we've cleaned up acceptably well. (XXX
* autovac launcher should be included here someday)
*/
- if (IsUnderPostmaster && !IsAutoVacuumLauncherProcess())
+ if (IsUnderPostmaster && !AmAutoVacuumLauncherProcess())
MarkPostmasterChildInactive();
/* wake autovac launcher if needed -- see comments in FreeWorkerInfo */
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 1a34bd3715f..38610b723af 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -3267,7 +3267,7 @@ ProcessInterrupts(void)
ereport(FATAL,
(errcode(ERRCODE_QUERY_CANCELED),
errmsg("canceling authentication due to timeout")));
- else if (IsAutoVacuumWorkerProcess())
+ else if (AmAutoVacuumWorkerProcess())
ereport(FATAL,
(errcode(ERRCODE_ADMIN_SHUTDOWN),
errmsg("terminating autovacuum process due to administrator command")));
@@ -3286,7 +3286,7 @@ ProcessInterrupts(void)
*/
proc_exit(1);
}
- else if (IsBackgroundWorker)
+ else if (AmBackgroundWorkerProcess())
ereport(FATAL,
(errcode(ERRCODE_ADMIN_SHUTDOWN),
errmsg("terminating background worker \"%s\" due to administrator command",
@@ -3386,7 +3386,7 @@ ProcessInterrupts(void)
(errcode(ERRCODE_QUERY_CANCELED),
errmsg("canceling statement due to statement timeout")));
}
- if (IsAutoVacuumWorkerProcess())
+ if (AmAutoVacuumWorkerProcess())
{
LockErrorCleanup();
ereport(ERROR,
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 111050725a6..b20a60b5a87 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -246,7 +246,7 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
*/
tabentry->ins_since_vacuum = 0;
- if (IsAutoVacuumWorkerProcess())
+ if (AmAutoVacuumWorkerProcess())
{
tabentry->last_autovacuum_time = ts;
tabentry->autovacuum_count++;
@@ -337,7 +337,7 @@ pgstat_report_analyze(Relation rel,
if (resetcounter)
tabentry->mod_since_analyze = 0;
- if (IsAutoVacuumWorkerProcess())
+ if (AmAutoVacuumWorkerProcess())
{
tabentry->last_autoanalyze_time = GetCurrentTimestamp();
tabentry->autoanalyze_count++;
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 88b03e8fa3c..ce1fa1092f2 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -114,7 +114,6 @@ pid_t PostmasterPid = 0;
bool IsPostmasterEnvironment = false;
bool IsUnderPostmaster = false;
bool IsBinaryUpgrade = false;
-bool IsBackgroundWorker = false;
bool ExitOnAnyError = false;
diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c
index 23f77a59e58..9c91e9a1324 100644
--- a/src/backend/utils/init/miscinit.c
+++ b/src/backend/utils/init/miscinit.c
@@ -837,7 +837,7 @@ InitializeSessionUserIdStandalone(void)
* This function should only be called in single-user mode, in autovacuum
* workers, and in background workers.
*/
- Assert(!IsUnderPostmaster || IsAutoVacuumWorkerProcess() || IsBackgroundWorker);
+ Assert(!IsUnderPostmaster || AmAutoVacuumWorkerProcess() || AmBackgroundWorkerProcess());
/* call only once */
Assert(!OidIsValid(AuthenticatedUserId));
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index e822cd61d5f..6bf71b2d4e2 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -344,7 +344,7 @@ CheckMyDatabase(const char *name, bool am_superuser, bool override_allow_connect
*
* We do not enforce them for autovacuum worker processes either.
*/
- if (IsUnderPostmaster && !IsAutoVacuumWorkerProcess())
+ if (IsUnderPostmaster && !AmAutoVacuumWorkerProcess())
{
/*
* Check that the database is currently allowing connections.
@@ -825,7 +825,7 @@ InitPostgres(const char *in_dbname, Oid dboid,
before_shmem_exit(ShutdownPostgres, 0);
/* The autovacuum launcher is done here */
- if (IsAutoVacuumLauncherProcess())
+ if (AmAutoVacuumLauncherProcess())
{
/* report this backend in the PgBackendStatus array */
pgstat_bestart();
@@ -869,7 +869,7 @@ InitPostgres(const char *in_dbname, Oid dboid,
* In standalone mode and in autovacuum worker processes, we use a fixed
* ID, otherwise we figure it out from the authenticated user name.
*/
- if (bootstrap || IsAutoVacuumWorkerProcess())
+ if (bootstrap || AmAutoVacuumWorkerProcess())
{
InitializeSessionUserIdStandalone();
am_superuser = true;
@@ -885,7 +885,7 @@ InitPostgres(const char *in_dbname, Oid dboid,
errhint("You should immediately run CREATE USER \"%s\" SUPERUSER;.",
username != NULL ? username : "postgres")));
}
- else if (IsBackgroundWorker)
+ else if (AmBackgroundWorkerProcess())
{
if (username == NULL && !OidIsValid(useroid))
{
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index dc0f6765e82..fe0f2b599a9 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -164,7 +164,6 @@ do { \
extern PGDLLIMPORT pid_t PostmasterPid;
extern PGDLLIMPORT bool IsPostmasterEnvironment;
extern PGDLLIMPORT bool IsUnderPostmaster;
-extern PGDLLIMPORT bool IsBackgroundWorker;
extern PGDLLIMPORT bool IsBinaryUpgrade;
extern PGDLLIMPORT bool ExitOnAnyError;
@@ -360,7 +359,12 @@ typedef enum BackendType
extern PGDLLIMPORT BackendType MyBackendType;
#define FIRST_AUX_PROC B_ARCHIVER
+#define IsAuxProcess(type) (MyBackendType >= FIRST_AUX_PROC)
+#define AmAutoVacuumLauncherProcess() (MyBackendType == B_AUTOVAC_LAUNCHER)
+#define AmAutoVacuumWorkerProcess() (MyBackendType == B_AUTOVAC_WORKER)
+#define AmBackgroundWorkerProcess() (MyBackendType == B_BG_WORKER)
+#define AmWalSenderProcess() (MyBackendType == B_WAL_SENDER)
#define AmStartupProcess() (MyBackendType == B_STARTUP)
#define AmBackgroundWriterProcess() (MyBackendType == B_BG_WRITER)
#define AmArchiverProcess() (MyBackendType == B_ARCHIVER)
@@ -369,8 +373,6 @@ extern PGDLLIMPORT BackendType MyBackendType;
#define AmWalReceiverProcess() (MyBackendType == B_WAL_RECEIVER)
#define AmWalSummarizerProcess() (MyBackendType == B_WAL_SUMMARIZER)
-#define IsAuxProcess(type) (MyBackendType >= FIRST_AUX_PROC)
-
extern const char *GetBackendTypeDesc(BackendType backendType);
extern void SetDatabasePath(const char *path);
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 1994aedef03..80cf4cdd969 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -49,11 +49,6 @@ extern PGDLLIMPORT int Log_autovacuum_min_duration;
/* Status inquiry functions */
extern bool AutoVacuumingActive(void);
-extern bool IsAutoVacuumLauncherProcess(void);
-extern bool IsAutoVacuumWorkerProcess(void);
-
-#define IsAnyAutoVacuumProcess() \
- (IsAutoVacuumLauncherProcess() || IsAutoVacuumWorkerProcess())
/* Functions to start autovacuum process, called from postmaster */
extern void autovac_init(void);
--
2.39.2
Re: Refactoring backend fork+exec code
On Thu, 2024-01-25 at 01:51 +0200, Heikki Linnakangas wrote:
And here we go. BackendID is now a 1-based index directly into the
PGPROC array.
Would it be worthwhile to also note in this comment FIRST_AUX_PROC's
and IsAuxProcess()'s dependency on B_ARCHIVER and it's location in the
enum table?
/*
¦* Auxiliary processes. These have PGPROC entries, but they are not
¦* attached to any particular database. There can be only one of each of
¦* these running at a time.
¦*
¦* If you modify these, make sure to update NUM_AUXILIARY_PROCS and the
¦* glossary in the docs.
¦*/
B_ARCHIVER,
B_BG_WRITER,
B_CHECKPOINTER,
B_STARTUP,
B_WAL_RECEIVER,
B_WAL_SUMMARIZER,
B_WAL_WRITER,
} BackendType;
#define BACKEND_NUM_TYPES (B_WAL_WRITER + 1)
extern PGDLLIMPORT BackendType MyBackendType;
#define FIRST_AUX_PROC B_ARCHIVER
#define IsAuxProcess(type) (MyBackendType >= FIRST_AUX_PROC)
Re: Refactoring backend fork+exec code
On 29/01/2024 17:54, reid.thompson@crunchydata.com wrote:
On Thu, 2024-01-25 at 01:51 +0200, Heikki Linnakangas wrote:
And here we go. BackendID is now a 1-based index directly into the
PGPROC array.Would it be worthwhile to also note in this comment FIRST_AUX_PROC's
and IsAuxProcess()'s dependency on B_ARCHIVER and it's location in the
enum table?
Yeah, that might be in order. Although looking closer, it's only used in
IsAuxProcess, which is only used in one sanity check in
AuxProcessMain(). And even that gets refactored away by the later
patches in this thread. So on second thoughts, I'll just remove it
altogether.
I spent some more time on the 'lastBackend' optimization in sinvaladt.c.
I realized that it became very useless with these patches, because aux
processes are allocated pgprocno's after all the slots for regular
backends. There are always aux processes running, so lastBackend would
always have a value close to the max anyway. I replaced that with a
dense 'pgprocnos' array that keeps track of the exact slots that are in
use. I'm not 100% sure this is worth the effort; does any real world
workload send shared invalidations so frequently that this matters? In
any case, this should avoid the regression if such a workload exists.
New patch set attached. I think these are ready to be committed, but
would appreciate a final review.
--
Heikki Linnakangas
Neon (https://neon.tech)
Attachments:
v8-0001-Remove-superfluous-pgprocno-field-from-PGPROC.patchtext/x-patch; charset=UTF-8; name=v8-0001-Remove-superfluous-pgprocno-field-from-PGPROC.patchDownload
From 54f22231bb2540fc5957c14005956161e6fc9dac Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 24 Jan 2024 23:15:55 +0200
Subject: [PATCH v8 1/5] Remove superfluous 'pgprocno' field from PGPROC
It was always just the index of the PGPROC entry from the beginning of
the proc array. Introduce a macro to compute it from the pointer
instead.
Reviewed-by: XXX
Discussion: https://www.postgresql.org/message-id/8171f1aa-496f-46a6-afc3-c46fe7a9b407%40iki.fi
---
src/backend/access/transam/clog.c | 4 ++--
src/backend/access/transam/twophase.c | 11 +++++------
src/backend/access/transam/xlog.c | 2 +-
src/backend/postmaster/bgwriter.c | 2 +-
src/backend/postmaster/pgarch.c | 2 +-
src/backend/postmaster/walsummarizer.c | 2 +-
src/backend/storage/buffer/bufmgr.c | 6 +++---
src/backend/storage/ipc/procarray.c | 6 +++---
src/backend/storage/lmgr/condition_variable.c | 12 ++++++------
src/backend/storage/lmgr/lwlock.c | 6 +++---
src/backend/storage/lmgr/predicate.c | 2 +-
src/backend/storage/lmgr/proc.c | 1 -
src/include/storage/lock.h | 2 +-
src/include/storage/proc.h | 6 +-----
14 files changed, 29 insertions(+), 35 deletions(-)
diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c
index f6e7da7ffc9..7550309c25a 100644
--- a/src/backend/access/transam/clog.c
+++ b/src/backend/access/transam/clog.c
@@ -458,7 +458,7 @@ TransactionGroupUpdateXidStatus(TransactionId xid, XidStatus status,
* less efficiently.
*/
if (nextidx != INVALID_PGPROCNO &&
- ProcGlobal->allProcs[nextidx].clogGroupMemberPage != proc->clogGroupMemberPage)
+ GetPGProcByNumber(nextidx)->clogGroupMemberPage != proc->clogGroupMemberPage)
{
/*
* Ensure that this proc is not a member of any clog group that
@@ -473,7 +473,7 @@ TransactionGroupUpdateXidStatus(TransactionId xid, XidStatus status,
if (pg_atomic_compare_exchange_u32(&procglobal->clogGroupFirst,
&nextidx,
- (uint32) proc->pgprocno))
+ (uint32) GetNumberFromPGProc(proc)))
break;
}
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 8426458f7f5..234c8d08ebc 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -284,7 +284,7 @@ TwoPhaseShmemInit(void)
TwoPhaseState->freeGXacts = &gxacts[i];
/* associate it with a PGPROC assigned by InitProcGlobal */
- gxacts[i].pgprocno = PreparedXactProcs[i].pgprocno;
+ gxacts[i].pgprocno = GetNumberFromPGProc(&PreparedXactProcs[i]);
/*
* Assign a unique ID for each dummy proc, so that the range of
@@ -461,7 +461,6 @@ MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid, const char *gid,
/* Initialize the PGPROC entry */
MemSet(proc, 0, sizeof(PGPROC));
- proc->pgprocno = gxact->pgprocno;
dlist_node_init(&proc->links);
proc->waitStatus = PROC_WAIT_STATUS_OK;
if (LocalTransactionIdIsValid(MyProc->lxid))
@@ -780,7 +779,7 @@ pg_prepared_xact(PG_FUNCTION_ARGS)
while (status->array != NULL && status->currIdx < status->ngxacts)
{
GlobalTransaction gxact = &status->array[status->currIdx++];
- PGPROC *proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ PGPROC *proc = GetPGProcByNumber(gxact->pgprocno);
Datum values[5] = {0};
bool nulls[5] = {0};
HeapTuple tuple;
@@ -935,7 +934,7 @@ TwoPhaseGetDummyProc(TransactionId xid, bool lock_held)
{
GlobalTransaction gxact = TwoPhaseGetGXact(xid, lock_held);
- return &ProcGlobal->allProcs[gxact->pgprocno];
+ return GetPGProcByNumber(gxact->pgprocno);
}
/************************************************************************/
@@ -1080,7 +1079,7 @@ save_state_data(const void *data, uint32 len)
void
StartPrepare(GlobalTransaction gxact)
{
- PGPROC *proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ PGPROC *proc = GetPGProcByNumber(gxact->pgprocno);
TransactionId xid = gxact->xid;
TwoPhaseFileHeader hdr;
TransactionId *children;
@@ -1539,7 +1538,7 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
* try to commit the same GID at once.
*/
gxact = LockGXact(gid, GetUserId());
- proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ proc = GetPGProcByNumber(gxact->pgprocno);
xid = gxact->xid;
/*
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 478377c4a23..be30a2388f5 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -1379,7 +1379,7 @@ WALInsertLockAcquire(void)
static int lockToTry = -1;
if (lockToTry == -1)
- lockToTry = MyProc->pgprocno % NUM_XLOGINSERT_LOCKS;
+ lockToTry = GetNumberFromPGProc(MyProc) % NUM_XLOGINSERT_LOCKS;
MyLockNo = lockToTry;
/*
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index d7d6cc0cd7b..0677e061216 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -331,7 +331,7 @@ BackgroundWriterMain(void)
if (rc == WL_TIMEOUT && can_hibernate && prev_hibernate)
{
/* Ask for notification at next buffer allocation */
- StrategyNotifyBgWriter(MyProc->pgprocno);
+ StrategyNotifyBgWriter(GetNumberFromPGProc(MyProc));
/* Sleep ... */
(void) WaitLatch(MyLatch,
WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH,
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 67693b05806..9bfe4fae046 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -242,7 +242,7 @@ PgArchiverMain(void)
* Advertise our pgprocno so that backends can use our latch to wake us up
* while we're sleeping.
*/
- PgArch->pgprocno = MyProc->pgprocno;
+ PgArch->pgprocno = GetNumberFromPGProc(MyProc);
/* Create workspace for pgarch_readyXlog() */
arch_files = palloc(sizeof(struct arch_files_state));
diff --git a/src/backend/postmaster/walsummarizer.c b/src/backend/postmaster/walsummarizer.c
index 9b883c21ca4..165bdd93fa3 100644
--- a/src/backend/postmaster/walsummarizer.c
+++ b/src/backend/postmaster/walsummarizer.c
@@ -248,7 +248,7 @@ WalSummarizerMain(void)
/* Advertise ourselves. */
on_shmem_exit(WalSummarizerShutdown, (Datum) 0);
LWLockAcquire(WALSummarizerLock, LW_EXCLUSIVE);
- WalSummarizerCtl->summarizer_pgprocno = MyProc->pgprocno;
+ WalSummarizerCtl->summarizer_pgprocno = GetNumberFromPGProc(MyProc);
LWLockRelease(WALSummarizerLock);
/* Create and switch to a memory context that we can reset on error. */
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 7d601bef6dd..e19d433a8d3 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -4792,7 +4792,7 @@ UnlockBuffers(void)
* got a cancel/die interrupt before getting the signal.
*/
if ((buf_state & BM_PIN_COUNT_WAITER) != 0 &&
- buf->wait_backend_pgprocno == MyProc->pgprocno)
+ buf->wait_backend_pgprocno == GetNumberFromPGProc(MyProc))
buf_state &= ~BM_PIN_COUNT_WAITER;
UnlockBufHdr(buf, buf_state);
@@ -4942,7 +4942,7 @@ LockBufferForCleanup(Buffer buffer)
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
elog(ERROR, "multiple backends attempting to wait for pincount 1");
}
- bufHdr->wait_backend_pgprocno = MyProc->pgprocno;
+ bufHdr->wait_backend_pgprocno = GetNumberFromPGProc(MyProc);
PinCountWaitBuf = bufHdr;
buf_state |= BM_PIN_COUNT_WAITER;
UnlockBufHdr(bufHdr, buf_state);
@@ -5006,7 +5006,7 @@ LockBufferForCleanup(Buffer buffer)
*/
buf_state = LockBufHdr(bufHdr);
if ((buf_state & BM_PIN_COUNT_WAITER) != 0 &&
- bufHdr->wait_backend_pgprocno == MyProc->pgprocno)
+ bufHdr->wait_backend_pgprocno == GetNumberFromPGProc(MyProc))
buf_state &= ~BM_PIN_COUNT_WAITER;
UnlockBufHdr(bufHdr, buf_state);
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index ee2d7f8585a..5a33eb7f7f7 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -505,7 +505,7 @@ ProcArrayAdd(PGPROC *proc)
Assert(allProcs[procno].pgxactoff == index);
/* If we have found our right position in the array, break */
- if (arrayP->pgprocnos[index] > proc->pgprocno)
+ if (arrayP->pgprocnos[index] > GetNumberFromPGProc(proc))
break;
}
@@ -523,7 +523,7 @@ ProcArrayAdd(PGPROC *proc)
&ProcGlobal->statusFlags[index],
movecount * sizeof(*ProcGlobal->statusFlags));
- arrayP->pgprocnos[index] = proc->pgprocno;
+ arrayP->pgprocnos[index] = GetNumberFromPGProc(proc);
proc->pgxactoff = index;
ProcGlobal->xids[index] = proc->xid;
ProcGlobal->subxidStates[index] = proc->subxidStatus;
@@ -808,7 +808,7 @@ ProcArrayGroupClearXid(PGPROC *proc, TransactionId latestXid)
if (pg_atomic_compare_exchange_u32(&procglobal->procArrayGroupFirst,
&nextidx,
- (uint32) proc->pgprocno))
+ (uint32) GetNumberFromPGProc(proc)))
break;
}
diff --git a/src/backend/storage/lmgr/condition_variable.c b/src/backend/storage/lmgr/condition_variable.c
index e2d6d685220..d4a50c70ea4 100644
--- a/src/backend/storage/lmgr/condition_variable.c
+++ b/src/backend/storage/lmgr/condition_variable.c
@@ -57,7 +57,7 @@ ConditionVariableInit(ConditionVariable *cv)
void
ConditionVariablePrepareToSleep(ConditionVariable *cv)
{
- int pgprocno = MyProc->pgprocno;
+ int pgprocno = GetNumberFromPGProc(MyProc);
/*
* If some other sleep is already prepared, cancel it; this is necessary
@@ -181,10 +181,10 @@ ConditionVariableTimedSleep(ConditionVariable *cv, long timeout,
* guarantee not to return spuriously, we'll avoid this obvious case.
*/
SpinLockAcquire(&cv->mutex);
- if (!proclist_contains(&cv->wakeup, MyProc->pgprocno, cvWaitLink))
+ if (!proclist_contains(&cv->wakeup, GetNumberFromPGProc(MyProc), cvWaitLink))
{
done = true;
- proclist_push_tail(&cv->wakeup, MyProc->pgprocno, cvWaitLink);
+ proclist_push_tail(&cv->wakeup, GetNumberFromPGProc(MyProc), cvWaitLink);
}
SpinLockRelease(&cv->mutex);
@@ -236,8 +236,8 @@ ConditionVariableCancelSleep(void)
return false;
SpinLockAcquire(&cv->mutex);
- if (proclist_contains(&cv->wakeup, MyProc->pgprocno, cvWaitLink))
- proclist_delete(&cv->wakeup, MyProc->pgprocno, cvWaitLink);
+ if (proclist_contains(&cv->wakeup, GetNumberFromPGProc(MyProc), cvWaitLink))
+ proclist_delete(&cv->wakeup, GetNumberFromPGProc(MyProc), cvWaitLink);
else
signaled = true;
SpinLockRelease(&cv->mutex);
@@ -281,7 +281,7 @@ ConditionVariableSignal(ConditionVariable *cv)
void
ConditionVariableBroadcast(ConditionVariable *cv)
{
- int pgprocno = MyProc->pgprocno;
+ int pgprocno = GetNumberFromPGProc(MyProc);
PGPROC *proc = NULL;
bool have_sentinel = false;
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index 98fa6035cc5..9b69e9f1e32 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -1056,9 +1056,9 @@ LWLockQueueSelf(LWLock *lock, LWLockMode mode)
/* LW_WAIT_UNTIL_FREE waiters are always at the front of the queue */
if (mode == LW_WAIT_UNTIL_FREE)
- proclist_push_head(&lock->waiters, MyProc->pgprocno, lwWaitLink);
+ proclist_push_head(&lock->waiters, GetNumberFromPGProc(MyProc), lwWaitLink);
else
- proclist_push_tail(&lock->waiters, MyProc->pgprocno, lwWaitLink);
+ proclist_push_tail(&lock->waiters, GetNumberFromPGProc(MyProc), lwWaitLink);
/* Can release the mutex now */
LWLockWaitListUnlock(lock);
@@ -1097,7 +1097,7 @@ LWLockDequeueSelf(LWLock *lock)
*/
on_waitlist = MyProc->lwWaiting == LW_WS_WAITING;
if (on_waitlist)
- proclist_delete(&lock->waiters, MyProc->pgprocno, lwWaitLink);
+ proclist_delete(&lock->waiters, GetNumberFromPGProc(MyProc), lwWaitLink);
if (proclist_is_empty(&lock->waiters) &&
(pg_atomic_read_u32(&lock->state) & LW_FLAG_HAS_WAITERS) != 0)
diff --git a/src/backend/storage/lmgr/predicate.c b/src/backend/storage/lmgr/predicate.c
index ee5ea1175c7..93e40ae22f0 100644
--- a/src/backend/storage/lmgr/predicate.c
+++ b/src/backend/storage/lmgr/predicate.c
@@ -1811,7 +1811,7 @@ GetSerializableTransactionSnapshotInt(Snapshot snapshot,
sxact->finishedBefore = InvalidTransactionId;
sxact->xmin = snapshot->xmin;
sxact->pid = MyProcPid;
- sxact->pgprocno = MyProc->pgprocno;
+ sxact->pgprocno = GetNumberFromPGProc(MyProc);
dlist_init(&sxact->predicateLocks);
dlist_node_init(&sxact->finishedLink);
sxact->flags = 0;
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index e5977548fe2..e355768634e 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -227,7 +227,6 @@ InitProcGlobal(void)
InitSharedLatch(&(proc->procLatch));
LWLockInitialize(&(proc->fpInfoLock), LWTRANCHE_LOCK_FASTPATH);
}
- proc->pgprocno = i;
/*
* Newly created PGPROCs for normal backends, autovacuum and bgworkers
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index 00679624f7d..ed6071f3286 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -540,7 +540,7 @@ typedef enum
* used for a given lock group is determined by the group leader's pgprocno.
*/
#define LockHashPartitionLockByProc(leader_pgproc) \
- LockHashPartitionLock((leader_pgproc)->pgprocno)
+ LockHashPartitionLock(GetNumberFromPGProc(leader_pgproc))
/*
* function prototypes
diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h
index 4bc226e36cd..37cf8b4067d 100644
--- a/src/include/storage/proc.h
+++ b/src/include/storage/proc.h
@@ -194,11 +194,6 @@ struct PGPROC
int pgxactoff; /* offset into various ProcGlobal->arrays with
* data mirrored from this PGPROC */
- int pgprocno; /* Number of this PGPROC in
- * ProcGlobal->allProcs array. This is set
- * once by InitProcGlobal().
- * ProcGlobal->allProcs[n].pgprocno == n */
-
/* These fields are zero while a backend is still starting up: */
BackendId backendId; /* This backend's backend ID (if assigned) */
Oid databaseId; /* OID of database this backend is using */
@@ -412,6 +407,7 @@ extern PGDLLIMPORT PGPROC *PreparedXactProcs;
/* Accessor for PGPROC given a pgprocno. */
#define GetPGProcByNumber(n) (&ProcGlobal->allProcs[(n)])
+#define GetNumberFromPGProc(proc) ((proc) - &ProcGlobal->allProcs[0])
/*
* We set aside some extra PGPROC structures for auxiliary processes,
--
2.39.2
v8-0002-Redefine-backend-ID-to-be-an-index-into-the-proc-.patchtext/x-patch; charset=UTF-8; name=v8-0002-Redefine-backend-ID-to-be-an-index-into-the-proc-.patchDownload
From 4e0121e064804b73ef8a5dc10be27b85968ea1af Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Mon, 29 Jan 2024 23:50:34 +0200
Subject: [PATCH v8 2/5] Redefine backend ID to be an index into the proc
array.
Previously, backend ID was an index into the ProcState array, in the
shared cache invalidation manager (sinvaladt.c). The entry in the
ProcState array was reserved at backend startup by scanning the array
for a free entry, and that was also when the backend got its backend
ID. Things becomes slightly simpler if we redefine backend ID to be
the index into the PGPROC array, and directly use it also as an index
to the ProcState array. This uses a little more memory, as we reserve
a few extra slots in the ProcState array for aux processes that don't
need them, but the simplicity is worth it.
Aux processes now also have a backend ID. This simplifies the
reservation of BackendStatusArray and ProcSignal slots.
You can now convert a backend ID into an index into the PGPROC array
simply by subtracting 1. We still use 0-based "pgprocnos" in various
places, for indexes into the PGPROC array, but the only difference now
is that backend IDs start at 1 while pgprocnos start at 0.
One potential downside of this patch is that the ProcState array might
get less densely packed, as we we don't try so hard to assign
low-numbered backend ID anymore. If it's less densely packed,
lastBackend will stay at a higher value, and SIInsertDataEntries() and
SICleanupQueue() need to scan over more unused entries. I think that's
fine. They are performance critical enough to matter, and there was no
guarantee on dense packing before either: If you launched a lot of
backends concurrently, and kept the last one open, lastBackend would
also stay at a high value.
Another potential downside of not assigning low backend IDs as
aggressively is that you might end up with more pg_temp namespaces, if
more distinct backend IDs are used. I think that's fine, and there
hasn't been any guarantees on that either. But if those things matter,
we could work on the management of free PGPROC entries in proc.c, to
assign PGPROC entries and hence backend IDs differently.
Reviewed-by: XXX
Discussion: https://www.postgresql.org/message-id/8171f1aa-496f-46a6-afc3-c46fe7a9b407@iki.fi
---
src/backend/access/transam/twophase.c | 29 +---
src/backend/access/transam/xact.c | 3 +-
src/backend/catalog/namespace.c | 2 +-
src/backend/postmaster/auxprocess.c | 12 +-
src/backend/storage/ipc/procarray.c | 60 +++++++-
src/backend/storage/ipc/procsignal.c | 27 ++--
src/backend/storage/ipc/sinvaladt.c | 145 ++++----------------
src/backend/storage/lmgr/lock.c | 16 +--
src/backend/storage/lmgr/proc.c | 14 +-
src/backend/utils/activity/backend_status.c | 52 +++----
src/backend/utils/adt/mcxtfuncs.c | 14 +-
src/backend/utils/error/csvlog.c | 4 +-
src/backend/utils/error/elog.c | 6 +-
src/backend/utils/error/jsonlog.c | 4 +-
src/backend/utils/init/postinit.c | 10 +-
src/backend/utils/time/snapmgr.c | 4 +-
src/include/miscadmin.h | 2 -
src/include/storage/backendid.h | 12 +-
src/include/storage/lock.h | 2 +-
src/include/storage/proc.h | 10 +-
src/include/storage/procarray.h | 4 +
src/include/storage/procsignal.h | 2 +-
src/include/storage/sinvaladt.h | 4 -
23 files changed, 173 insertions(+), 265 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 234c8d08ebc..0b6dd3b3cf0 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -151,7 +151,6 @@ typedef struct GlobalTransactionData
{
GlobalTransaction next; /* list link for free list */
int pgprocno; /* ID of associated dummy PGPROC */
- BackendId dummyBackendId; /* similar to backend id for backends */
TimestampTz prepared_at; /* time of preparation */
/*
@@ -285,20 +284,6 @@ TwoPhaseShmemInit(void)
/* associate it with a PGPROC assigned by InitProcGlobal */
gxacts[i].pgprocno = GetNumberFromPGProc(&PreparedXactProcs[i]);
-
- /*
- * Assign a unique ID for each dummy proc, so that the range of
- * dummy backend IDs immediately follows the range of normal
- * backend IDs. We don't dare to assign a real backend ID to dummy
- * procs, because prepared transactions don't take part in cache
- * invalidation like a real backend ID would imply, but having a
- * unique ID for them is nevertheless handy. This arrangement
- * allows you to allocate an array of size (MaxBackends +
- * max_prepared_xacts + 1), and have a slot for every backend and
- * prepared transaction. Currently multixact.c uses that
- * technique.
- */
- gxacts[i].dummyBackendId = MaxBackends + 1 + i;
}
}
else
@@ -457,7 +442,7 @@ MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid, const char *gid,
Assert(LWLockHeldByMeInMode(TwoPhaseStateLock, LW_EXCLUSIVE));
Assert(gxact != NULL);
- proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ proc = GetPGProcByNumber(gxact->pgprocno);
/* Initialize the PGPROC entry */
MemSet(proc, 0, sizeof(PGPROC));
@@ -467,14 +452,12 @@ MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid, const char *gid,
{
/* clone VXID, for TwoPhaseGetXidByVirtualXID() to find */
proc->lxid = MyProc->lxid;
- proc->backendId = MyBackendId;
}
else
{
Assert(AmStartupProcess() || !IsPostmasterEnvironment);
/* GetLockConflicts() uses this to specify a wait on the XID */
proc->lxid = xid;
- proc->backendId = InvalidBackendId;
}
proc->xid = xid;
Assert(proc->xmin == InvalidTransactionId);
@@ -522,7 +505,7 @@ static void
GXactLoadSubxactData(GlobalTransaction gxact, int nsubxacts,
TransactionId *children)
{
- PGPROC *proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ PGPROC *proc = GetPGProcByNumber(gxact->pgprocno);
/* We need no extra lock since the GXACT isn't valid yet */
if (nsubxacts > PGPROC_MAX_CACHED_SUBXIDS)
@@ -559,7 +542,7 @@ MarkAsPrepared(GlobalTransaction gxact, bool lock_held)
* Put it into the global ProcArray so TransactionIdIsInProgress considers
* the XID as still running.
*/
- ProcArrayAdd(&ProcGlobal->allProcs[gxact->pgprocno]);
+ ProcArrayAdd(GetPGProcByNumber(gxact->pgprocno));
}
/*
@@ -583,7 +566,7 @@ LockGXact(const char *gid, Oid user)
for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
{
GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
- PGPROC *proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ PGPROC *proc = GetPGProcByNumber(gxact->pgprocno);
/* Ignore not-yet-valid GIDs */
if (!gxact->valid)
@@ -884,7 +867,7 @@ TwoPhaseGetXidByVirtualXID(VirtualTransactionId vxid,
if (!gxact->valid)
continue;
- proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ proc = GetPGProcByNumber(gxact->pgprocno);
GET_VXID_FROM_PGPROC(proc_vxid, *proc);
if (VirtualTransactionIdEquals(vxid, proc_vxid))
{
@@ -919,7 +902,7 @@ TwoPhaseGetDummyBackendId(TransactionId xid, bool lock_held)
{
GlobalTransaction gxact = TwoPhaseGetGXact(xid, lock_held);
- return gxact->dummyBackendId;
+ return gxact->pgprocno + 1;
}
/*
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 464858117e0..6c5c6147f13 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -2097,9 +2097,8 @@ StartTransaction(void)
/*
* Advertise it in the proc array. We assume assignment of
- * localTransactionId is atomic, and the backendId should be set already.
+ * localTransactionId is atomic.
*/
- Assert(MyProc->backendId == vxid.backendId);
MyProc->lxid = vxid.localTransactionId;
TRACE_POSTGRESQL_TRANSACTION_START(vxid.localTransactionId);
diff --git a/src/backend/catalog/namespace.c b/src/backend/catalog/namespace.c
index b610aa62423..e6eea92abd6 100644
--- a/src/backend/catalog/namespace.c
+++ b/src/backend/catalog/namespace.c
@@ -49,7 +49,7 @@
#include "parser/parse_func.h"
#include "storage/ipc.h"
#include "storage/lmgr.h"
-#include "storage/sinvaladt.h"
+#include "storage/procarray.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
diff --git a/src/backend/postmaster/auxprocess.c b/src/backend/postmaster/auxprocess.c
index ab86e802f21..39171fea06b 100644
--- a/src/backend/postmaster/auxprocess.c
+++ b/src/backend/postmaster/auxprocess.c
@@ -107,17 +107,7 @@ AuxiliaryProcessMain(AuxProcType auxtype)
BaseInit();
- /*
- * Assign the ProcSignalSlot for an auxiliary process. Since it doesn't
- * have a BackendId, the slot is statically allocated based on the
- * auxiliary process type (MyAuxProcType). Backends use slots indexed in
- * the range from 1 to MaxBackends (inclusive), so we use MaxBackends +
- * AuxProcType + 1 as the index of the slot for an auxiliary process.
- *
- * This will need rethinking if we ever want more than one of a particular
- * auxiliary process type.
- */
- ProcSignalInit(MaxBackends + MyAuxProcType + 1);
+ ProcSignalInit();
/*
* Auxiliary processes don't run transactions, but they may need a
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 5a33eb7f7f7..cd127a5357b 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -2546,7 +2546,7 @@ ProcArrayInstallImportedXmin(TransactionId xmin,
continue;
/* We are only interested in the specific virtual transaction. */
- if (proc->backendId != sourcevxid->backendId)
+ if (GetBackendIdFromPGProc(proc) != sourcevxid->backendId)
continue;
if (proc->lxid != sourcevxid->localTransactionId)
continue;
@@ -3097,6 +3097,64 @@ HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids, int type)
return result;
}
+/*
+ * BackendIdGetProc -- get a backend's PGPROC given its backend ID
+ *
+ * The result may be out of date arbitrarily quickly, so the caller
+ * must be careful about how this information is used. NULL is
+ * returned if the backend is not active.
+ */
+PGPROC *
+BackendIdGetProc(int backendID)
+{
+ PGPROC *result;
+
+ if (backendID < 1 || backendID > ProcGlobal->allProcCount)
+ return NULL;
+ result = GetPGProcByBackendId(backendID);
+
+ if (result->pid == 0)
+ return NULL;
+
+ return result;
+}
+
+/*
+ * BackendIdGetTransactionIds -- get a backend's transaction status
+ *
+ * Get the xid, xmin, nsubxid and overflow status of the backend. The
+ * result may be out of date arbitrarily quickly, so the caller must be
+ * careful about how this information is used.
+ */
+void
+BackendIdGetTransactionIds(int backendID, TransactionId *xid,
+ TransactionId *xmin, int *nsubxid, bool *overflowed)
+{
+ PGPROC *proc;
+
+ *xid = InvalidTransactionId;
+ *xmin = InvalidTransactionId;
+ *nsubxid = 0;
+ *overflowed = false;
+
+ if (backendID < 1 || backendID > ProcGlobal->allProcCount)
+ return;
+ proc = GetPGProcByBackendId(backendID);
+
+ /* Need to lock out additions/removals of backends */
+ LWLockAcquire(ProcArrayLock, LW_SHARED);
+
+ if (proc->pid != 0)
+ {
+ *xid = proc->xid;
+ *xmin = proc->xmin;
+ *nsubxid = proc->subxidStatus.count;
+ *overflowed = proc->subxidStatus.overflowed;
+ }
+
+ LWLockRelease(ProcArrayLock);
+}
+
/*
* BackendPidGetProc -- get a backend's PGPROC given its PID
*
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index e84619e5a58..d1d5bf0c152 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -87,7 +87,7 @@ typedef struct
* possible auxiliary process type. (This scheme assumes there is not
* more than one of any auxiliary process type at a time.)
*/
-#define NumProcSignalSlots (MaxBackends + NUM_AUXPROCTYPES)
+#define NumProcSignalSlots (MaxBackends + NUM_AUXILIARY_PROCS)
/* Check whether the relevant type bit is set in the flags. */
#define BARRIER_SHOULD_CHECK(flags, type) \
@@ -154,24 +154,23 @@ ProcSignalShmemInit(void)
/*
* ProcSignalInit
* Register the current process in the ProcSignal array
- *
- * The passed index should be my BackendId if the process has one,
- * or MaxBackends + aux process type if not.
*/
void
-ProcSignalInit(int pss_idx)
+ProcSignalInit(void)
{
ProcSignalSlot *slot;
uint64 barrier_generation;
- Assert(pss_idx >= 1 && pss_idx <= NumProcSignalSlots);
-
- slot = &ProcSignal->psh_slot[pss_idx - 1];
+ if (MyBackendId <= 0)
+ elog(ERROR, "MyBackendId not set");
+ if (MyBackendId > NumProcSignalSlots)
+ elog(ERROR, "unexpected MyBackendId %d in ProcSignalInit (max %d)", MyBackendId, NumProcSignalSlots);
+ slot = &ProcSignal->psh_slot[MyBackendId - 1];
/* sanity check */
if (slot->pss_pid != 0)
elog(LOG, "process %d taking over ProcSignal slot %d, but it's not empty",
- MyProcPid, pss_idx);
+ MyProcPid, (int) (slot - ProcSignal->psh_slot));
/* Clear out any leftover signal reasons */
MemSet(slot->pss_signalFlags, 0, NUM_PROCSIGNALS * sizeof(sig_atomic_t));
@@ -200,7 +199,7 @@ ProcSignalInit(int pss_idx)
MyProcSignalSlot = slot;
/* Set up to release the slot on process exit */
- on_shmem_exit(CleanupProcSignalState, Int32GetDatum(pss_idx));
+ on_shmem_exit(CleanupProcSignalState, (Datum) 0);
}
/*
@@ -212,11 +211,7 @@ ProcSignalInit(int pss_idx)
static void
CleanupProcSignalState(int status, Datum arg)
{
- int pss_idx = DatumGetInt32(arg);
- ProcSignalSlot *slot;
-
- slot = &ProcSignal->psh_slot[pss_idx - 1];
- Assert(slot == MyProcSignalSlot);
+ ProcSignalSlot *slot = MyProcSignalSlot;
/*
* Clear MyProcSignalSlot, so that a SIGUSR1 received after this point
@@ -233,7 +228,7 @@ CleanupProcSignalState(int status, Datum arg)
* infinite loop trying to exit
*/
elog(LOG, "process %d releasing ProcSignal slot %d, but it contains %d",
- MyProcPid, pss_idx, (int) slot->pss_pid);
+ MyProcPid, (int) (slot - ProcSignal->psh_slot), (int) slot->pss_pid);
return; /* XXX better to zero the slot anyway? */
}
diff --git a/src/backend/storage/ipc/sinvaladt.c b/src/backend/storage/ipc/sinvaladt.c
index 748a792a854..8105717c578 100644
--- a/src/backend/storage/ipc/sinvaladt.c
+++ b/src/backend/storage/ipc/sinvaladt.c
@@ -139,7 +139,6 @@ typedef struct ProcState
{
/* procPid is zero in an inactive ProcState array entry. */
pid_t procPid; /* PID of backend, for signaling */
- PGPROC *proc; /* PGPROC of backend */
/* nextMsgNum is meaningless if procPid == 0 or resetState is true. */
int nextMsgNum; /* next message number to read */
bool resetState; /* backend needs to reset its state */
@@ -173,7 +172,6 @@ typedef struct SISeg
int maxMsgNum; /* next message number to be assigned */
int nextThreshold; /* # of messages to call SICleanupQueue */
int lastBackend; /* index of last active procState entry, +1 */
- int maxBackends; /* size of procState array */
slock_t msgnumLock; /* spinlock protecting maxMsgNum */
@@ -183,11 +181,18 @@ typedef struct SISeg
SharedInvalidationMessage buffer[MAXNUMMESSAGES];
/*
- * Per-backend invalidation state info (has MaxBackends entries).
+ * Per-backend invalidation state info (has NumProcStateSlots entries).
*/
ProcState procState[FLEXIBLE_ARRAY_MEMBER];
} SISeg;
+/*
+ * We reserve a slot for each possible BackendId, plus one for each
+ * possible auxiliary process type. (This scheme assumes there is not
+ * more than one of any auxiliary process type at a time.)
+ */
+#define NumProcStateSlots (MaxBackends + NUM_AUXILIARY_PROCS)
+
static SISeg *shmInvalBuffer; /* pointer to the shared inval buffer */
@@ -205,16 +210,7 @@ SInvalShmemSize(void)
Size size;
size = offsetof(SISeg, procState);
-
- /*
- * In Hot Standby mode, the startup process requests a procState array
- * slot using InitRecoveryTransactionEnvironment(). Even though
- * MaxBackends doesn't account for the startup process, it is guaranteed
- * to get a free slot. This is because the autovacuum launcher and worker
- * processes, which are included in MaxBackends, are not started in Hot
- * Standby mode.
- */
- size = add_size(size, mul_size(sizeof(ProcState), MaxBackends));
+ size = add_size(size, mul_size(sizeof(ProcState), NumProcStateSlots));
return size;
}
@@ -240,16 +236,14 @@ CreateSharedInvalidationState(void)
shmInvalBuffer->maxMsgNum = 0;
shmInvalBuffer->nextThreshold = CLEANUP_MIN;
shmInvalBuffer->lastBackend = 0;
- shmInvalBuffer->maxBackends = MaxBackends;
SpinLockInit(&shmInvalBuffer->msgnumLock);
/* The buffer[] array is initially all unused, so we need not fill it */
/* Mark all backends inactive, and initialize nextLXID */
- for (i = 0; i < shmInvalBuffer->maxBackends; i++)
+ for (i = 0; i < NumProcStateSlots; i++)
{
shmInvalBuffer->procState[i].procPid = 0; /* inactive */
- shmInvalBuffer->procState[i].proc = NULL;
shmInvalBuffer->procState[i].nextMsgNum = 0; /* meaningless */
shmInvalBuffer->procState[i].resetState = false;
shmInvalBuffer->procState[i].signaled = false;
@@ -265,10 +259,17 @@ CreateSharedInvalidationState(void)
void
SharedInvalBackendInit(bool sendOnly)
{
- int index;
- ProcState *stateP = NULL;
+ ProcState *stateP;
+ pid_t oldPid;
SISeg *segP = shmInvalBuffer;
+ if (MyBackendId <= 0)
+ elog(ERROR, "MyBackendId not set");
+ if (MyBackendId > NumProcStateSlots)
+ elog(PANIC, "unexpected MyBackendId %d in SharedInvalBackendInit (max %d)",
+ MyBackendId, NumProcStateSlots);
+ stateP = &segP->procState[MyBackendId - 1];
+
/*
* This can run in parallel with read operations, but not with write
* operations, since SIInsertDataEntries relies on lastBackend to set
@@ -276,48 +277,22 @@ SharedInvalBackendInit(bool sendOnly)
*/
LWLockAcquire(SInvalWriteLock, LW_EXCLUSIVE);
- /* Look for a free entry in the procState array */
- for (index = 0; index < segP->lastBackend; index++)
- {
- if (segP->procState[index].procPid == 0) /* inactive slot? */
- {
- stateP = &segP->procState[index];
- break;
- }
- }
-
- if (stateP == NULL)
+ oldPid = stateP->procPid;
+ if (oldPid != 0)
{
- if (segP->lastBackend < segP->maxBackends)
- {
- stateP = &segP->procState[segP->lastBackend];
- Assert(stateP->procPid == 0);
- segP->lastBackend++;
- }
- else
- {
- /*
- * out of procState slots: MaxBackends exceeded -- report normally
- */
- MyBackendId = InvalidBackendId;
- LWLockRelease(SInvalWriteLock);
- ereport(FATAL,
- (errcode(ERRCODE_TOO_MANY_CONNECTIONS),
- errmsg("sorry, too many clients already")));
- }
+ LWLockRelease(SInvalWriteLock);
+ elog(ERROR, "sinval slot for backend %d is already in use by process %d",
+ MyBackendId, oldPid);
}
- MyBackendId = (stateP - &segP->procState[0]) + 1;
-
- /* Advertise assigned backend ID in MyProc */
- MyProc->backendId = MyBackendId;
+ if (MyBackendId > segP->lastBackend)
+ segP->lastBackend = MyBackendId;
/* Fetch next local transaction ID into local memory */
nextLocalTransactionId = stateP->nextLXID;
/* mark myself active, with all extant messages already read */
stateP->procPid = MyProcPid;
- stateP->proc = MyProc;
stateP->nextMsgNum = segP->maxMsgNum;
stateP->resetState = false;
stateP->signaled = false;
@@ -328,8 +303,6 @@ SharedInvalBackendInit(bool sendOnly)
/* register exit routine to mark my entry inactive at exit */
on_shmem_exit(CleanupInvalidationState, PointerGetDatum(segP));
-
- elog(DEBUG4, "my backend ID is %d", MyBackendId);
}
/*
@@ -358,7 +331,6 @@ CleanupInvalidationState(int status, Datum arg)
/* Mark myself inactive */
stateP->procPid = 0;
- stateP->proc = NULL;
stateP->nextMsgNum = 0;
stateP->resetState = false;
stateP->signaled = false;
@@ -374,71 +346,6 @@ CleanupInvalidationState(int status, Datum arg)
LWLockRelease(SInvalWriteLock);
}
-/*
- * BackendIdGetProc
- * Get the PGPROC structure for a backend, given the backend ID.
- * The result may be out of date arbitrarily quickly, so the caller
- * must be careful about how this information is used. NULL is
- * returned if the backend is not active.
- */
-PGPROC *
-BackendIdGetProc(int backendID)
-{
- PGPROC *result = NULL;
- SISeg *segP = shmInvalBuffer;
-
- /* Need to lock out additions/removals of backends */
- LWLockAcquire(SInvalWriteLock, LW_SHARED);
-
- if (backendID > 0 && backendID <= segP->lastBackend)
- {
- ProcState *stateP = &segP->procState[backendID - 1];
-
- result = stateP->proc;
- }
-
- LWLockRelease(SInvalWriteLock);
-
- return result;
-}
-
-/*
- * BackendIdGetTransactionIds
- * Get the xid, xmin, nsubxid and overflow status of the backend. The
- * result may be out of date arbitrarily quickly, so the caller must be
- * careful about how this information is used.
- */
-void
-BackendIdGetTransactionIds(int backendID, TransactionId *xid,
- TransactionId *xmin, int *nsubxid, bool *overflowed)
-{
- SISeg *segP = shmInvalBuffer;
-
- *xid = InvalidTransactionId;
- *xmin = InvalidTransactionId;
- *nsubxid = 0;
- *overflowed = false;
-
- /* Need to lock out additions/removals of backends */
- LWLockAcquire(SInvalWriteLock, LW_SHARED);
-
- if (backendID > 0 && backendID <= segP->lastBackend)
- {
- ProcState *stateP = &segP->procState[backendID - 1];
- PGPROC *proc = stateP->proc;
-
- if (proc != NULL)
- {
- *xid = proc->xid;
- *xmin = proc->xmin;
- *nsubxid = proc->subxidStatus.count;
- *overflowed = proc->subxidStatus.overflowed;
- }
- }
-
- LWLockRelease(SInvalWriteLock);
-}
-
/*
* SIInsertDataEntries
* Add new invalidation message(s) to the buffer.
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index c70a1adb9ad..0252cacf0a7 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -3625,7 +3625,7 @@ GetLockStatusData(void)
proc->fpRelId[f]);
instance->holdMask = lockbits << FAST_PATH_LOCKNUMBER_OFFSET;
instance->waitLockMode = NoLock;
- instance->backend = proc->backendId;
+ instance->backend = GetBackendIdFromPGProc(proc);
instance->lxid = proc->lxid;
instance->pid = proc->pid;
instance->leaderPid = proc->pid;
@@ -3652,14 +3652,14 @@ GetLockStatusData(void)
repalloc(data->locks, sizeof(LockInstanceData) * els);
}
- vxid.backendId = proc->backendId;
+ vxid.backendId = GetBackendIdFromPGProc(proc);
vxid.localTransactionId = proc->fpLocalTransactionId;
instance = &data->locks[el];
SET_LOCKTAG_VIRTUALTRANSACTION(instance->locktag, vxid);
instance->holdMask = LOCKBIT_ON(ExclusiveLock);
instance->waitLockMode = NoLock;
- instance->backend = proc->backendId;
+ instance->backend = GetBackendIdFromPGProc(proc);
instance->lxid = proc->lxid;
instance->pid = proc->pid;
instance->leaderPid = proc->pid;
@@ -3712,7 +3712,7 @@ GetLockStatusData(void)
instance->waitLockMode = proc->waitLockMode;
else
instance->waitLockMode = NoLock;
- instance->backend = proc->backendId;
+ instance->backend = GetBackendIdFromPGProc(proc);
instance->lxid = proc->lxid;
instance->pid = proc->pid;
instance->leaderPid = proclock->groupLeader->pid;
@@ -3888,7 +3888,7 @@ GetSingleProcBlockerStatusData(PGPROC *blocked_proc, BlockedProcsData *data)
instance->waitLockMode = proc->waitLockMode;
else
instance->waitLockMode = NoLock;
- instance->backend = proc->backendId;
+ instance->backend = GetBackendIdFromPGProc(proc);
instance->lxid = proc->lxid;
instance->pid = proc->pid;
instance->leaderPid = proclock->groupLeader->pid;
@@ -4391,7 +4391,7 @@ VirtualXactLockTableInsert(VirtualTransactionId vxid)
LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE);
- Assert(MyProc->backendId == vxid.backendId);
+ Assert(MyBackendId == vxid.backendId);
Assert(MyProc->fpLocalTransactionId == InvalidLocalTransactionId);
Assert(MyProc->fpVXIDLock == false);
@@ -4413,8 +4413,6 @@ VirtualXactLockTableCleanup(void)
bool fastpath;
LocalTransactionId lxid;
- Assert(MyProc->backendId != InvalidBackendId);
-
/*
* Clean up shared memory state.
*/
@@ -4541,7 +4539,7 @@ VirtualXactLock(VirtualTransactionId vxid, bool wait)
*/
LWLockAcquire(&proc->fpInfoLock, LW_EXCLUSIVE);
- if (proc->backendId != vxid.backendId
+ if (GetBackendIdFromPGProc(proc) != vxid.backendId
|| proc->fpLocalTransactionId != vxid.localTransactionId)
{
/* VXID ended */
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index e355768634e..9c25f631c5d 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -239,25 +239,25 @@ InitProcGlobal(void)
if (i < MaxConnections)
{
/* PGPROC for normal backend, add to freeProcs list */
- dlist_push_head(&ProcGlobal->freeProcs, &proc->links);
+ dlist_push_tail(&ProcGlobal->freeProcs, &proc->links);
proc->procgloballist = &ProcGlobal->freeProcs;
}
else if (i < MaxConnections + autovacuum_max_workers + 1)
{
/* PGPROC for AV launcher/worker, add to autovacFreeProcs list */
- dlist_push_head(&ProcGlobal->autovacFreeProcs, &proc->links);
+ dlist_push_tail(&ProcGlobal->autovacFreeProcs, &proc->links);
proc->procgloballist = &ProcGlobal->autovacFreeProcs;
}
else if (i < MaxConnections + autovacuum_max_workers + 1 + max_worker_processes)
{
/* PGPROC for bgworker, add to bgworkerFreeProcs list */
- dlist_push_head(&ProcGlobal->bgworkerFreeProcs, &proc->links);
+ dlist_push_tail(&ProcGlobal->bgworkerFreeProcs, &proc->links);
proc->procgloballist = &ProcGlobal->bgworkerFreeProcs;
}
else if (i < MaxBackends)
{
/* PGPROC for walsender, add to walsenderFreeProcs list */
- dlist_push_head(&ProcGlobal->walsenderFreeProcs, &proc->links);
+ dlist_push_tail(&ProcGlobal->walsenderFreeProcs, &proc->links);
proc->procgloballist = &ProcGlobal->walsenderFreeProcs;
}
@@ -351,6 +351,7 @@ InitProcess(void)
(errcode(ERRCODE_TOO_MANY_CONNECTIONS),
errmsg("sorry, too many clients already")));
}
+ MyBackendId = GetBackendIdFromPGProc(MyProc);
/*
* Cross-check that the PGPROC is of the type we expect; if this were not
@@ -380,7 +381,6 @@ InitProcess(void)
MyProc->xmin = InvalidTransactionId;
MyProc->pid = MyProcPid;
/* backendId, databaseId and roleId will be filled in later */
- MyProc->backendId = InvalidBackendId;
MyProc->databaseId = InvalidOid;
MyProc->roleId = InvalidOid;
MyProc->tempNamespaceId = InvalidOid;
@@ -561,6 +561,7 @@ InitAuxiliaryProcess(void)
((volatile PGPROC *) auxproc)->pid = MyProcPid;
MyProc = auxproc;
+ MyBackendId = GetBackendIdFromPGProc(MyProc);
SpinLockRelease(ProcStructLock);
@@ -575,7 +576,6 @@ InitAuxiliaryProcess(void)
MyProc->fpLocalTransactionId = InvalidLocalTransactionId;
MyProc->xid = InvalidTransactionId;
MyProc->xmin = InvalidTransactionId;
- MyProc->backendId = InvalidBackendId;
MyProc->databaseId = InvalidOid;
MyProc->roleId = InvalidOid;
MyProc->tempNamespaceId = InvalidOid;
@@ -905,6 +905,7 @@ ProcKill(int code, Datum arg)
proc = MyProc;
MyProc = NULL;
+ MyBackendId = InvalidBackendId;
DisownLatch(&proc->procLatch);
procgloballist = proc->procgloballist;
@@ -976,6 +977,7 @@ AuxiliaryProcKill(int code, Datum arg)
proc = MyProc;
MyProc = NULL;
+ MyBackendId = InvalidBackendId;
DisownLatch(&proc->procLatch);
SpinLockAcquire(ProcStructLock);
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 1a1050c8da1..3d3f7b06723 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -19,6 +19,7 @@
#include "port/atomics.h" /* for memory barriers */
#include "storage/ipc.h"
#include "storage/proc.h" /* for MyProc */
+#include "storage/procarray.h"
#include "storage/sinvaladt.h"
#include "utils/ascii.h"
#include "utils/backend_status.h"
@@ -29,13 +30,12 @@
/* ----------
* Total number of backends including auxiliary
*
- * We reserve a slot for each possible BackendId, plus one for each
- * possible auxiliary process type. (This scheme assumes there is not
- * more than one of any auxiliary process type at a time.) MaxBackends
- * includes autovacuum workers and background workers as well.
+ * We reserve a slot for each possible PGPROC entry, including aux processes.
+ * (But not including PGPROC entries reserved for prepared xacts; they are not
+ * real processes.)
* ----------
*/
-#define NumBackendStatSlots (MaxBackends + NUM_AUXPROCTYPES)
+#define NumBackendStatSlots (MaxBackends + NUM_AUXILIARY_PROCS)
/* ----------
@@ -238,10 +238,9 @@ CreateSharedBackendStatus(void)
/*
* Initialize pgstats backend activity state, and set up our on-proc-exit
- * hook. Called from InitPostgres and AuxiliaryProcessMain. For auxiliary
- * process, MyBackendId is invalid. Otherwise, MyBackendId must be set, but we
- * must not have started any transaction yet (since the exit hook must run
- * after the last transaction exit).
+ * hook. Called from InitPostgres and AuxiliaryProcessMain. MyBackendId must
+ * be set, but we must not have started any transaction yet (since the exit
+ * hook must run after the last transaction exit).
*
* NOTE: MyDatabaseId isn't set yet; so the shutdown hook has to be careful.
*/
@@ -249,26 +248,9 @@ void
pgstat_beinit(void)
{
/* Initialize MyBEEntry */
- if (MyBackendId != InvalidBackendId)
- {
- Assert(MyBackendId >= 1 && MyBackendId <= MaxBackends);
- MyBEEntry = &BackendStatusArray[MyBackendId - 1];
- }
- else
- {
- /* Must be an auxiliary process */
- Assert(MyAuxProcType != NotAnAuxProcess);
-
- /*
- * Assign the MyBEEntry for an auxiliary process. Since it doesn't
- * have a BackendId, the slot is statically allocated based on the
- * auxiliary process type (MyAuxProcType). Backends use slots indexed
- * in the range from 0 to MaxBackends (exclusive), so we use
- * MaxBackends + AuxProcType as the index of the slot for an auxiliary
- * process.
- */
- MyBEEntry = &BackendStatusArray[MaxBackends + MyAuxProcType];
- }
+ Assert(MyBackendId != InvalidBackendId);
+ Assert(MyBackendId >= 1 && MyBackendId <= NumBackendStatSlots);
+ MyBEEntry = &BackendStatusArray[MyBackendId - 1];
/* Set up a process-exit hook to clean up */
on_shmem_exit(pgstat_beshutdown_hook, 0);
@@ -281,12 +263,12 @@ pgstat_beinit(void)
* Initialize this backend's entry in the PgBackendStatus array.
* Called from InitPostgres.
*
- * Apart from auxiliary processes, MyBackendId, MyDatabaseId,
- * session userid, and application_name must be set for a
- * backend (hence, this cannot be combined with pgstat_beinit).
- * Note also that we must be inside a transaction if this isn't an aux
- * process, as we may need to do encoding conversion on some strings.
- * ----------
+ * Apart from auxiliary processes, MyDatabaseId, session userid, and
+ * application_name must already be set (hence, this cannot be combined
+ * with pgstat_beinit). Note also that we must be inside a transaction
+ * if this isn't an aux process, as we may need to do encoding conversion
+ * on some strings.
+ *----------
*/
void
pgstat_bestart(void)
diff --git a/src/backend/utils/adt/mcxtfuncs.c b/src/backend/utils/adt/mcxtfuncs.c
index 4708d73f5fa..a7267dc15d1 100644
--- a/src/backend/utils/adt/mcxtfuncs.c
+++ b/src/backend/utils/adt/mcxtfuncs.c
@@ -148,19 +148,11 @@ pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
PGPROC *proc;
BackendId backendId = InvalidBackendId;
- proc = BackendPidGetProc(pid);
-
/*
* See if the process with given pid is a backend or an auxiliary process.
- *
- * If the given process is a backend, use its backend id in
- * SendProcSignal() later to speed up the operation. Otherwise, don't do
- * that because auxiliary processes (except the startup process) don't
- * have a valid backend id.
*/
- if (proc != NULL)
- backendId = proc->backendId;
- else
+ proc = BackendPidGetProc(pid);
+ if (proc == NULL)
proc = AuxiliaryPidGetProc(pid);
/*
@@ -183,6 +175,8 @@ pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(false);
}
+ if (proc != NULL)
+ backendId = GetBackendIdFromPGProc(proc);
if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, backendId) < 0)
{
/* Again, just a warning to allow loops */
diff --git a/src/backend/utils/error/csvlog.c b/src/backend/utils/error/csvlog.c
index 1b62b07f231..692ee3c1491 100644
--- a/src/backend/utils/error/csvlog.c
+++ b/src/backend/utils/error/csvlog.c
@@ -152,8 +152,8 @@ write_csvlog(ErrorData *edata)
/* Virtual transaction id */
/* keep VXID format in sync with lockfuncs.c */
- if (MyProc != NULL && MyProc->backendId != InvalidBackendId)
- appendStringInfo(&buf, "%d/%u", MyProc->backendId, MyProc->lxid);
+ if (MyProc != NULL)
+ appendStringInfo(&buf, "%d/%u", MyBackendId, MyProc->lxid);
appendStringInfoChar(&buf, ',');
/* Transaction id */
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 2c7a20e3d31..10249f140fe 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -3074,18 +3074,18 @@ log_status_format(StringInfo buf, const char *format, ErrorData *edata)
break;
case 'v':
/* keep VXID format in sync with lockfuncs.c */
- if (MyProc != NULL && MyProc->backendId != InvalidBackendId)
+ if (MyProc != NULL)
{
if (padding != 0)
{
char strfbuf[128];
snprintf(strfbuf, sizeof(strfbuf) - 1, "%d/%u",
- MyProc->backendId, MyProc->lxid);
+ MyBackendId, MyProc->lxid);
appendStringInfo(buf, "%*s", padding, strfbuf);
}
else
- appendStringInfo(buf, "%d/%u", MyProc->backendId, MyProc->lxid);
+ appendStringInfo(buf, "%d/%u", MyBackendId, MyProc->lxid);
}
else if (padding != 0)
appendStringInfoSpaces(buf,
diff --git a/src/backend/utils/error/jsonlog.c b/src/backend/utils/error/jsonlog.c
index 2903561f1c4..3ebe0b7b13c 100644
--- a/src/backend/utils/error/jsonlog.c
+++ b/src/backend/utils/error/jsonlog.c
@@ -197,8 +197,8 @@ write_jsonlog(ErrorData *edata)
/* Virtual transaction id */
/* keep VXID format in sync with lockfuncs.c */
- if (MyProc != NULL && MyProc->backendId != InvalidBackendId)
- appendJSONKeyValueFmt(&buf, "vxid", true, "%d/%u", MyProc->backendId,
+ if (MyProc != NULL)
+ appendJSONKeyValueFmt(&buf, "vxid", true, "%d/%u", MyBackendId,
MyProc->lxid);
/* Transaction id */
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index 1ad33671598..e822cd61d5f 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -740,18 +740,10 @@ InitPostgres(const char *in_dbname, Oid dboid,
/*
* Initialize my entry in the shared-invalidation manager's array of
* per-backend data.
- *
- * Sets up MyBackendId, a unique backend identifier.
*/
- MyBackendId = InvalidBackendId;
-
SharedInvalBackendInit(false);
- if (MyBackendId > MaxBackends || MyBackendId <= 0)
- elog(FATAL, "bad backend ID: %d", MyBackendId);
-
- /* Now that we have a BackendId, we can participate in ProcSignal */
- ProcSignalInit(MyBackendId);
+ ProcSignalInit();
/*
* Also set up timeout handlers needed for backend operation. We need
diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
index 675e81d82d7..94ddf68fa52 100644
--- a/src/backend/utils/time/snapmgr.c
+++ b/src/backend/utils/time/snapmgr.c
@@ -1154,7 +1154,7 @@ ExportSnapshot(Snapshot snapshot)
* inside the transaction from 1.
*/
snprintf(path, sizeof(path), SNAPSHOT_EXPORT_DIR "/%08X-%08X-%d",
- MyProc->backendId, MyProc->lxid, list_length(exportedSnapshots) + 1);
+ MyBackendId, MyProc->lxid, list_length(exportedSnapshots) + 1);
/*
* Copy the snapshot into TopTransactionContext, add it to the
@@ -1181,7 +1181,7 @@ ExportSnapshot(Snapshot snapshot)
*/
initStringInfo(&buf);
- appendStringInfo(&buf, "vxid:%d/%u\n", MyProc->backendId, MyProc->lxid);
+ appendStringInfo(&buf, "vxid:%d/%u\n", MyBackendId, MyProc->lxid);
appendStringInfo(&buf, "pid:%d\n", MyProcPid);
appendStringInfo(&buf, "dbid:%u\n", MyDatabaseId);
appendStringInfo(&buf, "iso:%d\n", XactIsoLevel);
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 0b01c1f0935..cbdc61b8576 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -444,8 +444,6 @@ typedef enum
WalWriterProcess,
WalReceiverProcess,
WalSummarizerProcess,
-
- NUM_AUXPROCTYPES /* Must be last! */
} AuxProcType;
extern PGDLLIMPORT AuxProcType MyAuxProcType;
diff --git a/src/include/storage/backendid.h b/src/include/storage/backendid.h
index 50ac982da19..01387723f79 100644
--- a/src/include/storage/backendid.h
+++ b/src/include/storage/backendid.h
@@ -14,11 +14,15 @@
#ifndef BACKENDID_H
#define BACKENDID_H
-/* ----------------
- * -cim 8/17/90
- * ----------------
+/*
+ * BackendId uniquely identifies an active backend or auxiliary process. It's
+ * assigned at backend startup after authentication. Note that a backend ID
+ * can be reused for a different backend immediately after a backend exits.
+ *
+ * Backend IDs are assigned starting from 1. For historical reasons, BackendId
+ * 0 is unused, but InvalidBackendId is defined as -1.
*/
-typedef int BackendId; /* unique currently active backend identifier */
+typedef int BackendId;
#define InvalidBackendId (-1)
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index ed6071f3286..03a920b2254 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -75,7 +75,7 @@ typedef struct
((vxid).backendId = InvalidBackendId, \
(vxid).localTransactionId = InvalidLocalTransactionId)
#define GET_VXID_FROM_PGPROC(vxid, proc) \
- ((vxid).backendId = (proc).backendId, \
+ ((vxid).backendId = GetBackendIdFromPGProc(&(proc)), \
(vxid).localTransactionId = (proc).lxid)
/* MAX_LOCKMODES cannot be larger than the # of bits in LOCKMASK */
diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h
index 37cf8b4067d..0d424a3334d 100644
--- a/src/include/storage/proc.h
+++ b/src/include/storage/proc.h
@@ -195,7 +195,6 @@ struct PGPROC
* data mirrored from this PGPROC */
/* These fields are zero while a backend is still starting up: */
- BackendId backendId; /* This backend's backend ID (if assigned) */
Oid databaseId; /* OID of database this backend is using */
Oid roleId; /* OID of role using this backend */
@@ -405,9 +404,16 @@ extern PGDLLIMPORT PROC_HDR *ProcGlobal;
extern PGDLLIMPORT PGPROC *PreparedXactProcs;
-/* Accessor for PGPROC given a pgprocno. */
+/*
+ * Accessors for getting PGPROC given a pgprocno or BackendId, and vice versa.
+ *
+ * For historical reasons, some code uses 0-based "proc numbers", while other
+ * code uses 1-based backend IDs.
+ */
#define GetPGProcByNumber(n) (&ProcGlobal->allProcs[(n)])
#define GetNumberFromPGProc(proc) ((proc) - &ProcGlobal->allProcs[0])
+#define GetPGProcByBackendId(n) (&ProcGlobal->allProcs[(n) - 1])
+#define GetBackendIdFromPGProc(proc) (GetNumberFromPGProc(proc) + 1)
/*
* We set aside some extra PGPROC structures for auxiliary processes,
diff --git a/src/include/storage/procarray.h b/src/include/storage/procarray.h
index f3eba9b7640..3af7577e8c6 100644
--- a/src/include/storage/procarray.h
+++ b/src/include/storage/procarray.h
@@ -64,6 +64,10 @@ extern VirtualTransactionId *GetVirtualXIDsDelayingChkpt(int *nvxids, int type);
extern bool HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids,
int nvxids, int type);
+extern PGPROC *BackendIdGetProc(int backendID);
+extern void BackendIdGetTransactionIds(int backendID, TransactionId *xid,
+ TransactionId *xmin, int *nsubxid,
+ bool *overflowed);
extern PGPROC *BackendPidGetProc(int pid);
extern PGPROC *BackendPidGetProcWithLock(int pid);
extern int BackendXidGetPid(TransactionId xid);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 52dcb4c2adf..febdda3611c 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -62,7 +62,7 @@ typedef enum
extern Size ProcSignalShmemSize(void);
extern void ProcSignalShmemInit(void);
-extern void ProcSignalInit(int pss_idx);
+extern void ProcSignalInit(void);
extern int SendProcSignal(pid_t pid, ProcSignalReason reason,
BackendId backendId);
diff --git a/src/include/storage/sinvaladt.h b/src/include/storage/sinvaladt.h
index aa3d203efca..c3c97b3f8b7 100644
--- a/src/include/storage/sinvaladt.h
+++ b/src/include/storage/sinvaladt.h
@@ -31,10 +31,6 @@
extern Size SInvalShmemSize(void);
extern void CreateSharedInvalidationState(void);
extern void SharedInvalBackendInit(bool sendOnly);
-extern PGPROC *BackendIdGetProc(int backendID);
-extern void BackendIdGetTransactionIds(int backendID, TransactionId *xid,
- TransactionId *xmin, int *nsubxid,
- bool *overflowed);
extern void SIInsertDataEntries(const SharedInvalidationMessage *data, int n);
extern int SIGetDataEntries(SharedInvalidationMessage *data, int datasize);
--
2.39.2
v8-0003-Replace-lastBackend-with-an-array-of-in-use-slots.patchtext/x-patch; charset=UTF-8; name=v8-0003-Replace-lastBackend-with-an-array-of-in-use-slots.patchDownload
From 94fd46c9ef30ba5e8ac1a8873fce577a4be425f4 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Mon, 29 Jan 2024 22:57:49 +0200
Subject: [PATCH v8 3/5] Replace 'lastBackend' with an array of in-use slots
Now that the procState array is indexed by pgprocno, the 'lastBackend'
optimization is useless, because aux processes are assigned PGPROC
slots and hence have numbers higher than max_connection. So
'lastBackend' was always set to almost the end of the array.
To replace that optimization, mantain a dense array of in-use
indexes. This's redundant with ProgGlobal->procarray, but I was afraid
of adding any more contention to ProcArrayLock, and this keeps the
code isolated to sinvaladt.c too.
It's not clear if we need that optimization at all. I was able to
write a test case that become slower without this: set max_connections
to a very high number (> 5000), and create+truncate a table in the
same transaction thousands of times to send invalidation messages,
with fsync=off. That became about 20% slower on my laptop. Arguably
that's so unrealistic that it doesn't matter, but nevertheless, this
commit restores the performance of that.
---
src/backend/storage/ipc/sinvaladt.c | 58 +++++++++++++++++------------
1 file changed, 35 insertions(+), 23 deletions(-)
diff --git a/src/backend/storage/ipc/sinvaladt.c b/src/backend/storage/ipc/sinvaladt.c
index 8105717c578..4d44b90e7b8 100644
--- a/src/backend/storage/ipc/sinvaladt.c
+++ b/src/backend/storage/ipc/sinvaladt.c
@@ -171,7 +171,6 @@ typedef struct SISeg
int minMsgNum; /* oldest message still needed */
int maxMsgNum; /* next message number to be assigned */
int nextThreshold; /* # of messages to call SICleanupQueue */
- int lastBackend; /* index of last active procState entry, +1 */
slock_t msgnumLock; /* spinlock protecting maxMsgNum */
@@ -181,8 +180,14 @@ typedef struct SISeg
SharedInvalidationMessage buffer[MAXNUMMESSAGES];
/*
- * Per-backend invalidation state info (has NumProcStateSlots entries).
+ * Per-backend invalidation state info.
+ *
+ * 'procState' has NumProcStateSlots entries, and is indexed by pgprocno.
+ * 'numProcs' is the number of slots currently in use, and 'pgprocnos' is
+ * a dense array of their indexes, to speed up scanning all in-use slots.
*/
+ int numProcs;
+ int *pgprocnos;
ProcState procState[FLEXIBLE_ARRAY_MEMBER];
} SISeg;
@@ -210,7 +215,8 @@ SInvalShmemSize(void)
Size size;
size = offsetof(SISeg, procState);
- size = add_size(size, mul_size(sizeof(ProcState), NumProcStateSlots));
+ size = add_size(size, mul_size(sizeof(ProcState), NumProcStateSlots)); /* procState */
+ size = add_size(size, mul_size(sizeof(int), NumProcStateSlots)); /* pgprocnos */
return size;
}
@@ -235,7 +241,6 @@ CreateSharedInvalidationState(void)
shmInvalBuffer->minMsgNum = 0;
shmInvalBuffer->maxMsgNum = 0;
shmInvalBuffer->nextThreshold = CLEANUP_MIN;
- shmInvalBuffer->lastBackend = 0;
SpinLockInit(&shmInvalBuffer->msgnumLock);
/* The buffer[] array is initially all unused, so we need not fill it */
@@ -250,6 +255,8 @@ CreateSharedInvalidationState(void)
shmInvalBuffer->procState[i].hasMessages = false;
shmInvalBuffer->procState[i].nextLXID = InvalidLocalTransactionId;
}
+ shmInvalBuffer->numProcs = 0;
+ shmInvalBuffer->pgprocnos = (int *) &shmInvalBuffer->procState[i];
}
/*
@@ -262,13 +269,15 @@ SharedInvalBackendInit(bool sendOnly)
ProcState *stateP;
pid_t oldPid;
SISeg *segP = shmInvalBuffer;
+ int pgprocno;
if (MyBackendId <= 0)
elog(ERROR, "MyBackendId not set");
if (MyBackendId > NumProcStateSlots)
elog(PANIC, "unexpected MyBackendId %d in SharedInvalBackendInit (max %d)",
MyBackendId, NumProcStateSlots);
- stateP = &segP->procState[MyBackendId - 1];
+ pgprocno = MyBackendId - 1;
+ stateP = &segP->procState[pgprocno];
/*
* This can run in parallel with read operations, but not with write
@@ -285,8 +294,7 @@ SharedInvalBackendInit(bool sendOnly)
MyBackendId, oldPid);
}
- if (MyBackendId > segP->lastBackend)
- segP->lastBackend = MyBackendId;
+ shmInvalBuffer->pgprocnos[shmInvalBuffer->numProcs++] = pgprocno;
/* Fetch next local transaction ID into local memory */
nextLocalTransactionId = stateP->nextLXID;
@@ -318,13 +326,14 @@ CleanupInvalidationState(int status, Datum arg)
{
SISeg *segP = (SISeg *) DatumGetPointer(arg);
ProcState *stateP;
+ int pgprocno = MyBackendId - 1;
int i;
Assert(PointerIsValid(segP));
LWLockAcquire(SInvalWriteLock, LW_EXCLUSIVE);
- stateP = &segP->procState[MyBackendId - 1];
+ stateP = &segP->procState[pgprocno];
/* Update next local transaction ID for next holder of this backendID */
stateP->nextLXID = nextLocalTransactionId;
@@ -335,13 +344,18 @@ CleanupInvalidationState(int status, Datum arg)
stateP->resetState = false;
stateP->signaled = false;
- /* Recompute index of last active backend */
- for (i = segP->lastBackend; i > 0; i--)
+ for (i = segP->numProcs - 1; i >= 0; i--)
{
- if (segP->procState[i - 1].procPid != 0)
+ if (segP->pgprocnos[i] == pgprocno)
+ {
+ if (i != segP->numProcs - 1)
+ segP->pgprocnos[i] = segP->pgprocnos[segP->numProcs - 1];
break;
+ }
}
- segP->lastBackend = i;
+ if (i < 0)
+ elog(PANIC, "could not find entry in sinval array");
+ segP->numProcs--;
LWLockRelease(SInvalWriteLock);
}
@@ -414,9 +428,9 @@ SIInsertDataEntries(const SharedInvalidationMessage *data, int n)
* these (unlocked) changes will be committed to memory before we exit
* the function.
*/
- for (i = 0; i < segP->lastBackend; i++)
+ for (i = 0; i < segP->numProcs; i++)
{
- ProcState *stateP = &segP->procState[i];
+ ProcState *stateP = &segP->procState[segP->pgprocnos[i]];
stateP->hasMessages = true;
}
@@ -584,13 +598,14 @@ SICleanupQueue(bool callerHasWriteLock, int minFree)
minsig = min - SIG_THRESHOLD;
lowbound = min - MAXNUMMESSAGES + minFree;
- for (i = 0; i < segP->lastBackend; i++)
+ for (i = 0; i < segP->numProcs; i++)
{
- ProcState *stateP = &segP->procState[i];
+ ProcState *stateP = &segP->procState[segP->pgprocnos[i]];
int n = stateP->nextMsgNum;
- /* Ignore if inactive or already in reset state */
- if (stateP->procPid == 0 || stateP->resetState || stateP->sendOnly)
+ /* Ignore if already in reset state */
+ Assert(stateP->procPid != 0);
+ if (stateP->resetState || stateP->sendOnly)
continue;
/*
@@ -626,11 +641,8 @@ SICleanupQueue(bool callerHasWriteLock, int minFree)
{
segP->minMsgNum -= MSGNUMWRAPAROUND;
segP->maxMsgNum -= MSGNUMWRAPAROUND;
- for (i = 0; i < segP->lastBackend; i++)
- {
- /* we don't bother skipping inactive entries here */
- segP->procState[i].nextMsgNum -= MSGNUMWRAPAROUND;
- }
+ for (i = 0; i < segP->numProcs; i++)
+ segP->procState[segP->pgprocnos[i]].nextMsgNum -= MSGNUMWRAPAROUND;
}
/*
--
2.39.2
v8-0004-Remove-MyAuxProcType-use-MyBackendType-instead.patchtext/x-patch; charset=UTF-8; name=v8-0004-Remove-MyAuxProcType-use-MyBackendType-instead.patchDownload
From fc86b8876e39fa3c2298fb03b716f0896d9ca5b8 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Mon, 29 Jan 2024 20:58:52 +0200
Subject: [PATCH v8 4/5] Remove MyAuxProcType, use MyBackendType instead
MyAuxProcType was redundant with MyBackendType.
Reviewed-by: Reid Thompson, Andres Freund
Discussion: https://www.postgresql.org/message-id/f3ecd4cb-85ee-4e54-8278-5fabfb3a4ed0@iki.fi
---
src/backend/postmaster/auxprocess.c | 58 +++++-------------------
src/backend/postmaster/postmaster.c | 36 +++++++--------
src/include/miscadmin.h | 69 ++++++++++++++---------------
src/include/postmaster/auxprocess.h | 2 +-
src/tools/pgindent/typedefs.list | 1 -
5 files changed, 63 insertions(+), 103 deletions(-)
diff --git a/src/backend/postmaster/auxprocess.c b/src/backend/postmaster/auxprocess.c
index 39171fea06b..fc13cd76321 100644
--- a/src/backend/postmaster/auxprocess.c
+++ b/src/backend/postmaster/auxprocess.c
@@ -38,14 +38,6 @@
static void ShutdownAuxiliaryProcess(int code, Datum arg);
-/* ----------------
- * global variables
- * ----------------
- */
-
-AuxProcType MyAuxProcType = NotAnAuxProcess; /* declared in miscadmin.h */
-
-
/*
* AuxiliaryProcessMain
*
@@ -55,39 +47,11 @@ AuxProcType MyAuxProcType = NotAnAuxProcess; /* declared in miscadmin.h */
* This code is here just because of historical reasons.
*/
void
-AuxiliaryProcessMain(AuxProcType auxtype)
+AuxiliaryProcessMain(BackendType auxtype)
{
Assert(IsUnderPostmaster);
- MyAuxProcType = auxtype;
-
- switch (MyAuxProcType)
- {
- case StartupProcess:
- MyBackendType = B_STARTUP;
- break;
- case ArchiverProcess:
- MyBackendType = B_ARCHIVER;
- break;
- case BgWriterProcess:
- MyBackendType = B_BG_WRITER;
- break;
- case CheckpointerProcess:
- MyBackendType = B_CHECKPOINTER;
- break;
- case WalWriterProcess:
- MyBackendType = B_WAL_WRITER;
- break;
- case WalReceiverProcess:
- MyBackendType = B_WAL_RECEIVER;
- break;
- case WalSummarizerProcess:
- MyBackendType = B_WAL_SUMMARIZER;
- break;
- default:
- elog(PANIC, "unrecognized process type: %d", (int) MyAuxProcType);
- MyBackendType = B_INVALID;
- }
+ MyBackendType = auxtype;
init_ps_display(NULL);
@@ -126,38 +90,38 @@ AuxiliaryProcessMain(AuxProcType auxtype)
SetProcessingMode(NormalProcessing);
- switch (MyAuxProcType)
+ switch (MyBackendType)
{
- case StartupProcess:
+ case B_STARTUP:
StartupProcessMain();
proc_exit(1);
- case ArchiverProcess:
+ case B_ARCHIVER:
PgArchiverMain();
proc_exit(1);
- case BgWriterProcess:
+ case B_BG_WRITER:
BackgroundWriterMain();
proc_exit(1);
- case CheckpointerProcess:
+ case B_CHECKPOINTER:
CheckpointerMain();
proc_exit(1);
- case WalWriterProcess:
+ case B_WAL_WRITER:
WalWriterMain();
proc_exit(1);
- case WalReceiverProcess:
+ case B_WAL_RECEIVER:
WalReceiverMain();
proc_exit(1);
- case WalSummarizerProcess:
+ case B_WAL_SUMMARIZER:
WalSummarizerMain();
proc_exit(1);
default:
- elog(PANIC, "unrecognized process type: %d", (int) MyAuxProcType);
+ elog(PANIC, "unrecognized process type: %d", (int) MyBackendType);
proc_exit(1);
}
}
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index feb471dd1df..c51b6a1376a 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -440,7 +440,7 @@ static int CountChildren(int target);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
-static pid_t StartChildProcess(AuxProcType type);
+static pid_t StartChildProcess(BackendType type);
static void StartAutovacuumWorker(void);
static void MaybeStartWalReceiver(void);
static void MaybeStartWalSummarizer(void);
@@ -561,13 +561,13 @@ static void ShmemBackendArrayAdd(Backend *bn);
static void ShmemBackendArrayRemove(Backend *bn);
#endif /* EXEC_BACKEND */
-#define StartupDataBase() StartChildProcess(StartupProcess)
-#define StartArchiver() StartChildProcess(ArchiverProcess)
-#define StartBackgroundWriter() StartChildProcess(BgWriterProcess)
-#define StartCheckpointer() StartChildProcess(CheckpointerProcess)
-#define StartWalWriter() StartChildProcess(WalWriterProcess)
-#define StartWalReceiver() StartChildProcess(WalReceiverProcess)
-#define StartWalSummarizer() StartChildProcess(WalSummarizerProcess)
+#define StartupDataBase() StartChildProcess(B_STARTUP)
+#define StartArchiver() StartChildProcess(B_ARCHIVER)
+#define StartBackgroundWriter() StartChildProcess(B_BG_WRITER)
+#define StartCheckpointer() StartChildProcess(B_CHECKPOINTER)
+#define StartWalWriter() StartChildProcess(B_WAL_WRITER)
+#define StartWalReceiver() StartChildProcess(B_WAL_RECEIVER)
+#define StartWalSummarizer() StartChildProcess(B_WAL_SUMMARIZER)
/* Macros to check exit status of a child process */
#define EXIT_STATUS_0(st) ((st) == 0)
@@ -4953,7 +4953,7 @@ SubPostmasterMain(int argc, char *argv[])
}
if (strcmp(argv[1], "--forkaux") == 0)
{
- AuxProcType auxtype;
+ BackendType auxtype;
Assert(argc == 4);
@@ -5292,7 +5292,7 @@ CountChildren(int target)
* to start subprocess.
*/
static pid_t
-StartChildProcess(AuxProcType type)
+StartChildProcess(BackendType type)
{
pid_t pid;
@@ -5344,31 +5344,31 @@ StartChildProcess(AuxProcType type)
errno = save_errno;
switch (type)
{
- case StartupProcess:
+ case B_STARTUP:
ereport(LOG,
(errmsg("could not fork startup process: %m")));
break;
- case ArchiverProcess:
+ case B_ARCHIVER:
ereport(LOG,
(errmsg("could not fork archiver process: %m")));
break;
- case BgWriterProcess:
+ case B_BG_WRITER:
ereport(LOG,
(errmsg("could not fork background writer process: %m")));
break;
- case CheckpointerProcess:
+ case B_CHECKPOINTER:
ereport(LOG,
(errmsg("could not fork checkpointer process: %m")));
break;
- case WalWriterProcess:
+ case B_WAL_WRITER:
ereport(LOG,
(errmsg("could not fork WAL writer process: %m")));
break;
- case WalReceiverProcess:
+ case B_WAL_RECEIVER:
ereport(LOG,
(errmsg("could not fork WAL receiver process: %m")));
break;
- case WalSummarizerProcess:
+ case B_WAL_SUMMARIZER:
ereport(LOG,
(errmsg("could not fork WAL summarizer process: %m")));
break;
@@ -5382,7 +5382,7 @@ StartChildProcess(AuxProcType type)
* fork failure is fatal during startup, but there's no need to choke
* immediately if starting other child types fails.
*/
- if (type == StartupProcess)
+ if (type == B_STARTUP)
ExitPostmaster(1);
return 0;
}
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index cbdc61b8576..4ecae4f834e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -321,29 +321,57 @@ extern void InitProcessLocalLatch(void);
extern void SwitchToSharedLatch(void);
extern void SwitchBackToLocalLatch(void);
+/*
+ * MyBackendType indicates what kind of a backend this is.
+ */
typedef enum BackendType
{
B_INVALID = 0,
- B_ARCHIVER,
+
+ /* Backends and other backend-like processes */
+ B_BACKEND,
B_AUTOVAC_LAUNCHER,
B_AUTOVAC_WORKER,
- B_BACKEND,
B_BG_WORKER,
+ B_WAL_SENDER,
+
+ B_STANDALONE_BACKEND,
+
+ /*
+ * Auxiliary processes. These have PGPROC entries, but they are not
+ * attached to any particular database. There can be only one of each of
+ * these running at a time.
+ *
+ * If you modify these, make sure to update NUM_AUXILIARY_PROCS and the
+ * glossary in the docs.
+ */
+ B_ARCHIVER,
B_BG_WRITER,
B_CHECKPOINTER,
- B_LOGGER,
- B_STANDALONE_BACKEND,
B_STARTUP,
B_WAL_RECEIVER,
- B_WAL_SENDER,
B_WAL_SUMMARIZER,
B_WAL_WRITER,
+
+ /*
+ * Logger is not connected to shared memory and does not have a PGPROC
+ * entry.
+ */
+ B_LOGGER,
} BackendType;
#define BACKEND_NUM_TYPES (B_WAL_WRITER + 1)
extern PGDLLIMPORT BackendType MyBackendType;
+#define AmStartupProcess() (MyBackendType == B_STARTUP)
+#define AmBackgroundWriterProcess() (MyBackendType == B_BG_WRITER)
+#define AmArchiverProcess() (MyBackendType == B_ARCHIVER)
+#define AmCheckpointerProcess() (MyBackendType == B_CHECKPOINTER)
+#define AmWalWriterProcess() (MyBackendType == B_WAL_WRITER)
+#define AmWalReceiverProcess() (MyBackendType == B_WAL_RECEIVER)
+#define AmWalSummarizerProcess() (MyBackendType == B_WAL_SUMMARIZER)
+
extern const char *GetBackendTypeDesc(BackendType backendType);
extern void SetDatabasePath(const char *path);
@@ -426,37 +454,6 @@ extern PGDLLIMPORT ProcessingMode Mode;
} while(0)
-/*
- * Auxiliary-process type identifiers. These used to be in bootstrap.h
- * but it seems saner to have them here, with the ProcessingMode stuff.
- * The MyAuxProcType global is defined and set in auxprocess.c.
- *
- * Make sure to list in the glossary any items you add here.
- */
-
-typedef enum
-{
- NotAnAuxProcess = -1,
- StartupProcess = 0,
- BgWriterProcess,
- ArchiverProcess,
- CheckpointerProcess,
- WalWriterProcess,
- WalReceiverProcess,
- WalSummarizerProcess,
-} AuxProcType;
-
-extern PGDLLIMPORT AuxProcType MyAuxProcType;
-
-#define AmStartupProcess() (MyAuxProcType == StartupProcess)
-#define AmBackgroundWriterProcess() (MyAuxProcType == BgWriterProcess)
-#define AmArchiverProcess() (MyAuxProcType == ArchiverProcess)
-#define AmCheckpointerProcess() (MyAuxProcType == CheckpointerProcess)
-#define AmWalWriterProcess() (MyAuxProcType == WalWriterProcess)
-#define AmWalReceiverProcess() (MyAuxProcType == WalReceiverProcess)
-#define AmWalSummarizerProcess() (MyAuxProcType == WalSummarizerProcess)
-
-
/*****************************************************************************
* pinit.h -- *
* POSTGRES initialization and cleanup definitions. *
diff --git a/src/include/postmaster/auxprocess.h b/src/include/postmaster/auxprocess.h
index 1fdde3bb77b..3e443edde70 100644
--- a/src/include/postmaster/auxprocess.h
+++ b/src/include/postmaster/auxprocess.h
@@ -15,6 +15,6 @@
#include "miscadmin.h"
-extern void AuxiliaryProcessMain(AuxProcType auxtype) pg_attribute_noreturn();
+extern void AuxiliaryProcessMain(BackendType auxtype) pg_attribute_noreturn();
#endif /* AUXPROCESS_H */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 90b37b919c2..9645c7b735c 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -173,7 +173,6 @@ AutoVacOpts
AutoVacuumShmemStruct
AutoVacuumWorkItem
AutoVacuumWorkItemType
-AuxProcType
BF_ctx
BF_key
BF_word
--
2.39.2
v8-0005-Use-MyBackendType-in-more-places-to-check-what-pr.patchtext/x-patch; charset=UTF-8; name=v8-0005-Use-MyBackendType-in-more-places-to-check-what-pr.patchDownload
From 91e3bdbc3f4d7ab91769972fe03f22d6e0717549 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 10 Jan 2024 12:59:48 +0200
Subject: [PATCH v8 5/5] Use MyBackendType in more places to check what process
this is
Remove IsBackgroundWorker, IsAutoVacuumLauncherProcess() and
IsAutoVacuumWorkerProcess() in favor of new Am*Process() macros that
use MyBackendType. For consistency with the existing Am*Process()
macros.
Reviewed-by: XXX
Discussion: XXX
---
src/backend/access/gin/ginfast.c | 2 +-
src/backend/access/gin/ginvacuum.c | 6 ++---
src/backend/access/heap/vacuumlazy.c | 4 +--
src/backend/commands/analyze.c | 4 +--
src/backend/commands/vacuum.c | 8 +++---
src/backend/postmaster/autovacuum.c | 26 --------------------
src/backend/postmaster/bgworker.c | 2 --
src/backend/postmaster/postmaster.c | 3 ---
src/backend/statistics/extended_stats.c | 2 +-
src/backend/storage/ipc/ipc.c | 2 +-
src/backend/storage/lmgr/proc.c | 19 +++++++-------
src/backend/tcop/postgres.c | 6 ++---
src/backend/utils/activity/pgstat_relation.c | 4 +--
src/backend/utils/init/globals.c | 1 -
src/backend/utils/init/miscinit.c | 2 +-
src/backend/utils/init/postinit.c | 8 +++---
src/include/miscadmin.h | 5 +++-
src/include/postmaster/autovacuum.h | 5 ----
18 files changed, 37 insertions(+), 72 deletions(-)
diff --git a/src/backend/access/gin/ginfast.c b/src/backend/access/gin/ginfast.c
index cff6850ef86..e118cecb9a4 100644
--- a/src/backend/access/gin/ginfast.c
+++ b/src/backend/access/gin/ginfast.c
@@ -812,7 +812,7 @@ ginInsertCleanup(GinState *ginstate, bool full_clean,
*/
LockPage(index, GIN_METAPAGE_BLKNO, ExclusiveLock);
workMemory =
- (IsAutoVacuumWorkerProcess() && autovacuum_work_mem != -1) ?
+ (AmAutoVacuumWorkerProcess() && autovacuum_work_mem != -1) ?
autovacuum_work_mem : maintenance_work_mem;
}
else
diff --git a/src/backend/access/gin/ginvacuum.c b/src/backend/access/gin/ginvacuum.c
index aee6aab44ae..b3f415e2849 100644
--- a/src/backend/access/gin/ginvacuum.c
+++ b/src/backend/access/gin/ginvacuum.c
@@ -590,7 +590,7 @@ ginbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
/*
* and cleanup any pending inserts
*/
- ginInsertCleanup(&gvs.ginstate, !IsAutoVacuumWorkerProcess(),
+ ginInsertCleanup(&gvs.ginstate, !AmAutoVacuumWorkerProcess(),
false, true, stats);
}
@@ -701,7 +701,7 @@ ginvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats)
*/
if (info->analyze_only)
{
- if (IsAutoVacuumWorkerProcess())
+ if (AmAutoVacuumWorkerProcess())
{
initGinState(&ginstate, index);
ginInsertCleanup(&ginstate, false, true, true, stats);
@@ -717,7 +717,7 @@ ginvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats)
{
stats = (IndexBulkDeleteResult *) palloc0(sizeof(IndexBulkDeleteResult));
initGinState(&ginstate, index);
- ginInsertCleanup(&ginstate, !IsAutoVacuumWorkerProcess(),
+ ginInsertCleanup(&ginstate, !AmAutoVacuumWorkerProcess(),
false, true, stats);
}
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index fa56480808b..5acea4a43a4 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -307,7 +307,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
char **indnames = NULL;
verbose = (params->options & VACOPT_VERBOSE) != 0;
- instrument = (verbose || (IsAutoVacuumWorkerProcess() &&
+ instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
params->log_min_duration >= 0));
if (instrument)
{
@@ -3087,7 +3087,7 @@ static int
dead_items_max_items(LVRelState *vacrel)
{
int64 max_items;
- int vac_work_mem = IsAutoVacuumWorkerProcess() &&
+ int vac_work_mem = AmAutoVacuumWorkerProcess() &&
autovacuum_work_mem != -1 ?
autovacuum_work_mem : maintenance_work_mem;
diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c
index a03495d6c95..d105d2fad7b 100644
--- a/src/backend/commands/analyze.c
+++ b/src/backend/commands/analyze.c
@@ -351,7 +351,7 @@ do_analyze_rel(Relation onerel, VacuumParams *params,
save_nestlevel = NewGUCNestLevel();
/* measure elapsed time iff autovacuum logging requires it */
- if (IsAutoVacuumWorkerProcess() && params->log_min_duration >= 0)
+ if (AmAutoVacuumWorkerProcess() && params->log_min_duration >= 0)
{
if (track_io_timing)
{
@@ -729,7 +729,7 @@ do_analyze_rel(Relation onerel, VacuumParams *params,
vac_close_indexes(nindexes, Irel, NoLock);
/* Log the action if appropriate */
- if (IsAutoVacuumWorkerProcess() && params->log_min_duration >= 0)
+ if (AmAutoVacuumWorkerProcess() && params->log_min_duration >= 0)
{
TimestampTz endtime = GetCurrentTimestamp();
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 64da8486276..e40cef6566e 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -564,7 +564,7 @@ vacuum(List *relations, VacuumParams *params, BufferAccessStrategy bstrategy,
else
{
Assert(params->options & VACOPT_ANALYZE);
- if (IsAutoVacuumWorkerProcess())
+ if (AmAutoVacuumWorkerProcess())
use_own_xacts = true;
else if (in_outer_xact)
use_own_xacts = false;
@@ -809,7 +809,7 @@ vacuum_open_relation(Oid relid, RangeVar *relation, bits32 options,
* statements in the permission checks; otherwise, only log if the caller
* so requested.
*/
- if (!IsAutoVacuumWorkerProcess())
+ if (!AmAutoVacuumWorkerProcess())
elevel = WARNING;
else if (verbose)
elevel = LOG;
@@ -896,7 +896,7 @@ expand_vacuum_rel(VacuumRelation *vrel, MemoryContext vac_context,
* Since autovacuum workers supply OIDs when calling vacuum(), no
* autovacuum worker should reach this code.
*/
- Assert(!IsAutoVacuumWorkerProcess());
+ Assert(!AmAutoVacuumWorkerProcess());
/*
* We transiently take AccessShareLock to protect the syscache lookup
@@ -2336,7 +2336,7 @@ vacuum_delay_point(void)
* [autovacuum_]vacuum_cost_delay to take effect while a table is being
* vacuumed or analyzed.
*/
- if (ConfigReloadPending && IsAutoVacuumWorkerProcess())
+ if (ConfigReloadPending && AmAutoVacuumWorkerProcess())
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 2c3099f76f1..cecbaaa5b91 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -135,10 +135,6 @@ int Log_autovacuum_min_duration = 600000;
#define MIN_AUTOVAC_SLEEPTIME 100.0 /* milliseconds */
#define MAX_AUTOVAC_SLEEPTIME 300 /* seconds */
-/* Flags to tell if we are in an autovacuum process */
-static bool am_autovacuum_launcher = false;
-static bool am_autovacuum_worker = false;
-
/*
* Variables to save the cost-related storage parameters for the current
* relation being vacuumed by this autovacuum worker. Using these, we can
@@ -435,8 +431,6 @@ AutoVacLauncherMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
- am_autovacuum_launcher = true;
-
MyBackendType = B_AUTOVAC_LAUNCHER;
init_ps_display(NULL);
@@ -1495,8 +1489,6 @@ AutoVacWorkerMain(int argc, char *argv[])
sigjmp_buf local_sigjmp_buf;
Oid dbid;
- am_autovacuum_worker = true;
-
MyBackendType = B_AUTOVAC_WORKER;
init_ps_display(NULL);
@@ -3355,24 +3347,6 @@ autovac_init(void)
errhint("Enable the \"track_counts\" option.")));
}
-/*
- * IsAutoVacuum functions
- * Return whether this is either a launcher autovacuum process or a worker
- * process.
- */
-bool
-IsAutoVacuumLauncherProcess(void)
-{
- return am_autovacuum_launcher;
-}
-
-bool
-IsAutoVacuumWorkerProcess(void)
-{
- return am_autovacuum_worker;
-}
-
-
/*
* AutoVacuumShmemSize
* Compute space needed for autovacuum-related shared memory
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index 67f92c24db1..d8e89de7494 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -731,8 +731,6 @@ BackgroundWorkerMain(void)
if (worker == NULL)
elog(FATAL, "unable to find bgworker entry");
- IsBackgroundWorker = true;
-
MyBackendType = B_BG_WORKER;
init_ps_display(worker->bgw_name);
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index c51b6a1376a..8aa2ddf0c46 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -4979,9 +4979,6 @@ SubPostmasterMain(int argc, char *argv[])
}
if (strcmp(argv[1], "--forkbgworker") == 0)
{
- /* do this as early as possible; in particular, before InitProcess() */
- IsBackgroundWorker = true;
-
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
diff --git a/src/backend/statistics/extended_stats.c b/src/backend/statistics/extended_stats.c
index c5461514d8f..135151a2723 100644
--- a/src/backend/statistics/extended_stats.c
+++ b/src/backend/statistics/extended_stats.c
@@ -173,7 +173,7 @@ BuildRelationExtStatistics(Relation onerel, bool inh, double totalrows,
natts, vacattrstats);
if (!stats)
{
- if (!IsAutoVacuumWorkerProcess())
+ if (!AmAutoVacuumWorkerProcess())
ereport(WARNING,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("statistics object \"%s.%s\" could not be computed for relation \"%s.%s\"",
diff --git a/src/backend/storage/ipc/ipc.c b/src/backend/storage/ipc/ipc.c
index 2681e4cdff5..b06e4b84528 100644
--- a/src/backend/storage/ipc/ipc.c
+++ b/src/backend/storage/ipc/ipc.c
@@ -136,7 +136,7 @@ proc_exit(int code)
*/
char gprofDirName[32];
- if (IsAutoVacuumWorkerProcess())
+ if (AmAutoVacuumWorkerProcess())
snprintf(gprofDirName, 32, "gprof/avworker");
else
snprintf(gprofDirName, 32, "gprof/%d", (int) getpid());
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index 9c25f631c5d..c0e4e1cf39b 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -41,7 +41,6 @@
#include "postmaster/autovacuum.h"
#include "replication/slot.h"
#include "replication/syncrep.h"
-#include "replication/walsender.h"
#include "storage/condition_variable.h"
#include "storage/ipc.h"
#include "storage/lmgr.h"
@@ -308,11 +307,11 @@ InitProcess(void)
elog(ERROR, "you already exist");
/* Decide which list should supply our PGPROC. */
- if (IsAnyAutoVacuumProcess())
+ if (AmAutoVacuumLauncherProcess() || AmAutoVacuumWorkerProcess())
procgloballist = &ProcGlobal->autovacFreeProcs;
- else if (IsBackgroundWorker)
+ else if (AmBackgroundWorkerProcess())
procgloballist = &ProcGlobal->bgworkerFreeProcs;
- else if (am_walsender)
+ else if (AmWalSenderProcess())
procgloballist = &ProcGlobal->walsenderFreeProcs;
else
procgloballist = &ProcGlobal->freeProcs;
@@ -342,7 +341,7 @@ InitProcess(void)
* in the autovacuum case?
*/
SpinLockRelease(ProcStructLock);
- if (am_walsender)
+ if (AmWalSenderProcess())
ereport(FATAL,
(errcode(ERRCODE_TOO_MANY_CONNECTIONS),
errmsg("number of requested standby connections exceeds max_wal_senders (currently %d)",
@@ -365,7 +364,7 @@ InitProcess(void)
* cleaning up. (XXX autovac launcher currently doesn't participate in
* this; it probably should.)
*/
- if (IsUnderPostmaster && !IsAutoVacuumLauncherProcess())
+ if (IsUnderPostmaster && !AmAutoVacuumLauncherProcess())
MarkPostmasterChildActive();
/*
@@ -384,11 +383,11 @@ InitProcess(void)
MyProc->databaseId = InvalidOid;
MyProc->roleId = InvalidOid;
MyProc->tempNamespaceId = InvalidOid;
- MyProc->isBackgroundWorker = IsBackgroundWorker;
+ MyProc->isBackgroundWorker = AmBackgroundWorkerProcess();
MyProc->delayChkptFlags = 0;
MyProc->statusFlags = 0;
/* NB -- autovac launcher intentionally does not set IS_AUTOVACUUM */
- if (IsAutoVacuumWorkerProcess())
+ if (AmAutoVacuumWorkerProcess())
MyProc->statusFlags |= PROC_IS_AUTOVACUUM;
MyProc->lwWaiting = LW_WS_NOT_WAITING;
MyProc->lwWaitMode = 0;
@@ -579,7 +578,7 @@ InitAuxiliaryProcess(void)
MyProc->databaseId = InvalidOid;
MyProc->roleId = InvalidOid;
MyProc->tempNamespaceId = InvalidOid;
- MyProc->isBackgroundWorker = IsBackgroundWorker;
+ MyProc->isBackgroundWorker = AmBackgroundWorkerProcess();
MyProc->delayChkptFlags = 0;
MyProc->statusFlags = 0;
MyProc->lwWaiting = LW_WS_NOT_WAITING;
@@ -935,7 +934,7 @@ ProcKill(int code, Datum arg)
* way, so tell the postmaster we've cleaned up acceptably well. (XXX
* autovac launcher should be included here someday)
*/
- if (IsUnderPostmaster && !IsAutoVacuumLauncherProcess())
+ if (IsUnderPostmaster && !AmAutoVacuumLauncherProcess())
MarkPostmasterChildInactive();
/* wake autovac launcher if needed -- see comments in FreeWorkerInfo */
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 1a34bd3715f..38610b723af 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -3267,7 +3267,7 @@ ProcessInterrupts(void)
ereport(FATAL,
(errcode(ERRCODE_QUERY_CANCELED),
errmsg("canceling authentication due to timeout")));
- else if (IsAutoVacuumWorkerProcess())
+ else if (AmAutoVacuumWorkerProcess())
ereport(FATAL,
(errcode(ERRCODE_ADMIN_SHUTDOWN),
errmsg("terminating autovacuum process due to administrator command")));
@@ -3286,7 +3286,7 @@ ProcessInterrupts(void)
*/
proc_exit(1);
}
- else if (IsBackgroundWorker)
+ else if (AmBackgroundWorkerProcess())
ereport(FATAL,
(errcode(ERRCODE_ADMIN_SHUTDOWN),
errmsg("terminating background worker \"%s\" due to administrator command",
@@ -3386,7 +3386,7 @@ ProcessInterrupts(void)
(errcode(ERRCODE_QUERY_CANCELED),
errmsg("canceling statement due to statement timeout")));
}
- if (IsAutoVacuumWorkerProcess())
+ if (AmAutoVacuumWorkerProcess())
{
LockErrorCleanup();
ereport(ERROR,
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 111050725a6..b20a60b5a87 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -246,7 +246,7 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
*/
tabentry->ins_since_vacuum = 0;
- if (IsAutoVacuumWorkerProcess())
+ if (AmAutoVacuumWorkerProcess())
{
tabentry->last_autovacuum_time = ts;
tabentry->autovacuum_count++;
@@ -337,7 +337,7 @@ pgstat_report_analyze(Relation rel,
if (resetcounter)
tabentry->mod_since_analyze = 0;
- if (IsAutoVacuumWorkerProcess())
+ if (AmAutoVacuumWorkerProcess())
{
tabentry->last_autoanalyze_time = GetCurrentTimestamp();
tabentry->autoanalyze_count++;
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 88b03e8fa3c..ce1fa1092f2 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -114,7 +114,6 @@ pid_t PostmasterPid = 0;
bool IsPostmasterEnvironment = false;
bool IsUnderPostmaster = false;
bool IsBinaryUpgrade = false;
-bool IsBackgroundWorker = false;
bool ExitOnAnyError = false;
diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c
index 23f77a59e58..9c91e9a1324 100644
--- a/src/backend/utils/init/miscinit.c
+++ b/src/backend/utils/init/miscinit.c
@@ -837,7 +837,7 @@ InitializeSessionUserIdStandalone(void)
* This function should only be called in single-user mode, in autovacuum
* workers, and in background workers.
*/
- Assert(!IsUnderPostmaster || IsAutoVacuumWorkerProcess() || IsBackgroundWorker);
+ Assert(!IsUnderPostmaster || AmAutoVacuumWorkerProcess() || AmBackgroundWorkerProcess());
/* call only once */
Assert(!OidIsValid(AuthenticatedUserId));
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index e822cd61d5f..6bf71b2d4e2 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -344,7 +344,7 @@ CheckMyDatabase(const char *name, bool am_superuser, bool override_allow_connect
*
* We do not enforce them for autovacuum worker processes either.
*/
- if (IsUnderPostmaster && !IsAutoVacuumWorkerProcess())
+ if (IsUnderPostmaster && !AmAutoVacuumWorkerProcess())
{
/*
* Check that the database is currently allowing connections.
@@ -825,7 +825,7 @@ InitPostgres(const char *in_dbname, Oid dboid,
before_shmem_exit(ShutdownPostgres, 0);
/* The autovacuum launcher is done here */
- if (IsAutoVacuumLauncherProcess())
+ if (AmAutoVacuumLauncherProcess())
{
/* report this backend in the PgBackendStatus array */
pgstat_bestart();
@@ -869,7 +869,7 @@ InitPostgres(const char *in_dbname, Oid dboid,
* In standalone mode and in autovacuum worker processes, we use a fixed
* ID, otherwise we figure it out from the authenticated user name.
*/
- if (bootstrap || IsAutoVacuumWorkerProcess())
+ if (bootstrap || AmAutoVacuumWorkerProcess())
{
InitializeSessionUserIdStandalone();
am_superuser = true;
@@ -885,7 +885,7 @@ InitPostgres(const char *in_dbname, Oid dboid,
errhint("You should immediately run CREATE USER \"%s\" SUPERUSER;.",
username != NULL ? username : "postgres")));
}
- else if (IsBackgroundWorker)
+ else if (AmBackgroundWorkerProcess())
{
if (username == NULL && !OidIsValid(useroid))
{
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 4ecae4f834e..631ff28cb20 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -164,7 +164,6 @@ do { \
extern PGDLLIMPORT pid_t PostmasterPid;
extern PGDLLIMPORT bool IsPostmasterEnvironment;
extern PGDLLIMPORT bool IsUnderPostmaster;
-extern PGDLLIMPORT bool IsBackgroundWorker;
extern PGDLLIMPORT bool IsBinaryUpgrade;
extern PGDLLIMPORT bool ExitOnAnyError;
@@ -364,6 +363,10 @@ typedef enum BackendType
extern PGDLLIMPORT BackendType MyBackendType;
+#define AmAutoVacuumLauncherProcess() (MyBackendType == B_AUTOVAC_LAUNCHER)
+#define AmAutoVacuumWorkerProcess() (MyBackendType == B_AUTOVAC_WORKER)
+#define AmBackgroundWorkerProcess() (MyBackendType == B_BG_WORKER)
+#define AmWalSenderProcess() (MyBackendType == B_WAL_SENDER)
#define AmStartupProcess() (MyBackendType == B_STARTUP)
#define AmBackgroundWriterProcess() (MyBackendType == B_BG_WRITER)
#define AmArchiverProcess() (MyBackendType == B_ARCHIVER)
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 1994aedef03..80cf4cdd969 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -49,11 +49,6 @@ extern PGDLLIMPORT int Log_autovacuum_min_duration;
/* Status inquiry functions */
extern bool AutoVacuumingActive(void);
-extern bool IsAutoVacuumLauncherProcess(void);
-extern bool IsAutoVacuumWorkerProcess(void);
-
-#define IsAnyAutoVacuumProcess() \
- (IsAutoVacuumLauncherProcess() || IsAutoVacuumWorkerProcess())
/* Functions to start autovacuum process, called from postmaster */
extern void autovac_init(void);
--
2.39.2
Re: Refactoring backend fork+exec code
On 30/01/2024 02:08, Heikki Linnakangas wrote:
On 29/01/2024 17:54, reid.thompson@crunchydata.com wrote:
On Thu, 2024-01-25 at 01:51 +0200, Heikki Linnakangas wrote:
And here we go. BackendID is now a 1-based index directly into the
PGPROC array.Would it be worthwhile to also note in this comment FIRST_AUX_PROC's
and IsAuxProcess()'s dependency on B_ARCHIVER and it's location in the
enum table?Yeah, that might be in order. Although looking closer, it's only used in
IsAuxProcess, which is only used in one sanity check in
AuxProcessMain(). And even that gets refactored away by the later
patches in this thread. So on second thoughts, I'll just remove it
altogether.I spent some more time on the 'lastBackend' optimization in sinvaladt.c.
I realized that it became very useless with these patches, because aux
processes are allocated pgprocno's after all the slots for regular
backends. There are always aux processes running, so lastBackend would
always have a value close to the max anyway. I replaced that with a
dense 'pgprocnos' array that keeps track of the exact slots that are in
use. I'm not 100% sure this is worth the effort; does any real world
workload send shared invalidations so frequently that this matters? In
any case, this should avoid the regression if such a workload exists.New patch set attached. I think these are ready to be committed, but
would appreciate a final review.
contrib/amcheck 003_cic_2pc.pl test failures revealed a bug that
required some reworking:
In a PGPROC entry for a prepared xact, the PGPROC's backendID needs to
be the original backend's ID, because the prepared xact is holding the
lock on the original virtual transaction id. When a transaction's
ownership is moved from the original backend's PGPROC entry to the
prepared xact PGPROC entry, the backendID needs to be copied over. My
patch removed the field altogether, so it was not copied over, which
made it look like it the original VXID lock was released at prepare.
I fixed that by adding back the backendID field. For regular backends,
it's always equal to pgprocno + 1, but for prepared xacts, it's the
original backend's ID. To make that less confusing, I moved the
backendID and lxid fields together under a 'vxid' struct. The two fields
together form the virtual transaction ID, and that's the only context
where the 'backendID' field should now be looked at.
I also squashed the 'lastBackend' changes in sinvaladt.c to the main patch.
--
Heikki Linnakangas
Neon (https://neon.tech)
Attachments:
v9-0001-Remove-superfluous-pgprocno-field-from-PGPROC.patchtext/x-patch; charset=UTF-8; name=v9-0001-Remove-superfluous-pgprocno-field-from-PGPROC.patchDownload
From 96c583b32db843fb07d38fd78f1e205882a78b01 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 24 Jan 2024 23:15:55 +0200
Subject: [PATCH v9 1/4] Remove superfluous 'pgprocno' field from PGPROC
It was always just the index of the PGPROC entry from the beginning of
the proc array. Introduce a macro to compute it from the pointer
instead.
Reviewed-by: XXX
Discussion: https://www.postgresql.org/message-id/8171f1aa-496f-46a6-afc3-c46fe7a9b407%40iki.fi
---
src/backend/access/transam/clog.c | 4 ++--
src/backend/access/transam/twophase.c | 11 +++++------
src/backend/access/transam/xlog.c | 2 +-
src/backend/postmaster/bgwriter.c | 2 +-
src/backend/postmaster/pgarch.c | 2 +-
src/backend/postmaster/walsummarizer.c | 2 +-
src/backend/storage/buffer/bufmgr.c | 6 +++---
src/backend/storage/ipc/procarray.c | 6 +++---
src/backend/storage/lmgr/condition_variable.c | 12 ++++++------
src/backend/storage/lmgr/lwlock.c | 6 +++---
src/backend/storage/lmgr/predicate.c | 2 +-
src/backend/storage/lmgr/proc.c | 1 -
src/include/storage/lock.h | 2 +-
src/include/storage/proc.h | 6 +-----
14 files changed, 29 insertions(+), 35 deletions(-)
diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c
index f6e7da7ffc9..7550309c25a 100644
--- a/src/backend/access/transam/clog.c
+++ b/src/backend/access/transam/clog.c
@@ -458,7 +458,7 @@ TransactionGroupUpdateXidStatus(TransactionId xid, XidStatus status,
* less efficiently.
*/
if (nextidx != INVALID_PGPROCNO &&
- ProcGlobal->allProcs[nextidx].clogGroupMemberPage != proc->clogGroupMemberPage)
+ GetPGProcByNumber(nextidx)->clogGroupMemberPage != proc->clogGroupMemberPage)
{
/*
* Ensure that this proc is not a member of any clog group that
@@ -473,7 +473,7 @@ TransactionGroupUpdateXidStatus(TransactionId xid, XidStatus status,
if (pg_atomic_compare_exchange_u32(&procglobal->clogGroupFirst,
&nextidx,
- (uint32) proc->pgprocno))
+ (uint32) GetNumberFromPGProc(proc)))
break;
}
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 8426458f7f5..234c8d08ebc 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -284,7 +284,7 @@ TwoPhaseShmemInit(void)
TwoPhaseState->freeGXacts = &gxacts[i];
/* associate it with a PGPROC assigned by InitProcGlobal */
- gxacts[i].pgprocno = PreparedXactProcs[i].pgprocno;
+ gxacts[i].pgprocno = GetNumberFromPGProc(&PreparedXactProcs[i]);
/*
* Assign a unique ID for each dummy proc, so that the range of
@@ -461,7 +461,6 @@ MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid, const char *gid,
/* Initialize the PGPROC entry */
MemSet(proc, 0, sizeof(PGPROC));
- proc->pgprocno = gxact->pgprocno;
dlist_node_init(&proc->links);
proc->waitStatus = PROC_WAIT_STATUS_OK;
if (LocalTransactionIdIsValid(MyProc->lxid))
@@ -780,7 +779,7 @@ pg_prepared_xact(PG_FUNCTION_ARGS)
while (status->array != NULL && status->currIdx < status->ngxacts)
{
GlobalTransaction gxact = &status->array[status->currIdx++];
- PGPROC *proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ PGPROC *proc = GetPGProcByNumber(gxact->pgprocno);
Datum values[5] = {0};
bool nulls[5] = {0};
HeapTuple tuple;
@@ -935,7 +934,7 @@ TwoPhaseGetDummyProc(TransactionId xid, bool lock_held)
{
GlobalTransaction gxact = TwoPhaseGetGXact(xid, lock_held);
- return &ProcGlobal->allProcs[gxact->pgprocno];
+ return GetPGProcByNumber(gxact->pgprocno);
}
/************************************************************************/
@@ -1080,7 +1079,7 @@ save_state_data(const void *data, uint32 len)
void
StartPrepare(GlobalTransaction gxact)
{
- PGPROC *proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ PGPROC *proc = GetPGProcByNumber(gxact->pgprocno);
TransactionId xid = gxact->xid;
TwoPhaseFileHeader hdr;
TransactionId *children;
@@ -1539,7 +1538,7 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
* try to commit the same GID at once.
*/
gxact = LockGXact(gid, GetUserId());
- proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ proc = GetPGProcByNumber(gxact->pgprocno);
xid = gxact->xid;
/*
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 478377c4a23..be30a2388f5 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -1379,7 +1379,7 @@ WALInsertLockAcquire(void)
static int lockToTry = -1;
if (lockToTry == -1)
- lockToTry = MyProc->pgprocno % NUM_XLOGINSERT_LOCKS;
+ lockToTry = GetNumberFromPGProc(MyProc) % NUM_XLOGINSERT_LOCKS;
MyLockNo = lockToTry;
/*
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index f11ce27084e..8f1415cd565 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -326,7 +326,7 @@ BackgroundWriterMain(void)
if (rc == WL_TIMEOUT && can_hibernate && prev_hibernate)
{
/* Ask for notification at next buffer allocation */
- StrategyNotifyBgWriter(MyProc->pgprocno);
+ StrategyNotifyBgWriter(GetNumberFromPGProc(MyProc));
/* Sleep ... */
(void) WaitLatch(MyLatch,
WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH,
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 67693b05806..9bfe4fae046 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -242,7 +242,7 @@ PgArchiverMain(void)
* Advertise our pgprocno so that backends can use our latch to wake us up
* while we're sleeping.
*/
- PgArch->pgprocno = MyProc->pgprocno;
+ PgArch->pgprocno = GetNumberFromPGProc(MyProc);
/* Create workspace for pgarch_readyXlog() */
arch_files = palloc(sizeof(struct arch_files_state));
diff --git a/src/backend/postmaster/walsummarizer.c b/src/backend/postmaster/walsummarizer.c
index 9b883c21ca4..165bdd93fa3 100644
--- a/src/backend/postmaster/walsummarizer.c
+++ b/src/backend/postmaster/walsummarizer.c
@@ -248,7 +248,7 @@ WalSummarizerMain(void)
/* Advertise ourselves. */
on_shmem_exit(WalSummarizerShutdown, (Datum) 0);
LWLockAcquire(WALSummarizerLock, LW_EXCLUSIVE);
- WalSummarizerCtl->summarizer_pgprocno = MyProc->pgprocno;
+ WalSummarizerCtl->summarizer_pgprocno = GetNumberFromPGProc(MyProc);
LWLockRelease(WALSummarizerLock);
/* Create and switch to a memory context that we can reset on error. */
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index d06014bfb84..c4c28bb37b3 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -4780,7 +4780,7 @@ UnlockBuffers(void)
* got a cancel/die interrupt before getting the signal.
*/
if ((buf_state & BM_PIN_COUNT_WAITER) != 0 &&
- buf->wait_backend_pgprocno == MyProc->pgprocno)
+ buf->wait_backend_pgprocno == GetNumberFromPGProc(MyProc))
buf_state &= ~BM_PIN_COUNT_WAITER;
UnlockBufHdr(buf, buf_state);
@@ -4930,7 +4930,7 @@ LockBufferForCleanup(Buffer buffer)
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
elog(ERROR, "multiple backends attempting to wait for pincount 1");
}
- bufHdr->wait_backend_pgprocno = MyProc->pgprocno;
+ bufHdr->wait_backend_pgprocno = GetNumberFromPGProc(MyProc);
PinCountWaitBuf = bufHdr;
buf_state |= BM_PIN_COUNT_WAITER;
UnlockBufHdr(bufHdr, buf_state);
@@ -4994,7 +4994,7 @@ LockBufferForCleanup(Buffer buffer)
*/
buf_state = LockBufHdr(bufHdr);
if ((buf_state & BM_PIN_COUNT_WAITER) != 0 &&
- bufHdr->wait_backend_pgprocno == MyProc->pgprocno)
+ bufHdr->wait_backend_pgprocno == GetNumberFromPGProc(MyProc))
buf_state &= ~BM_PIN_COUNT_WAITER;
UnlockBufHdr(bufHdr, buf_state);
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index ee2d7f8585a..5a33eb7f7f7 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -505,7 +505,7 @@ ProcArrayAdd(PGPROC *proc)
Assert(allProcs[procno].pgxactoff == index);
/* If we have found our right position in the array, break */
- if (arrayP->pgprocnos[index] > proc->pgprocno)
+ if (arrayP->pgprocnos[index] > GetNumberFromPGProc(proc))
break;
}
@@ -523,7 +523,7 @@ ProcArrayAdd(PGPROC *proc)
&ProcGlobal->statusFlags[index],
movecount * sizeof(*ProcGlobal->statusFlags));
- arrayP->pgprocnos[index] = proc->pgprocno;
+ arrayP->pgprocnos[index] = GetNumberFromPGProc(proc);
proc->pgxactoff = index;
ProcGlobal->xids[index] = proc->xid;
ProcGlobal->subxidStates[index] = proc->subxidStatus;
@@ -808,7 +808,7 @@ ProcArrayGroupClearXid(PGPROC *proc, TransactionId latestXid)
if (pg_atomic_compare_exchange_u32(&procglobal->procArrayGroupFirst,
&nextidx,
- (uint32) proc->pgprocno))
+ (uint32) GetNumberFromPGProc(proc)))
break;
}
diff --git a/src/backend/storage/lmgr/condition_variable.c b/src/backend/storage/lmgr/condition_variable.c
index e2d6d685220..d4a50c70ea4 100644
--- a/src/backend/storage/lmgr/condition_variable.c
+++ b/src/backend/storage/lmgr/condition_variable.c
@@ -57,7 +57,7 @@ ConditionVariableInit(ConditionVariable *cv)
void
ConditionVariablePrepareToSleep(ConditionVariable *cv)
{
- int pgprocno = MyProc->pgprocno;
+ int pgprocno = GetNumberFromPGProc(MyProc);
/*
* If some other sleep is already prepared, cancel it; this is necessary
@@ -181,10 +181,10 @@ ConditionVariableTimedSleep(ConditionVariable *cv, long timeout,
* guarantee not to return spuriously, we'll avoid this obvious case.
*/
SpinLockAcquire(&cv->mutex);
- if (!proclist_contains(&cv->wakeup, MyProc->pgprocno, cvWaitLink))
+ if (!proclist_contains(&cv->wakeup, GetNumberFromPGProc(MyProc), cvWaitLink))
{
done = true;
- proclist_push_tail(&cv->wakeup, MyProc->pgprocno, cvWaitLink);
+ proclist_push_tail(&cv->wakeup, GetNumberFromPGProc(MyProc), cvWaitLink);
}
SpinLockRelease(&cv->mutex);
@@ -236,8 +236,8 @@ ConditionVariableCancelSleep(void)
return false;
SpinLockAcquire(&cv->mutex);
- if (proclist_contains(&cv->wakeup, MyProc->pgprocno, cvWaitLink))
- proclist_delete(&cv->wakeup, MyProc->pgprocno, cvWaitLink);
+ if (proclist_contains(&cv->wakeup, GetNumberFromPGProc(MyProc), cvWaitLink))
+ proclist_delete(&cv->wakeup, GetNumberFromPGProc(MyProc), cvWaitLink);
else
signaled = true;
SpinLockRelease(&cv->mutex);
@@ -281,7 +281,7 @@ ConditionVariableSignal(ConditionVariable *cv)
void
ConditionVariableBroadcast(ConditionVariable *cv)
{
- int pgprocno = MyProc->pgprocno;
+ int pgprocno = GetNumberFromPGProc(MyProc);
PGPROC *proc = NULL;
bool have_sentinel = false;
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index 98fa6035cc5..9b69e9f1e32 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -1056,9 +1056,9 @@ LWLockQueueSelf(LWLock *lock, LWLockMode mode)
/* LW_WAIT_UNTIL_FREE waiters are always at the front of the queue */
if (mode == LW_WAIT_UNTIL_FREE)
- proclist_push_head(&lock->waiters, MyProc->pgprocno, lwWaitLink);
+ proclist_push_head(&lock->waiters, GetNumberFromPGProc(MyProc), lwWaitLink);
else
- proclist_push_tail(&lock->waiters, MyProc->pgprocno, lwWaitLink);
+ proclist_push_tail(&lock->waiters, GetNumberFromPGProc(MyProc), lwWaitLink);
/* Can release the mutex now */
LWLockWaitListUnlock(lock);
@@ -1097,7 +1097,7 @@ LWLockDequeueSelf(LWLock *lock)
*/
on_waitlist = MyProc->lwWaiting == LW_WS_WAITING;
if (on_waitlist)
- proclist_delete(&lock->waiters, MyProc->pgprocno, lwWaitLink);
+ proclist_delete(&lock->waiters, GetNumberFromPGProc(MyProc), lwWaitLink);
if (proclist_is_empty(&lock->waiters) &&
(pg_atomic_read_u32(&lock->state) & LW_FLAG_HAS_WAITERS) != 0)
diff --git a/src/backend/storage/lmgr/predicate.c b/src/backend/storage/lmgr/predicate.c
index eed63a05ed8..3eedaf70b27 100644
--- a/src/backend/storage/lmgr/predicate.c
+++ b/src/backend/storage/lmgr/predicate.c
@@ -1830,7 +1830,7 @@ GetSerializableTransactionSnapshotInt(Snapshot snapshot,
sxact->finishedBefore = InvalidTransactionId;
sxact->xmin = snapshot->xmin;
sxact->pid = MyProcPid;
- sxact->pgprocno = MyProc->pgprocno;
+ sxact->pgprocno = GetNumberFromPGProc(MyProc);
dlist_init(&sxact->predicateLocks);
dlist_node_init(&sxact->finishedLink);
sxact->flags = 0;
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index e5977548fe2..e355768634e 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -227,7 +227,6 @@ InitProcGlobal(void)
InitSharedLatch(&(proc->procLatch));
LWLockInitialize(&(proc->fpInfoLock), LWTRANCHE_LOCK_FASTPATH);
}
- proc->pgprocno = i;
/*
* Newly created PGPROCs for normal backends, autovacuum and bgworkers
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index 00679624f7d..ed6071f3286 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -540,7 +540,7 @@ typedef enum
* used for a given lock group is determined by the group leader's pgprocno.
*/
#define LockHashPartitionLockByProc(leader_pgproc) \
- LockHashPartitionLock((leader_pgproc)->pgprocno)
+ LockHashPartitionLock(GetNumberFromPGProc(leader_pgproc))
/*
* function prototypes
diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h
index 4bc226e36cd..37cf8b4067d 100644
--- a/src/include/storage/proc.h
+++ b/src/include/storage/proc.h
@@ -194,11 +194,6 @@ struct PGPROC
int pgxactoff; /* offset into various ProcGlobal->arrays with
* data mirrored from this PGPROC */
- int pgprocno; /* Number of this PGPROC in
- * ProcGlobal->allProcs array. This is set
- * once by InitProcGlobal().
- * ProcGlobal->allProcs[n].pgprocno == n */
-
/* These fields are zero while a backend is still starting up: */
BackendId backendId; /* This backend's backend ID (if assigned) */
Oid databaseId; /* OID of database this backend is using */
@@ -412,6 +407,7 @@ extern PGDLLIMPORT PGPROC *PreparedXactProcs;
/* Accessor for PGPROC given a pgprocno. */
#define GetPGProcByNumber(n) (&ProcGlobal->allProcs[(n)])
+#define GetNumberFromPGProc(proc) ((proc) - &ProcGlobal->allProcs[0])
/*
* We set aside some extra PGPROC structures for auxiliary processes,
--
2.39.2
v9-0002-Redefine-backend-ID-to-be-an-index-into-the-proc-.patchtext/x-patch; charset=UTF-8; name=v9-0002-Redefine-backend-ID-to-be-an-index-into-the-proc-.patchDownload
From 5b19b778bcd1128690cbef6fb1a918b7687ae9db Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Thu, 1 Feb 2024 15:32:35 +0200
Subject: [PATCH v9 2/4] Redefine backend ID to be an index into the proc
array.
Previously, backend ID was an index into the ProcState array, in the
shared cache invalidation manager (sinvaladt.c). The entry in the
ProcState array was reserved at backend startup by scanning the array
for a free entry, and that was also when the backend got its backend
ID. Things becomes slightly simpler if we redefine backend ID to be
the index into the PGPROC array, and directly use it also as an index
to the ProcState array. This uses a little more memory, as we reserve
a few extra slots in the ProcState array for aux processes that don't
need them, but the simplicity is worth it.
Aux processes now also have a backend ID. This simplifies the
reservation of BackendStatusArray and ProcSignal slots.
You can now convert a backend ID into an index into the PGPROC array
simply by subtracting 1. We still use 0-based "pgprocnos" in various
places, for indexes into the PGPROC array, but the only difference now
is that backend IDs start at 1 while pgprocnos start at 0.
There is still a 'backendId' field in PGPROC, now part of 'vxid' which
encapsulates the backend ID and local transaction ID together. It's
needed for prepared xacts. For regular backends, the backendId is
always equal to pgprocno + 1, but for prepared xact PGPROC entries,
it's the ID of the original backend that processed the transaction.
Reviewed-by: XXX
Discussion: https://www.postgresql.org/message-id/8171f1aa-496f-46a6-afc3-c46fe7a9b407@iki.fi
---
src/backend/access/transam/twophase.c | 37 ++--
src/backend/access/transam/xact.c | 12 +-
src/backend/catalog/namespace.c | 2 +-
src/backend/commands/sequence.c | 2 +-
src/backend/executor/functions.c | 4 +-
src/backend/postmaster/auxprocess.c | 12 +-
src/backend/storage/ipc/procarray.c | 73 +++++++-
src/backend/storage/ipc/procsignal.c | 27 ++-
src/backend/storage/ipc/sinvaladt.c | 191 ++++++--------------
src/backend/storage/ipc/standby.c | 1 +
src/backend/storage/lmgr/lock.c | 28 +--
src/backend/storage/lmgr/proc.c | 31 ++--
src/backend/utils/activity/backend_status.c | 52 ++----
src/backend/utils/adt/lockfuncs.c | 2 +-
src/backend/utils/adt/mcxtfuncs.c | 14 +-
src/backend/utils/error/csvlog.c | 4 +-
src/backend/utils/error/elog.c | 6 +-
src/backend/utils/error/jsonlog.c | 6 +-
src/backend/utils/init/postinit.c | 10 +-
src/backend/utils/time/snapmgr.c | 5 +-
src/include/miscadmin.h | 2 -
src/include/storage/backendid.h | 12 +-
src/include/storage/lock.h | 9 +-
src/include/storage/proc.h | 32 +++-
src/include/storage/procarray.h | 4 +
src/include/storage/procsignal.h | 2 +-
src/include/storage/sinvaladt.h | 4 -
src/pl/plpgsql/src/pl_exec.c | 10 +-
28 files changed, 274 insertions(+), 320 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 234c8d08ebc..5c282002900 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -151,7 +151,6 @@ typedef struct GlobalTransactionData
{
GlobalTransaction next; /* list link for free list */
int pgprocno; /* ID of associated dummy PGPROC */
- BackendId dummyBackendId; /* similar to backend id for backends */
TimestampTz prepared_at; /* time of preparation */
/*
@@ -285,20 +284,6 @@ TwoPhaseShmemInit(void)
/* associate it with a PGPROC assigned by InitProcGlobal */
gxacts[i].pgprocno = GetNumberFromPGProc(&PreparedXactProcs[i]);
-
- /*
- * Assign a unique ID for each dummy proc, so that the range of
- * dummy backend IDs immediately follows the range of normal
- * backend IDs. We don't dare to assign a real backend ID to dummy
- * procs, because prepared transactions don't take part in cache
- * invalidation like a real backend ID would imply, but having a
- * unique ID for them is nevertheless handy. This arrangement
- * allows you to allocate an array of size (MaxBackends +
- * max_prepared_xacts + 1), and have a slot for every backend and
- * prepared transaction. Currently multixact.c uses that
- * technique.
- */
- gxacts[i].dummyBackendId = MaxBackends + 1 + i;
}
}
else
@@ -457,24 +442,24 @@ MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid, const char *gid,
Assert(LWLockHeldByMeInMode(TwoPhaseStateLock, LW_EXCLUSIVE));
Assert(gxact != NULL);
- proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ proc = GetPGProcByNumber(gxact->pgprocno);
/* Initialize the PGPROC entry */
MemSet(proc, 0, sizeof(PGPROC));
dlist_node_init(&proc->links);
proc->waitStatus = PROC_WAIT_STATUS_OK;
- if (LocalTransactionIdIsValid(MyProc->lxid))
+ if (LocalTransactionIdIsValid(MyProc->vxid.lxid))
{
/* clone VXID, for TwoPhaseGetXidByVirtualXID() to find */
- proc->lxid = MyProc->lxid;
- proc->backendId = MyBackendId;
+ proc->vxid.lxid = MyProc->vxid.lxid;
+ proc->vxid.backendId = MyBackendId;
}
else
{
Assert(AmStartupProcess() || !IsPostmasterEnvironment);
/* GetLockConflicts() uses this to specify a wait on the XID */
- proc->lxid = xid;
- proc->backendId = InvalidBackendId;
+ proc->vxid.lxid = xid;
+ proc->vxid.backendId = InvalidBackendId;
}
proc->xid = xid;
Assert(proc->xmin == InvalidTransactionId);
@@ -522,7 +507,7 @@ static void
GXactLoadSubxactData(GlobalTransaction gxact, int nsubxacts,
TransactionId *children)
{
- PGPROC *proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ PGPROC *proc = GetPGProcByNumber(gxact->pgprocno);
/* We need no extra lock since the GXACT isn't valid yet */
if (nsubxacts > PGPROC_MAX_CACHED_SUBXIDS)
@@ -559,7 +544,7 @@ MarkAsPrepared(GlobalTransaction gxact, bool lock_held)
* Put it into the global ProcArray so TransactionIdIsInProgress considers
* the XID as still running.
*/
- ProcArrayAdd(&ProcGlobal->allProcs[gxact->pgprocno]);
+ ProcArrayAdd(GetPGProcByNumber(gxact->pgprocno));
}
/*
@@ -583,7 +568,7 @@ LockGXact(const char *gid, Oid user)
for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
{
GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
- PGPROC *proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ PGPROC *proc = GetPGProcByNumber(gxact->pgprocno);
/* Ignore not-yet-valid GIDs */
if (!gxact->valid)
@@ -884,7 +869,7 @@ TwoPhaseGetXidByVirtualXID(VirtualTransactionId vxid,
if (!gxact->valid)
continue;
- proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ proc = GetPGProcByNumber(gxact->pgprocno);
GET_VXID_FROM_PGPROC(proc_vxid, *proc);
if (VirtualTransactionIdEquals(vxid, proc_vxid))
{
@@ -919,7 +904,7 @@ TwoPhaseGetDummyBackendId(TransactionId xid, bool lock_held)
{
GlobalTransaction gxact = TwoPhaseGetGXact(xid, lock_held);
- return gxact->dummyBackendId;
+ return gxact->pgprocno + 1;
}
/*
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 464858117e0..f791340d3df 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -600,9 +600,9 @@ GetStableLatestTransactionId(void)
static LocalTransactionId lxid = InvalidLocalTransactionId;
static TransactionId stablexid = InvalidTransactionId;
- if (lxid != MyProc->lxid)
+ if (lxid != MyProc->vxid.lxid)
{
- lxid = MyProc->lxid;
+ lxid = MyProc->vxid.lxid;
stablexid = GetTopTransactionIdIfAny();
if (!TransactionIdIsValid(stablexid))
stablexid = ReadNextTransactionId();
@@ -2099,8 +2099,8 @@ StartTransaction(void)
* Advertise it in the proc array. We assume assignment of
* localTransactionId is atomic, and the backendId should be set already.
*/
- Assert(MyProc->backendId == vxid.backendId);
- MyProc->lxid = vxid.localTransactionId;
+ Assert(MyProc->vxid.backendId == vxid.backendId);
+ MyProc->vxid.lxid = vxid.localTransactionId;
TRACE_POSTGRESQL_TRANSACTION_START(vxid.localTransactionId);
@@ -2281,7 +2281,7 @@ CommitTransaction(void)
ParallelWorkerReportLastRecEnd(XactLastRecEnd);
}
- TRACE_POSTGRESQL_TRANSACTION_COMMIT(MyProc->lxid);
+ TRACE_POSTGRESQL_TRANSACTION_COMMIT(MyProc->vxid.lxid);
/*
* Let others know about no transaction in progress by me. Note that this
@@ -2824,7 +2824,7 @@ AbortTransaction(void)
XLogSetAsyncXactLSN(XactLastRecEnd);
}
- TRACE_POSTGRESQL_TRANSACTION_ABORT(MyProc->lxid);
+ TRACE_POSTGRESQL_TRANSACTION_ABORT(MyProc->vxid.lxid);
/*
* Let others know about no transaction in progress by me. Note that this
diff --git a/src/backend/catalog/namespace.c b/src/backend/catalog/namespace.c
index b610aa62423..e6eea92abd6 100644
--- a/src/backend/catalog/namespace.c
+++ b/src/backend/catalog/namespace.c
@@ -49,7 +49,7 @@
#include "parser/parse_func.h"
#include "storage/ipc.h"
#include "storage/lmgr.h"
-#include "storage/sinvaladt.h"
+#include "storage/procarray.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index f7803744a5b..f99dc48e8cf 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1076,7 +1076,7 @@ setval3_oid(PG_FUNCTION_ARGS)
static Relation
lock_and_open_sequence(SeqTable seq)
{
- LocalTransactionId thislxid = MyProc->lxid;
+ LocalTransactionId thislxid = MyProc->vxid.lxid;
/* Get the lock if not already held in this xact */
if (seq->lxid != thislxid)
diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c
index 0f811fd2fc9..a4b6e1effdb 100644
--- a/src/backend/executor/functions.c
+++ b/src/backend/executor/functions.c
@@ -799,7 +799,7 @@ init_sql_fcache(FunctionCallInfo fcinfo, Oid collation, bool lazyEvalOK)
lazyEvalOK);
/* Mark fcache with time of creation to show it's valid */
- fcache->lxid = MyProc->lxid;
+ fcache->lxid = MyProc->vxid.lxid;
fcache->subxid = GetCurrentSubTransactionId();
ReleaseSysCache(procedureTuple);
@@ -1081,7 +1081,7 @@ fmgr_sql(PG_FUNCTION_ARGS)
if (fcache != NULL)
{
- if (fcache->lxid != MyProc->lxid ||
+ if (fcache->lxid != MyProc->vxid.lxid ||
!SubTransactionIsActive(fcache->subxid))
{
/* It's stale; unlink and delete */
diff --git a/src/backend/postmaster/auxprocess.c b/src/backend/postmaster/auxprocess.c
index ab86e802f21..39171fea06b 100644
--- a/src/backend/postmaster/auxprocess.c
+++ b/src/backend/postmaster/auxprocess.c
@@ -107,17 +107,7 @@ AuxiliaryProcessMain(AuxProcType auxtype)
BaseInit();
- /*
- * Assign the ProcSignalSlot for an auxiliary process. Since it doesn't
- * have a BackendId, the slot is statically allocated based on the
- * auxiliary process type (MyAuxProcType). Backends use slots indexed in
- * the range from 1 to MaxBackends (inclusive), so we use MaxBackends +
- * AuxProcType + 1 as the index of the slot for an auxiliary process.
- *
- * This will need rethinking if we ever want more than one of a particular
- * auxiliary process type.
- */
- ProcSignalInit(MaxBackends + MyAuxProcType + 1);
+ ProcSignalInit();
/*
* Auxiliary processes don't run transactions, but they may need a
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 5a33eb7f7f7..15044e1e14b 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -700,7 +700,7 @@ ProcArrayEndTransaction(PGPROC *proc, TransactionId latestXid)
Assert(proc->subxidStatus.count == 0);
Assert(!proc->subxidStatus.overflowed);
- proc->lxid = InvalidLocalTransactionId;
+ proc->vxid.lxid = InvalidLocalTransactionId;
proc->xmin = InvalidTransactionId;
/* be sure this is cleared in abort */
@@ -742,7 +742,7 @@ ProcArrayEndTransactionInternal(PGPROC *proc, TransactionId latestXid)
ProcGlobal->xids[pgxactoff] = InvalidTransactionId;
proc->xid = InvalidTransactionId;
- proc->lxid = InvalidLocalTransactionId;
+ proc->vxid.lxid = InvalidLocalTransactionId;
proc->xmin = InvalidTransactionId;
/* be sure this is cleared in abort */
@@ -928,7 +928,7 @@ ProcArrayClearTransaction(PGPROC *proc)
ProcGlobal->xids[pgxactoff] = InvalidTransactionId;
proc->xid = InvalidTransactionId;
- proc->lxid = InvalidLocalTransactionId;
+ proc->vxid.lxid = InvalidLocalTransactionId;
proc->xmin = InvalidTransactionId;
proc->recoveryConflictPending = false;
@@ -2534,6 +2534,11 @@ ProcArrayInstallImportedXmin(TransactionId xmin,
/* Get lock so source xact can't end while we're doing this */
LWLockAcquire(ProcArrayLock, LW_SHARED);
+ /*
+ * Find the PGPROC entry of the source transaction. (This could use
+ * GetPGProcByBackendId(), unless it's a prepared xact. But this isn't
+ * performance critical.)
+ */
for (index = 0; index < arrayP->numProcs; index++)
{
int pgprocno = arrayP->pgprocnos[index];
@@ -2546,9 +2551,9 @@ ProcArrayInstallImportedXmin(TransactionId xmin,
continue;
/* We are only interested in the specific virtual transaction. */
- if (proc->backendId != sourcevxid->backendId)
+ if (proc->vxid.backendId != sourcevxid->backendId)
continue;
- if (proc->lxid != sourcevxid->localTransactionId)
+ if (proc->vxid.lxid != sourcevxid->localTransactionId)
continue;
/*
@@ -3097,6 +3102,64 @@ HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids, int type)
return result;
}
+/*
+ * BackendIdGetProc -- get a backend's PGPROC given its backend ID
+ *
+ * The result may be out of date arbitrarily quickly, so the caller
+ * must be careful about how this information is used. NULL is
+ * returned if the backend is not active.
+ */
+PGPROC *
+BackendIdGetProc(int backendID)
+{
+ PGPROC *result;
+
+ if (backendID < 1 || backendID > ProcGlobal->allProcCount)
+ return NULL;
+ result = GetPGProcByBackendId(backendID);
+
+ if (result->pid == 0)
+ return NULL;
+
+ return result;
+}
+
+/*
+ * BackendIdGetTransactionIds -- get a backend's transaction status
+ *
+ * Get the xid, xmin, nsubxid and overflow status of the backend. The
+ * result may be out of date arbitrarily quickly, so the caller must be
+ * careful about how this information is used.
+ */
+void
+BackendIdGetTransactionIds(int backendID, TransactionId *xid,
+ TransactionId *xmin, int *nsubxid, bool *overflowed)
+{
+ PGPROC *proc;
+
+ *xid = InvalidTransactionId;
+ *xmin = InvalidTransactionId;
+ *nsubxid = 0;
+ *overflowed = false;
+
+ if (backendID < 1 || backendID > ProcGlobal->allProcCount)
+ return;
+ proc = GetPGProcByBackendId(backendID);
+
+ /* Need to lock out additions/removals of backends */
+ LWLockAcquire(ProcArrayLock, LW_SHARED);
+
+ if (proc->pid != 0)
+ {
+ *xid = proc->xid;
+ *xmin = proc->xmin;
+ *nsubxid = proc->subxidStatus.count;
+ *overflowed = proc->subxidStatus.overflowed;
+ }
+
+ LWLockRelease(ProcArrayLock);
+}
+
/*
* BackendPidGetProc -- get a backend's PGPROC given its PID
*
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index e84619e5a58..d1d5bf0c152 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -87,7 +87,7 @@ typedef struct
* possible auxiliary process type. (This scheme assumes there is not
* more than one of any auxiliary process type at a time.)
*/
-#define NumProcSignalSlots (MaxBackends + NUM_AUXPROCTYPES)
+#define NumProcSignalSlots (MaxBackends + NUM_AUXILIARY_PROCS)
/* Check whether the relevant type bit is set in the flags. */
#define BARRIER_SHOULD_CHECK(flags, type) \
@@ -154,24 +154,23 @@ ProcSignalShmemInit(void)
/*
* ProcSignalInit
* Register the current process in the ProcSignal array
- *
- * The passed index should be my BackendId if the process has one,
- * or MaxBackends + aux process type if not.
*/
void
-ProcSignalInit(int pss_idx)
+ProcSignalInit(void)
{
ProcSignalSlot *slot;
uint64 barrier_generation;
- Assert(pss_idx >= 1 && pss_idx <= NumProcSignalSlots);
-
- slot = &ProcSignal->psh_slot[pss_idx - 1];
+ if (MyBackendId <= 0)
+ elog(ERROR, "MyBackendId not set");
+ if (MyBackendId > NumProcSignalSlots)
+ elog(ERROR, "unexpected MyBackendId %d in ProcSignalInit (max %d)", MyBackendId, NumProcSignalSlots);
+ slot = &ProcSignal->psh_slot[MyBackendId - 1];
/* sanity check */
if (slot->pss_pid != 0)
elog(LOG, "process %d taking over ProcSignal slot %d, but it's not empty",
- MyProcPid, pss_idx);
+ MyProcPid, (int) (slot - ProcSignal->psh_slot));
/* Clear out any leftover signal reasons */
MemSet(slot->pss_signalFlags, 0, NUM_PROCSIGNALS * sizeof(sig_atomic_t));
@@ -200,7 +199,7 @@ ProcSignalInit(int pss_idx)
MyProcSignalSlot = slot;
/* Set up to release the slot on process exit */
- on_shmem_exit(CleanupProcSignalState, Int32GetDatum(pss_idx));
+ on_shmem_exit(CleanupProcSignalState, (Datum) 0);
}
/*
@@ -212,11 +211,7 @@ ProcSignalInit(int pss_idx)
static void
CleanupProcSignalState(int status, Datum arg)
{
- int pss_idx = DatumGetInt32(arg);
- ProcSignalSlot *slot;
-
- slot = &ProcSignal->psh_slot[pss_idx - 1];
- Assert(slot == MyProcSignalSlot);
+ ProcSignalSlot *slot = MyProcSignalSlot;
/*
* Clear MyProcSignalSlot, so that a SIGUSR1 received after this point
@@ -233,7 +228,7 @@ CleanupProcSignalState(int status, Datum arg)
* infinite loop trying to exit
*/
elog(LOG, "process %d releasing ProcSignal slot %d, but it contains %d",
- MyProcPid, pss_idx, (int) slot->pss_pid);
+ MyProcPid, (int) (slot - ProcSignal->psh_slot), (int) slot->pss_pid);
return; /* XXX better to zero the slot anyway? */
}
diff --git a/src/backend/storage/ipc/sinvaladt.c b/src/backend/storage/ipc/sinvaladt.c
index 748a792a854..a12aaae601a 100644
--- a/src/backend/storage/ipc/sinvaladt.c
+++ b/src/backend/storage/ipc/sinvaladt.c
@@ -139,7 +139,6 @@ typedef struct ProcState
{
/* procPid is zero in an inactive ProcState array entry. */
pid_t procPid; /* PID of backend, for signaling */
- PGPROC *proc; /* PGPROC of backend */
/* nextMsgNum is meaningless if procPid == 0 or resetState is true. */
int nextMsgNum; /* next message number to read */
bool resetState; /* backend needs to reset its state */
@@ -172,8 +171,6 @@ typedef struct SISeg
int minMsgNum; /* oldest message still needed */
int maxMsgNum; /* next message number to be assigned */
int nextThreshold; /* # of messages to call SICleanupQueue */
- int lastBackend; /* index of last active procState entry, +1 */
- int maxBackends; /* size of procState array */
slock_t msgnumLock; /* spinlock protecting maxMsgNum */
@@ -183,11 +180,24 @@ typedef struct SISeg
SharedInvalidationMessage buffer[MAXNUMMESSAGES];
/*
- * Per-backend invalidation state info (has MaxBackends entries).
+ * Per-backend invalidation state info.
+ *
+ * 'procState' has NumProcStateSlots entries, and is indexed by pgprocno.
+ * 'numProcs' is the number of slots currently in use, and 'pgprocnos' is
+ * a dense array of their indexes, to speed up scanning all in-use slots.
*/
+ int numProcs;
+ int *pgprocnos;
ProcState procState[FLEXIBLE_ARRAY_MEMBER];
} SISeg;
+/*
+ * We reserve a slot for each possible BackendId, plus one for each
+ * possible auxiliary process type. (This scheme assumes there is not
+ * more than one of any auxiliary process type at a time.)
+ */
+#define NumProcStateSlots (MaxBackends + NUM_AUXILIARY_PROCS)
+
static SISeg *shmInvalBuffer; /* pointer to the shared inval buffer */
@@ -205,16 +215,8 @@ SInvalShmemSize(void)
Size size;
size = offsetof(SISeg, procState);
-
- /*
- * In Hot Standby mode, the startup process requests a procState array
- * slot using InitRecoveryTransactionEnvironment(). Even though
- * MaxBackends doesn't account for the startup process, it is guaranteed
- * to get a free slot. This is because the autovacuum launcher and worker
- * processes, which are included in MaxBackends, are not started in Hot
- * Standby mode.
- */
- size = add_size(size, mul_size(sizeof(ProcState), MaxBackends));
+ size = add_size(size, mul_size(sizeof(ProcState), NumProcStateSlots)); /* procState */
+ size = add_size(size, mul_size(sizeof(int), NumProcStateSlots)); /* pgprocnos */
return size;
}
@@ -239,23 +241,22 @@ CreateSharedInvalidationState(void)
shmInvalBuffer->minMsgNum = 0;
shmInvalBuffer->maxMsgNum = 0;
shmInvalBuffer->nextThreshold = CLEANUP_MIN;
- shmInvalBuffer->lastBackend = 0;
- shmInvalBuffer->maxBackends = MaxBackends;
SpinLockInit(&shmInvalBuffer->msgnumLock);
/* The buffer[] array is initially all unused, so we need not fill it */
/* Mark all backends inactive, and initialize nextLXID */
- for (i = 0; i < shmInvalBuffer->maxBackends; i++)
+ for (i = 0; i < NumProcStateSlots; i++)
{
shmInvalBuffer->procState[i].procPid = 0; /* inactive */
- shmInvalBuffer->procState[i].proc = NULL;
shmInvalBuffer->procState[i].nextMsgNum = 0; /* meaningless */
shmInvalBuffer->procState[i].resetState = false;
shmInvalBuffer->procState[i].signaled = false;
shmInvalBuffer->procState[i].hasMessages = false;
shmInvalBuffer->procState[i].nextLXID = InvalidLocalTransactionId;
}
+ shmInvalBuffer->numProcs = 0;
+ shmInvalBuffer->pgprocnos = (int *) &shmInvalBuffer->procState[i];
}
/*
@@ -265,9 +266,18 @@ CreateSharedInvalidationState(void)
void
SharedInvalBackendInit(bool sendOnly)
{
- int index;
- ProcState *stateP = NULL;
+ ProcState *stateP;
+ pid_t oldPid;
SISeg *segP = shmInvalBuffer;
+ int pgprocno;
+
+ if (MyBackendId <= 0)
+ elog(ERROR, "MyBackendId not set");
+ if (MyBackendId > NumProcStateSlots)
+ elog(PANIC, "unexpected MyBackendId %d in SharedInvalBackendInit (max %d)",
+ MyBackendId, NumProcStateSlots);
+ pgprocno = MyBackendId - 1;
+ stateP = &segP->procState[pgprocno];
/*
* This can run in parallel with read operations, but not with write
@@ -276,48 +286,21 @@ SharedInvalBackendInit(bool sendOnly)
*/
LWLockAcquire(SInvalWriteLock, LW_EXCLUSIVE);
- /* Look for a free entry in the procState array */
- for (index = 0; index < segP->lastBackend; index++)
- {
- if (segP->procState[index].procPid == 0) /* inactive slot? */
- {
- stateP = &segP->procState[index];
- break;
- }
- }
-
- if (stateP == NULL)
+ oldPid = stateP->procPid;
+ if (oldPid != 0)
{
- if (segP->lastBackend < segP->maxBackends)
- {
- stateP = &segP->procState[segP->lastBackend];
- Assert(stateP->procPid == 0);
- segP->lastBackend++;
- }
- else
- {
- /*
- * out of procState slots: MaxBackends exceeded -- report normally
- */
- MyBackendId = InvalidBackendId;
- LWLockRelease(SInvalWriteLock);
- ereport(FATAL,
- (errcode(ERRCODE_TOO_MANY_CONNECTIONS),
- errmsg("sorry, too many clients already")));
- }
+ LWLockRelease(SInvalWriteLock);
+ elog(ERROR, "sinval slot for backend %d is already in use by process %d",
+ MyBackendId, (int) oldPid);
}
- MyBackendId = (stateP - &segP->procState[0]) + 1;
-
- /* Advertise assigned backend ID in MyProc */
- MyProc->backendId = MyBackendId;
+ shmInvalBuffer->pgprocnos[shmInvalBuffer->numProcs++] = pgprocno;
/* Fetch next local transaction ID into local memory */
nextLocalTransactionId = stateP->nextLXID;
/* mark myself active, with all extant messages already read */
stateP->procPid = MyProcPid;
- stateP->proc = MyProc;
stateP->nextMsgNum = segP->maxMsgNum;
stateP->resetState = false;
stateP->signaled = false;
@@ -328,8 +311,6 @@ SharedInvalBackendInit(bool sendOnly)
/* register exit routine to mark my entry inactive at exit */
on_shmem_exit(CleanupInvalidationState, PointerGetDatum(segP));
-
- elog(DEBUG4, "my backend ID is %d", MyBackendId);
}
/*
@@ -345,96 +326,36 @@ CleanupInvalidationState(int status, Datum arg)
{
SISeg *segP = (SISeg *) DatumGetPointer(arg);
ProcState *stateP;
+ int pgprocno = MyBackendId - 1;
int i;
Assert(PointerIsValid(segP));
LWLockAcquire(SInvalWriteLock, LW_EXCLUSIVE);
- stateP = &segP->procState[MyBackendId - 1];
+ stateP = &segP->procState[pgprocno];
/* Update next local transaction ID for next holder of this backendID */
stateP->nextLXID = nextLocalTransactionId;
/* Mark myself inactive */
stateP->procPid = 0;
- stateP->proc = NULL;
stateP->nextMsgNum = 0;
stateP->resetState = false;
stateP->signaled = false;
- /* Recompute index of last active backend */
- for (i = segP->lastBackend; i > 0; i--)
- {
- if (segP->procState[i - 1].procPid != 0)
- break;
- }
- segP->lastBackend = i;
-
- LWLockRelease(SInvalWriteLock);
-}
-
-/*
- * BackendIdGetProc
- * Get the PGPROC structure for a backend, given the backend ID.
- * The result may be out of date arbitrarily quickly, so the caller
- * must be careful about how this information is used. NULL is
- * returned if the backend is not active.
- */
-PGPROC *
-BackendIdGetProc(int backendID)
-{
- PGPROC *result = NULL;
- SISeg *segP = shmInvalBuffer;
-
- /* Need to lock out additions/removals of backends */
- LWLockAcquire(SInvalWriteLock, LW_SHARED);
-
- if (backendID > 0 && backendID <= segP->lastBackend)
+ for (i = segP->numProcs - 1; i >= 0; i--)
{
- ProcState *stateP = &segP->procState[backendID - 1];
-
- result = stateP->proc;
- }
-
- LWLockRelease(SInvalWriteLock);
-
- return result;
-}
-
-/*
- * BackendIdGetTransactionIds
- * Get the xid, xmin, nsubxid and overflow status of the backend. The
- * result may be out of date arbitrarily quickly, so the caller must be
- * careful about how this information is used.
- */
-void
-BackendIdGetTransactionIds(int backendID, TransactionId *xid,
- TransactionId *xmin, int *nsubxid, bool *overflowed)
-{
- SISeg *segP = shmInvalBuffer;
-
- *xid = InvalidTransactionId;
- *xmin = InvalidTransactionId;
- *nsubxid = 0;
- *overflowed = false;
-
- /* Need to lock out additions/removals of backends */
- LWLockAcquire(SInvalWriteLock, LW_SHARED);
-
- if (backendID > 0 && backendID <= segP->lastBackend)
- {
- ProcState *stateP = &segP->procState[backendID - 1];
- PGPROC *proc = stateP->proc;
-
- if (proc != NULL)
+ if (segP->pgprocnos[i] == pgprocno)
{
- *xid = proc->xid;
- *xmin = proc->xmin;
- *nsubxid = proc->subxidStatus.count;
- *overflowed = proc->subxidStatus.overflowed;
+ if (i != segP->numProcs - 1)
+ segP->pgprocnos[i] = segP->pgprocnos[segP->numProcs - 1];
+ break;
}
}
+ if (i < 0)
+ elog(PANIC, "could not find entry in sinval array");
+ segP->numProcs--;
LWLockRelease(SInvalWriteLock);
}
@@ -507,9 +428,9 @@ SIInsertDataEntries(const SharedInvalidationMessage *data, int n)
* these (unlocked) changes will be committed to memory before we exit
* the function.
*/
- for (i = 0; i < segP->lastBackend; i++)
+ for (i = 0; i < segP->numProcs; i++)
{
- ProcState *stateP = &segP->procState[i];
+ ProcState *stateP = &segP->procState[segP->pgprocnos[i]];
stateP->hasMessages = true;
}
@@ -677,13 +598,14 @@ SICleanupQueue(bool callerHasWriteLock, int minFree)
minsig = min - SIG_THRESHOLD;
lowbound = min - MAXNUMMESSAGES + minFree;
- for (i = 0; i < segP->lastBackend; i++)
+ for (i = 0; i < segP->numProcs; i++)
{
- ProcState *stateP = &segP->procState[i];
+ ProcState *stateP = &segP->procState[segP->pgprocnos[i]];
int n = stateP->nextMsgNum;
- /* Ignore if inactive or already in reset state */
- if (stateP->procPid == 0 || stateP->resetState || stateP->sendOnly)
+ /* Ignore if already in reset state */
+ Assert(stateP->procPid != 0);
+ if (stateP->resetState || stateP->sendOnly)
continue;
/*
@@ -719,11 +641,8 @@ SICleanupQueue(bool callerHasWriteLock, int minFree)
{
segP->minMsgNum -= MSGNUMWRAPAROUND;
segP->maxMsgNum -= MSGNUMWRAPAROUND;
- for (i = 0; i < segP->lastBackend; i++)
- {
- /* we don't bother skipping inactive entries here */
- segP->procState[i].nextMsgNum -= MSGNUMWRAPAROUND;
- }
+ for (i = 0; i < segP->numProcs; i++)
+ segP->procState[segP->pgprocnos[i]].nextMsgNum -= MSGNUMWRAPAROUND;
}
/*
diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c
index d8755a106d5..97d1ab65740 100644
--- a/src/backend/storage/ipc/standby.c
+++ b/src/backend/storage/ipc/standby.c
@@ -137,6 +137,7 @@ InitRecoveryTransactionEnvironment(void)
* are held by vxids and row level locks are held by xids. All queries
* hold AccessShareLocks so never block while we write or lock new rows.
*/
+ MyProc->vxid.backendId = MyBackendId;
vxid.backendId = MyBackendId;
vxid.localTransactionId = GetNextLocalTransactionId();
VirtualXactLockTableInsert(vxid);
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index c70a1adb9ad..e62968b4a86 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -3625,8 +3625,8 @@ GetLockStatusData(void)
proc->fpRelId[f]);
instance->holdMask = lockbits << FAST_PATH_LOCKNUMBER_OFFSET;
instance->waitLockMode = NoLock;
- instance->backend = proc->backendId;
- instance->lxid = proc->lxid;
+ instance->vxid.backendId = proc->vxid.backendId;
+ instance->vxid.localTransactionId = proc->vxid.lxid;
instance->pid = proc->pid;
instance->leaderPid = proc->pid;
instance->fastpath = true;
@@ -3652,15 +3652,15 @@ GetLockStatusData(void)
repalloc(data->locks, sizeof(LockInstanceData) * els);
}
- vxid.backendId = proc->backendId;
+ vxid.backendId = proc->vxid.backendId;
vxid.localTransactionId = proc->fpLocalTransactionId;
instance = &data->locks[el];
SET_LOCKTAG_VIRTUALTRANSACTION(instance->locktag, vxid);
instance->holdMask = LOCKBIT_ON(ExclusiveLock);
instance->waitLockMode = NoLock;
- instance->backend = proc->backendId;
- instance->lxid = proc->lxid;
+ instance->vxid.backendId = proc->vxid.backendId;
+ instance->vxid.localTransactionId = proc->vxid.lxid;
instance->pid = proc->pid;
instance->leaderPid = proc->pid;
instance->fastpath = true;
@@ -3712,8 +3712,8 @@ GetLockStatusData(void)
instance->waitLockMode = proc->waitLockMode;
else
instance->waitLockMode = NoLock;
- instance->backend = proc->backendId;
- instance->lxid = proc->lxid;
+ instance->vxid.backendId = proc->vxid.backendId;
+ instance->vxid.localTransactionId = proc->vxid.lxid;
instance->pid = proc->pid;
instance->leaderPid = proclock->groupLeader->pid;
instance->fastpath = false;
@@ -3888,8 +3888,8 @@ GetSingleProcBlockerStatusData(PGPROC *blocked_proc, BlockedProcsData *data)
instance->waitLockMode = proc->waitLockMode;
else
instance->waitLockMode = NoLock;
- instance->backend = proc->backendId;
- instance->lxid = proc->lxid;
+ instance->vxid.backendId = proc->vxid.backendId;
+ instance->vxid.localTransactionId = proc->vxid.lxid;
instance->pid = proc->pid;
instance->leaderPid = proclock->groupLeader->pid;
instance->fastpath = false;
@@ -4374,8 +4374,8 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
* lockers, as we haven't advertised this vxid via the ProcArray yet.
*
* Since MyProc->fpLocalTransactionId will normally contain the same data
- * as MyProc->lxid, you might wonder if we really need both. The
- * difference is that MyProc->lxid is set and cleared unlocked, and
+ * as MyProc->vxid.lxid, you might wonder if we really need both. The
+ * difference is that MyProc->vxid.lxid is set and cleared unlocked, and
* examined by procarray.c, while fpLocalTransactionId is protected by
* fpInfoLock and is used only by the locking subsystem. Doing it this
* way makes it easier to verify that there are no funny race conditions.
@@ -4391,7 +4391,7 @@ VirtualXactLockTableInsert(VirtualTransactionId vxid)
LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE);
- Assert(MyProc->backendId == vxid.backendId);
+ Assert(MyProc->vxid.backendId == vxid.backendId);
Assert(MyProc->fpLocalTransactionId == InvalidLocalTransactionId);
Assert(MyProc->fpVXIDLock == false);
@@ -4413,7 +4413,7 @@ VirtualXactLockTableCleanup(void)
bool fastpath;
LocalTransactionId lxid;
- Assert(MyProc->backendId != InvalidBackendId);
+ Assert(MyProc->vxid.backendId != InvalidBackendId);
/*
* Clean up shared memory state.
@@ -4541,7 +4541,7 @@ VirtualXactLock(VirtualTransactionId vxid, bool wait)
*/
LWLockAcquire(&proc->fpInfoLock, LW_EXCLUSIVE);
- if (proc->backendId != vxid.backendId
+ if (proc->vxid.backendId != vxid.backendId
|| proc->fpLocalTransactionId != vxid.localTransactionId)
{
/* VXID ended */
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index e355768634e..4c568cd6850 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -239,25 +239,25 @@ InitProcGlobal(void)
if (i < MaxConnections)
{
/* PGPROC for normal backend, add to freeProcs list */
- dlist_push_head(&ProcGlobal->freeProcs, &proc->links);
+ dlist_push_tail(&ProcGlobal->freeProcs, &proc->links);
proc->procgloballist = &ProcGlobal->freeProcs;
}
else if (i < MaxConnections + autovacuum_max_workers + 1)
{
/* PGPROC for AV launcher/worker, add to autovacFreeProcs list */
- dlist_push_head(&ProcGlobal->autovacFreeProcs, &proc->links);
+ dlist_push_tail(&ProcGlobal->autovacFreeProcs, &proc->links);
proc->procgloballist = &ProcGlobal->autovacFreeProcs;
}
else if (i < MaxConnections + autovacuum_max_workers + 1 + max_worker_processes)
{
/* PGPROC for bgworker, add to bgworkerFreeProcs list */
- dlist_push_head(&ProcGlobal->bgworkerFreeProcs, &proc->links);
+ dlist_push_tail(&ProcGlobal->bgworkerFreeProcs, &proc->links);
proc->procgloballist = &ProcGlobal->bgworkerFreeProcs;
}
else if (i < MaxBackends)
{
/* PGPROC for walsender, add to walsenderFreeProcs list */
- dlist_push_head(&ProcGlobal->walsenderFreeProcs, &proc->links);
+ dlist_push_tail(&ProcGlobal->walsenderFreeProcs, &proc->links);
proc->procgloballist = &ProcGlobal->walsenderFreeProcs;
}
@@ -351,6 +351,7 @@ InitProcess(void)
(errcode(ERRCODE_TOO_MANY_CONNECTIONS),
errmsg("sorry, too many clients already")));
}
+ MyBackendId = GetBackendIdFromPGProc(MyProc);
/*
* Cross-check that the PGPROC is of the type we expect; if this were not
@@ -373,14 +374,14 @@ InitProcess(void)
*/
dlist_node_init(&MyProc->links);
MyProc->waitStatus = PROC_WAIT_STATUS_OK;
- MyProc->lxid = InvalidLocalTransactionId;
MyProc->fpVXIDLock = false;
MyProc->fpLocalTransactionId = InvalidLocalTransactionId;
MyProc->xid = InvalidTransactionId;
MyProc->xmin = InvalidTransactionId;
MyProc->pid = MyProcPid;
- /* backendId, databaseId and roleId will be filled in later */
- MyProc->backendId = InvalidBackendId;
+ MyProc->vxid.backendId = MyBackendId;
+ MyProc->vxid.lxid = InvalidLocalTransactionId;
+ /* databaseId and roleId will be filled in later */
MyProc->databaseId = InvalidOid;
MyProc->roleId = InvalidOid;
MyProc->tempNamespaceId = InvalidOid;
@@ -560,22 +561,23 @@ InitAuxiliaryProcess(void)
/* use volatile pointer to prevent code rearrangement */
((volatile PGPROC *) auxproc)->pid = MyProcPid;
- MyProc = auxproc;
-
SpinLockRelease(ProcStructLock);
+ MyProc = auxproc;
+ MyBackendId = GetBackendIdFromPGProc(MyProc);
+
/*
* Initialize all fields of MyProc, except for those previously
* initialized by InitProcGlobal.
*/
dlist_node_init(&MyProc->links);
MyProc->waitStatus = PROC_WAIT_STATUS_OK;
- MyProc->lxid = InvalidLocalTransactionId;
MyProc->fpVXIDLock = false;
MyProc->fpLocalTransactionId = InvalidLocalTransactionId;
MyProc->xid = InvalidTransactionId;
MyProc->xmin = InvalidTransactionId;
- MyProc->backendId = InvalidBackendId;
+ MyProc->vxid.backendId = InvalidBackendId;
+ MyProc->vxid.lxid = InvalidLocalTransactionId;
MyProc->databaseId = InvalidOid;
MyProc->roleId = InvalidOid;
MyProc->tempNamespaceId = InvalidOid;
@@ -905,8 +907,14 @@ ProcKill(int code, Datum arg)
proc = MyProc;
MyProc = NULL;
+ MyBackendId = InvalidBackendId;
DisownLatch(&proc->procLatch);
+ /* Mark the proc no longer in use */
+ proc->pid = 0;
+ proc->vxid.backendId = InvalidBackendId;
+ proc->vxid.lxid = InvalidTransactionId;
+
procgloballist = proc->procgloballist;
SpinLockAcquire(ProcStructLock);
@@ -976,6 +984,7 @@ AuxiliaryProcKill(int code, Datum arg)
proc = MyProc;
MyProc = NULL;
+ MyBackendId = InvalidBackendId;
DisownLatch(&proc->procLatch);
SpinLockAcquire(ProcStructLock);
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 1a1050c8da1..3d3f7b06723 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -19,6 +19,7 @@
#include "port/atomics.h" /* for memory barriers */
#include "storage/ipc.h"
#include "storage/proc.h" /* for MyProc */
+#include "storage/procarray.h"
#include "storage/sinvaladt.h"
#include "utils/ascii.h"
#include "utils/backend_status.h"
@@ -29,13 +30,12 @@
/* ----------
* Total number of backends including auxiliary
*
- * We reserve a slot for each possible BackendId, plus one for each
- * possible auxiliary process type. (This scheme assumes there is not
- * more than one of any auxiliary process type at a time.) MaxBackends
- * includes autovacuum workers and background workers as well.
+ * We reserve a slot for each possible PGPROC entry, including aux processes.
+ * (But not including PGPROC entries reserved for prepared xacts; they are not
+ * real processes.)
* ----------
*/
-#define NumBackendStatSlots (MaxBackends + NUM_AUXPROCTYPES)
+#define NumBackendStatSlots (MaxBackends + NUM_AUXILIARY_PROCS)
/* ----------
@@ -238,10 +238,9 @@ CreateSharedBackendStatus(void)
/*
* Initialize pgstats backend activity state, and set up our on-proc-exit
- * hook. Called from InitPostgres and AuxiliaryProcessMain. For auxiliary
- * process, MyBackendId is invalid. Otherwise, MyBackendId must be set, but we
- * must not have started any transaction yet (since the exit hook must run
- * after the last transaction exit).
+ * hook. Called from InitPostgres and AuxiliaryProcessMain. MyBackendId must
+ * be set, but we must not have started any transaction yet (since the exit
+ * hook must run after the last transaction exit).
*
* NOTE: MyDatabaseId isn't set yet; so the shutdown hook has to be careful.
*/
@@ -249,26 +248,9 @@ void
pgstat_beinit(void)
{
/* Initialize MyBEEntry */
- if (MyBackendId != InvalidBackendId)
- {
- Assert(MyBackendId >= 1 && MyBackendId <= MaxBackends);
- MyBEEntry = &BackendStatusArray[MyBackendId - 1];
- }
- else
- {
- /* Must be an auxiliary process */
- Assert(MyAuxProcType != NotAnAuxProcess);
-
- /*
- * Assign the MyBEEntry for an auxiliary process. Since it doesn't
- * have a BackendId, the slot is statically allocated based on the
- * auxiliary process type (MyAuxProcType). Backends use slots indexed
- * in the range from 0 to MaxBackends (exclusive), so we use
- * MaxBackends + AuxProcType as the index of the slot for an auxiliary
- * process.
- */
- MyBEEntry = &BackendStatusArray[MaxBackends + MyAuxProcType];
- }
+ Assert(MyBackendId != InvalidBackendId);
+ Assert(MyBackendId >= 1 && MyBackendId <= NumBackendStatSlots);
+ MyBEEntry = &BackendStatusArray[MyBackendId - 1];
/* Set up a process-exit hook to clean up */
on_shmem_exit(pgstat_beshutdown_hook, 0);
@@ -281,12 +263,12 @@ pgstat_beinit(void)
* Initialize this backend's entry in the PgBackendStatus array.
* Called from InitPostgres.
*
- * Apart from auxiliary processes, MyBackendId, MyDatabaseId,
- * session userid, and application_name must be set for a
- * backend (hence, this cannot be combined with pgstat_beinit).
- * Note also that we must be inside a transaction if this isn't an aux
- * process, as we may need to do encoding conversion on some strings.
- * ----------
+ * Apart from auxiliary processes, MyDatabaseId, session userid, and
+ * application_name must already be set (hence, this cannot be combined
+ * with pgstat_beinit). Note also that we must be inside a transaction
+ * if this isn't an aux process, as we may need to do encoding conversion
+ * on some strings.
+ *----------
*/
void
pgstat_bestart(void)
diff --git a/src/backend/utils/adt/lockfuncs.c b/src/backend/utils/adt/lockfuncs.c
index 4b49f7fe3d8..bbe5cc0806e 100644
--- a/src/backend/utils/adt/lockfuncs.c
+++ b/src/backend/utils/adt/lockfuncs.c
@@ -353,7 +353,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
break;
}
- values[10] = VXIDGetDatum(instance->backend, instance->lxid);
+ values[10] = VXIDGetDatum(instance->vxid.backendId, instance->vxid.localTransactionId);
if (instance->pid != 0)
values[11] = Int32GetDatum(instance->pid);
else
diff --git a/src/backend/utils/adt/mcxtfuncs.c b/src/backend/utils/adt/mcxtfuncs.c
index 4708d73f5fa..a7267dc15d1 100644
--- a/src/backend/utils/adt/mcxtfuncs.c
+++ b/src/backend/utils/adt/mcxtfuncs.c
@@ -148,19 +148,11 @@ pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
PGPROC *proc;
BackendId backendId = InvalidBackendId;
- proc = BackendPidGetProc(pid);
-
/*
* See if the process with given pid is a backend or an auxiliary process.
- *
- * If the given process is a backend, use its backend id in
- * SendProcSignal() later to speed up the operation. Otherwise, don't do
- * that because auxiliary processes (except the startup process) don't
- * have a valid backend id.
*/
- if (proc != NULL)
- backendId = proc->backendId;
- else
+ proc = BackendPidGetProc(pid);
+ if (proc == NULL)
proc = AuxiliaryPidGetProc(pid);
/*
@@ -183,6 +175,8 @@ pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(false);
}
+ if (proc != NULL)
+ backendId = GetBackendIdFromPGProc(proc);
if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, backendId) < 0)
{
/* Again, just a warning to allow loops */
diff --git a/src/backend/utils/error/csvlog.c b/src/backend/utils/error/csvlog.c
index 1b62b07f231..46d5b621301 100644
--- a/src/backend/utils/error/csvlog.c
+++ b/src/backend/utils/error/csvlog.c
@@ -152,8 +152,8 @@ write_csvlog(ErrorData *edata)
/* Virtual transaction id */
/* keep VXID format in sync with lockfuncs.c */
- if (MyProc != NULL && MyProc->backendId != InvalidBackendId)
- appendStringInfo(&buf, "%d/%u", MyProc->backendId, MyProc->lxid);
+ if (MyProc != NULL)
+ appendStringInfo(&buf, "%d/%u", MyProc->vxid.backendId, MyProc->vxid.lxid);
appendStringInfoChar(&buf, ',');
/* Transaction id */
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 2c7a20e3d31..4e93d93549e 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -3074,18 +3074,18 @@ log_status_format(StringInfo buf, const char *format, ErrorData *edata)
break;
case 'v':
/* keep VXID format in sync with lockfuncs.c */
- if (MyProc != NULL && MyProc->backendId != InvalidBackendId)
+ if (MyProc != NULL)
{
if (padding != 0)
{
char strfbuf[128];
snprintf(strfbuf, sizeof(strfbuf) - 1, "%d/%u",
- MyProc->backendId, MyProc->lxid);
+ MyProc->vxid.backendId, MyProc->vxid.lxid);
appendStringInfo(buf, "%*s", padding, strfbuf);
}
else
- appendStringInfo(buf, "%d/%u", MyProc->backendId, MyProc->lxid);
+ appendStringInfo(buf, "%d/%u", MyProc->vxid.backendId, MyProc->vxid.lxid);
}
else if (padding != 0)
appendStringInfoSpaces(buf,
diff --git a/src/backend/utils/error/jsonlog.c b/src/backend/utils/error/jsonlog.c
index 2903561f1c4..067d9e30b16 100644
--- a/src/backend/utils/error/jsonlog.c
+++ b/src/backend/utils/error/jsonlog.c
@@ -197,9 +197,9 @@ write_jsonlog(ErrorData *edata)
/* Virtual transaction id */
/* keep VXID format in sync with lockfuncs.c */
- if (MyProc != NULL && MyProc->backendId != InvalidBackendId)
- appendJSONKeyValueFmt(&buf, "vxid", true, "%d/%u", MyProc->backendId,
- MyProc->lxid);
+ if (MyProc != NULL && MyProc->vxid.backendId != InvalidBackendId)
+ appendJSONKeyValueFmt(&buf, "vxid", true, "%d/%u",
+ MyProc->vxid.backendId, MyProc->vxid.lxid);
/* Transaction id */
appendJSONKeyValueFmt(&buf, "txid", false, "%u",
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index 1ad33671598..e822cd61d5f 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -740,18 +740,10 @@ InitPostgres(const char *in_dbname, Oid dboid,
/*
* Initialize my entry in the shared-invalidation manager's array of
* per-backend data.
- *
- * Sets up MyBackendId, a unique backend identifier.
*/
- MyBackendId = InvalidBackendId;
-
SharedInvalBackendInit(false);
- if (MyBackendId > MaxBackends || MyBackendId <= 0)
- elog(FATAL, "bad backend ID: %d", MyBackendId);
-
- /* Now that we have a BackendId, we can participate in ProcSignal */
- ProcSignalInit(MyBackendId);
+ ProcSignalInit();
/*
* Also set up timeout handlers needed for backend operation. We need
diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
index 675e81d82d7..a0916959b17 100644
--- a/src/backend/utils/time/snapmgr.c
+++ b/src/backend/utils/time/snapmgr.c
@@ -1154,7 +1154,8 @@ ExportSnapshot(Snapshot snapshot)
* inside the transaction from 1.
*/
snprintf(path, sizeof(path), SNAPSHOT_EXPORT_DIR "/%08X-%08X-%d",
- MyProc->backendId, MyProc->lxid, list_length(exportedSnapshots) + 1);
+ MyProc->vxid.backendId, MyProc->vxid.lxid,
+ list_length(exportedSnapshots) + 1);
/*
* Copy the snapshot into TopTransactionContext, add it to the
@@ -1181,7 +1182,7 @@ ExportSnapshot(Snapshot snapshot)
*/
initStringInfo(&buf);
- appendStringInfo(&buf, "vxid:%d/%u\n", MyProc->backendId, MyProc->lxid);
+ appendStringInfo(&buf, "vxid:%d/%u\n", MyProc->vxid.backendId, MyProc->vxid.lxid);
appendStringInfo(&buf, "pid:%d\n", MyProcPid);
appendStringInfo(&buf, "dbid:%u\n", MyDatabaseId);
appendStringInfo(&buf, "iso:%d\n", XactIsoLevel);
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 0b01c1f0935..cbdc61b8576 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -444,8 +444,6 @@ typedef enum
WalWriterProcess,
WalReceiverProcess,
WalSummarizerProcess,
-
- NUM_AUXPROCTYPES /* Must be last! */
} AuxProcType;
extern PGDLLIMPORT AuxProcType MyAuxProcType;
diff --git a/src/include/storage/backendid.h b/src/include/storage/backendid.h
index 50ac982da19..01387723f79 100644
--- a/src/include/storage/backendid.h
+++ b/src/include/storage/backendid.h
@@ -14,11 +14,15 @@
#ifndef BACKENDID_H
#define BACKENDID_H
-/* ----------------
- * -cim 8/17/90
- * ----------------
+/*
+ * BackendId uniquely identifies an active backend or auxiliary process. It's
+ * assigned at backend startup after authentication. Note that a backend ID
+ * can be reused for a different backend immediately after a backend exits.
+ *
+ * Backend IDs are assigned starting from 1. For historical reasons, BackendId
+ * 0 is unused, but InvalidBackendId is defined as -1.
*/
-typedef int BackendId; /* unique currently active backend identifier */
+typedef int BackendId;
#define InvalidBackendId (-1)
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index ed6071f3286..c5c84201378 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -74,9 +74,9 @@ typedef struct
#define SetInvalidVirtualTransactionId(vxid) \
((vxid).backendId = InvalidBackendId, \
(vxid).localTransactionId = InvalidLocalTransactionId)
-#define GET_VXID_FROM_PGPROC(vxid, proc) \
- ((vxid).backendId = (proc).backendId, \
- (vxid).localTransactionId = (proc).lxid)
+#define GET_VXID_FROM_PGPROC(vxid_dst, proc) \
+ ((vxid_dst).backendId = (proc).vxid.backendId, \
+ (vxid_dst).localTransactionId = (proc).vxid.lxid)
/* MAX_LOCKMODES cannot be larger than the # of bits in LOCKMASK */
#define MAX_LOCKMODES 10
@@ -454,8 +454,7 @@ typedef struct LockInstanceData
LOCKTAG locktag; /* tag for locked object */
LOCKMASK holdMask; /* locks held by this PGPROC */
LOCKMODE waitLockMode; /* lock awaited by this PGPROC, if any */
- BackendId backend; /* backend ID of this PGPROC */
- LocalTransactionId lxid; /* local transaction ID of this PGPROC */
+ VirtualTransactionId vxid; /* local transaction ID of this PGPROC */
TimestampTz waitStart; /* time at which this PGPROC started waiting
* for lock */
int pid; /* pid of this PGPROC */
diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h
index 37cf8b4067d..b550e07bbce 100644
--- a/src/include/storage/proc.h
+++ b/src/include/storage/proc.h
@@ -186,16 +186,31 @@ struct PGPROC
* vacuum must not remove tuples deleted by
* xid >= xmin ! */
- LocalTransactionId lxid; /* local id of top-level transaction currently
- * being executed by this proc, if running;
- * else InvalidLocalTransactionId */
int pid; /* Backend's process ID; 0 if prepared xact */
int pgxactoff; /* offset into various ProcGlobal->arrays with
* data mirrored from this PGPROC */
+ /*
+ * Currently running top-level transaction's virtual xid. Together these
+ * form a VirtualTransactionId, but we don't use that struct because this
+ * is not atomically assignable as whole, and we want to enforce code to
+ * consider both parts separately. See comments at VirtualTransactionId.
+ */
+ struct {
+ BackendId backendId; /* For regular backends, equal to
+ * GetBackendIdFromPGProc(proc). For
+ * prepared xacts, ID of the original
+ * backend that processed the
+ * transaction. For unused PGPROC entries,
+ * InvalidbackendID. */
+ LocalTransactionId lxid; /* local id of top-level transaction
+ * currently * being executed by this
+ * proc, if running; else
+ * InvalidLocaltransactionId */
+ } vxid;
+
/* These fields are zero while a backend is still starting up: */
- BackendId backendId; /* This backend's backend ID (if assigned) */
Oid databaseId; /* OID of database this backend is using */
Oid roleId; /* OID of role using this backend */
@@ -405,9 +420,16 @@ extern PGDLLIMPORT PROC_HDR *ProcGlobal;
extern PGDLLIMPORT PGPROC *PreparedXactProcs;
-/* Accessor for PGPROC given a pgprocno. */
+/*
+ * Accessors for getting PGPROC given a pgprocno or BackendId, and vice versa.
+ *
+ * For historical reasons, some code uses 0-based "proc numbers", while other
+ * code uses 1-based backend IDs.
+ */
#define GetPGProcByNumber(n) (&ProcGlobal->allProcs[(n)])
#define GetNumberFromPGProc(proc) ((proc) - &ProcGlobal->allProcs[0])
+#define GetPGProcByBackendId(n) (&ProcGlobal->allProcs[(n) - 1])
+#define GetBackendIdFromPGProc(proc) (GetNumberFromPGProc(proc) + 1)
/*
* We set aside some extra PGPROC structures for auxiliary processes,
diff --git a/src/include/storage/procarray.h b/src/include/storage/procarray.h
index f3eba9b7640..3af7577e8c6 100644
--- a/src/include/storage/procarray.h
+++ b/src/include/storage/procarray.h
@@ -64,6 +64,10 @@ extern VirtualTransactionId *GetVirtualXIDsDelayingChkpt(int *nvxids, int type);
extern bool HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids,
int nvxids, int type);
+extern PGPROC *BackendIdGetProc(int backendID);
+extern void BackendIdGetTransactionIds(int backendID, TransactionId *xid,
+ TransactionId *xmin, int *nsubxid,
+ bool *overflowed);
extern PGPROC *BackendPidGetProc(int pid);
extern PGPROC *BackendPidGetProcWithLock(int pid);
extern int BackendXidGetPid(TransactionId xid);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 52dcb4c2adf..febdda3611c 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -62,7 +62,7 @@ typedef enum
extern Size ProcSignalShmemSize(void);
extern void ProcSignalShmemInit(void);
-extern void ProcSignalInit(int pss_idx);
+extern void ProcSignalInit(void);
extern int SendProcSignal(pid_t pid, ProcSignalReason reason,
BackendId backendId);
diff --git a/src/include/storage/sinvaladt.h b/src/include/storage/sinvaladt.h
index aa3d203efca..c3c97b3f8b7 100644
--- a/src/include/storage/sinvaladt.h
+++ b/src/include/storage/sinvaladt.h
@@ -31,10 +31,6 @@
extern Size SInvalShmemSize(void);
extern void CreateSharedInvalidationState(void);
extern void SharedInvalBackendInit(bool sendOnly);
-extern PGPROC *BackendIdGetProc(int backendID);
-extern void BackendIdGetTransactionIds(int backendID, TransactionId *xid,
- TransactionId *xmin, int *nsubxid,
- bool *overflowed);
extern void SIInsertDataEntries(const SharedInvalidationMessage *data, int n);
extern int SIGetDataEntries(SharedInvalidationMessage *data, int datasize);
diff --git a/src/pl/plpgsql/src/pl_exec.c b/src/pl/plpgsql/src/pl_exec.c
index 6d1691340c5..ed51694428a 100644
--- a/src/pl/plpgsql/src/pl_exec.c
+++ b/src/pl/plpgsql/src/pl_exec.c
@@ -2211,7 +2211,7 @@ exec_stmt_call(PLpgSQL_execstate *estate, PLpgSQL_stmt_call *stmt)
paramLI = setup_param_list(estate, expr);
- before_lxid = MyProc->lxid;
+ before_lxid = MyProc->vxid.lxid;
/*
* If we have a procedure-lifespan resowner, use that to hold the refcount
@@ -2232,7 +2232,7 @@ exec_stmt_call(PLpgSQL_execstate *estate, PLpgSQL_stmt_call *stmt)
elog(ERROR, "SPI_execute_plan_extended failed executing query \"%s\": %s",
expr->query, SPI_result_code_string(rc));
- after_lxid = MyProc->lxid;
+ after_lxid = MyProc->vxid.lxid;
if (before_lxid != after_lxid)
{
@@ -6037,7 +6037,7 @@ exec_eval_simple_expr(PLpgSQL_execstate *estate,
int32 *rettypmod)
{
ExprContext *econtext = estate->eval_econtext;
- LocalTransactionId curlxid = MyProc->lxid;
+ LocalTransactionId curlxid = MyProc->vxid.lxid;
ParamListInfo paramLI;
void *save_setup_arg;
bool need_snapshot;
@@ -7943,7 +7943,7 @@ get_cast_hashentry(PLpgSQL_execstate *estate,
* functions do; DO blocks have private simple_eval_estates, and private
* cast hash tables to go with them.)
*/
- curlxid = MyProc->lxid;
+ curlxid = MyProc->vxid.lxid;
if (cast_entry->cast_lxid != curlxid || cast_entry->cast_in_use)
{
oldcontext = MemoryContextSwitchTo(estate->simple_eval_estate->es_query_cxt);
@@ -8070,7 +8070,7 @@ exec_simple_check_plan(PLpgSQL_execstate *estate, PLpgSQL_expr *expr)
/* Remember that we have the refcount */
expr->expr_simple_plansource = plansource;
expr->expr_simple_plan = cplan;
- expr->expr_simple_plan_lxid = MyProc->lxid;
+ expr->expr_simple_plan_lxid = MyProc->vxid.lxid;
/* Share the remaining work with the replan code path */
exec_save_simple_expr(expr, cplan);
--
2.39.2
v9-0003-Remove-MyAuxProcType-use-MyBackendType-instead.patchtext/x-patch; charset=UTF-8; name=v9-0003-Remove-MyAuxProcType-use-MyBackendType-instead.patchDownload
From 27467c98c7986c5ef674779290bbf911d138018b Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Mon, 29 Jan 2024 20:58:52 +0200
Subject: [PATCH v9 3/4] Remove MyAuxProcType, use MyBackendType instead
MyAuxProcType was redundant with MyBackendType.
Reviewed-by: Reid Thompson, Andres Freund
Discussion: https://www.postgresql.org/message-id/f3ecd4cb-85ee-4e54-8278-5fabfb3a4ed0@iki.fi
---
src/backend/postmaster/auxprocess.c | 58 +++++-------------------
src/backend/postmaster/postmaster.c | 36 +++++++--------
src/include/miscadmin.h | 69 ++++++++++++++---------------
src/include/postmaster/auxprocess.h | 2 +-
src/tools/pgindent/typedefs.list | 1 -
5 files changed, 63 insertions(+), 103 deletions(-)
diff --git a/src/backend/postmaster/auxprocess.c b/src/backend/postmaster/auxprocess.c
index 39171fea06b..fc13cd76321 100644
--- a/src/backend/postmaster/auxprocess.c
+++ b/src/backend/postmaster/auxprocess.c
@@ -38,14 +38,6 @@
static void ShutdownAuxiliaryProcess(int code, Datum arg);
-/* ----------------
- * global variables
- * ----------------
- */
-
-AuxProcType MyAuxProcType = NotAnAuxProcess; /* declared in miscadmin.h */
-
-
/*
* AuxiliaryProcessMain
*
@@ -55,39 +47,11 @@ AuxProcType MyAuxProcType = NotAnAuxProcess; /* declared in miscadmin.h */
* This code is here just because of historical reasons.
*/
void
-AuxiliaryProcessMain(AuxProcType auxtype)
+AuxiliaryProcessMain(BackendType auxtype)
{
Assert(IsUnderPostmaster);
- MyAuxProcType = auxtype;
-
- switch (MyAuxProcType)
- {
- case StartupProcess:
- MyBackendType = B_STARTUP;
- break;
- case ArchiverProcess:
- MyBackendType = B_ARCHIVER;
- break;
- case BgWriterProcess:
- MyBackendType = B_BG_WRITER;
- break;
- case CheckpointerProcess:
- MyBackendType = B_CHECKPOINTER;
- break;
- case WalWriterProcess:
- MyBackendType = B_WAL_WRITER;
- break;
- case WalReceiverProcess:
- MyBackendType = B_WAL_RECEIVER;
- break;
- case WalSummarizerProcess:
- MyBackendType = B_WAL_SUMMARIZER;
- break;
- default:
- elog(PANIC, "unrecognized process type: %d", (int) MyAuxProcType);
- MyBackendType = B_INVALID;
- }
+ MyBackendType = auxtype;
init_ps_display(NULL);
@@ -126,38 +90,38 @@ AuxiliaryProcessMain(AuxProcType auxtype)
SetProcessingMode(NormalProcessing);
- switch (MyAuxProcType)
+ switch (MyBackendType)
{
- case StartupProcess:
+ case B_STARTUP:
StartupProcessMain();
proc_exit(1);
- case ArchiverProcess:
+ case B_ARCHIVER:
PgArchiverMain();
proc_exit(1);
- case BgWriterProcess:
+ case B_BG_WRITER:
BackgroundWriterMain();
proc_exit(1);
- case CheckpointerProcess:
+ case B_CHECKPOINTER:
CheckpointerMain();
proc_exit(1);
- case WalWriterProcess:
+ case B_WAL_WRITER:
WalWriterMain();
proc_exit(1);
- case WalReceiverProcess:
+ case B_WAL_RECEIVER:
WalReceiverMain();
proc_exit(1);
- case WalSummarizerProcess:
+ case B_WAL_SUMMARIZER:
WalSummarizerMain();
proc_exit(1);
default:
- elog(PANIC, "unrecognized process type: %d", (int) MyAuxProcType);
+ elog(PANIC, "unrecognized process type: %d", (int) MyBackendType);
proc_exit(1);
}
}
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index feb471dd1df..c51b6a1376a 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -440,7 +440,7 @@ static int CountChildren(int target);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
-static pid_t StartChildProcess(AuxProcType type);
+static pid_t StartChildProcess(BackendType type);
static void StartAutovacuumWorker(void);
static void MaybeStartWalReceiver(void);
static void MaybeStartWalSummarizer(void);
@@ -561,13 +561,13 @@ static void ShmemBackendArrayAdd(Backend *bn);
static void ShmemBackendArrayRemove(Backend *bn);
#endif /* EXEC_BACKEND */
-#define StartupDataBase() StartChildProcess(StartupProcess)
-#define StartArchiver() StartChildProcess(ArchiverProcess)
-#define StartBackgroundWriter() StartChildProcess(BgWriterProcess)
-#define StartCheckpointer() StartChildProcess(CheckpointerProcess)
-#define StartWalWriter() StartChildProcess(WalWriterProcess)
-#define StartWalReceiver() StartChildProcess(WalReceiverProcess)
-#define StartWalSummarizer() StartChildProcess(WalSummarizerProcess)
+#define StartupDataBase() StartChildProcess(B_STARTUP)
+#define StartArchiver() StartChildProcess(B_ARCHIVER)
+#define StartBackgroundWriter() StartChildProcess(B_BG_WRITER)
+#define StartCheckpointer() StartChildProcess(B_CHECKPOINTER)
+#define StartWalWriter() StartChildProcess(B_WAL_WRITER)
+#define StartWalReceiver() StartChildProcess(B_WAL_RECEIVER)
+#define StartWalSummarizer() StartChildProcess(B_WAL_SUMMARIZER)
/* Macros to check exit status of a child process */
#define EXIT_STATUS_0(st) ((st) == 0)
@@ -4953,7 +4953,7 @@ SubPostmasterMain(int argc, char *argv[])
}
if (strcmp(argv[1], "--forkaux") == 0)
{
- AuxProcType auxtype;
+ BackendType auxtype;
Assert(argc == 4);
@@ -5292,7 +5292,7 @@ CountChildren(int target)
* to start subprocess.
*/
static pid_t
-StartChildProcess(AuxProcType type)
+StartChildProcess(BackendType type)
{
pid_t pid;
@@ -5344,31 +5344,31 @@ StartChildProcess(AuxProcType type)
errno = save_errno;
switch (type)
{
- case StartupProcess:
+ case B_STARTUP:
ereport(LOG,
(errmsg("could not fork startup process: %m")));
break;
- case ArchiverProcess:
+ case B_ARCHIVER:
ereport(LOG,
(errmsg("could not fork archiver process: %m")));
break;
- case BgWriterProcess:
+ case B_BG_WRITER:
ereport(LOG,
(errmsg("could not fork background writer process: %m")));
break;
- case CheckpointerProcess:
+ case B_CHECKPOINTER:
ereport(LOG,
(errmsg("could not fork checkpointer process: %m")));
break;
- case WalWriterProcess:
+ case B_WAL_WRITER:
ereport(LOG,
(errmsg("could not fork WAL writer process: %m")));
break;
- case WalReceiverProcess:
+ case B_WAL_RECEIVER:
ereport(LOG,
(errmsg("could not fork WAL receiver process: %m")));
break;
- case WalSummarizerProcess:
+ case B_WAL_SUMMARIZER:
ereport(LOG,
(errmsg("could not fork WAL summarizer process: %m")));
break;
@@ -5382,7 +5382,7 @@ StartChildProcess(AuxProcType type)
* fork failure is fatal during startup, but there's no need to choke
* immediately if starting other child types fails.
*/
- if (type == StartupProcess)
+ if (type == B_STARTUP)
ExitPostmaster(1);
return 0;
}
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index cbdc61b8576..4ecae4f834e 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -321,29 +321,57 @@ extern void InitProcessLocalLatch(void);
extern void SwitchToSharedLatch(void);
extern void SwitchBackToLocalLatch(void);
+/*
+ * MyBackendType indicates what kind of a backend this is.
+ */
typedef enum BackendType
{
B_INVALID = 0,
- B_ARCHIVER,
+
+ /* Backends and other backend-like processes */
+ B_BACKEND,
B_AUTOVAC_LAUNCHER,
B_AUTOVAC_WORKER,
- B_BACKEND,
B_BG_WORKER,
+ B_WAL_SENDER,
+
+ B_STANDALONE_BACKEND,
+
+ /*
+ * Auxiliary processes. These have PGPROC entries, but they are not
+ * attached to any particular database. There can be only one of each of
+ * these running at a time.
+ *
+ * If you modify these, make sure to update NUM_AUXILIARY_PROCS and the
+ * glossary in the docs.
+ */
+ B_ARCHIVER,
B_BG_WRITER,
B_CHECKPOINTER,
- B_LOGGER,
- B_STANDALONE_BACKEND,
B_STARTUP,
B_WAL_RECEIVER,
- B_WAL_SENDER,
B_WAL_SUMMARIZER,
B_WAL_WRITER,
+
+ /*
+ * Logger is not connected to shared memory and does not have a PGPROC
+ * entry.
+ */
+ B_LOGGER,
} BackendType;
#define BACKEND_NUM_TYPES (B_WAL_WRITER + 1)
extern PGDLLIMPORT BackendType MyBackendType;
+#define AmStartupProcess() (MyBackendType == B_STARTUP)
+#define AmBackgroundWriterProcess() (MyBackendType == B_BG_WRITER)
+#define AmArchiverProcess() (MyBackendType == B_ARCHIVER)
+#define AmCheckpointerProcess() (MyBackendType == B_CHECKPOINTER)
+#define AmWalWriterProcess() (MyBackendType == B_WAL_WRITER)
+#define AmWalReceiverProcess() (MyBackendType == B_WAL_RECEIVER)
+#define AmWalSummarizerProcess() (MyBackendType == B_WAL_SUMMARIZER)
+
extern const char *GetBackendTypeDesc(BackendType backendType);
extern void SetDatabasePath(const char *path);
@@ -426,37 +454,6 @@ extern PGDLLIMPORT ProcessingMode Mode;
} while(0)
-/*
- * Auxiliary-process type identifiers. These used to be in bootstrap.h
- * but it seems saner to have them here, with the ProcessingMode stuff.
- * The MyAuxProcType global is defined and set in auxprocess.c.
- *
- * Make sure to list in the glossary any items you add here.
- */
-
-typedef enum
-{
- NotAnAuxProcess = -1,
- StartupProcess = 0,
- BgWriterProcess,
- ArchiverProcess,
- CheckpointerProcess,
- WalWriterProcess,
- WalReceiverProcess,
- WalSummarizerProcess,
-} AuxProcType;
-
-extern PGDLLIMPORT AuxProcType MyAuxProcType;
-
-#define AmStartupProcess() (MyAuxProcType == StartupProcess)
-#define AmBackgroundWriterProcess() (MyAuxProcType == BgWriterProcess)
-#define AmArchiverProcess() (MyAuxProcType == ArchiverProcess)
-#define AmCheckpointerProcess() (MyAuxProcType == CheckpointerProcess)
-#define AmWalWriterProcess() (MyAuxProcType == WalWriterProcess)
-#define AmWalReceiverProcess() (MyAuxProcType == WalReceiverProcess)
-#define AmWalSummarizerProcess() (MyAuxProcType == WalSummarizerProcess)
-
-
/*****************************************************************************
* pinit.h -- *
* POSTGRES initialization and cleanup definitions. *
diff --git a/src/include/postmaster/auxprocess.h b/src/include/postmaster/auxprocess.h
index 1fdde3bb77b..3e443edde70 100644
--- a/src/include/postmaster/auxprocess.h
+++ b/src/include/postmaster/auxprocess.h
@@ -15,6 +15,6 @@
#include "miscadmin.h"
-extern void AuxiliaryProcessMain(AuxProcType auxtype) pg_attribute_noreturn();
+extern void AuxiliaryProcessMain(BackendType auxtype) pg_attribute_noreturn();
#endif /* AUXPROCESS_H */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 91433d439b7..5b0d80f0daf 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -173,7 +173,6 @@ AutoVacOpts
AutoVacuumShmemStruct
AutoVacuumWorkItem
AutoVacuumWorkItemType
-AuxProcType
BF_ctx
BF_key
BF_word
--
2.39.2
v9-0004-Use-MyBackendType-in-more-places-to-check-what-pr.patchtext/x-patch; charset=UTF-8; name=v9-0004-Use-MyBackendType-in-more-places-to-check-what-pr.patchDownload
From ef39b4f427979a9988e255f95f896513f4d9e3a5 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 10 Jan 2024 12:59:48 +0200
Subject: [PATCH v9 4/4] Use MyBackendType in more places to check what process
this is
Remove IsBackgroundWorker, IsAutoVacuumLauncherProcess() and
IsAutoVacuumWorkerProcess() in favor of new Am*Process() macros that
use MyBackendType. For consistency with the existing Am*Process()
macros.
Reviewed-by: XXX
Discussion: XXX
---
src/backend/access/gin/ginfast.c | 2 +-
src/backend/access/gin/ginvacuum.c | 6 ++---
src/backend/access/heap/vacuumlazy.c | 4 +--
src/backend/commands/analyze.c | 4 +--
src/backend/commands/vacuum.c | 8 +++---
src/backend/postmaster/autovacuum.c | 26 --------------------
src/backend/postmaster/bgworker.c | 2 --
src/backend/postmaster/postmaster.c | 3 ---
src/backend/statistics/extended_stats.c | 2 +-
src/backend/storage/ipc/ipc.c | 2 +-
src/backend/storage/lmgr/proc.c | 19 +++++++-------
src/backend/tcop/postgres.c | 6 ++---
src/backend/utils/activity/pgstat_relation.c | 4 +--
src/backend/utils/init/globals.c | 1 -
src/backend/utils/init/miscinit.c | 2 +-
src/backend/utils/init/postinit.c | 8 +++---
src/include/miscadmin.h | 5 +++-
src/include/postmaster/autovacuum.h | 5 ----
18 files changed, 37 insertions(+), 72 deletions(-)
diff --git a/src/backend/access/gin/ginfast.c b/src/backend/access/gin/ginfast.c
index cff6850ef86..e118cecb9a4 100644
--- a/src/backend/access/gin/ginfast.c
+++ b/src/backend/access/gin/ginfast.c
@@ -812,7 +812,7 @@ ginInsertCleanup(GinState *ginstate, bool full_clean,
*/
LockPage(index, GIN_METAPAGE_BLKNO, ExclusiveLock);
workMemory =
- (IsAutoVacuumWorkerProcess() && autovacuum_work_mem != -1) ?
+ (AmAutoVacuumWorkerProcess() && autovacuum_work_mem != -1) ?
autovacuum_work_mem : maintenance_work_mem;
}
else
diff --git a/src/backend/access/gin/ginvacuum.c b/src/backend/access/gin/ginvacuum.c
index aee6aab44ae..b3f415e2849 100644
--- a/src/backend/access/gin/ginvacuum.c
+++ b/src/backend/access/gin/ginvacuum.c
@@ -590,7 +590,7 @@ ginbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
/*
* and cleanup any pending inserts
*/
- ginInsertCleanup(&gvs.ginstate, !IsAutoVacuumWorkerProcess(),
+ ginInsertCleanup(&gvs.ginstate, !AmAutoVacuumWorkerProcess(),
false, true, stats);
}
@@ -701,7 +701,7 @@ ginvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats)
*/
if (info->analyze_only)
{
- if (IsAutoVacuumWorkerProcess())
+ if (AmAutoVacuumWorkerProcess())
{
initGinState(&ginstate, index);
ginInsertCleanup(&ginstate, false, true, true, stats);
@@ -717,7 +717,7 @@ ginvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats)
{
stats = (IndexBulkDeleteResult *) palloc0(sizeof(IndexBulkDeleteResult));
initGinState(&ginstate, index);
- ginInsertCleanup(&ginstate, !IsAutoVacuumWorkerProcess(),
+ ginInsertCleanup(&ginstate, !AmAutoVacuumWorkerProcess(),
false, true, stats);
}
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index fa56480808b..5acea4a43a4 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -307,7 +307,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
char **indnames = NULL;
verbose = (params->options & VACOPT_VERBOSE) != 0;
- instrument = (verbose || (IsAutoVacuumWorkerProcess() &&
+ instrument = (verbose || (AmAutoVacuumWorkerProcess() &&
params->log_min_duration >= 0));
if (instrument)
{
@@ -3087,7 +3087,7 @@ static int
dead_items_max_items(LVRelState *vacrel)
{
int64 max_items;
- int vac_work_mem = IsAutoVacuumWorkerProcess() &&
+ int vac_work_mem = AmAutoVacuumWorkerProcess() &&
autovacuum_work_mem != -1 ?
autovacuum_work_mem : maintenance_work_mem;
diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c
index a03495d6c95..d105d2fad7b 100644
--- a/src/backend/commands/analyze.c
+++ b/src/backend/commands/analyze.c
@@ -351,7 +351,7 @@ do_analyze_rel(Relation onerel, VacuumParams *params,
save_nestlevel = NewGUCNestLevel();
/* measure elapsed time iff autovacuum logging requires it */
- if (IsAutoVacuumWorkerProcess() && params->log_min_duration >= 0)
+ if (AmAutoVacuumWorkerProcess() && params->log_min_duration >= 0)
{
if (track_io_timing)
{
@@ -729,7 +729,7 @@ do_analyze_rel(Relation onerel, VacuumParams *params,
vac_close_indexes(nindexes, Irel, NoLock);
/* Log the action if appropriate */
- if (IsAutoVacuumWorkerProcess() && params->log_min_duration >= 0)
+ if (AmAutoVacuumWorkerProcess() && params->log_min_duration >= 0)
{
TimestampTz endtime = GetCurrentTimestamp();
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 64da8486276..e40cef6566e 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -564,7 +564,7 @@ vacuum(List *relations, VacuumParams *params, BufferAccessStrategy bstrategy,
else
{
Assert(params->options & VACOPT_ANALYZE);
- if (IsAutoVacuumWorkerProcess())
+ if (AmAutoVacuumWorkerProcess())
use_own_xacts = true;
else if (in_outer_xact)
use_own_xacts = false;
@@ -809,7 +809,7 @@ vacuum_open_relation(Oid relid, RangeVar *relation, bits32 options,
* statements in the permission checks; otherwise, only log if the caller
* so requested.
*/
- if (!IsAutoVacuumWorkerProcess())
+ if (!AmAutoVacuumWorkerProcess())
elevel = WARNING;
else if (verbose)
elevel = LOG;
@@ -896,7 +896,7 @@ expand_vacuum_rel(VacuumRelation *vrel, MemoryContext vac_context,
* Since autovacuum workers supply OIDs when calling vacuum(), no
* autovacuum worker should reach this code.
*/
- Assert(!IsAutoVacuumWorkerProcess());
+ Assert(!AmAutoVacuumWorkerProcess());
/*
* We transiently take AccessShareLock to protect the syscache lookup
@@ -2336,7 +2336,7 @@ vacuum_delay_point(void)
* [autovacuum_]vacuum_cost_delay to take effect while a table is being
* vacuumed or analyzed.
*/
- if (ConfigReloadPending && IsAutoVacuumWorkerProcess())
+ if (ConfigReloadPending && AmAutoVacuumWorkerProcess())
{
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 2c3099f76f1..cecbaaa5b91 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -135,10 +135,6 @@ int Log_autovacuum_min_duration = 600000;
#define MIN_AUTOVAC_SLEEPTIME 100.0 /* milliseconds */
#define MAX_AUTOVAC_SLEEPTIME 300 /* seconds */
-/* Flags to tell if we are in an autovacuum process */
-static bool am_autovacuum_launcher = false;
-static bool am_autovacuum_worker = false;
-
/*
* Variables to save the cost-related storage parameters for the current
* relation being vacuumed by this autovacuum worker. Using these, we can
@@ -435,8 +431,6 @@ AutoVacLauncherMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
- am_autovacuum_launcher = true;
-
MyBackendType = B_AUTOVAC_LAUNCHER;
init_ps_display(NULL);
@@ -1495,8 +1489,6 @@ AutoVacWorkerMain(int argc, char *argv[])
sigjmp_buf local_sigjmp_buf;
Oid dbid;
- am_autovacuum_worker = true;
-
MyBackendType = B_AUTOVAC_WORKER;
init_ps_display(NULL);
@@ -3355,24 +3347,6 @@ autovac_init(void)
errhint("Enable the \"track_counts\" option.")));
}
-/*
- * IsAutoVacuum functions
- * Return whether this is either a launcher autovacuum process or a worker
- * process.
- */
-bool
-IsAutoVacuumLauncherProcess(void)
-{
- return am_autovacuum_launcher;
-}
-
-bool
-IsAutoVacuumWorkerProcess(void)
-{
- return am_autovacuum_worker;
-}
-
-
/*
* AutoVacuumShmemSize
* Compute space needed for autovacuum-related shared memory
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index 67f92c24db1..d8e89de7494 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -731,8 +731,6 @@ BackgroundWorkerMain(void)
if (worker == NULL)
elog(FATAL, "unable to find bgworker entry");
- IsBackgroundWorker = true;
-
MyBackendType = B_BG_WORKER;
init_ps_display(worker->bgw_name);
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index c51b6a1376a..8aa2ddf0c46 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -4979,9 +4979,6 @@ SubPostmasterMain(int argc, char *argv[])
}
if (strcmp(argv[1], "--forkbgworker") == 0)
{
- /* do this as early as possible; in particular, before InitProcess() */
- IsBackgroundWorker = true;
-
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
diff --git a/src/backend/statistics/extended_stats.c b/src/backend/statistics/extended_stats.c
index c5461514d8f..135151a2723 100644
--- a/src/backend/statistics/extended_stats.c
+++ b/src/backend/statistics/extended_stats.c
@@ -173,7 +173,7 @@ BuildRelationExtStatistics(Relation onerel, bool inh, double totalrows,
natts, vacattrstats);
if (!stats)
{
- if (!IsAutoVacuumWorkerProcess())
+ if (!AmAutoVacuumWorkerProcess())
ereport(WARNING,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("statistics object \"%s.%s\" could not be computed for relation \"%s.%s\"",
diff --git a/src/backend/storage/ipc/ipc.c b/src/backend/storage/ipc/ipc.c
index 2681e4cdff5..b06e4b84528 100644
--- a/src/backend/storage/ipc/ipc.c
+++ b/src/backend/storage/ipc/ipc.c
@@ -136,7 +136,7 @@ proc_exit(int code)
*/
char gprofDirName[32];
- if (IsAutoVacuumWorkerProcess())
+ if (AmAutoVacuumWorkerProcess())
snprintf(gprofDirName, 32, "gprof/avworker");
else
snprintf(gprofDirName, 32, "gprof/%d", (int) getpid());
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index 4c568cd6850..541a9304e00 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -41,7 +41,6 @@
#include "postmaster/autovacuum.h"
#include "replication/slot.h"
#include "replication/syncrep.h"
-#include "replication/walsender.h"
#include "storage/condition_variable.h"
#include "storage/ipc.h"
#include "storage/lmgr.h"
@@ -308,11 +307,11 @@ InitProcess(void)
elog(ERROR, "you already exist");
/* Decide which list should supply our PGPROC. */
- if (IsAnyAutoVacuumProcess())
+ if (AmAutoVacuumLauncherProcess() || AmAutoVacuumWorkerProcess())
procgloballist = &ProcGlobal->autovacFreeProcs;
- else if (IsBackgroundWorker)
+ else if (AmBackgroundWorkerProcess())
procgloballist = &ProcGlobal->bgworkerFreeProcs;
- else if (am_walsender)
+ else if (AmWalSenderProcess())
procgloballist = &ProcGlobal->walsenderFreeProcs;
else
procgloballist = &ProcGlobal->freeProcs;
@@ -342,7 +341,7 @@ InitProcess(void)
* in the autovacuum case?
*/
SpinLockRelease(ProcStructLock);
- if (am_walsender)
+ if (AmWalSenderProcess())
ereport(FATAL,
(errcode(ERRCODE_TOO_MANY_CONNECTIONS),
errmsg("number of requested standby connections exceeds max_wal_senders (currently %d)",
@@ -365,7 +364,7 @@ InitProcess(void)
* cleaning up. (XXX autovac launcher currently doesn't participate in
* this; it probably should.)
*/
- if (IsUnderPostmaster && !IsAutoVacuumLauncherProcess())
+ if (IsUnderPostmaster && !AmAutoVacuumLauncherProcess())
MarkPostmasterChildActive();
/*
@@ -385,11 +384,11 @@ InitProcess(void)
MyProc->databaseId = InvalidOid;
MyProc->roleId = InvalidOid;
MyProc->tempNamespaceId = InvalidOid;
- MyProc->isBackgroundWorker = IsBackgroundWorker;
+ MyProc->isBackgroundWorker = AmBackgroundWorkerProcess();
MyProc->delayChkptFlags = 0;
MyProc->statusFlags = 0;
/* NB -- autovac launcher intentionally does not set IS_AUTOVACUUM */
- if (IsAutoVacuumWorkerProcess())
+ if (AmAutoVacuumWorkerProcess())
MyProc->statusFlags |= PROC_IS_AUTOVACUUM;
MyProc->lwWaiting = LW_WS_NOT_WAITING;
MyProc->lwWaitMode = 0;
@@ -581,7 +580,7 @@ InitAuxiliaryProcess(void)
MyProc->databaseId = InvalidOid;
MyProc->roleId = InvalidOid;
MyProc->tempNamespaceId = InvalidOid;
- MyProc->isBackgroundWorker = IsBackgroundWorker;
+ MyProc->isBackgroundWorker = AmBackgroundWorkerProcess();
MyProc->delayChkptFlags = 0;
MyProc->statusFlags = 0;
MyProc->lwWaiting = LW_WS_NOT_WAITING;
@@ -942,7 +941,7 @@ ProcKill(int code, Datum arg)
* way, so tell the postmaster we've cleaned up acceptably well. (XXX
* autovac launcher should be included here someday)
*/
- if (IsUnderPostmaster && !IsAutoVacuumLauncherProcess())
+ if (IsUnderPostmaster && !AmAutoVacuumLauncherProcess())
MarkPostmasterChildInactive();
/* wake autovac launcher if needed -- see comments in FreeWorkerInfo */
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 1a34bd3715f..38610b723af 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -3267,7 +3267,7 @@ ProcessInterrupts(void)
ereport(FATAL,
(errcode(ERRCODE_QUERY_CANCELED),
errmsg("canceling authentication due to timeout")));
- else if (IsAutoVacuumWorkerProcess())
+ else if (AmAutoVacuumWorkerProcess())
ereport(FATAL,
(errcode(ERRCODE_ADMIN_SHUTDOWN),
errmsg("terminating autovacuum process due to administrator command")));
@@ -3286,7 +3286,7 @@ ProcessInterrupts(void)
*/
proc_exit(1);
}
- else if (IsBackgroundWorker)
+ else if (AmBackgroundWorkerProcess())
ereport(FATAL,
(errcode(ERRCODE_ADMIN_SHUTDOWN),
errmsg("terminating background worker \"%s\" due to administrator command",
@@ -3386,7 +3386,7 @@ ProcessInterrupts(void)
(errcode(ERRCODE_QUERY_CANCELED),
errmsg("canceling statement due to statement timeout")));
}
- if (IsAutoVacuumWorkerProcess())
+ if (AmAutoVacuumWorkerProcess())
{
LockErrorCleanup();
ereport(ERROR,
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 111050725a6..b20a60b5a87 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -246,7 +246,7 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
*/
tabentry->ins_since_vacuum = 0;
- if (IsAutoVacuumWorkerProcess())
+ if (AmAutoVacuumWorkerProcess())
{
tabentry->last_autovacuum_time = ts;
tabentry->autovacuum_count++;
@@ -337,7 +337,7 @@ pgstat_report_analyze(Relation rel,
if (resetcounter)
tabentry->mod_since_analyze = 0;
- if (IsAutoVacuumWorkerProcess())
+ if (AmAutoVacuumWorkerProcess())
{
tabentry->last_autoanalyze_time = GetCurrentTimestamp();
tabentry->autoanalyze_count++;
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 88b03e8fa3c..ce1fa1092f2 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -114,7 +114,6 @@ pid_t PostmasterPid = 0;
bool IsPostmasterEnvironment = false;
bool IsUnderPostmaster = false;
bool IsBinaryUpgrade = false;
-bool IsBackgroundWorker = false;
bool ExitOnAnyError = false;
diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c
index 23f77a59e58..9c91e9a1324 100644
--- a/src/backend/utils/init/miscinit.c
+++ b/src/backend/utils/init/miscinit.c
@@ -837,7 +837,7 @@ InitializeSessionUserIdStandalone(void)
* This function should only be called in single-user mode, in autovacuum
* workers, and in background workers.
*/
- Assert(!IsUnderPostmaster || IsAutoVacuumWorkerProcess() || IsBackgroundWorker);
+ Assert(!IsUnderPostmaster || AmAutoVacuumWorkerProcess() || AmBackgroundWorkerProcess());
/* call only once */
Assert(!OidIsValid(AuthenticatedUserId));
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index e822cd61d5f..6bf71b2d4e2 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -344,7 +344,7 @@ CheckMyDatabase(const char *name, bool am_superuser, bool override_allow_connect
*
* We do not enforce them for autovacuum worker processes either.
*/
- if (IsUnderPostmaster && !IsAutoVacuumWorkerProcess())
+ if (IsUnderPostmaster && !AmAutoVacuumWorkerProcess())
{
/*
* Check that the database is currently allowing connections.
@@ -825,7 +825,7 @@ InitPostgres(const char *in_dbname, Oid dboid,
before_shmem_exit(ShutdownPostgres, 0);
/* The autovacuum launcher is done here */
- if (IsAutoVacuumLauncherProcess())
+ if (AmAutoVacuumLauncherProcess())
{
/* report this backend in the PgBackendStatus array */
pgstat_bestart();
@@ -869,7 +869,7 @@ InitPostgres(const char *in_dbname, Oid dboid,
* In standalone mode and in autovacuum worker processes, we use a fixed
* ID, otherwise we figure it out from the authenticated user name.
*/
- if (bootstrap || IsAutoVacuumWorkerProcess())
+ if (bootstrap || AmAutoVacuumWorkerProcess())
{
InitializeSessionUserIdStandalone();
am_superuser = true;
@@ -885,7 +885,7 @@ InitPostgres(const char *in_dbname, Oid dboid,
errhint("You should immediately run CREATE USER \"%s\" SUPERUSER;.",
username != NULL ? username : "postgres")));
}
- else if (IsBackgroundWorker)
+ else if (AmBackgroundWorkerProcess())
{
if (username == NULL && !OidIsValid(useroid))
{
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 4ecae4f834e..631ff28cb20 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -164,7 +164,6 @@ do { \
extern PGDLLIMPORT pid_t PostmasterPid;
extern PGDLLIMPORT bool IsPostmasterEnvironment;
extern PGDLLIMPORT bool IsUnderPostmaster;
-extern PGDLLIMPORT bool IsBackgroundWorker;
extern PGDLLIMPORT bool IsBinaryUpgrade;
extern PGDLLIMPORT bool ExitOnAnyError;
@@ -364,6 +363,10 @@ typedef enum BackendType
extern PGDLLIMPORT BackendType MyBackendType;
+#define AmAutoVacuumLauncherProcess() (MyBackendType == B_AUTOVAC_LAUNCHER)
+#define AmAutoVacuumWorkerProcess() (MyBackendType == B_AUTOVAC_WORKER)
+#define AmBackgroundWorkerProcess() (MyBackendType == B_BG_WORKER)
+#define AmWalSenderProcess() (MyBackendType == B_WAL_SENDER)
#define AmStartupProcess() (MyBackendType == B_STARTUP)
#define AmBackgroundWriterProcess() (MyBackendType == B_BG_WRITER)
#define AmArchiverProcess() (MyBackendType == B_ARCHIVER)
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 1994aedef03..80cf4cdd969 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -49,11 +49,6 @@ extern PGDLLIMPORT int Log_autovacuum_min_duration;
/* Status inquiry functions */
extern bool AutoVacuumingActive(void);
-extern bool IsAutoVacuumLauncherProcess(void);
-extern bool IsAutoVacuumWorkerProcess(void);
-
-#define IsAnyAutoVacuumProcess() \
- (IsAutoVacuumLauncherProcess() || IsAutoVacuumWorkerProcess())
/* Functions to start autovacuum process, called from postmaster */
extern void autovac_init(void);
--
2.39.2
Re: Refactoring backend fork+exec code
Hi,
On 2024-01-30 02:08:36 +0200, Heikki Linnakangas wrote:
I spent some more time on the 'lastBackend' optimization in sinvaladt.c. I
realized that it became very useless with these patches, because aux
processes are allocated pgprocno's after all the slots for regular backends.
There are always aux processes running, so lastBackend would always have a
value close to the max anyway. I replaced that with a dense 'pgprocnos'
array that keeps track of the exact slots that are in use. I'm not 100% sure
this is worth the effort; does any real world workload send shared
invalidations so frequently that this matters? In any case, this should
avoid the regression if such a workload exists.New patch set attached. I think these are ready to be committed, but would
appreciate a final review.
From 54f22231bb2540fc5957c14005956161e6fc9dac Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 24 Jan 2024 23:15:55 +0200
Subject: [PATCH v8 1/5] Remove superfluous 'pgprocno' field from PGPROCIt was always just the index of the PGPROC entry from the beginning of
the proc array. Introduce a macro to compute it from the pointer
instead.
Hm. The pointer math here is bit more expensive than in some other cases, as
the struct is fairly large and sizeof(PGPROC) isn't a power of two. Adding
more math into loops like in TransactionGroupUpdateXidStatus() might end up
showing up.
I've been thinking that we likely should pad PGPROC to some more convenient
boundary, but...
Is this really related to the rest of the series?
From 4e0121e064804b73ef8a5dc10be27b85968ea1af Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Mon, 29 Jan 2024 23:50:34 +0200
Subject: [PATCH v8 2/5] Redefine backend ID to be an index into the proc
array.Previously, backend ID was an index into the ProcState array, in the
shared cache invalidation manager (sinvaladt.c). The entry in the
ProcState array was reserved at backend startup by scanning the array
for a free entry, and that was also when the backend got its backend
ID. Things becomes slightly simpler if we redefine backend ID to be
the index into the PGPROC array, and directly use it also as an index
to the ProcState array. This uses a little more memory, as we reserve
a few extra slots in the ProcState array for aux processes that don't
need them, but the simplicity is worth it.
Aux processes now also have a backend ID. This simplifies the
reservation of BackendStatusArray and ProcSignal slots.You can now convert a backend ID into an index into the PGPROC array
simply by subtracting 1. We still use 0-based "pgprocnos" in various
places, for indexes into the PGPROC array, but the only difference now
is that backend IDs start at 1 while pgprocnos start at 0.
Why aren't we using 0-based indexing for both? InvalidBackendId is -1, so
there'd not be a conflict, right?
One potential downside of this patch is that the ProcState array might
get less densely packed, as we we don't try so hard to assign
low-numbered backend ID anymore. If it's less densely packed,
lastBackend will stay at a higher value, and SIInsertDataEntries() and
SICleanupQueue() need to scan over more unused entries. I think that's
fine. They are performance critical enough to matter, and there was no
guarantee on dense packing before either: If you launched a lot of
backends concurrently, and kept the last one open, lastBackend would
also stay at a high value.
It's perhaps worth noting here that there's a future patch that also addresses
this to some degree?
@@ -457,7 +442,7 @@ MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid, const char *gid,
Assert(LWLockHeldByMeInMode(TwoPhaseStateLock, LW_EXCLUSIVE));Assert(gxact != NULL); - proc = &ProcGlobal->allProcs[gxact->pgprocno]; + proc = GetPGProcByNumber(gxact->pgprocno);/* Initialize the PGPROC entry */
MemSet(proc, 0, sizeof(PGPROC));
This set of changes is independent of this commit, isn't it?
diff --git a/src/backend/postmaster/auxprocess.c b/src/backend/postmaster/auxprocess.c index ab86e802f21..39171fea06b 100644 --- a/src/backend/postmaster/auxprocess.c +++ b/src/backend/postmaster/auxprocess.c @@ -107,17 +107,7 @@ AuxiliaryProcessMain(AuxProcType auxtype)BaseInit();
- /* - * Assign the ProcSignalSlot for an auxiliary process. Since it doesn't - * have a BackendId, the slot is statically allocated based on the - * auxiliary process type (MyAuxProcType). Backends use slots indexed in - * the range from 1 to MaxBackends (inclusive), so we use MaxBackends + - * AuxProcType + 1 as the index of the slot for an auxiliary process. - * - * This will need rethinking if we ever want more than one of a particular - * auxiliary process type. - */ - ProcSignalInit(MaxBackends + MyAuxProcType + 1); + ProcSignalInit();
Now that we don't need the offset here, we could move ProcSignalInit() into
BsaeInit() I think?
+/* + * BackendIdGetProc -- get a backend's PGPROC given its backend ID + * + * The result may be out of date arbitrarily quickly, so the caller + * must be careful about how this information is used. NULL is + * returned if the backend is not active. + */ +PGPROC * +BackendIdGetProc(int backendID) +{ + PGPROC *result; + + if (backendID < 1 || backendID > ProcGlobal->allProcCount) + return NULL;
Hm, doesn't calling BackendIdGetProc() with these values a bug? That's not
about being out of date or such.
+/* + * BackendIdGetTransactionIds -- get a backend's transaction status + * + * Get the xid, xmin, nsubxid and overflow status of the backend. The + * result may be out of date arbitrarily quickly, so the caller must be + * careful about how this information is used. + */ +void +BackendIdGetTransactionIds(int backendID, TransactionId *xid, + TransactionId *xmin, int *nsubxid, bool *overflowed) +{ + PGPROC *proc; + + *xid = InvalidTransactionId; + *xmin = InvalidTransactionId; + *nsubxid = 0; + *overflowed = false; + + if (backendID < 1 || backendID > ProcGlobal->allProcCount) + return; + proc = GetPGProcByBackendId(backendID); + + /* Need to lock out additions/removals of backends */ + LWLockAcquire(ProcArrayLock, LW_SHARED); + + if (proc->pid != 0) + { + *xid = proc->xid; + *xmin = proc->xmin; + *nsubxid = proc->subxidStatus.count; + *overflowed = proc->subxidStatus.overflowed; + } + + LWLockRelease(ProcArrayLock); +}
Hm, I'm not sure about the locking here. For one, previously we weren't
holding ProcArrayLock. For another, holding ProcArrayLock guarantees that the
backend doesn't end its transaction, but it can still assign xids etc. And,
for that matter, the backendid could have been recycled between the caller
acquiring the backendId and calling BackendIdGetTransactionIds().
--- a/src/backend/utils/error/elog.c +++ b/src/backend/utils/error/elog.c @@ -3074,18 +3074,18 @@ log_status_format(StringInfo buf, const char *format, ErrorData *edata) break; case 'v': /* keep VXID format in sync with lockfuncs.c */ - if (MyProc != NULL && MyProc->backendId != InvalidBackendId) + if (MyProc != NULL)
Doesn't this mean we'll include a vxid in more cases now, particularly
including aux processes? That might be ok, but I also suspect that it'll never
have meaningful values...
From 94fd46c9ef30ba5e8ac1a8873fce577a4be425f4 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Mon, 29 Jan 2024 22:57:49 +0200
Subject: [PATCH v8 3/5] Replace 'lastBackend' with an array of in-use slotsNow that the procState array is indexed by pgprocno, the 'lastBackend'
optimization is useless, because aux processes are assigned PGPROC
slots and hence have numbers higher than max_connection. So
'lastBackend' was always set to almost the end of the array.To replace that optimization, mantain a dense array of in-use
indexes. This's redundant with ProgGlobal->procarray, but I was afraid
of adding any more contention to ProcArrayLock, and this keeps the
code isolated to sinvaladt.c too.
I think it'd be good to include that explanation and justification in the code
as well.
I suspect we'll need to split out "procarray membership" locking from
ProcArrayLock at some point in some form (vagueness alert). To reduce
contention we already have to hold both ProcArrayLock and XidGenLock when
changing membership, so that holding either of the locks prevents the set of
members to change. This, kinda and differently, adds yet another lock to that.
It's not clear if we need that optimization at all. I was able to
write a test case that become slower without this: set max_connections
to a very high number (> 5000), and create+truncate a table in the
same transaction thousands of times to send invalidation messages,
with fsync=off. That became about 20% slower on my laptop. Arguably
that's so unrealistic that it doesn't matter, but nevertheless, this
commit restores the performance of that.
I think it's unfortunately not that uncommon to be bottlenecked by sinval
performance, so I think it's good that you're addressing it.
Greetings,
Andres Freund
Re: Refactoring backend fork+exec code
On 07/02/2024 20:25, Andres Freund wrote:
On 2024-01-30 02:08:36 +0200, Heikki Linnakangas wrote:
From 54f22231bb2540fc5957c14005956161e6fc9dac Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 24 Jan 2024 23:15:55 +0200
Subject: [PATCH v8 1/5] Remove superfluous 'pgprocno' field from PGPROCIt was always just the index of the PGPROC entry from the beginning of
the proc array. Introduce a macro to compute it from the pointer
instead.Hm. The pointer math here is bit more expensive than in some other cases, as
the struct is fairly large and sizeof(PGPROC) isn't a power of two. Adding
more math into loops like in TransactionGroupUpdateXidStatus() might end up
showing up.
I added a MyProcNumber global variable that is set to
GetNumberFromPGProc(MyProc). I'm not really concerned about the extra
math, but with MyProcNumber it should definitely not be an issue. The
few GetNumberFromPGProc() invocations that remain are in less
performance-critical paths.
(In later patch, I switch backend ids to 0-based indexing, which
replaces MyProcNumber references with MyBackendId)
Is this really related to the rest of the series?
It's not strictly necessary, but it felt prudent to remove it now, since
I'm removing the backendID field too.
You can now convert a backend ID into an index into the PGPROC array
simply by subtracting 1. We still use 0-based "pgprocnos" in various
places, for indexes into the PGPROC array, but the only difference now
is that backend IDs start at 1 while pgprocnos start at 0.Why aren't we using 0-based indexing for both? InvalidBackendId is -1, so
there'd not be a conflict, right?
Correct. I was being conservative and didn't dare to change the old
convention. The backend ids are visible in a few places like "pg_temp_0"
schema names, and pg_stat_get_*() functions.
One alternative would be to reserve and waste allProcs[0]. Then pgprocno
and backend ID could both be direct indexes to the array, but 0 would
not be used.
If we switch to 0-based indexing, it begs the question: why don't we
merge the concepts of "pgprocno" and "BackendId" completely and call it
the same thing everywhere? It probably would be best in the long run. It
feels like a lot of churn though.
Anyway, I switched to 0-based indexing in the attached new version, to
see what it looks like.
@@ -457,7 +442,7 @@ MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid, const char *gid,
Assert(LWLockHeldByMeInMode(TwoPhaseStateLock, LW_EXCLUSIVE));Assert(gxact != NULL); - proc = &ProcGlobal->allProcs[gxact->pgprocno]; + proc = GetPGProcByNumber(gxact->pgprocno);/* Initialize the PGPROC entry */
MemSet(proc, 0, sizeof(PGPROC));This set of changes is independent of this commit, isn't it?
Yes. It's just for symmetry, now that we use GetNumberFromPGProc() to
get the pgprocno.
diff --git a/src/backend/postmaster/auxprocess.c b/src/backend/postmaster/auxprocess.c index ab86e802f21..39171fea06b 100644 --- a/src/backend/postmaster/auxprocess.c +++ b/src/backend/postmaster/auxprocess.c @@ -107,17 +107,7 @@ AuxiliaryProcessMain(AuxProcType auxtype)BaseInit();
- /* - * Assign the ProcSignalSlot for an auxiliary process. Since it doesn't - * have a BackendId, the slot is statically allocated based on the - * auxiliary process type (MyAuxProcType). Backends use slots indexed in - * the range from 1 to MaxBackends (inclusive), so we use MaxBackends + - * AuxProcType + 1 as the index of the slot for an auxiliary process. - * - * This will need rethinking if we ever want more than one of a particular - * auxiliary process type. - */ - ProcSignalInit(MaxBackends + MyAuxProcType + 1); + ProcSignalInit();Now that we don't need the offset here, we could move ProcSignalInit() into
BsaeInit() I think?
Hmm, doesn't feel right to me. BaseInit() is mostly concerned with
setting up backend-private structures, and it's also called for a
standalone backend.
I feel the process initialization codepaths could use some cleanup in
general. Not sure what exactly.
+/* + * BackendIdGetProc -- get a backend's PGPROC given its backend ID + * + * The result may be out of date arbitrarily quickly, so the caller + * must be careful about how this information is used. NULL is + * returned if the backend is not active. + */ +PGPROC * +BackendIdGetProc(int backendID) +{ + PGPROC *result; + + if (backendID < 1 || backendID > ProcGlobal->allProcCount) + return NULL;Hm, doesn't calling BackendIdGetProc() with these values a bug? That's not
about being out of date or such.
Perhaps. I just followed the example of the old implementation, which
also returns NULL on bogus inputs.
+/* + * BackendIdGetTransactionIds -- get a backend's transaction status + * + * Get the xid, xmin, nsubxid and overflow status of the backend. The + * result may be out of date arbitrarily quickly, so the caller must be + * careful about how this information is used. + */ +void +BackendIdGetTransactionIds(int backendID, TransactionId *xid, + TransactionId *xmin, int *nsubxid, bool *overflowed) +{ + PGPROC *proc; + + *xid = InvalidTransactionId; + *xmin = InvalidTransactionId; + *nsubxid = 0; + *overflowed = false; + + if (backendID < 1 || backendID > ProcGlobal->allProcCount) + return; + proc = GetPGProcByBackendId(backendID); + + /* Need to lock out additions/removals of backends */ + LWLockAcquire(ProcArrayLock, LW_SHARED); + + if (proc->pid != 0) + { + *xid = proc->xid; + *xmin = proc->xmin; + *nsubxid = proc->subxidStatus.count; + *overflowed = proc->subxidStatus.overflowed; + } + + LWLockRelease(ProcArrayLock); +}Hm, I'm not sure about the locking here. For one, previously we weren't
holding ProcArrayLock. For another, holding ProcArrayLock guarantees that the
backend doesn't end its transaction, but it can still assign xids etc. And,
for that matter, the backendid could have been recycled between the caller
acquiring the backendId and calling BackendIdGetTransactionIds().
Yeah, the returned values could be out-of-date and even inconsistent
with each other. I just faithfully copied the old implementation.
Perhaps this should just skip the ProcArrayLock altogether.
--- a/src/backend/utils/error/elog.c +++ b/src/backend/utils/error/elog.c @@ -3074,18 +3074,18 @@ log_status_format(StringInfo buf, const char *format, ErrorData *edata) break; case 'v': /* keep VXID format in sync with lockfuncs.c */ - if (MyProc != NULL && MyProc->backendId != InvalidBackendId) + if (MyProc != NULL)Doesn't this mean we'll include a vxid in more cases now, particularly
including aux processes? That might be ok, but I also suspect that it'll never
have meaningful values...
Fixed. (I thought I changed that back already in the last patch version,
but apparently I only did it in jsonlog.c)
From 94fd46c9ef30ba5e8ac1a8873fce577a4be425f4 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Mon, 29 Jan 2024 22:57:49 +0200
Subject: [PATCH v8 3/5] Replace 'lastBackend' with an array of in-use slotsNow that the procState array is indexed by pgprocno, the 'lastBackend'
optimization is useless, because aux processes are assigned PGPROC
slots and hence have numbers higher than max_connection. So
'lastBackend' was always set to almost the end of the array.To replace that optimization, mantain a dense array of in-use
indexes. This's redundant with ProgGlobal->procarray, but I was afraid
of adding any more contention to ProcArrayLock, and this keeps the
code isolated to sinvaladt.c too.I think it'd be good to include that explanation and justification in the code
as well.
Added a comment.
Attached is a new version of these BackendId changes. I kept it as three
separate patches to highlight the changes from switching to 0-based
indexing, but I think they should be squashed together before pushing.
I think the last remaining question here is about the 0- vs 1-based
indexing of BackendIds. Is it a good idea to switch to 0-based indexing?
And if we do it, should we reserve PGPROC 0. I'm on the fence on this one.
And if we switch to 0-based indexing, should we do a more comprehensive
search & replace of "pgprocno" to "backendId", or something like that.
My vote is no, the code churn doesn't feel worth it. And it can also be
done separately later if we want to.
--
Heikki Linnakangas
Neon (https://neon.tech)
Attachments:
v10-0001-Remove-superfluous-pgprocno-field-from-PGPROC.patchtext/x-patch; charset=UTF-8; name=v10-0001-Remove-superfluous-pgprocno-field-from-PGPROC.patchDownload
From 2f0649151e28e9f2fdd572f7c4e329991bdc2570 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Thu, 8 Feb 2024 12:57:39 +0200
Subject: [PATCH v10 1/3] Remove superfluous 'pgprocno' field from PGPROC
It was always just the index of the PGPROC entry from the beginning of
the proc array. Introduce a macro to compute it from the pointer
instead.
Reviewed-by: Andres Freund
Discussion: https://www.postgresql.org/message-id/8171f1aa-496f-46a6-afc3-c46fe7a9b407%40iki.fi
---
src/backend/access/transam/clog.c | 5 +++--
src/backend/access/transam/twophase.c | 11 +++++------
src/backend/access/transam/xlog.c | 2 +-
src/backend/postmaster/bgwriter.c | 2 +-
src/backend/postmaster/pgarch.c | 2 +-
src/backend/postmaster/walsummarizer.c | 2 +-
src/backend/storage/buffer/bufmgr.c | 6 +++---
src/backend/storage/ipc/procarray.c | 14 ++++++++------
src/backend/storage/lmgr/condition_variable.c | 12 ++++++------
src/backend/storage/lmgr/lwlock.c | 6 +++---
src/backend/storage/lmgr/predicate.c | 2 +-
src/backend/storage/lmgr/proc.c | 7 ++++++-
src/include/storage/lock.h | 2 +-
src/include/storage/proc.h | 7 ++-----
14 files changed, 42 insertions(+), 38 deletions(-)
diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c
index 06fc2989bab..97f7434da34 100644
--- a/src/backend/access/transam/clog.c
+++ b/src/backend/access/transam/clog.c
@@ -425,6 +425,7 @@ TransactionGroupUpdateXidStatus(TransactionId xid, XidStatus status,
{
volatile PROC_HDR *procglobal = ProcGlobal;
PGPROC *proc = MyProc;
+ int pgprocno = MyProcNumber;
uint32 nextidx;
uint32 wakeidx;
@@ -458,7 +459,7 @@ TransactionGroupUpdateXidStatus(TransactionId xid, XidStatus status,
* less efficiently.
*/
if (nextidx != INVALID_PGPROCNO &&
- ProcGlobal->allProcs[nextidx].clogGroupMemberPage != proc->clogGroupMemberPage)
+ GetPGProcByNumber(nextidx)->clogGroupMemberPage != proc->clogGroupMemberPage)
{
/*
* Ensure that this proc is not a member of any clog group that
@@ -473,7 +474,7 @@ TransactionGroupUpdateXidStatus(TransactionId xid, XidStatus status,
if (pg_atomic_compare_exchange_u32(&procglobal->clogGroupFirst,
&nextidx,
- (uint32) proc->pgprocno))
+ (uint32) pgprocno))
break;
}
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 8426458f7f5..234c8d08ebc 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -284,7 +284,7 @@ TwoPhaseShmemInit(void)
TwoPhaseState->freeGXacts = &gxacts[i];
/* associate it with a PGPROC assigned by InitProcGlobal */
- gxacts[i].pgprocno = PreparedXactProcs[i].pgprocno;
+ gxacts[i].pgprocno = GetNumberFromPGProc(&PreparedXactProcs[i]);
/*
* Assign a unique ID for each dummy proc, so that the range of
@@ -461,7 +461,6 @@ MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid, const char *gid,
/* Initialize the PGPROC entry */
MemSet(proc, 0, sizeof(PGPROC));
- proc->pgprocno = gxact->pgprocno;
dlist_node_init(&proc->links);
proc->waitStatus = PROC_WAIT_STATUS_OK;
if (LocalTransactionIdIsValid(MyProc->lxid))
@@ -780,7 +779,7 @@ pg_prepared_xact(PG_FUNCTION_ARGS)
while (status->array != NULL && status->currIdx < status->ngxacts)
{
GlobalTransaction gxact = &status->array[status->currIdx++];
- PGPROC *proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ PGPROC *proc = GetPGProcByNumber(gxact->pgprocno);
Datum values[5] = {0};
bool nulls[5] = {0};
HeapTuple tuple;
@@ -935,7 +934,7 @@ TwoPhaseGetDummyProc(TransactionId xid, bool lock_held)
{
GlobalTransaction gxact = TwoPhaseGetGXact(xid, lock_held);
- return &ProcGlobal->allProcs[gxact->pgprocno];
+ return GetPGProcByNumber(gxact->pgprocno);
}
/************************************************************************/
@@ -1080,7 +1079,7 @@ save_state_data(const void *data, uint32 len)
void
StartPrepare(GlobalTransaction gxact)
{
- PGPROC *proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ PGPROC *proc = GetPGProcByNumber(gxact->pgprocno);
TransactionId xid = gxact->xid;
TwoPhaseFileHeader hdr;
TransactionId *children;
@@ -1539,7 +1538,7 @@ FinishPreparedTransaction(const char *gid, bool isCommit)
* try to commit the same GID at once.
*/
gxact = LockGXact(gid, GetUserId());
- proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ proc = GetPGProcByNumber(gxact->pgprocno);
xid = gxact->xid;
/*
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 478377c4a23..1e4abbca8be 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -1379,7 +1379,7 @@ WALInsertLockAcquire(void)
static int lockToTry = -1;
if (lockToTry == -1)
- lockToTry = MyProc->pgprocno % NUM_XLOGINSERT_LOCKS;
+ lockToTry = MyProcNumber % NUM_XLOGINSERT_LOCKS;
MyLockNo = lockToTry;
/*
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index f11ce27084e..6364b16261f 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -326,7 +326,7 @@ BackgroundWriterMain(void)
if (rc == WL_TIMEOUT && can_hibernate && prev_hibernate)
{
/* Ask for notification at next buffer allocation */
- StrategyNotifyBgWriter(MyProc->pgprocno);
+ StrategyNotifyBgWriter(MyProcNumber);
/* Sleep ... */
(void) WaitLatch(MyLatch,
WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH,
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 67693b05806..2aa3e9890a2 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -242,7 +242,7 @@ PgArchiverMain(void)
* Advertise our pgprocno so that backends can use our latch to wake us up
* while we're sleeping.
*/
- PgArch->pgprocno = MyProc->pgprocno;
+ PgArch->pgprocno = MyProcNumber;
/* Create workspace for pgarch_readyXlog() */
arch_files = palloc(sizeof(struct arch_files_state));
diff --git a/src/backend/postmaster/walsummarizer.c b/src/backend/postmaster/walsummarizer.c
index 9b883c21ca4..5d61d1517e8 100644
--- a/src/backend/postmaster/walsummarizer.c
+++ b/src/backend/postmaster/walsummarizer.c
@@ -248,7 +248,7 @@ WalSummarizerMain(void)
/* Advertise ourselves. */
on_shmem_exit(WalSummarizerShutdown, (Datum) 0);
LWLockAcquire(WALSummarizerLock, LW_EXCLUSIVE);
- WalSummarizerCtl->summarizer_pgprocno = MyProc->pgprocno;
+ WalSummarizerCtl->summarizer_pgprocno = MyProcNumber;
LWLockRelease(WALSummarizerLock);
/* Create and switch to a memory context that we can reset on error. */
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index eb1ec3b86df..87a31745864 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -4780,7 +4780,7 @@ UnlockBuffers(void)
* got a cancel/die interrupt before getting the signal.
*/
if ((buf_state & BM_PIN_COUNT_WAITER) != 0 &&
- buf->wait_backend_pgprocno == MyProc->pgprocno)
+ buf->wait_backend_pgprocno == MyProcNumber)
buf_state &= ~BM_PIN_COUNT_WAITER;
UnlockBufHdr(buf, buf_state);
@@ -4930,7 +4930,7 @@ LockBufferForCleanup(Buffer buffer)
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
elog(ERROR, "multiple backends attempting to wait for pincount 1");
}
- bufHdr->wait_backend_pgprocno = MyProc->pgprocno;
+ bufHdr->wait_backend_pgprocno = MyProcNumber;
PinCountWaitBuf = bufHdr;
buf_state |= BM_PIN_COUNT_WAITER;
UnlockBufHdr(bufHdr, buf_state);
@@ -4994,7 +4994,7 @@ LockBufferForCleanup(Buffer buffer)
*/
buf_state = LockBufHdr(bufHdr);
if ((buf_state & BM_PIN_COUNT_WAITER) != 0 &&
- bufHdr->wait_backend_pgprocno == MyProc->pgprocno)
+ bufHdr->wait_backend_pgprocno == MyProcNumber)
buf_state &= ~BM_PIN_COUNT_WAITER;
UnlockBufHdr(bufHdr, buf_state);
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index ee2d7f8585a..dd329a86ef4 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -468,6 +468,7 @@ CreateSharedProcArray(void)
void
ProcArrayAdd(PGPROC *proc)
{
+ int pgprocno = GetNumberFromPGProc(proc);
ProcArrayStruct *arrayP = procArray;
int index;
int movecount;
@@ -499,13 +500,13 @@ ProcArrayAdd(PGPROC *proc)
*/
for (index = 0; index < arrayP->numProcs; index++)
{
- int procno PG_USED_FOR_ASSERTS_ONLY = arrayP->pgprocnos[index];
+ int this_procno = arrayP->pgprocnos[index];
- Assert(procno >= 0 && procno < (arrayP->maxProcs + NUM_AUXILIARY_PROCS));
- Assert(allProcs[procno].pgxactoff == index);
+ Assert(this_procno >= 0 && this_procno < (arrayP->maxProcs + NUM_AUXILIARY_PROCS));
+ Assert(allProcs[this_procno].pgxactoff == index);
/* If we have found our right position in the array, break */
- if (arrayP->pgprocnos[index] > proc->pgprocno)
+ if (this_procno > pgprocno)
break;
}
@@ -523,7 +524,7 @@ ProcArrayAdd(PGPROC *proc)
&ProcGlobal->statusFlags[index],
movecount * sizeof(*ProcGlobal->statusFlags));
- arrayP->pgprocnos[index] = proc->pgprocno;
+ arrayP->pgprocnos[index] = GetNumberFromPGProc(proc);
proc->pgxactoff = index;
ProcGlobal->xids[index] = proc->xid;
ProcGlobal->subxidStates[index] = proc->subxidStatus;
@@ -791,6 +792,7 @@ ProcArrayEndTransactionInternal(PGPROC *proc, TransactionId latestXid)
static void
ProcArrayGroupClearXid(PGPROC *proc, TransactionId latestXid)
{
+ int pgprocno = GetNumberFromPGProc(proc);
PROC_HDR *procglobal = ProcGlobal;
uint32 nextidx;
uint32 wakeidx;
@@ -808,7 +810,7 @@ ProcArrayGroupClearXid(PGPROC *proc, TransactionId latestXid)
if (pg_atomic_compare_exchange_u32(&procglobal->procArrayGroupFirst,
&nextidx,
- (uint32) proc->pgprocno))
+ (uint32) pgprocno))
break;
}
diff --git a/src/backend/storage/lmgr/condition_variable.c b/src/backend/storage/lmgr/condition_variable.c
index e2d6d685220..10fdae19dcc 100644
--- a/src/backend/storage/lmgr/condition_variable.c
+++ b/src/backend/storage/lmgr/condition_variable.c
@@ -57,7 +57,7 @@ ConditionVariableInit(ConditionVariable *cv)
void
ConditionVariablePrepareToSleep(ConditionVariable *cv)
{
- int pgprocno = MyProc->pgprocno;
+ int pgprocno = MyProcNumber;
/*
* If some other sleep is already prepared, cancel it; this is necessary
@@ -181,10 +181,10 @@ ConditionVariableTimedSleep(ConditionVariable *cv, long timeout,
* guarantee not to return spuriously, we'll avoid this obvious case.
*/
SpinLockAcquire(&cv->mutex);
- if (!proclist_contains(&cv->wakeup, MyProc->pgprocno, cvWaitLink))
+ if (!proclist_contains(&cv->wakeup, MyProcNumber, cvWaitLink))
{
done = true;
- proclist_push_tail(&cv->wakeup, MyProc->pgprocno, cvWaitLink);
+ proclist_push_tail(&cv->wakeup, MyProcNumber, cvWaitLink);
}
SpinLockRelease(&cv->mutex);
@@ -236,8 +236,8 @@ ConditionVariableCancelSleep(void)
return false;
SpinLockAcquire(&cv->mutex);
- if (proclist_contains(&cv->wakeup, MyProc->pgprocno, cvWaitLink))
- proclist_delete(&cv->wakeup, MyProc->pgprocno, cvWaitLink);
+ if (proclist_contains(&cv->wakeup, MyProcNumber, cvWaitLink))
+ proclist_delete(&cv->wakeup, MyProcNumber, cvWaitLink);
else
signaled = true;
SpinLockRelease(&cv->mutex);
@@ -281,7 +281,7 @@ ConditionVariableSignal(ConditionVariable *cv)
void
ConditionVariableBroadcast(ConditionVariable *cv)
{
- int pgprocno = MyProc->pgprocno;
+ int pgprocno = MyProcNumber;
PGPROC *proc = NULL;
bool have_sentinel = false;
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index 71677cf0317..997857679ed 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -1056,9 +1056,9 @@ LWLockQueueSelf(LWLock *lock, LWLockMode mode)
/* LW_WAIT_UNTIL_FREE waiters are always at the front of the queue */
if (mode == LW_WAIT_UNTIL_FREE)
- proclist_push_head(&lock->waiters, MyProc->pgprocno, lwWaitLink);
+ proclist_push_head(&lock->waiters, MyProcNumber, lwWaitLink);
else
- proclist_push_tail(&lock->waiters, MyProc->pgprocno, lwWaitLink);
+ proclist_push_tail(&lock->waiters, MyProcNumber, lwWaitLink);
/* Can release the mutex now */
LWLockWaitListUnlock(lock);
@@ -1097,7 +1097,7 @@ LWLockDequeueSelf(LWLock *lock)
*/
on_waitlist = MyProc->lwWaiting == LW_WS_WAITING;
if (on_waitlist)
- proclist_delete(&lock->waiters, MyProc->pgprocno, lwWaitLink);
+ proclist_delete(&lock->waiters, MyProcNumber, lwWaitLink);
if (proclist_is_empty(&lock->waiters) &&
(pg_atomic_read_u32(&lock->state) & LW_FLAG_HAS_WAITERS) != 0)
diff --git a/src/backend/storage/lmgr/predicate.c b/src/backend/storage/lmgr/predicate.c
index eed63a05ed8..d62060d58c8 100644
--- a/src/backend/storage/lmgr/predicate.c
+++ b/src/backend/storage/lmgr/predicate.c
@@ -1830,7 +1830,7 @@ GetSerializableTransactionSnapshotInt(Snapshot snapshot,
sxact->finishedBefore = InvalidTransactionId;
sxact->xmin = snapshot->xmin;
sxact->pid = MyProcPid;
- sxact->pgprocno = MyProc->pgprocno;
+ sxact->pgprocno = MyProcNumber;
dlist_init(&sxact->predicateLocks);
dlist_node_init(&sxact->finishedLink);
sxact->flags = 0;
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index e5977548fe2..fddf44015fd 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -64,6 +64,7 @@ bool log_lock_waits = false;
/* Pointer to this process's PGPROC struct, if any */
PGPROC *MyProc = NULL;
+int MyProcNumber = INVALID_PGPROCNO;
/*
* This spinlock protects the freelist of recycled PGPROC structures.
@@ -227,7 +228,6 @@ InitProcGlobal(void)
InitSharedLatch(&(proc->procLatch));
LWLockInitialize(&(proc->fpInfoLock), LWTRANCHE_LOCK_FASTPATH);
}
- proc->pgprocno = i;
/*
* Newly created PGPROCs for normal backends, autovacuum and bgworkers
@@ -352,6 +352,7 @@ InitProcess(void)
(errcode(ERRCODE_TOO_MANY_CONNECTIONS),
errmsg("sorry, too many clients already")));
}
+ MyProcNumber = GetNumberFromPGProc(MyProc);
/*
* Cross-check that the PGPROC is of the type we expect; if this were not
@@ -565,6 +566,8 @@ InitAuxiliaryProcess(void)
SpinLockRelease(ProcStructLock);
+ MyProcNumber = GetNumberFromPGProc(MyProc);
+
/*
* Initialize all fields of MyProc, except for those previously
* initialized by InitProcGlobal.
@@ -906,6 +909,7 @@ ProcKill(int code, Datum arg)
proc = MyProc;
MyProc = NULL;
+ MyProcNumber = INVALID_PGPROCNO;
DisownLatch(&proc->procLatch);
procgloballist = proc->procgloballist;
@@ -977,6 +981,7 @@ AuxiliaryProcKill(int code, Datum arg)
proc = MyProc;
MyProc = NULL;
+ MyProcNumber = INVALID_PGPROCNO;
DisownLatch(&proc->procLatch);
SpinLockAcquire(ProcStructLock);
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index 00679624f7d..ed6071f3286 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -540,7 +540,7 @@ typedef enum
* used for a given lock group is determined by the group leader's pgprocno.
*/
#define LockHashPartitionLockByProc(leader_pgproc) \
- LockHashPartitionLock((leader_pgproc)->pgprocno)
+ LockHashPartitionLock(GetNumberFromPGProc(leader_pgproc))
/*
* function prototypes
diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h
index 4bc226e36cd..ca21518f522 100644
--- a/src/include/storage/proc.h
+++ b/src/include/storage/proc.h
@@ -194,11 +194,6 @@ struct PGPROC
int pgxactoff; /* offset into various ProcGlobal->arrays with
* data mirrored from this PGPROC */
- int pgprocno; /* Number of this PGPROC in
- * ProcGlobal->allProcs array. This is set
- * once by InitProcGlobal().
- * ProcGlobal->allProcs[n].pgprocno == n */
-
/* These fields are zero while a backend is still starting up: */
BackendId backendId; /* This backend's backend ID (if assigned) */
Oid databaseId; /* OID of database this backend is using */
@@ -307,6 +302,7 @@ struct PGPROC
extern PGDLLIMPORT PGPROC *MyProc;
+extern PGDLLIMPORT int MyProcNumber; /* same as GetNumberFromPGProc(MyProc) */
/*
* There is one ProcGlobal struct for the whole database cluster.
@@ -412,6 +408,7 @@ extern PGDLLIMPORT PGPROC *PreparedXactProcs;
/* Accessor for PGPROC given a pgprocno. */
#define GetPGProcByNumber(n) (&ProcGlobal->allProcs[(n)])
+#define GetNumberFromPGProc(proc) ((proc) - &ProcGlobal->allProcs[0])
/*
* We set aside some extra PGPROC structures for auxiliary processes,
--
2.39.2
v10-0002-Redefine-backend-ID-to-be-an-index-into-the-proc.patchtext/x-patch; charset=UTF-8; name=v10-0002-Redefine-backend-ID-to-be-an-index-into-the-proc.patchDownload
From d4cf14da8400cfb43d8aee1592f8af3486fa8de9 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Thu, 8 Feb 2024 12:57:54 +0200
Subject: [PATCH v10 2/3] Redefine backend ID to be an index into the proc
array.
Previously, backend ID was an index into the ProcState array, in the
shared cache invalidation manager (sinvaladt.c). The entry in the
ProcState array was reserved at backend startup by scanning the array
for a free entry, and that was also when the backend got its backend
ID. Things becomes slightly simpler if we redefine backend ID to be
the index into the PGPROC array, and directly use it also as an index
to the ProcState array. This uses a little more memory, as we reserve
a few extra slots in the ProcState array for aux processes that don't
need them, but the simplicity is worth it.
Aux processes now also have a backend ID. This simplifies the
reservation of BackendStatusArray and ProcSignal slots.
You can now convert a backend ID into an index into the PGPROC array
simply by subtracting 1. We still use 0-based "pgprocnos" in various
places, for indexes into the PGPROC array, but the only difference now
is that backend IDs start at 1 while pgprocnos start at 0.
There is still a 'backendId' field in PGPROC, now part of 'vxid' which
encapsulates the backend ID and local transaction ID together. It's
needed for prepared xacts. For regular backends, the backendId is
always equal to pgprocno + 1, but for prepared xact PGPROC entries,
it's the ID of the original backend that processed the transaction.
Reviewed-by: Andres Freund
Discussion: https://www.postgresql.org/message-id/8171f1aa-496f-46a6-afc3-c46fe7a9b407@iki.fi
---
src/backend/access/transam/twophase.c | 37 ++--
src/backend/access/transam/xact.c | 12 +-
src/backend/catalog/namespace.c | 2 +-
src/backend/commands/sequence.c | 2 +-
src/backend/executor/functions.c | 4 +-
src/backend/postmaster/auxprocess.c | 12 +-
src/backend/storage/ipc/procarray.c | 73 ++++++-
src/backend/storage/ipc/procsignal.c | 27 ++-
src/backend/storage/ipc/sinvaladt.c | 200 ++++++--------------
src/backend/storage/ipc/standby.c | 1 +
src/backend/storage/lmgr/lock.c | 28 +--
src/backend/storage/lmgr/proc.c | 30 +--
src/backend/utils/activity/backend_status.c | 52 ++---
src/backend/utils/adt/lockfuncs.c | 2 +-
src/backend/utils/adt/mcxtfuncs.c | 14 +-
src/backend/utils/error/csvlog.c | 4 +-
src/backend/utils/error/elog.c | 6 +-
src/backend/utils/error/jsonlog.c | 6 +-
src/backend/utils/init/postinit.c | 10 +-
src/backend/utils/time/snapmgr.c | 5 +-
src/include/miscadmin.h | 2 -
src/include/storage/backendid.h | 12 +-
src/include/storage/lock.h | 9 +-
src/include/storage/proc.h | 32 +++-
src/include/storage/procarray.h | 4 +
src/include/storage/procsignal.h | 2 +-
src/include/storage/sinvaladt.h | 4 -
src/pl/plpgsql/src/pl_exec.c | 10 +-
28 files changed, 280 insertions(+), 322 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 234c8d08ebc..5c282002900 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -151,7 +151,6 @@ typedef struct GlobalTransactionData
{
GlobalTransaction next; /* list link for free list */
int pgprocno; /* ID of associated dummy PGPROC */
- BackendId dummyBackendId; /* similar to backend id for backends */
TimestampTz prepared_at; /* time of preparation */
/*
@@ -285,20 +284,6 @@ TwoPhaseShmemInit(void)
/* associate it with a PGPROC assigned by InitProcGlobal */
gxacts[i].pgprocno = GetNumberFromPGProc(&PreparedXactProcs[i]);
-
- /*
- * Assign a unique ID for each dummy proc, so that the range of
- * dummy backend IDs immediately follows the range of normal
- * backend IDs. We don't dare to assign a real backend ID to dummy
- * procs, because prepared transactions don't take part in cache
- * invalidation like a real backend ID would imply, but having a
- * unique ID for them is nevertheless handy. This arrangement
- * allows you to allocate an array of size (MaxBackends +
- * max_prepared_xacts + 1), and have a slot for every backend and
- * prepared transaction. Currently multixact.c uses that
- * technique.
- */
- gxacts[i].dummyBackendId = MaxBackends + 1 + i;
}
}
else
@@ -457,24 +442,24 @@ MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid, const char *gid,
Assert(LWLockHeldByMeInMode(TwoPhaseStateLock, LW_EXCLUSIVE));
Assert(gxact != NULL);
- proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ proc = GetPGProcByNumber(gxact->pgprocno);
/* Initialize the PGPROC entry */
MemSet(proc, 0, sizeof(PGPROC));
dlist_node_init(&proc->links);
proc->waitStatus = PROC_WAIT_STATUS_OK;
- if (LocalTransactionIdIsValid(MyProc->lxid))
+ if (LocalTransactionIdIsValid(MyProc->vxid.lxid))
{
/* clone VXID, for TwoPhaseGetXidByVirtualXID() to find */
- proc->lxid = MyProc->lxid;
- proc->backendId = MyBackendId;
+ proc->vxid.lxid = MyProc->vxid.lxid;
+ proc->vxid.backendId = MyBackendId;
}
else
{
Assert(AmStartupProcess() || !IsPostmasterEnvironment);
/* GetLockConflicts() uses this to specify a wait on the XID */
- proc->lxid = xid;
- proc->backendId = InvalidBackendId;
+ proc->vxid.lxid = xid;
+ proc->vxid.backendId = InvalidBackendId;
}
proc->xid = xid;
Assert(proc->xmin == InvalidTransactionId);
@@ -522,7 +507,7 @@ static void
GXactLoadSubxactData(GlobalTransaction gxact, int nsubxacts,
TransactionId *children)
{
- PGPROC *proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ PGPROC *proc = GetPGProcByNumber(gxact->pgprocno);
/* We need no extra lock since the GXACT isn't valid yet */
if (nsubxacts > PGPROC_MAX_CACHED_SUBXIDS)
@@ -559,7 +544,7 @@ MarkAsPrepared(GlobalTransaction gxact, bool lock_held)
* Put it into the global ProcArray so TransactionIdIsInProgress considers
* the XID as still running.
*/
- ProcArrayAdd(&ProcGlobal->allProcs[gxact->pgprocno]);
+ ProcArrayAdd(GetPGProcByNumber(gxact->pgprocno));
}
/*
@@ -583,7 +568,7 @@ LockGXact(const char *gid, Oid user)
for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
{
GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
- PGPROC *proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ PGPROC *proc = GetPGProcByNumber(gxact->pgprocno);
/* Ignore not-yet-valid GIDs */
if (!gxact->valid)
@@ -884,7 +869,7 @@ TwoPhaseGetXidByVirtualXID(VirtualTransactionId vxid,
if (!gxact->valid)
continue;
- proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ proc = GetPGProcByNumber(gxact->pgprocno);
GET_VXID_FROM_PGPROC(proc_vxid, *proc);
if (VirtualTransactionIdEquals(vxid, proc_vxid))
{
@@ -919,7 +904,7 @@ TwoPhaseGetDummyBackendId(TransactionId xid, bool lock_held)
{
GlobalTransaction gxact = TwoPhaseGetGXact(xid, lock_held);
- return gxact->dummyBackendId;
+ return gxact->pgprocno + 1;
}
/*
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 464858117e0..f791340d3df 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -600,9 +600,9 @@ GetStableLatestTransactionId(void)
static LocalTransactionId lxid = InvalidLocalTransactionId;
static TransactionId stablexid = InvalidTransactionId;
- if (lxid != MyProc->lxid)
+ if (lxid != MyProc->vxid.lxid)
{
- lxid = MyProc->lxid;
+ lxid = MyProc->vxid.lxid;
stablexid = GetTopTransactionIdIfAny();
if (!TransactionIdIsValid(stablexid))
stablexid = ReadNextTransactionId();
@@ -2099,8 +2099,8 @@ StartTransaction(void)
* Advertise it in the proc array. We assume assignment of
* localTransactionId is atomic, and the backendId should be set already.
*/
- Assert(MyProc->backendId == vxid.backendId);
- MyProc->lxid = vxid.localTransactionId;
+ Assert(MyProc->vxid.backendId == vxid.backendId);
+ MyProc->vxid.lxid = vxid.localTransactionId;
TRACE_POSTGRESQL_TRANSACTION_START(vxid.localTransactionId);
@@ -2281,7 +2281,7 @@ CommitTransaction(void)
ParallelWorkerReportLastRecEnd(XactLastRecEnd);
}
- TRACE_POSTGRESQL_TRANSACTION_COMMIT(MyProc->lxid);
+ TRACE_POSTGRESQL_TRANSACTION_COMMIT(MyProc->vxid.lxid);
/*
* Let others know about no transaction in progress by me. Note that this
@@ -2824,7 +2824,7 @@ AbortTransaction(void)
XLogSetAsyncXactLSN(XactLastRecEnd);
}
- TRACE_POSTGRESQL_TRANSACTION_ABORT(MyProc->lxid);
+ TRACE_POSTGRESQL_TRANSACTION_ABORT(MyProc->vxid.lxid);
/*
* Let others know about no transaction in progress by me. Note that this
diff --git a/src/backend/catalog/namespace.c b/src/backend/catalog/namespace.c
index 8df30b24401..620ce7e75d2 100644
--- a/src/backend/catalog/namespace.c
+++ b/src/backend/catalog/namespace.c
@@ -49,7 +49,7 @@
#include "parser/parse_func.h"
#include "storage/ipc.h"
#include "storage/lmgr.h"
-#include "storage/sinvaladt.h"
+#include "storage/procarray.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index f7803744a5b..f99dc48e8cf 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1076,7 +1076,7 @@ setval3_oid(PG_FUNCTION_ARGS)
static Relation
lock_and_open_sequence(SeqTable seq)
{
- LocalTransactionId thislxid = MyProc->lxid;
+ LocalTransactionId thislxid = MyProc->vxid.lxid;
/* Get the lock if not already held in this xact */
if (seq->lxid != thislxid)
diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c
index 0f811fd2fc9..a4b6e1effdb 100644
--- a/src/backend/executor/functions.c
+++ b/src/backend/executor/functions.c
@@ -799,7 +799,7 @@ init_sql_fcache(FunctionCallInfo fcinfo, Oid collation, bool lazyEvalOK)
lazyEvalOK);
/* Mark fcache with time of creation to show it's valid */
- fcache->lxid = MyProc->lxid;
+ fcache->lxid = MyProc->vxid.lxid;
fcache->subxid = GetCurrentSubTransactionId();
ReleaseSysCache(procedureTuple);
@@ -1081,7 +1081,7 @@ fmgr_sql(PG_FUNCTION_ARGS)
if (fcache != NULL)
{
- if (fcache->lxid != MyProc->lxid ||
+ if (fcache->lxid != MyProc->vxid.lxid ||
!SubTransactionIsActive(fcache->subxid))
{
/* It's stale; unlink and delete */
diff --git a/src/backend/postmaster/auxprocess.c b/src/backend/postmaster/auxprocess.c
index ab86e802f21..39171fea06b 100644
--- a/src/backend/postmaster/auxprocess.c
+++ b/src/backend/postmaster/auxprocess.c
@@ -107,17 +107,7 @@ AuxiliaryProcessMain(AuxProcType auxtype)
BaseInit();
- /*
- * Assign the ProcSignalSlot for an auxiliary process. Since it doesn't
- * have a BackendId, the slot is statically allocated based on the
- * auxiliary process type (MyAuxProcType). Backends use slots indexed in
- * the range from 1 to MaxBackends (inclusive), so we use MaxBackends +
- * AuxProcType + 1 as the index of the slot for an auxiliary process.
- *
- * This will need rethinking if we ever want more than one of a particular
- * auxiliary process type.
- */
- ProcSignalInit(MaxBackends + MyAuxProcType + 1);
+ ProcSignalInit();
/*
* Auxiliary processes don't run transactions, but they may need a
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index dd329a86ef4..d96606ebba5 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -701,7 +701,7 @@ ProcArrayEndTransaction(PGPROC *proc, TransactionId latestXid)
Assert(proc->subxidStatus.count == 0);
Assert(!proc->subxidStatus.overflowed);
- proc->lxid = InvalidLocalTransactionId;
+ proc->vxid.lxid = InvalidLocalTransactionId;
proc->xmin = InvalidTransactionId;
/* be sure this is cleared in abort */
@@ -743,7 +743,7 @@ ProcArrayEndTransactionInternal(PGPROC *proc, TransactionId latestXid)
ProcGlobal->xids[pgxactoff] = InvalidTransactionId;
proc->xid = InvalidTransactionId;
- proc->lxid = InvalidLocalTransactionId;
+ proc->vxid.lxid = InvalidLocalTransactionId;
proc->xmin = InvalidTransactionId;
/* be sure this is cleared in abort */
@@ -930,7 +930,7 @@ ProcArrayClearTransaction(PGPROC *proc)
ProcGlobal->xids[pgxactoff] = InvalidTransactionId;
proc->xid = InvalidTransactionId;
- proc->lxid = InvalidLocalTransactionId;
+ proc->vxid.lxid = InvalidLocalTransactionId;
proc->xmin = InvalidTransactionId;
proc->recoveryConflictPending = false;
@@ -2536,6 +2536,11 @@ ProcArrayInstallImportedXmin(TransactionId xmin,
/* Get lock so source xact can't end while we're doing this */
LWLockAcquire(ProcArrayLock, LW_SHARED);
+ /*
+ * Find the PGPROC entry of the source transaction. (This could use
+ * GetPGProcByBackendId(), unless it's a prepared xact. But this isn't
+ * performance critical.)
+ */
for (index = 0; index < arrayP->numProcs; index++)
{
int pgprocno = arrayP->pgprocnos[index];
@@ -2548,9 +2553,9 @@ ProcArrayInstallImportedXmin(TransactionId xmin,
continue;
/* We are only interested in the specific virtual transaction. */
- if (proc->backendId != sourcevxid->backendId)
+ if (proc->vxid.backendId != sourcevxid->backendId)
continue;
- if (proc->lxid != sourcevxid->localTransactionId)
+ if (proc->vxid.lxid != sourcevxid->localTransactionId)
continue;
/*
@@ -3099,6 +3104,64 @@ HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids, int type)
return result;
}
+/*
+ * BackendIdGetProc -- get a backend's PGPROC given its backend ID
+ *
+ * The result may be out of date arbitrarily quickly, so the caller
+ * must be careful about how this information is used. NULL is
+ * returned if the backend is not active.
+ */
+PGPROC *
+BackendIdGetProc(int backendID)
+{
+ PGPROC *result;
+
+ if (backendID < 1 || backendID > ProcGlobal->allProcCount)
+ return NULL;
+ result = GetPGProcByBackendId(backendID);
+
+ if (result->pid == 0)
+ return NULL;
+
+ return result;
+}
+
+/*
+ * BackendIdGetTransactionIds -- get a backend's transaction status
+ *
+ * Get the xid, xmin, nsubxid and overflow status of the backend. The
+ * result may be out of date arbitrarily quickly, so the caller must be
+ * careful about how this information is used.
+ */
+void
+BackendIdGetTransactionIds(int backendID, TransactionId *xid,
+ TransactionId *xmin, int *nsubxid, bool *overflowed)
+{
+ PGPROC *proc;
+
+ *xid = InvalidTransactionId;
+ *xmin = InvalidTransactionId;
+ *nsubxid = 0;
+ *overflowed = false;
+
+ if (backendID < 1 || backendID > ProcGlobal->allProcCount)
+ return;
+ proc = GetPGProcByBackendId(backendID);
+
+ /* Need to lock out additions/removals of backends */
+ LWLockAcquire(ProcArrayLock, LW_SHARED);
+
+ if (proc->pid != 0)
+ {
+ *xid = proc->xid;
+ *xmin = proc->xmin;
+ *nsubxid = proc->subxidStatus.count;
+ *overflowed = proc->subxidStatus.overflowed;
+ }
+
+ LWLockRelease(ProcArrayLock);
+}
+
/*
* BackendPidGetProc -- get a backend's PGPROC given its PID
*
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index e84619e5a58..d1d5bf0c152 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -87,7 +87,7 @@ typedef struct
* possible auxiliary process type. (This scheme assumes there is not
* more than one of any auxiliary process type at a time.)
*/
-#define NumProcSignalSlots (MaxBackends + NUM_AUXPROCTYPES)
+#define NumProcSignalSlots (MaxBackends + NUM_AUXILIARY_PROCS)
/* Check whether the relevant type bit is set in the flags. */
#define BARRIER_SHOULD_CHECK(flags, type) \
@@ -154,24 +154,23 @@ ProcSignalShmemInit(void)
/*
* ProcSignalInit
* Register the current process in the ProcSignal array
- *
- * The passed index should be my BackendId if the process has one,
- * or MaxBackends + aux process type if not.
*/
void
-ProcSignalInit(int pss_idx)
+ProcSignalInit(void)
{
ProcSignalSlot *slot;
uint64 barrier_generation;
- Assert(pss_idx >= 1 && pss_idx <= NumProcSignalSlots);
-
- slot = &ProcSignal->psh_slot[pss_idx - 1];
+ if (MyBackendId <= 0)
+ elog(ERROR, "MyBackendId not set");
+ if (MyBackendId > NumProcSignalSlots)
+ elog(ERROR, "unexpected MyBackendId %d in ProcSignalInit (max %d)", MyBackendId, NumProcSignalSlots);
+ slot = &ProcSignal->psh_slot[MyBackendId - 1];
/* sanity check */
if (slot->pss_pid != 0)
elog(LOG, "process %d taking over ProcSignal slot %d, but it's not empty",
- MyProcPid, pss_idx);
+ MyProcPid, (int) (slot - ProcSignal->psh_slot));
/* Clear out any leftover signal reasons */
MemSet(slot->pss_signalFlags, 0, NUM_PROCSIGNALS * sizeof(sig_atomic_t));
@@ -200,7 +199,7 @@ ProcSignalInit(int pss_idx)
MyProcSignalSlot = slot;
/* Set up to release the slot on process exit */
- on_shmem_exit(CleanupProcSignalState, Int32GetDatum(pss_idx));
+ on_shmem_exit(CleanupProcSignalState, (Datum) 0);
}
/*
@@ -212,11 +211,7 @@ ProcSignalInit(int pss_idx)
static void
CleanupProcSignalState(int status, Datum arg)
{
- int pss_idx = DatumGetInt32(arg);
- ProcSignalSlot *slot;
-
- slot = &ProcSignal->psh_slot[pss_idx - 1];
- Assert(slot == MyProcSignalSlot);
+ ProcSignalSlot *slot = MyProcSignalSlot;
/*
* Clear MyProcSignalSlot, so that a SIGUSR1 received after this point
@@ -233,7 +228,7 @@ CleanupProcSignalState(int status, Datum arg)
* infinite loop trying to exit
*/
elog(LOG, "process %d releasing ProcSignal slot %d, but it contains %d",
- MyProcPid, pss_idx, (int) slot->pss_pid);
+ MyProcPid, (int) (slot - ProcSignal->psh_slot), (int) slot->pss_pid);
return; /* XXX better to zero the slot anyway? */
}
diff --git a/src/backend/storage/ipc/sinvaladt.c b/src/backend/storage/ipc/sinvaladt.c
index 748a792a854..f624bfc7d78 100644
--- a/src/backend/storage/ipc/sinvaladt.c
+++ b/src/backend/storage/ipc/sinvaladt.c
@@ -139,7 +139,6 @@ typedef struct ProcState
{
/* procPid is zero in an inactive ProcState array entry. */
pid_t procPid; /* PID of backend, for signaling */
- PGPROC *proc; /* PGPROC of backend */
/* nextMsgNum is meaningless if procPid == 0 or resetState is true. */
int nextMsgNum; /* next message number to read */
bool resetState; /* backend needs to reset its state */
@@ -172,8 +171,6 @@ typedef struct SISeg
int minMsgNum; /* oldest message still needed */
int maxMsgNum; /* next message number to be assigned */
int nextThreshold; /* # of messages to call SICleanupQueue */
- int lastBackend; /* index of last active procState entry, +1 */
- int maxBackends; /* size of procState array */
slock_t msgnumLock; /* spinlock protecting maxMsgNum */
@@ -183,11 +180,29 @@ typedef struct SISeg
SharedInvalidationMessage buffer[MAXNUMMESSAGES];
/*
- * Per-backend invalidation state info (has MaxBackends entries).
+ * Per-backend invalidation state info.
+ *
+ * 'procState' has NumProcStateSlots entries, and is indexed by pgprocno.
+ * 'numProcs' is the number of slots currently in use, and 'pgprocnos' is
+ * a dense array of their indexes, to speed up scanning all in-use slots.
+ *
+ * 'pgprocnos' is largely redundant with ProcArrayStruct->pgprocnos, but
+ * having our separate copy avoids contention on ProcArrayLock, and allows
+ * us to track only the processes that participate in shared cache
+ * invalidations.
*/
+ int numProcs;
+ int *pgprocnos;
ProcState procState[FLEXIBLE_ARRAY_MEMBER];
} SISeg;
+/*
+ * We reserve a slot for each possible BackendId, plus one for each
+ * possible auxiliary process type. (This scheme assumes there is not
+ * more than one of any auxiliary process type at a time.)
+ */
+#define NumProcStateSlots (MaxBackends + NUM_AUXILIARY_PROCS)
+
static SISeg *shmInvalBuffer; /* pointer to the shared inval buffer */
@@ -205,16 +220,8 @@ SInvalShmemSize(void)
Size size;
size = offsetof(SISeg, procState);
-
- /*
- * In Hot Standby mode, the startup process requests a procState array
- * slot using InitRecoveryTransactionEnvironment(). Even though
- * MaxBackends doesn't account for the startup process, it is guaranteed
- * to get a free slot. This is because the autovacuum launcher and worker
- * processes, which are included in MaxBackends, are not started in Hot
- * Standby mode.
- */
- size = add_size(size, mul_size(sizeof(ProcState), MaxBackends));
+ size = add_size(size, mul_size(sizeof(ProcState), NumProcStateSlots)); /* procState */
+ size = add_size(size, mul_size(sizeof(int), NumProcStateSlots)); /* pgprocnos */
return size;
}
@@ -239,23 +246,22 @@ CreateSharedInvalidationState(void)
shmInvalBuffer->minMsgNum = 0;
shmInvalBuffer->maxMsgNum = 0;
shmInvalBuffer->nextThreshold = CLEANUP_MIN;
- shmInvalBuffer->lastBackend = 0;
- shmInvalBuffer->maxBackends = MaxBackends;
SpinLockInit(&shmInvalBuffer->msgnumLock);
/* The buffer[] array is initially all unused, so we need not fill it */
/* Mark all backends inactive, and initialize nextLXID */
- for (i = 0; i < shmInvalBuffer->maxBackends; i++)
+ for (i = 0; i < NumProcStateSlots; i++)
{
shmInvalBuffer->procState[i].procPid = 0; /* inactive */
- shmInvalBuffer->procState[i].proc = NULL;
shmInvalBuffer->procState[i].nextMsgNum = 0; /* meaningless */
shmInvalBuffer->procState[i].resetState = false;
shmInvalBuffer->procState[i].signaled = false;
shmInvalBuffer->procState[i].hasMessages = false;
shmInvalBuffer->procState[i].nextLXID = InvalidLocalTransactionId;
}
+ shmInvalBuffer->numProcs = 0;
+ shmInvalBuffer->pgprocnos = (int *) &shmInvalBuffer->procState[i];
}
/*
@@ -265,59 +271,41 @@ CreateSharedInvalidationState(void)
void
SharedInvalBackendInit(bool sendOnly)
{
- int index;
- ProcState *stateP = NULL;
+ ProcState *stateP;
+ pid_t oldPid;
SISeg *segP = shmInvalBuffer;
+ int pgprocno;
+
+ if (MyBackendId <= 0)
+ elog(ERROR, "MyBackendId not set");
+ if (MyBackendId > NumProcStateSlots)
+ elog(PANIC, "unexpected MyBackendId %d in SharedInvalBackendInit (max %d)",
+ MyBackendId, NumProcStateSlots);
+ pgprocno = MyBackendId - 1;
+ stateP = &segP->procState[pgprocno];
/*
* This can run in parallel with read operations, but not with write
- * operations, since SIInsertDataEntries relies on lastBackend to set
- * hasMessages appropriately.
+ * operations, since SIInsertDataEntries relies on the pgprocnos array to
+ * set hasMessages appropriately.
*/
LWLockAcquire(SInvalWriteLock, LW_EXCLUSIVE);
- /* Look for a free entry in the procState array */
- for (index = 0; index < segP->lastBackend; index++)
- {
- if (segP->procState[index].procPid == 0) /* inactive slot? */
- {
- stateP = &segP->procState[index];
- break;
- }
- }
-
- if (stateP == NULL)
+ oldPid = stateP->procPid;
+ if (oldPid != 0)
{
- if (segP->lastBackend < segP->maxBackends)
- {
- stateP = &segP->procState[segP->lastBackend];
- Assert(stateP->procPid == 0);
- segP->lastBackend++;
- }
- else
- {
- /*
- * out of procState slots: MaxBackends exceeded -- report normally
- */
- MyBackendId = InvalidBackendId;
- LWLockRelease(SInvalWriteLock);
- ereport(FATAL,
- (errcode(ERRCODE_TOO_MANY_CONNECTIONS),
- errmsg("sorry, too many clients already")));
- }
+ LWLockRelease(SInvalWriteLock);
+ elog(ERROR, "sinval slot for backend %d is already in use by process %d",
+ MyBackendId, (int) oldPid);
}
- MyBackendId = (stateP - &segP->procState[0]) + 1;
-
- /* Advertise assigned backend ID in MyProc */
- MyProc->backendId = MyBackendId;
+ shmInvalBuffer->pgprocnos[shmInvalBuffer->numProcs++] = pgprocno;
/* Fetch next local transaction ID into local memory */
nextLocalTransactionId = stateP->nextLXID;
/* mark myself active, with all extant messages already read */
stateP->procPid = MyProcPid;
- stateP->proc = MyProc;
stateP->nextMsgNum = segP->maxMsgNum;
stateP->resetState = false;
stateP->signaled = false;
@@ -328,8 +316,6 @@ SharedInvalBackendInit(bool sendOnly)
/* register exit routine to mark my entry inactive at exit */
on_shmem_exit(CleanupInvalidationState, PointerGetDatum(segP));
-
- elog(DEBUG4, "my backend ID is %d", MyBackendId);
}
/*
@@ -345,96 +331,36 @@ CleanupInvalidationState(int status, Datum arg)
{
SISeg *segP = (SISeg *) DatumGetPointer(arg);
ProcState *stateP;
+ int pgprocno = MyBackendId - 1;
int i;
Assert(PointerIsValid(segP));
LWLockAcquire(SInvalWriteLock, LW_EXCLUSIVE);
- stateP = &segP->procState[MyBackendId - 1];
+ stateP = &segP->procState[pgprocno];
/* Update next local transaction ID for next holder of this backendID */
stateP->nextLXID = nextLocalTransactionId;
/* Mark myself inactive */
stateP->procPid = 0;
- stateP->proc = NULL;
stateP->nextMsgNum = 0;
stateP->resetState = false;
stateP->signaled = false;
- /* Recompute index of last active backend */
- for (i = segP->lastBackend; i > 0; i--)
+ for (i = segP->numProcs - 1; i >= 0; i--)
{
- if (segP->procState[i - 1].procPid != 0)
- break;
- }
- segP->lastBackend = i;
-
- LWLockRelease(SInvalWriteLock);
-}
-
-/*
- * BackendIdGetProc
- * Get the PGPROC structure for a backend, given the backend ID.
- * The result may be out of date arbitrarily quickly, so the caller
- * must be careful about how this information is used. NULL is
- * returned if the backend is not active.
- */
-PGPROC *
-BackendIdGetProc(int backendID)
-{
- PGPROC *result = NULL;
- SISeg *segP = shmInvalBuffer;
-
- /* Need to lock out additions/removals of backends */
- LWLockAcquire(SInvalWriteLock, LW_SHARED);
-
- if (backendID > 0 && backendID <= segP->lastBackend)
- {
- ProcState *stateP = &segP->procState[backendID - 1];
-
- result = stateP->proc;
- }
-
- LWLockRelease(SInvalWriteLock);
-
- return result;
-}
-
-/*
- * BackendIdGetTransactionIds
- * Get the xid, xmin, nsubxid and overflow status of the backend. The
- * result may be out of date arbitrarily quickly, so the caller must be
- * careful about how this information is used.
- */
-void
-BackendIdGetTransactionIds(int backendID, TransactionId *xid,
- TransactionId *xmin, int *nsubxid, bool *overflowed)
-{
- SISeg *segP = shmInvalBuffer;
-
- *xid = InvalidTransactionId;
- *xmin = InvalidTransactionId;
- *nsubxid = 0;
- *overflowed = false;
-
- /* Need to lock out additions/removals of backends */
- LWLockAcquire(SInvalWriteLock, LW_SHARED);
-
- if (backendID > 0 && backendID <= segP->lastBackend)
- {
- ProcState *stateP = &segP->procState[backendID - 1];
- PGPROC *proc = stateP->proc;
-
- if (proc != NULL)
+ if (segP->pgprocnos[i] == pgprocno)
{
- *xid = proc->xid;
- *xmin = proc->xmin;
- *nsubxid = proc->subxidStatus.count;
- *overflowed = proc->subxidStatus.overflowed;
+ if (i != segP->numProcs - 1)
+ segP->pgprocnos[i] = segP->pgprocnos[segP->numProcs - 1];
+ break;
}
}
+ if (i < 0)
+ elog(PANIC, "could not find entry in sinval array");
+ segP->numProcs--;
LWLockRelease(SInvalWriteLock);
}
@@ -507,9 +433,9 @@ SIInsertDataEntries(const SharedInvalidationMessage *data, int n)
* these (unlocked) changes will be committed to memory before we exit
* the function.
*/
- for (i = 0; i < segP->lastBackend; i++)
+ for (i = 0; i < segP->numProcs; i++)
{
- ProcState *stateP = &segP->procState[i];
+ ProcState *stateP = &segP->procState[segP->pgprocnos[i]];
stateP->hasMessages = true;
}
@@ -677,13 +603,14 @@ SICleanupQueue(bool callerHasWriteLock, int minFree)
minsig = min - SIG_THRESHOLD;
lowbound = min - MAXNUMMESSAGES + minFree;
- for (i = 0; i < segP->lastBackend; i++)
+ for (i = 0; i < segP->numProcs; i++)
{
- ProcState *stateP = &segP->procState[i];
+ ProcState *stateP = &segP->procState[segP->pgprocnos[i]];
int n = stateP->nextMsgNum;
- /* Ignore if inactive or already in reset state */
- if (stateP->procPid == 0 || stateP->resetState || stateP->sendOnly)
+ /* Ignore if already in reset state */
+ Assert(stateP->procPid != 0);
+ if (stateP->resetState || stateP->sendOnly)
continue;
/*
@@ -719,11 +646,8 @@ SICleanupQueue(bool callerHasWriteLock, int minFree)
{
segP->minMsgNum -= MSGNUMWRAPAROUND;
segP->maxMsgNum -= MSGNUMWRAPAROUND;
- for (i = 0; i < segP->lastBackend; i++)
- {
- /* we don't bother skipping inactive entries here */
- segP->procState[i].nextMsgNum -= MSGNUMWRAPAROUND;
- }
+ for (i = 0; i < segP->numProcs; i++)
+ segP->procState[segP->pgprocnos[i]].nextMsgNum -= MSGNUMWRAPAROUND;
}
/*
diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c
index d8755a106d5..97d1ab65740 100644
--- a/src/backend/storage/ipc/standby.c
+++ b/src/backend/storage/ipc/standby.c
@@ -137,6 +137,7 @@ InitRecoveryTransactionEnvironment(void)
* are held by vxids and row level locks are held by xids. All queries
* hold AccessShareLocks so never block while we write or lock new rows.
*/
+ MyProc->vxid.backendId = MyBackendId;
vxid.backendId = MyBackendId;
vxid.localTransactionId = GetNextLocalTransactionId();
VirtualXactLockTableInsert(vxid);
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index c70a1adb9ad..e62968b4a86 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -3625,8 +3625,8 @@ GetLockStatusData(void)
proc->fpRelId[f]);
instance->holdMask = lockbits << FAST_PATH_LOCKNUMBER_OFFSET;
instance->waitLockMode = NoLock;
- instance->backend = proc->backendId;
- instance->lxid = proc->lxid;
+ instance->vxid.backendId = proc->vxid.backendId;
+ instance->vxid.localTransactionId = proc->vxid.lxid;
instance->pid = proc->pid;
instance->leaderPid = proc->pid;
instance->fastpath = true;
@@ -3652,15 +3652,15 @@ GetLockStatusData(void)
repalloc(data->locks, sizeof(LockInstanceData) * els);
}
- vxid.backendId = proc->backendId;
+ vxid.backendId = proc->vxid.backendId;
vxid.localTransactionId = proc->fpLocalTransactionId;
instance = &data->locks[el];
SET_LOCKTAG_VIRTUALTRANSACTION(instance->locktag, vxid);
instance->holdMask = LOCKBIT_ON(ExclusiveLock);
instance->waitLockMode = NoLock;
- instance->backend = proc->backendId;
- instance->lxid = proc->lxid;
+ instance->vxid.backendId = proc->vxid.backendId;
+ instance->vxid.localTransactionId = proc->vxid.lxid;
instance->pid = proc->pid;
instance->leaderPid = proc->pid;
instance->fastpath = true;
@@ -3712,8 +3712,8 @@ GetLockStatusData(void)
instance->waitLockMode = proc->waitLockMode;
else
instance->waitLockMode = NoLock;
- instance->backend = proc->backendId;
- instance->lxid = proc->lxid;
+ instance->vxid.backendId = proc->vxid.backendId;
+ instance->vxid.localTransactionId = proc->vxid.lxid;
instance->pid = proc->pid;
instance->leaderPid = proclock->groupLeader->pid;
instance->fastpath = false;
@@ -3888,8 +3888,8 @@ GetSingleProcBlockerStatusData(PGPROC *blocked_proc, BlockedProcsData *data)
instance->waitLockMode = proc->waitLockMode;
else
instance->waitLockMode = NoLock;
- instance->backend = proc->backendId;
- instance->lxid = proc->lxid;
+ instance->vxid.backendId = proc->vxid.backendId;
+ instance->vxid.localTransactionId = proc->vxid.lxid;
instance->pid = proc->pid;
instance->leaderPid = proclock->groupLeader->pid;
instance->fastpath = false;
@@ -4374,8 +4374,8 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
* lockers, as we haven't advertised this vxid via the ProcArray yet.
*
* Since MyProc->fpLocalTransactionId will normally contain the same data
- * as MyProc->lxid, you might wonder if we really need both. The
- * difference is that MyProc->lxid is set and cleared unlocked, and
+ * as MyProc->vxid.lxid, you might wonder if we really need both. The
+ * difference is that MyProc->vxid.lxid is set and cleared unlocked, and
* examined by procarray.c, while fpLocalTransactionId is protected by
* fpInfoLock and is used only by the locking subsystem. Doing it this
* way makes it easier to verify that there are no funny race conditions.
@@ -4391,7 +4391,7 @@ VirtualXactLockTableInsert(VirtualTransactionId vxid)
LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE);
- Assert(MyProc->backendId == vxid.backendId);
+ Assert(MyProc->vxid.backendId == vxid.backendId);
Assert(MyProc->fpLocalTransactionId == InvalidLocalTransactionId);
Assert(MyProc->fpVXIDLock == false);
@@ -4413,7 +4413,7 @@ VirtualXactLockTableCleanup(void)
bool fastpath;
LocalTransactionId lxid;
- Assert(MyProc->backendId != InvalidBackendId);
+ Assert(MyProc->vxid.backendId != InvalidBackendId);
/*
* Clean up shared memory state.
@@ -4541,7 +4541,7 @@ VirtualXactLock(VirtualTransactionId vxid, bool wait)
*/
LWLockAcquire(&proc->fpInfoLock, LW_EXCLUSIVE);
- if (proc->backendId != vxid.backendId
+ if (proc->vxid.backendId != vxid.backendId
|| proc->fpLocalTransactionId != vxid.localTransactionId)
{
/* VXID ended */
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index fddf44015fd..4226e41d80c 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -240,25 +240,25 @@ InitProcGlobal(void)
if (i < MaxConnections)
{
/* PGPROC for normal backend, add to freeProcs list */
- dlist_push_head(&ProcGlobal->freeProcs, &proc->links);
+ dlist_push_tail(&ProcGlobal->freeProcs, &proc->links);
proc->procgloballist = &ProcGlobal->freeProcs;
}
else if (i < MaxConnections + autovacuum_max_workers + 1)
{
/* PGPROC for AV launcher/worker, add to autovacFreeProcs list */
- dlist_push_head(&ProcGlobal->autovacFreeProcs, &proc->links);
+ dlist_push_tail(&ProcGlobal->autovacFreeProcs, &proc->links);
proc->procgloballist = &ProcGlobal->autovacFreeProcs;
}
else if (i < MaxConnections + autovacuum_max_workers + 1 + max_worker_processes)
{
/* PGPROC for bgworker, add to bgworkerFreeProcs list */
- dlist_push_head(&ProcGlobal->bgworkerFreeProcs, &proc->links);
+ dlist_push_tail(&ProcGlobal->bgworkerFreeProcs, &proc->links);
proc->procgloballist = &ProcGlobal->bgworkerFreeProcs;
}
else if (i < MaxBackends)
{
/* PGPROC for walsender, add to walsenderFreeProcs list */
- dlist_push_head(&ProcGlobal->walsenderFreeProcs, &proc->links);
+ dlist_push_tail(&ProcGlobal->walsenderFreeProcs, &proc->links);
proc->procgloballist = &ProcGlobal->walsenderFreeProcs;
}
@@ -353,6 +353,7 @@ InitProcess(void)
errmsg("sorry, too many clients already")));
}
MyProcNumber = GetNumberFromPGProc(MyProc);
+ MyBackendId = GetBackendIdFromPGProc(MyProc);
/*
* Cross-check that the PGPROC is of the type we expect; if this were not
@@ -375,14 +376,14 @@ InitProcess(void)
*/
dlist_node_init(&MyProc->links);
MyProc->waitStatus = PROC_WAIT_STATUS_OK;
- MyProc->lxid = InvalidLocalTransactionId;
MyProc->fpVXIDLock = false;
MyProc->fpLocalTransactionId = InvalidLocalTransactionId;
MyProc->xid = InvalidTransactionId;
MyProc->xmin = InvalidTransactionId;
MyProc->pid = MyProcPid;
- /* backendId, databaseId and roleId will be filled in later */
- MyProc->backendId = InvalidBackendId;
+ MyProc->vxid.backendId = MyBackendId;
+ MyProc->vxid.lxid = InvalidLocalTransactionId;
+ /* databaseId and roleId will be filled in later */
MyProc->databaseId = InvalidOid;
MyProc->roleId = InvalidOid;
MyProc->tempNamespaceId = InvalidOid;
@@ -562,11 +563,11 @@ InitAuxiliaryProcess(void)
/* use volatile pointer to prevent code rearrangement */
((volatile PGPROC *) auxproc)->pid = MyProcPid;
- MyProc = auxproc;
-
SpinLockRelease(ProcStructLock);
+ MyProc = auxproc;
MyProcNumber = GetNumberFromPGProc(MyProc);
+ MyBackendId = GetBackendIdFromPGProc(MyProc);
/*
* Initialize all fields of MyProc, except for those previously
@@ -574,12 +575,12 @@ InitAuxiliaryProcess(void)
*/
dlist_node_init(&MyProc->links);
MyProc->waitStatus = PROC_WAIT_STATUS_OK;
- MyProc->lxid = InvalidLocalTransactionId;
MyProc->fpVXIDLock = false;
MyProc->fpLocalTransactionId = InvalidLocalTransactionId;
MyProc->xid = InvalidTransactionId;
MyProc->xmin = InvalidTransactionId;
- MyProc->backendId = InvalidBackendId;
+ MyProc->vxid.backendId = InvalidBackendId;
+ MyProc->vxid.lxid = InvalidLocalTransactionId;
MyProc->databaseId = InvalidOid;
MyProc->roleId = InvalidOid;
MyProc->tempNamespaceId = InvalidOid;
@@ -910,8 +911,14 @@ ProcKill(int code, Datum arg)
proc = MyProc;
MyProc = NULL;
MyProcNumber = INVALID_PGPROCNO;
+ MyBackendId = InvalidBackendId;
DisownLatch(&proc->procLatch);
+ /* Mark the proc no longer in use */
+ proc->pid = 0;
+ proc->vxid.backendId = InvalidBackendId;
+ proc->vxid.lxid = InvalidTransactionId;
+
procgloballist = proc->procgloballist;
SpinLockAcquire(ProcStructLock);
@@ -982,6 +989,7 @@ AuxiliaryProcKill(int code, Datum arg)
proc = MyProc;
MyProc = NULL;
MyProcNumber = INVALID_PGPROCNO;
+ MyBackendId = InvalidBackendId;
DisownLatch(&proc->procLatch);
SpinLockAcquire(ProcStructLock);
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 1a1050c8da1..3d3f7b06723 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -19,6 +19,7 @@
#include "port/atomics.h" /* for memory barriers */
#include "storage/ipc.h"
#include "storage/proc.h" /* for MyProc */
+#include "storage/procarray.h"
#include "storage/sinvaladt.h"
#include "utils/ascii.h"
#include "utils/backend_status.h"
@@ -29,13 +30,12 @@
/* ----------
* Total number of backends including auxiliary
*
- * We reserve a slot for each possible BackendId, plus one for each
- * possible auxiliary process type. (This scheme assumes there is not
- * more than one of any auxiliary process type at a time.) MaxBackends
- * includes autovacuum workers and background workers as well.
+ * We reserve a slot for each possible PGPROC entry, including aux processes.
+ * (But not including PGPROC entries reserved for prepared xacts; they are not
+ * real processes.)
* ----------
*/
-#define NumBackendStatSlots (MaxBackends + NUM_AUXPROCTYPES)
+#define NumBackendStatSlots (MaxBackends + NUM_AUXILIARY_PROCS)
/* ----------
@@ -238,10 +238,9 @@ CreateSharedBackendStatus(void)
/*
* Initialize pgstats backend activity state, and set up our on-proc-exit
- * hook. Called from InitPostgres and AuxiliaryProcessMain. For auxiliary
- * process, MyBackendId is invalid. Otherwise, MyBackendId must be set, but we
- * must not have started any transaction yet (since the exit hook must run
- * after the last transaction exit).
+ * hook. Called from InitPostgres and AuxiliaryProcessMain. MyBackendId must
+ * be set, but we must not have started any transaction yet (since the exit
+ * hook must run after the last transaction exit).
*
* NOTE: MyDatabaseId isn't set yet; so the shutdown hook has to be careful.
*/
@@ -249,26 +248,9 @@ void
pgstat_beinit(void)
{
/* Initialize MyBEEntry */
- if (MyBackendId != InvalidBackendId)
- {
- Assert(MyBackendId >= 1 && MyBackendId <= MaxBackends);
- MyBEEntry = &BackendStatusArray[MyBackendId - 1];
- }
- else
- {
- /* Must be an auxiliary process */
- Assert(MyAuxProcType != NotAnAuxProcess);
-
- /*
- * Assign the MyBEEntry for an auxiliary process. Since it doesn't
- * have a BackendId, the slot is statically allocated based on the
- * auxiliary process type (MyAuxProcType). Backends use slots indexed
- * in the range from 0 to MaxBackends (exclusive), so we use
- * MaxBackends + AuxProcType as the index of the slot for an auxiliary
- * process.
- */
- MyBEEntry = &BackendStatusArray[MaxBackends + MyAuxProcType];
- }
+ Assert(MyBackendId != InvalidBackendId);
+ Assert(MyBackendId >= 1 && MyBackendId <= NumBackendStatSlots);
+ MyBEEntry = &BackendStatusArray[MyBackendId - 1];
/* Set up a process-exit hook to clean up */
on_shmem_exit(pgstat_beshutdown_hook, 0);
@@ -281,12 +263,12 @@ pgstat_beinit(void)
* Initialize this backend's entry in the PgBackendStatus array.
* Called from InitPostgres.
*
- * Apart from auxiliary processes, MyBackendId, MyDatabaseId,
- * session userid, and application_name must be set for a
- * backend (hence, this cannot be combined with pgstat_beinit).
- * Note also that we must be inside a transaction if this isn't an aux
- * process, as we may need to do encoding conversion on some strings.
- * ----------
+ * Apart from auxiliary processes, MyDatabaseId, session userid, and
+ * application_name must already be set (hence, this cannot be combined
+ * with pgstat_beinit). Note also that we must be inside a transaction
+ * if this isn't an aux process, as we may need to do encoding conversion
+ * on some strings.
+ *----------
*/
void
pgstat_bestart(void)
diff --git a/src/backend/utils/adt/lockfuncs.c b/src/backend/utils/adt/lockfuncs.c
index 4b49f7fe3d8..bbe5cc0806e 100644
--- a/src/backend/utils/adt/lockfuncs.c
+++ b/src/backend/utils/adt/lockfuncs.c
@@ -353,7 +353,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
break;
}
- values[10] = VXIDGetDatum(instance->backend, instance->lxid);
+ values[10] = VXIDGetDatum(instance->vxid.backendId, instance->vxid.localTransactionId);
if (instance->pid != 0)
values[11] = Int32GetDatum(instance->pid);
else
diff --git a/src/backend/utils/adt/mcxtfuncs.c b/src/backend/utils/adt/mcxtfuncs.c
index 4708d73f5fa..a7267dc15d1 100644
--- a/src/backend/utils/adt/mcxtfuncs.c
+++ b/src/backend/utils/adt/mcxtfuncs.c
@@ -148,19 +148,11 @@ pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
PGPROC *proc;
BackendId backendId = InvalidBackendId;
- proc = BackendPidGetProc(pid);
-
/*
* See if the process with given pid is a backend or an auxiliary process.
- *
- * If the given process is a backend, use its backend id in
- * SendProcSignal() later to speed up the operation. Otherwise, don't do
- * that because auxiliary processes (except the startup process) don't
- * have a valid backend id.
*/
- if (proc != NULL)
- backendId = proc->backendId;
- else
+ proc = BackendPidGetProc(pid);
+ if (proc == NULL)
proc = AuxiliaryPidGetProc(pid);
/*
@@ -183,6 +175,8 @@ pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(false);
}
+ if (proc != NULL)
+ backendId = GetBackendIdFromPGProc(proc);
if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, backendId) < 0)
{
/* Again, just a warning to allow loops */
diff --git a/src/backend/utils/error/csvlog.c b/src/backend/utils/error/csvlog.c
index 1b62b07f231..1d44d8a6a31 100644
--- a/src/backend/utils/error/csvlog.c
+++ b/src/backend/utils/error/csvlog.c
@@ -152,8 +152,8 @@ write_csvlog(ErrorData *edata)
/* Virtual transaction id */
/* keep VXID format in sync with lockfuncs.c */
- if (MyProc != NULL && MyProc->backendId != InvalidBackendId)
- appendStringInfo(&buf, "%d/%u", MyProc->backendId, MyProc->lxid);
+ if (MyProc != NULL && MyProc->vxid.backendId != InvalidBackendId)
+ appendStringInfo(&buf, "%d/%u", MyProc->vxid.backendId, MyProc->vxid.lxid);
appendStringInfoChar(&buf, ',');
/* Transaction id */
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 700fbde6db4..208e1bc0ee4 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -3078,18 +3078,18 @@ log_status_format(StringInfo buf, const char *format, ErrorData *edata)
break;
case 'v':
/* keep VXID format in sync with lockfuncs.c */
- if (MyProc != NULL && MyProc->backendId != InvalidBackendId)
+ if (MyProc != NULL && MyProc->vxid.backendId != InvalidBackendId)
{
if (padding != 0)
{
char strfbuf[128];
snprintf(strfbuf, sizeof(strfbuf) - 1, "%d/%u",
- MyProc->backendId, MyProc->lxid);
+ MyProc->vxid.backendId, MyProc->vxid.lxid);
appendStringInfo(buf, "%*s", padding, strfbuf);
}
else
- appendStringInfo(buf, "%d/%u", MyProc->backendId, MyProc->lxid);
+ appendStringInfo(buf, "%d/%u", MyProc->vxid.backendId, MyProc->vxid.lxid);
}
else if (padding != 0)
appendStringInfoSpaces(buf,
diff --git a/src/backend/utils/error/jsonlog.c b/src/backend/utils/error/jsonlog.c
index 2903561f1c4..067d9e30b16 100644
--- a/src/backend/utils/error/jsonlog.c
+++ b/src/backend/utils/error/jsonlog.c
@@ -197,9 +197,9 @@ write_jsonlog(ErrorData *edata)
/* Virtual transaction id */
/* keep VXID format in sync with lockfuncs.c */
- if (MyProc != NULL && MyProc->backendId != InvalidBackendId)
- appendJSONKeyValueFmt(&buf, "vxid", true, "%d/%u", MyProc->backendId,
- MyProc->lxid);
+ if (MyProc != NULL && MyProc->vxid.backendId != InvalidBackendId)
+ appendJSONKeyValueFmt(&buf, "vxid", true, "%d/%u",
+ MyProc->vxid.backendId, MyProc->vxid.lxid);
/* Transaction id */
appendJSONKeyValueFmt(&buf, "txid", false, "%u",
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index 1ad33671598..e822cd61d5f 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -740,18 +740,10 @@ InitPostgres(const char *in_dbname, Oid dboid,
/*
* Initialize my entry in the shared-invalidation manager's array of
* per-backend data.
- *
- * Sets up MyBackendId, a unique backend identifier.
*/
- MyBackendId = InvalidBackendId;
-
SharedInvalBackendInit(false);
- if (MyBackendId > MaxBackends || MyBackendId <= 0)
- elog(FATAL, "bad backend ID: %d", MyBackendId);
-
- /* Now that we have a BackendId, we can participate in ProcSignal */
- ProcSignalInit(MyBackendId);
+ ProcSignalInit();
/*
* Also set up timeout handlers needed for backend operation. We need
diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
index 675e81d82d7..a0916959b17 100644
--- a/src/backend/utils/time/snapmgr.c
+++ b/src/backend/utils/time/snapmgr.c
@@ -1154,7 +1154,8 @@ ExportSnapshot(Snapshot snapshot)
* inside the transaction from 1.
*/
snprintf(path, sizeof(path), SNAPSHOT_EXPORT_DIR "/%08X-%08X-%d",
- MyProc->backendId, MyProc->lxid, list_length(exportedSnapshots) + 1);
+ MyProc->vxid.backendId, MyProc->vxid.lxid,
+ list_length(exportedSnapshots) + 1);
/*
* Copy the snapshot into TopTransactionContext, add it to the
@@ -1181,7 +1182,7 @@ ExportSnapshot(Snapshot snapshot)
*/
initStringInfo(&buf);
- appendStringInfo(&buf, "vxid:%d/%u\n", MyProc->backendId, MyProc->lxid);
+ appendStringInfo(&buf, "vxid:%d/%u\n", MyProc->vxid.backendId, MyProc->vxid.lxid);
appendStringInfo(&buf, "pid:%d\n", MyProcPid);
appendStringInfo(&buf, "dbid:%u\n", MyDatabaseId);
appendStringInfo(&buf, "iso:%d\n", XactIsoLevel);
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 0b01c1f0935..cbdc61b8576 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -444,8 +444,6 @@ typedef enum
WalWriterProcess,
WalReceiverProcess,
WalSummarizerProcess,
-
- NUM_AUXPROCTYPES /* Must be last! */
} AuxProcType;
extern PGDLLIMPORT AuxProcType MyAuxProcType;
diff --git a/src/include/storage/backendid.h b/src/include/storage/backendid.h
index 50ac982da19..01387723f79 100644
--- a/src/include/storage/backendid.h
+++ b/src/include/storage/backendid.h
@@ -14,11 +14,15 @@
#ifndef BACKENDID_H
#define BACKENDID_H
-/* ----------------
- * -cim 8/17/90
- * ----------------
+/*
+ * BackendId uniquely identifies an active backend or auxiliary process. It's
+ * assigned at backend startup after authentication. Note that a backend ID
+ * can be reused for a different backend immediately after a backend exits.
+ *
+ * Backend IDs are assigned starting from 1. For historical reasons, BackendId
+ * 0 is unused, but InvalidBackendId is defined as -1.
*/
-typedef int BackendId; /* unique currently active backend identifier */
+typedef int BackendId;
#define InvalidBackendId (-1)
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index ed6071f3286..c5c84201378 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -74,9 +74,9 @@ typedef struct
#define SetInvalidVirtualTransactionId(vxid) \
((vxid).backendId = InvalidBackendId, \
(vxid).localTransactionId = InvalidLocalTransactionId)
-#define GET_VXID_FROM_PGPROC(vxid, proc) \
- ((vxid).backendId = (proc).backendId, \
- (vxid).localTransactionId = (proc).lxid)
+#define GET_VXID_FROM_PGPROC(vxid_dst, proc) \
+ ((vxid_dst).backendId = (proc).vxid.backendId, \
+ (vxid_dst).localTransactionId = (proc).vxid.lxid)
/* MAX_LOCKMODES cannot be larger than the # of bits in LOCKMASK */
#define MAX_LOCKMODES 10
@@ -454,8 +454,7 @@ typedef struct LockInstanceData
LOCKTAG locktag; /* tag for locked object */
LOCKMASK holdMask; /* locks held by this PGPROC */
LOCKMODE waitLockMode; /* lock awaited by this PGPROC, if any */
- BackendId backend; /* backend ID of this PGPROC */
- LocalTransactionId lxid; /* local transaction ID of this PGPROC */
+ VirtualTransactionId vxid; /* local transaction ID of this PGPROC */
TimestampTz waitStart; /* time at which this PGPROC started waiting
* for lock */
int pid; /* pid of this PGPROC */
diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h
index ca21518f522..d1e50380b7c 100644
--- a/src/include/storage/proc.h
+++ b/src/include/storage/proc.h
@@ -186,16 +186,31 @@ struct PGPROC
* vacuum must not remove tuples deleted by
* xid >= xmin ! */
- LocalTransactionId lxid; /* local id of top-level transaction currently
- * being executed by this proc, if running;
- * else InvalidLocalTransactionId */
int pid; /* Backend's process ID; 0 if prepared xact */
int pgxactoff; /* offset into various ProcGlobal->arrays with
* data mirrored from this PGPROC */
+ /*
+ * Currently running top-level transaction's virtual xid. Together these
+ * form a VirtualTransactionId, but we don't use that struct because this
+ * is not atomically assignable as whole, and we want to enforce code to
+ * consider both parts separately. See comments at VirtualTransactionId.
+ */
+ struct {
+ BackendId backendId; /* For regular backends, equal to
+ * GetBackendIdFromPGProc(proc). For
+ * prepared xacts, ID of the original
+ * backend that processed the
+ * transaction. For unused PGPROC entries,
+ * InvalidbackendID. */
+ LocalTransactionId lxid; /* local id of top-level transaction
+ * currently * being executed by this
+ * proc, if running; else
+ * InvalidLocaltransactionId */
+ } vxid;
+
/* These fields are zero while a backend is still starting up: */
- BackendId backendId; /* This backend's backend ID (if assigned) */
Oid databaseId; /* OID of database this backend is using */
Oid roleId; /* OID of role using this backend */
@@ -406,9 +421,16 @@ extern PGDLLIMPORT PROC_HDR *ProcGlobal;
extern PGDLLIMPORT PGPROC *PreparedXactProcs;
-/* Accessor for PGPROC given a pgprocno. */
+/*
+ * Accessors for getting PGPROC given a pgprocno or BackendId, and vice versa.
+ *
+ * For historical reasons, some code uses 0-based "proc numbers", while other
+ * code uses 1-based backend IDs.
+ */
#define GetPGProcByNumber(n) (&ProcGlobal->allProcs[(n)])
#define GetNumberFromPGProc(proc) ((proc) - &ProcGlobal->allProcs[0])
+#define GetPGProcByBackendId(n) (&ProcGlobal->allProcs[(n) - 1])
+#define GetBackendIdFromPGProc(proc) (GetNumberFromPGProc(proc) + 1)
/*
* We set aside some extra PGPROC structures for auxiliary processes,
diff --git a/src/include/storage/procarray.h b/src/include/storage/procarray.h
index f3eba9b7640..3af7577e8c6 100644
--- a/src/include/storage/procarray.h
+++ b/src/include/storage/procarray.h
@@ -64,6 +64,10 @@ extern VirtualTransactionId *GetVirtualXIDsDelayingChkpt(int *nvxids, int type);
extern bool HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids,
int nvxids, int type);
+extern PGPROC *BackendIdGetProc(int backendID);
+extern void BackendIdGetTransactionIds(int backendID, TransactionId *xid,
+ TransactionId *xmin, int *nsubxid,
+ bool *overflowed);
extern PGPROC *BackendPidGetProc(int pid);
extern PGPROC *BackendPidGetProcWithLock(int pid);
extern int BackendXidGetPid(TransactionId xid);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 52dcb4c2adf..febdda3611c 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -62,7 +62,7 @@ typedef enum
extern Size ProcSignalShmemSize(void);
extern void ProcSignalShmemInit(void);
-extern void ProcSignalInit(int pss_idx);
+extern void ProcSignalInit(void);
extern int SendProcSignal(pid_t pid, ProcSignalReason reason,
BackendId backendId);
diff --git a/src/include/storage/sinvaladt.h b/src/include/storage/sinvaladt.h
index aa3d203efca..c3c97b3f8b7 100644
--- a/src/include/storage/sinvaladt.h
+++ b/src/include/storage/sinvaladt.h
@@ -31,10 +31,6 @@
extern Size SInvalShmemSize(void);
extern void CreateSharedInvalidationState(void);
extern void SharedInvalBackendInit(bool sendOnly);
-extern PGPROC *BackendIdGetProc(int backendID);
-extern void BackendIdGetTransactionIds(int backendID, TransactionId *xid,
- TransactionId *xmin, int *nsubxid,
- bool *overflowed);
extern void SIInsertDataEntries(const SharedInvalidationMessage *data, int n);
extern int SIGetDataEntries(SharedInvalidationMessage *data, int datasize);
diff --git a/src/pl/plpgsql/src/pl_exec.c b/src/pl/plpgsql/src/pl_exec.c
index 6d1691340c5..ed51694428a 100644
--- a/src/pl/plpgsql/src/pl_exec.c
+++ b/src/pl/plpgsql/src/pl_exec.c
@@ -2211,7 +2211,7 @@ exec_stmt_call(PLpgSQL_execstate *estate, PLpgSQL_stmt_call *stmt)
paramLI = setup_param_list(estate, expr);
- before_lxid = MyProc->lxid;
+ before_lxid = MyProc->vxid.lxid;
/*
* If we have a procedure-lifespan resowner, use that to hold the refcount
@@ -2232,7 +2232,7 @@ exec_stmt_call(PLpgSQL_execstate *estate, PLpgSQL_stmt_call *stmt)
elog(ERROR, "SPI_execute_plan_extended failed executing query \"%s\": %s",
expr->query, SPI_result_code_string(rc));
- after_lxid = MyProc->lxid;
+ after_lxid = MyProc->vxid.lxid;
if (before_lxid != after_lxid)
{
@@ -6037,7 +6037,7 @@ exec_eval_simple_expr(PLpgSQL_execstate *estate,
int32 *rettypmod)
{
ExprContext *econtext = estate->eval_econtext;
- LocalTransactionId curlxid = MyProc->lxid;
+ LocalTransactionId curlxid = MyProc->vxid.lxid;
ParamListInfo paramLI;
void *save_setup_arg;
bool need_snapshot;
@@ -7943,7 +7943,7 @@ get_cast_hashentry(PLpgSQL_execstate *estate,
* functions do; DO blocks have private simple_eval_estates, and private
* cast hash tables to go with them.)
*/
- curlxid = MyProc->lxid;
+ curlxid = MyProc->vxid.lxid;
if (cast_entry->cast_lxid != curlxid || cast_entry->cast_in_use)
{
oldcontext = MemoryContextSwitchTo(estate->simple_eval_estate->es_query_cxt);
@@ -8070,7 +8070,7 @@ exec_simple_check_plan(PLpgSQL_execstate *estate, PLpgSQL_expr *expr)
/* Remember that we have the refcount */
expr->expr_simple_plansource = plansource;
expr->expr_simple_plan = cplan;
- expr->expr_simple_plan_lxid = MyProc->lxid;
+ expr->expr_simple_plan_lxid = MyProc->vxid.lxid;
/* Share the remaining work with the replan code path */
exec_save_simple_expr(expr, cplan);
--
2.39.2
v10-0003-Use-0-based-indexing-for-BackendIds.patchtext/x-patch; charset=UTF-8; name=v10-0003-Use-0-based-indexing-for-BackendIds.patchDownload
From b00703d869aaf5c0e76f6e8a15adeb1e8b99fbc1 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Thu, 8 Feb 2024 12:58:08 +0200
Subject: [PATCH v10 3/3] Use 0-based indexing for BackendIds
---
doc/src/sgml/monitoring.sgml | 2 +-
src/backend/access/transam/clog.c | 2 +-
src/backend/access/transam/multixact.c | 19 ++++++++-----------
src/backend/access/transam/twophase.c | 4 ++--
src/backend/access/transam/xlog.c | 2 +-
src/backend/commands/async.c | 12 ++++++------
src/backend/postmaster/bgwriter.c | 2 +-
src/backend/postmaster/pgarch.c | 2 +-
src/backend/postmaster/walsummarizer.c | 2 +-
src/backend/storage/buffer/bufmgr.c | 6 +++---
src/backend/storage/ipc/procarray.c | 10 +++++-----
src/backend/storage/ipc/procsignal.c | 8 ++++----
src/backend/storage/ipc/sinvaladt.c | 19 ++++++++-----------
src/backend/storage/lmgr/condition_variable.c | 12 ++++++------
src/backend/storage/lmgr/lwlock.c | 6 +++---
src/backend/storage/lmgr/predicate.c | 2 +-
src/backend/storage/lmgr/proc.c | 9 ++-------
src/backend/utils/activity/backend_status.c | 12 ++++++------
src/backend/utils/adt/mcxtfuncs.c | 2 +-
src/include/storage/backendid.h | 6 ++++--
src/include/storage/proc.h | 11 +++--------
21 files changed, 68 insertions(+), 82 deletions(-)
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 5cf9363ac82..adec1b92a31 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4933,7 +4933,7 @@ description | Waiting for a newly initialized WAL file to reach durable storage
access functions can be used; these are shown in <xref
linkend="monitoring-stats-backend-funcs-table"/>.
These access functions use the session's backend ID number, which is a
- small positive integer that is distinct from the backend ID of any
+ small non-negative integer that is distinct from the backend ID of any
concurrent session, although a session's ID can be recycled as soon as
it exits. The backend ID is used, among other things, to identify the
session's temporary schema if it has one.
diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c
index 97f7434da34..7b79067efb1 100644
--- a/src/backend/access/transam/clog.c
+++ b/src/backend/access/transam/clog.c
@@ -425,7 +425,7 @@ TransactionGroupUpdateXidStatus(TransactionId xid, XidStatus status,
{
volatile PROC_HDR *procglobal = ProcGlobal;
PGPROC *proc = MyProc;
- int pgprocno = MyProcNumber;
+ int pgprocno = MyBackendId;
uint32 nextidx;
uint32 wakeidx;
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index febc429f724..f577008f4bd 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -238,9 +238,8 @@ typedef struct MultiXactStateData
* immediately following the MultiXactStateData struct. Each is indexed by
* BackendId.
*
- * In both arrays, there's a slot for all normal backends (1..MaxBackends)
- * followed by a slot for max_prepared_xacts prepared transactions. Valid
- * BackendIds start from 1; element zero of each array is never used.
+ * In both arrays, there's a slot for all normal backends (0..MaxBackends-1)
+ * followed by a slot for max_prepared_xacts prepared transactions.
*
* OldestMemberMXactId[k] is the oldest MultiXactId each backend's current
* transaction(s) could possibly be a member of, or InvalidMultiXactId
@@ -284,8 +283,7 @@ typedef struct MultiXactStateData
} MultiXactStateData;
/*
- * Last element of OldestMemberMXactId and OldestVisibleMXactId arrays.
- * Valid elements are (1..MaxOldestSlot); element 0 is never used.
+ * Size of OldestMemberMXactId and OldestVisibleMXactId arrays.
*/
#define MaxOldestSlot (MaxBackends + max_prepared_xacts)
@@ -698,7 +696,7 @@ MultiXactIdSetOldestVisible(void)
if (oldestMXact < FirstMultiXactId)
oldestMXact = FirstMultiXactId;
- for (i = 1; i <= MaxOldestSlot; i++)
+ for (i = 0; i < MaxOldestSlot; i++)
{
MultiXactId thisoldest = OldestMemberMXactId[i];
@@ -1828,9 +1826,9 @@ MultiXactShmemSize(void)
{
Size size;
- /* We need 2*MaxOldestSlot + 1 perBackendXactIds[] entries */
+ /* We need 2*MaxOldestSlot perBackendXactIds[] entries */
#define SHARED_MULTIXACT_STATE_SIZE \
- add_size(offsetof(MultiXactStateData, perBackendXactIds) + sizeof(MultiXactId), \
+ add_size(offsetof(MultiXactStateData, perBackendXactIds), \
mul_size(sizeof(MultiXactId) * 2, MaxOldestSlot))
size = SHARED_MULTIXACT_STATE_SIZE;
@@ -1880,8 +1878,7 @@ MultiXactShmemInit(void)
Assert(found);
/*
- * Set up array pointers. Note that perBackendXactIds[0] is wasted space
- * since we only use indexes 1..MaxOldestSlot in each array.
+ * Set up array pointers.
*/
OldestMemberMXactId = MultiXactState->perBackendXactIds;
OldestVisibleMXactId = OldestMemberMXactId + MaxOldestSlot;
@@ -2533,7 +2530,7 @@ GetOldestMultiXactId(void)
nextMXact = FirstMultiXactId;
oldestMXact = nextMXact;
- for (i = 1; i <= MaxOldestSlot; i++)
+ for (i = 0; i < MaxOldestSlot; i++)
{
MultiXactId thisoldest;
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 5c282002900..709353d4056 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -895,7 +895,7 @@ TwoPhaseGetXidByVirtualXID(VirtualTransactionId vxid,
* Get the dummy backend ID for prepared transaction specified by XID
*
* Dummy backend IDs are similar to real backend IDs of real backends.
- * They start at MaxBackends + 1, and are unique across all currently active
+ * They start at MaxBackends, and are unique across all currently active
* real backends and prepared transactions. If lock_held is set to true,
* TwoPhaseStateLock will not be taken, so the caller had better hold it.
*/
@@ -904,7 +904,7 @@ TwoPhaseGetDummyBackendId(TransactionId xid, bool lock_held)
{
GlobalTransaction gxact = TwoPhaseGetGXact(xid, lock_held);
- return gxact->pgprocno + 1;
+ return gxact->pgprocno;
}
/*
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 1e4abbca8be..e401684f82f 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -1379,7 +1379,7 @@ WALInsertLockAcquire(void)
static int lockToTry = -1;
if (lockToTry == -1)
- lockToTry = MyProcNumber % NUM_XLOGINSERT_LOCKS;
+ lockToTry = MyBackendId % NUM_XLOGINSERT_LOCKS;
MyLockNo = lockToTry;
/*
diff --git a/src/backend/commands/async.c b/src/backend/commands/async.c
index 8b24b222931..5aa893c9141 100644
--- a/src/backend/commands/async.c
+++ b/src/backend/commands/async.c
@@ -1094,7 +1094,7 @@ Exec_ListenPreCommit(void)
head = QUEUE_HEAD;
max = QUEUE_TAIL;
prevListener = InvalidBackendId;
- for (BackendId i = QUEUE_FIRST_LISTENER; i > 0; i = QUEUE_NEXT_LISTENER(i))
+ for (BackendId i = QUEUE_FIRST_LISTENER; i != InvalidBackendId; i = QUEUE_NEXT_LISTENER(i))
{
if (QUEUE_BACKEND_DBOID(i) == MyDatabaseId)
max = QUEUE_POS_MAX(max, QUEUE_BACKEND_POS(i));
@@ -1106,7 +1106,7 @@ Exec_ListenPreCommit(void)
QUEUE_BACKEND_PID(MyBackendId) = MyProcPid;
QUEUE_BACKEND_DBOID(MyBackendId) = MyDatabaseId;
/* Insert backend into list of listeners at correct position */
- if (prevListener > 0)
+ if (prevListener != InvalidBackendId)
{
QUEUE_NEXT_LISTENER(MyBackendId) = QUEUE_NEXT_LISTENER(prevListener);
QUEUE_NEXT_LISTENER(prevListener) = MyBackendId;
@@ -1253,7 +1253,7 @@ asyncQueueUnregister(void)
QUEUE_FIRST_LISTENER = QUEUE_NEXT_LISTENER(MyBackendId);
else
{
- for (BackendId i = QUEUE_FIRST_LISTENER; i > 0; i = QUEUE_NEXT_LISTENER(i))
+ for (BackendId i = QUEUE_FIRST_LISTENER; i != InvalidBackendId; i = QUEUE_NEXT_LISTENER(i))
{
if (QUEUE_NEXT_LISTENER(i) == MyBackendId)
{
@@ -1533,7 +1533,7 @@ asyncQueueFillWarning(void)
QueuePosition min = QUEUE_HEAD;
int32 minPid = InvalidPid;
- for (BackendId i = QUEUE_FIRST_LISTENER; i > 0; i = QUEUE_NEXT_LISTENER(i))
+ for (BackendId i = QUEUE_FIRST_LISTENER; i != InvalidBackendId; i = QUEUE_NEXT_LISTENER(i))
{
Assert(QUEUE_BACKEND_PID(i) != InvalidPid);
min = QUEUE_POS_MIN(min, QUEUE_BACKEND_POS(i));
@@ -1589,7 +1589,7 @@ SignalBackends(void)
count = 0;
LWLockAcquire(NotifyQueueLock, LW_EXCLUSIVE);
- for (BackendId i = QUEUE_FIRST_LISTENER; i > 0; i = QUEUE_NEXT_LISTENER(i))
+ for (BackendId i = QUEUE_FIRST_LISTENER; i != InvalidBackendId; i = QUEUE_NEXT_LISTENER(i))
{
int32 pid = QUEUE_BACKEND_PID(i);
QueuePosition pos;
@@ -2126,7 +2126,7 @@ asyncQueueAdvanceTail(void)
*/
LWLockAcquire(NotifyQueueLock, LW_EXCLUSIVE);
min = QUEUE_HEAD;
- for (BackendId i = QUEUE_FIRST_LISTENER; i > 0; i = QUEUE_NEXT_LISTENER(i))
+ for (BackendId i = QUEUE_FIRST_LISTENER; i != InvalidBackendId; i = QUEUE_NEXT_LISTENER(i))
{
Assert(QUEUE_BACKEND_PID(i) != InvalidPid);
min = QUEUE_POS_MIN(min, QUEUE_BACKEND_POS(i));
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index 6364b16261f..6cef901045d 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -326,7 +326,7 @@ BackgroundWriterMain(void)
if (rc == WL_TIMEOUT && can_hibernate && prev_hibernate)
{
/* Ask for notification at next buffer allocation */
- StrategyNotifyBgWriter(MyProcNumber);
+ StrategyNotifyBgWriter(MyBackendId);
/* Sleep ... */
(void) WaitLatch(MyLatch,
WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH,
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 2aa3e9890a2..213c9669fda 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -242,7 +242,7 @@ PgArchiverMain(void)
* Advertise our pgprocno so that backends can use our latch to wake us up
* while we're sleeping.
*/
- PgArch->pgprocno = MyProcNumber;
+ PgArch->pgprocno = MyBackendId;
/* Create workspace for pgarch_readyXlog() */
arch_files = palloc(sizeof(struct arch_files_state));
diff --git a/src/backend/postmaster/walsummarizer.c b/src/backend/postmaster/walsummarizer.c
index 5d61d1517e8..71560e8b808 100644
--- a/src/backend/postmaster/walsummarizer.c
+++ b/src/backend/postmaster/walsummarizer.c
@@ -248,7 +248,7 @@ WalSummarizerMain(void)
/* Advertise ourselves. */
on_shmem_exit(WalSummarizerShutdown, (Datum) 0);
LWLockAcquire(WALSummarizerLock, LW_EXCLUSIVE);
- WalSummarizerCtl->summarizer_pgprocno = MyProcNumber;
+ WalSummarizerCtl->summarizer_pgprocno = MyBackendId;
LWLockRelease(WALSummarizerLock);
/* Create and switch to a memory context that we can reset on error. */
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 87a31745864..bb9da98c9f5 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -4780,7 +4780,7 @@ UnlockBuffers(void)
* got a cancel/die interrupt before getting the signal.
*/
if ((buf_state & BM_PIN_COUNT_WAITER) != 0 &&
- buf->wait_backend_pgprocno == MyProcNumber)
+ buf->wait_backend_pgprocno == MyBackendId)
buf_state &= ~BM_PIN_COUNT_WAITER;
UnlockBufHdr(buf, buf_state);
@@ -4930,7 +4930,7 @@ LockBufferForCleanup(Buffer buffer)
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
elog(ERROR, "multiple backends attempting to wait for pincount 1");
}
- bufHdr->wait_backend_pgprocno = MyProcNumber;
+ bufHdr->wait_backend_pgprocno = MyBackendId;
PinCountWaitBuf = bufHdr;
buf_state |= BM_PIN_COUNT_WAITER;
UnlockBufHdr(bufHdr, buf_state);
@@ -4994,7 +4994,7 @@ LockBufferForCleanup(Buffer buffer)
*/
buf_state = LockBufHdr(bufHdr);
if ((buf_state & BM_PIN_COUNT_WAITER) != 0 &&
- bufHdr->wait_backend_pgprocno == MyProcNumber)
+ bufHdr->wait_backend_pgprocno == MyBackendId)
buf_state &= ~BM_PIN_COUNT_WAITER;
UnlockBufHdr(bufHdr, buf_state);
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index d96606ebba5..7ed08d95355 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -2538,7 +2538,7 @@ ProcArrayInstallImportedXmin(TransactionId xmin,
/*
* Find the PGPROC entry of the source transaction. (This could use
- * GetPGProcByBackendId(), unless it's a prepared xact. But this isn't
+ * GetPGProcByNumber(), unless it's a prepared xact. But this isn't
* performance critical.)
*/
for (index = 0; index < arrayP->numProcs; index++)
@@ -3116,9 +3116,9 @@ BackendIdGetProc(int backendID)
{
PGPROC *result;
- if (backendID < 1 || backendID > ProcGlobal->allProcCount)
+ if (backendID < 0 || backendID >= ProcGlobal->allProcCount)
return NULL;
- result = GetPGProcByBackendId(backendID);
+ result = GetPGProcByNumber(backendID);
if (result->pid == 0)
return NULL;
@@ -3144,9 +3144,9 @@ BackendIdGetTransactionIds(int backendID, TransactionId *xid,
*nsubxid = 0;
*overflowed = false;
- if (backendID < 1 || backendID > ProcGlobal->allProcCount)
+ if (backendID < 0 || backendID >= ProcGlobal->allProcCount)
return;
- proc = GetPGProcByBackendId(backendID);
+ proc = GetPGProcByNumber(backendID);
/* Need to lock out additions/removals of backends */
LWLockAcquire(ProcArrayLock, LW_SHARED);
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index d1d5bf0c152..0bf577b8bc0 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -161,11 +161,11 @@ ProcSignalInit(void)
ProcSignalSlot *slot;
uint64 barrier_generation;
- if (MyBackendId <= 0)
+ if (MyBackendId < 0)
elog(ERROR, "MyBackendId not set");
- if (MyBackendId > NumProcSignalSlots)
+ if (MyBackendId >= NumProcSignalSlots)
elog(ERROR, "unexpected MyBackendId %d in ProcSignalInit (max %d)", MyBackendId, NumProcSignalSlots);
- slot = &ProcSignal->psh_slot[MyBackendId - 1];
+ slot = &ProcSignal->psh_slot[MyBackendId];
/* sanity check */
if (slot->pss_pid != 0)
@@ -260,7 +260,7 @@ SendProcSignal(pid_t pid, ProcSignalReason reason, BackendId backendId)
if (backendId != InvalidBackendId)
{
- slot = &ProcSignal->psh_slot[backendId - 1];
+ slot = &ProcSignal->psh_slot[backendId];
/*
* Note: Since there's no locking, it's possible that the target
diff --git a/src/backend/storage/ipc/sinvaladt.c b/src/backend/storage/ipc/sinvaladt.c
index f624bfc7d78..f3ef3eb1ab0 100644
--- a/src/backend/storage/ipc/sinvaladt.c
+++ b/src/backend/storage/ipc/sinvaladt.c
@@ -274,15 +274,13 @@ SharedInvalBackendInit(bool sendOnly)
ProcState *stateP;
pid_t oldPid;
SISeg *segP = shmInvalBuffer;
- int pgprocno;
- if (MyBackendId <= 0)
+ if (MyBackendId < 0)
elog(ERROR, "MyBackendId not set");
- if (MyBackendId > NumProcStateSlots)
+ if (MyBackendId >= NumProcStateSlots)
elog(PANIC, "unexpected MyBackendId %d in SharedInvalBackendInit (max %d)",
MyBackendId, NumProcStateSlots);
- pgprocno = MyBackendId - 1;
- stateP = &segP->procState[pgprocno];
+ stateP = &segP->procState[MyBackendId];
/*
* This can run in parallel with read operations, but not with write
@@ -299,7 +297,7 @@ SharedInvalBackendInit(bool sendOnly)
MyBackendId, (int) oldPid);
}
- shmInvalBuffer->pgprocnos[shmInvalBuffer->numProcs++] = pgprocno;
+ shmInvalBuffer->pgprocnos[shmInvalBuffer->numProcs++] = MyBackendId;
/* Fetch next local transaction ID into local memory */
nextLocalTransactionId = stateP->nextLXID;
@@ -331,14 +329,13 @@ CleanupInvalidationState(int status, Datum arg)
{
SISeg *segP = (SISeg *) DatumGetPointer(arg);
ProcState *stateP;
- int pgprocno = MyBackendId - 1;
int i;
Assert(PointerIsValid(segP));
LWLockAcquire(SInvalWriteLock, LW_EXCLUSIVE);
- stateP = &segP->procState[pgprocno];
+ stateP = &segP->procState[MyBackendId];
/* Update next local transaction ID for next holder of this backendID */
stateP->nextLXID = nextLocalTransactionId;
@@ -351,7 +348,7 @@ CleanupInvalidationState(int status, Datum arg)
for (i = segP->numProcs - 1; i >= 0; i--)
{
- if (segP->pgprocnos[i] == pgprocno)
+ if (segP->pgprocnos[i] == MyBackendId)
{
if (i != segP->numProcs - 1)
segP->pgprocnos[i] = segP->pgprocnos[segP->numProcs - 1];
@@ -481,7 +478,7 @@ SIGetDataEntries(SharedInvalidationMessage *data, int datasize)
int n;
segP = shmInvalBuffer;
- stateP = &segP->procState[MyBackendId - 1];
+ stateP = &segP->procState[MyBackendId];
/*
* Before starting to take locks, do a quick, unlocked test to see whether
@@ -668,7 +665,7 @@ SICleanupQueue(bool callerHasWriteLock, int minFree)
if (needSig)
{
pid_t his_pid = needSig->procPid;
- BackendId his_backendId = (needSig - &segP->procState[0]) + 1;
+ BackendId his_backendId = (needSig - &segP->procState[0]);
needSig->signaled = true;
LWLockRelease(SInvalReadLock);
diff --git a/src/backend/storage/lmgr/condition_variable.c b/src/backend/storage/lmgr/condition_variable.c
index 10fdae19dcc..daff1368c1f 100644
--- a/src/backend/storage/lmgr/condition_variable.c
+++ b/src/backend/storage/lmgr/condition_variable.c
@@ -57,7 +57,7 @@ ConditionVariableInit(ConditionVariable *cv)
void
ConditionVariablePrepareToSleep(ConditionVariable *cv)
{
- int pgprocno = MyProcNumber;
+ int pgprocno = MyBackendId;
/*
* If some other sleep is already prepared, cancel it; this is necessary
@@ -181,10 +181,10 @@ ConditionVariableTimedSleep(ConditionVariable *cv, long timeout,
* guarantee not to return spuriously, we'll avoid this obvious case.
*/
SpinLockAcquire(&cv->mutex);
- if (!proclist_contains(&cv->wakeup, MyProcNumber, cvWaitLink))
+ if (!proclist_contains(&cv->wakeup, MyBackendId, cvWaitLink))
{
done = true;
- proclist_push_tail(&cv->wakeup, MyProcNumber, cvWaitLink);
+ proclist_push_tail(&cv->wakeup, MyBackendId, cvWaitLink);
}
SpinLockRelease(&cv->mutex);
@@ -236,8 +236,8 @@ ConditionVariableCancelSleep(void)
return false;
SpinLockAcquire(&cv->mutex);
- if (proclist_contains(&cv->wakeup, MyProcNumber, cvWaitLink))
- proclist_delete(&cv->wakeup, MyProcNumber, cvWaitLink);
+ if (proclist_contains(&cv->wakeup, MyBackendId, cvWaitLink))
+ proclist_delete(&cv->wakeup, MyBackendId, cvWaitLink);
else
signaled = true;
SpinLockRelease(&cv->mutex);
@@ -281,7 +281,7 @@ ConditionVariableSignal(ConditionVariable *cv)
void
ConditionVariableBroadcast(ConditionVariable *cv)
{
- int pgprocno = MyProcNumber;
+ int pgprocno = MyBackendId;
PGPROC *proc = NULL;
bool have_sentinel = false;
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index 997857679ed..291ac8d58f8 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -1056,9 +1056,9 @@ LWLockQueueSelf(LWLock *lock, LWLockMode mode)
/* LW_WAIT_UNTIL_FREE waiters are always at the front of the queue */
if (mode == LW_WAIT_UNTIL_FREE)
- proclist_push_head(&lock->waiters, MyProcNumber, lwWaitLink);
+ proclist_push_head(&lock->waiters, MyBackendId, lwWaitLink);
else
- proclist_push_tail(&lock->waiters, MyProcNumber, lwWaitLink);
+ proclist_push_tail(&lock->waiters, MyBackendId, lwWaitLink);
/* Can release the mutex now */
LWLockWaitListUnlock(lock);
@@ -1097,7 +1097,7 @@ LWLockDequeueSelf(LWLock *lock)
*/
on_waitlist = MyProc->lwWaiting == LW_WS_WAITING;
if (on_waitlist)
- proclist_delete(&lock->waiters, MyProcNumber, lwWaitLink);
+ proclist_delete(&lock->waiters, MyBackendId, lwWaitLink);
if (proclist_is_empty(&lock->waiters) &&
(pg_atomic_read_u32(&lock->state) & LW_FLAG_HAS_WAITERS) != 0)
diff --git a/src/backend/storage/lmgr/predicate.c b/src/backend/storage/lmgr/predicate.c
index d62060d58c8..d3dec91a68d 100644
--- a/src/backend/storage/lmgr/predicate.c
+++ b/src/backend/storage/lmgr/predicate.c
@@ -1830,7 +1830,7 @@ GetSerializableTransactionSnapshotInt(Snapshot snapshot,
sxact->finishedBefore = InvalidTransactionId;
sxact->xmin = snapshot->xmin;
sxact->pid = MyProcPid;
- sxact->pgprocno = MyProcNumber;
+ sxact->pgprocno = MyBackendId;
dlist_init(&sxact->predicateLocks);
dlist_node_init(&sxact->finishedLink);
sxact->flags = 0;
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index 4226e41d80c..402ab53a061 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -64,7 +64,6 @@ bool log_lock_waits = false;
/* Pointer to this process's PGPROC struct, if any */
PGPROC *MyProc = NULL;
-int MyProcNumber = INVALID_PGPROCNO;
/*
* This spinlock protects the freelist of recycled PGPROC structures.
@@ -352,8 +351,7 @@ InitProcess(void)
(errcode(ERRCODE_TOO_MANY_CONNECTIONS),
errmsg("sorry, too many clients already")));
}
- MyProcNumber = GetNumberFromPGProc(MyProc);
- MyBackendId = GetBackendIdFromPGProc(MyProc);
+ MyBackendId = GetNumberFromPGProc(MyProc);
/*
* Cross-check that the PGPROC is of the type we expect; if this were not
@@ -566,8 +564,7 @@ InitAuxiliaryProcess(void)
SpinLockRelease(ProcStructLock);
MyProc = auxproc;
- MyProcNumber = GetNumberFromPGProc(MyProc);
- MyBackendId = GetBackendIdFromPGProc(MyProc);
+ MyBackendId = GetNumberFromPGProc(MyProc);
/*
* Initialize all fields of MyProc, except for those previously
@@ -910,7 +907,6 @@ ProcKill(int code, Datum arg)
proc = MyProc;
MyProc = NULL;
- MyProcNumber = INVALID_PGPROCNO;
MyBackendId = InvalidBackendId;
DisownLatch(&proc->procLatch);
@@ -988,7 +984,6 @@ AuxiliaryProcKill(int code, Datum arg)
proc = MyProc;
MyProc = NULL;
- MyProcNumber = INVALID_PGPROCNO;
MyBackendId = InvalidBackendId;
DisownLatch(&proc->procLatch);
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 3d3f7b06723..8a4a9cd219b 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -249,8 +249,8 @@ pgstat_beinit(void)
{
/* Initialize MyBEEntry */
Assert(MyBackendId != InvalidBackendId);
- Assert(MyBackendId >= 1 && MyBackendId <= NumBackendStatSlots);
- MyBEEntry = &BackendStatusArray[MyBackendId - 1];
+ Assert(MyBackendId >= 0 && MyBackendId < NumBackendStatSlots);
+ MyBEEntry = &BackendStatusArray[MyBackendId];
/* Set up a process-exit hook to clean up */
on_shmem_exit(pgstat_beshutdown_hook, 0);
@@ -721,7 +721,7 @@ pgstat_read_current_status(void)
#ifdef ENABLE_GSS
PgBackendGSSStatus *localgssstatus;
#endif
- int i;
+ int backendId;
if (localBackendStatusTable)
return; /* already done */
@@ -764,7 +764,7 @@ pgstat_read_current_status(void)
beentry = BackendStatusArray;
localentry = localtable;
- for (i = 1; i <= NumBackendStatSlots; i++)
+ for (backendId = 0; backendId < NumBackendStatSlots; backendId++)
{
/*
* Follow the protocol of retrying if st_changecount changes while we
@@ -835,8 +835,8 @@ pgstat_read_current_status(void)
* is in order by backend_id. pgstat_get_beentry_by_backend_id()
* depends on that.
*/
- localentry->backend_id = i;
- BackendIdGetTransactionIds(i,
+ localentry->backend_id = backendId;
+ BackendIdGetTransactionIds(backendId,
&localentry->backend_xid,
&localentry->backend_xmin,
&localentry->backend_subxact_count,
diff --git a/src/backend/utils/adt/mcxtfuncs.c b/src/backend/utils/adt/mcxtfuncs.c
index a7267dc15d1..a3239ab126b 100644
--- a/src/backend/utils/adt/mcxtfuncs.c
+++ b/src/backend/utils/adt/mcxtfuncs.c
@@ -176,7 +176,7 @@ pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
}
if (proc != NULL)
- backendId = GetBackendIdFromPGProc(proc);
+ backendId = GetNumberFromPGProc(proc);
if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, backendId) < 0)
{
/* Again, just a warning to allow loops */
diff --git a/src/include/storage/backendid.h b/src/include/storage/backendid.h
index 01387723f79..de242e2c039 100644
--- a/src/include/storage/backendid.h
+++ b/src/include/storage/backendid.h
@@ -19,8 +19,10 @@
* assigned at backend startup after authentication. Note that a backend ID
* can be reused for a different backend immediately after a backend exits.
*
- * Backend IDs are assigned starting from 1. For historical reasons, BackendId
- * 0 is unused, but InvalidBackendId is defined as -1.
+ * Backend IDs are assigned starting from 0. A backend ID is also an index
+ * into the proc array, and is synonymous with "proc numbers" used in other
+ * parts of the code. (Before version 17, backend IDs were assigned
+ * separately from proc numbers, and started from 1.)
*/
typedef int BackendId;
diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h
index d1e50380b7c..eca0a5809e1 100644
--- a/src/include/storage/proc.h
+++ b/src/include/storage/proc.h
@@ -199,7 +199,7 @@ struct PGPROC
*/
struct {
BackendId backendId; /* For regular backends, equal to
- * GetBackendIdFromPGProc(proc). For
+ * GetNumberFromPGProc(proc). For
* prepared xacts, ID of the original
* backend that processed the
* transaction. For unused PGPROC entries,
@@ -317,7 +317,6 @@ struct PGPROC
extern PGDLLIMPORT PGPROC *MyProc;
-extern PGDLLIMPORT int MyProcNumber; /* same as GetNumberFromPGProc(MyProc) */
/*
* There is one ProcGlobal struct for the whole database cluster.
@@ -422,15 +421,11 @@ extern PGDLLIMPORT PROC_HDR *ProcGlobal;
extern PGDLLIMPORT PGPROC *PreparedXactProcs;
/*
- * Accessors for getting PGPROC given a pgprocno or BackendId, and vice versa.
- *
- * For historical reasons, some code uses 0-based "proc numbers", while other
- * code uses 1-based backend IDs.
+ * Accessors for getting PGPROC given a pgprocno (or BackendId which is the
+ * same thing since version 17) and vice versa.
*/
#define GetPGProcByNumber(n) (&ProcGlobal->allProcs[(n)])
#define GetNumberFromPGProc(proc) ((proc) - &ProcGlobal->allProcs[0])
-#define GetPGProcByBackendId(n) (&ProcGlobal->allProcs[(n) - 1])
-#define GetBackendIdFromPGProc(proc) (GetNumberFromPGProc(proc) + 1)
/*
* We set aside some extra PGPROC structures for auxiliary processes,
--
2.39.2
Re: Refactoring backend fork+exec code
Hi,
On 2024-02-08 13:19:53 +0200, Heikki Linnakangas wrote:
- /* - * Assign the ProcSignalSlot for an auxiliary process. Since it doesn't - * have a BackendId, the slot is statically allocated based on the - * auxiliary process type (MyAuxProcType). Backends use slots indexed in - * the range from 1 to MaxBackends (inclusive), so we use MaxBackends + - * AuxProcType + 1 as the index of the slot for an auxiliary process. - * - * This will need rethinking if we ever want more than one of a particular - * auxiliary process type. - */ - ProcSignalInit(MaxBackends + MyAuxProcType + 1); + ProcSignalInit();Now that we don't need the offset here, we could move ProcSignalInit() into
BsaeInit() I think?Hmm, doesn't feel right to me. BaseInit() is mostly concerned with setting
up backend-private structures, and it's also called for a standalone
backend.
It already initializes a lot of shared subsystems (pgstat, replication slots
and arguable things like the buffer pool, temporary file access and WAL). And
note that it already requires that MyProc is already set (but it's not yet
"added" to the procarray, i.e. doesn't do visibility stuff at that stage).
I don't think that BaseInit() being called by standalone backends really poses
a problem? So is InitPostgres(), which does call ProcSignalInit() in
standalone processes.
My mental model is that BaseInit() is for stuff that's shared between
processes that do attach to databases and those that don't. Right now the
initialization flow is something like this ascii diagram:
standalone: \ /-> StartupXLOG() \
-> InitProcess() -\ /-> ProcArrayAdd() -> SharedInvalBackendInit() -> ProcSignalInit()- -> pgstat_beinit() -> attach to db -> pgstat_bestart()
normal backend: / \ /
-> BaseInit() -
aux process: InitAuxiliaryProcess() -/ \-- -> ProcSignalInit() -> pgstat_beinit() -> pgstat_bestart()
The only reason ProcSignalInit() happens kinda late is that historically we
used BackendIds as the index, which were only assigned in
SharedInvalBackendInit() for normal processes. But that doesn't make sense
anymore after your changes.
Similarly, we do pgstat_beinit() quite late, but that's again only because it
uses MyBackendId, which today is only assigned during
SharedInvalBackendInit(). I don't think we can do pgstat_bestart() earlier
though, which is a shame, given the four calls to it inside InitPostgres().
I feel the process initialization codepaths could use some cleanup in
general. Not sure what exactly.
Very much agreed.
+/* + * BackendIdGetProc -- get a backend's PGPROC given its backend ID + * + * The result may be out of date arbitrarily quickly, so the caller + * must be careful about how this information is used. NULL is + * returned if the backend is not active. + */ +PGPROC * +BackendIdGetProc(int backendID) +{ + PGPROC *result; + + if (backendID < 1 || backendID > ProcGlobal->allProcCount) + return NULL;Hm, doesn't calling BackendIdGetProc() with these values a bug? That's not
about being out of date or such.Perhaps. I just followed the example of the old implementation, which also
returns NULL on bogus inputs.
Fair enough. Makes it harder to not notice bugs, but that's not on this patchset to fix...
I think the last remaining question here is about the 0- vs 1-based indexing
of BackendIds. Is it a good idea to switch to 0-based indexing? And if we do
it, should we reserve PGPROC 0. I'm on the fence on this one.
I lean towards it being a good idea. Having two internal indexing schemes was
bad enough so far, but at least one would fairly quickly notice if one used
the wrong one. If they're just offset by 1, it might end up taking longer,
because that'll often also be a valid id. But I think you have the author's
prerogative on this one.
If we do so, I think it might be better to standardize on MyProcNumber instead
of MyBackendId. That'll force looking at code where indexing shifts by 1 - and
it also seems more descriptive, as inside postgres it's imo clearer what a
"proc number" is than what a "backend id" is. Particularly because the latter
is also used for things that aren't backends...
The only exception are SQL level users, for those I think it might make sense
to keep the current 1 based indexing, there's just a few functions where we'd
need to translate.
@@ -791,6 +792,7 @@ ProcArrayEndTransactionInternal(PGPROC *proc, TransactionId latestXid) static void ProcArrayGroupClearXid(PGPROC *proc, TransactionId latestXid) { + int pgprocno = GetNumberFromPGProc(proc); PROC_HDR *procglobal = ProcGlobal; uint32 nextidx; uint32 wakeidx;
This one is the only one where I could see the additional math done in
GetNumberFromPGProc() hurting. Which is somewhat silly, because the proc
passed in is always MyProc. In the most unrealistic workload imaginable (many
backends doing nothing but assigning xids and committing, server-side), it
indeed seems to make a tiny difference. But not enough to worry about, I think.
FWIW, if I use GetNumberFromPGProc(MyProc) instead of MyProcNumber in
LWLockQueueSelf(), that does show up a bit more noticeable.
void
-ProcSignalInit(int pss_idx)
+ProcSignalInit(void)
{
ProcSignalSlot *slot;
uint64 barrier_generation;- Assert(pss_idx >= 1 && pss_idx <= NumProcSignalSlots); - - slot = &ProcSignal->psh_slot[pss_idx - 1]; + if (MyBackendId <= 0) + elog(ERROR, "MyBackendId not set"); + if (MyBackendId > NumProcSignalSlots) + elog(ERROR, "unexpected MyBackendId %d in ProcSignalInit (max %d)", MyBackendId, NumProcSignalSlots); + slot = &ProcSignal->psh_slot[MyBackendId - 1];/* sanity check */ if (slot->pss_pid != 0) elog(LOG, "process %d taking over ProcSignal slot %d, but it's not empty", - MyProcPid, pss_idx); + MyProcPid, (int) (slot - ProcSignal->psh_slot));
Hm, why not use MyBackendId - 1 as above? Am I missing something?
/* @@ -212,11 +211,7 @@ ProcSignalInit(int pss_idx) static void CleanupProcSignalState(int status, Datum arg) { - int pss_idx = DatumGetInt32(arg); - ProcSignalSlot *slot; - - slot = &ProcSignal->psh_slot[pss_idx - 1]; - Assert(slot == MyProcSignalSlot); + ProcSignalSlot *slot = MyProcSignalSlot;
Maybe worth asserting that MyProcSignalSlot isn't NULL? Previously that was
checked via the assertion above.
+ if (i != segP->numProcs - 1) + segP->pgprocnos[i] = segP->pgprocnos[segP->numProcs - 1]; + break;
Hm. This means the list will be out-of-order more and more over time, leading
to less cache efficient access patterns. Perhaps we should keep this sorted,
like we do for ProcGlobal->xids etc?
@@ -148,19 +148,11 @@ pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
PGPROC *proc;
BackendId backendId = InvalidBackendId;- proc = BackendPidGetProc(pid); - /* * See if the process with given pid is a backend or an auxiliary process. - * - * If the given process is a backend, use its backend id in - * SendProcSignal() later to speed up the operation. Otherwise, don't do - * that because auxiliary processes (except the startup process) don't - * have a valid backend id. */ - if (proc != NULL) - backendId = proc->backendId; - else + proc = BackendPidGetProc(pid); + if (proc == NULL) proc = AuxiliaryPidGetProc(pid);/*
@@ -183,6 +175,8 @@ pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(false);
}+ if (proc != NULL) + backendId = GetBackendIdFromPGProc(proc);
How can proc be NULL here?
Greetings,
Andres Freund
Re: Refactoring backend fork+exec code
On Thu, Feb 15, 2024 at 3:07 AM Andres Freund <andres@anarazel.de> wrote:
I think the last remaining question here is about the 0- vs 1-based indexing
of BackendIds. Is it a good idea to switch to 0-based indexing? And if we do
it, should we reserve PGPROC 0. I'm on the fence on this one.I lean towards it being a good idea. Having two internal indexing schemes was
bad enough so far, but at least one would fairly quickly notice if one used
the wrong one. If they're just offset by 1, it might end up taking longer,
because that'll often also be a valid id.
Yeah, I think making everything 0-based is probably the best way
forward long term. It might require more cleanup work to get there,
but it's just a lot simpler in the end, IMHO.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: Refactoring backend fork+exec code
On 15/02/2024 07:09, Robert Haas wrote:
On Thu, Feb 15, 2024 at 3:07 AM Andres Freund <andres@anarazel.de> wrote:
I think the last remaining question here is about the 0- vs 1-based indexing
of BackendIds. Is it a good idea to switch to 0-based indexing? And if we do
it, should we reserve PGPROC 0. I'm on the fence on this one.I lean towards it being a good idea. Having two internal indexing schemes was
bad enough so far, but at least one would fairly quickly notice if one used
the wrong one. If they're just offset by 1, it might end up taking longer,
because that'll often also be a valid id.Yeah, I think making everything 0-based is probably the best way
forward long term. It might require more cleanup work to get there,
but it's just a lot simpler in the end, IMHO.
Here's another patch version that does that. Yeah, I agree it's nicer in
the end.
I'm pretty happy with this now. I'll read through these patches myself
again after sleeping over it and try to get this committed by the end of
the week, but another pair of eyes wouldn't hurt.
On 14/02/2024 23:37, Andres Freund wrote:
void
-ProcSignalInit(int pss_idx)
+ProcSignalInit(void)
{
ProcSignalSlot *slot;
uint64 barrier_generation;- Assert(pss_idx >= 1 && pss_idx <= NumProcSignalSlots); - - slot = &ProcSignal->psh_slot[pss_idx - 1]; + if (MyBackendId <= 0) + elog(ERROR, "MyBackendId not set"); + if (MyBackendId > NumProcSignalSlots) + elog(ERROR, "unexpected MyBackendId %d in ProcSignalInit (max %d)", MyBackendId, NumProcSignalSlots); + slot = &ProcSignal->psh_slot[MyBackendId - 1];/* sanity check */ if (slot->pss_pid != 0) elog(LOG, "process %d taking over ProcSignal slot %d, but it's not empty", - MyProcPid, pss_idx); + MyProcPid, (int) (slot - ProcSignal->psh_slot));Hm, why not use MyBackendId - 1 as above? Am I missing something?
You're right, fixed.
/* @@ -212,11 +211,7 @@ ProcSignalInit(int pss_idx) static void CleanupProcSignalState(int status, Datum arg) { - int pss_idx = DatumGetInt32(arg); - ProcSignalSlot *slot; - - slot = &ProcSignal->psh_slot[pss_idx - 1]; - Assert(slot == MyProcSignalSlot); + ProcSignalSlot *slot = MyProcSignalSlot;Maybe worth asserting that MyProcSignalSlot isn't NULL? Previously that was
checked via the assertion above.
Added.
+ if (i != segP->numProcs - 1) + segP->pgprocnos[i] = segP->pgprocnos[segP->numProcs - 1]; + break;Hm. This means the list will be out-of-order more and more over time, leading
to less cache efficient access patterns. Perhaps we should keep this sorted,
like we do for ProcGlobal->xids etc?
Perhaps, although these are accessed much less frequently so I'm not
convinced it's worth the trouble.
I haven't found a good performance test case that where the shared cache
invalidation would show up. Would you happen to have one?
@@ -183,6 +175,8 @@ pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(false);
}+ if (proc != NULL) + backendId = GetBackendIdFromPGProc(proc);How can proc be NULL here?
Fixed.
--
Heikki Linnakangas
Neon (https://neon.tech)
Attachments:
v11-0001-Redefine-backend-ID-to-be-an-index-into-the-proc.patchtext/x-patch; charset=UTF-8; name=v11-0001-Redefine-backend-ID-to-be-an-index-into-the-proc.patchDownload
From 209306f255bf2c56645cdb527e82be243619c416 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Thu, 22 Feb 2024 02:11:34 +0200
Subject: [PATCH v11 1/2] Redefine backend ID to be an index into the proc
array.
Previously, backend ID was an index into the ProcState array, in the
shared cache invalidation manager (sinvaladt.c). The entry in the
ProcState array was reserved at backend startup by scanning the array
for a free entry, and that was also when the backend got its backend
ID. Things becomes slightly simpler if we redefine backend ID to be
the index into the PGPROC array, and directly use it also as an index
to the ProcState array. This uses a little more memory, as we reserve
a few extra slots in the ProcState array for aux processes that don't
need them, but the simplicity is worth it.
Aux processes now also have a backend ID. This simplifies the
reservation of BackendStatusArray and ProcSignal slots.
You can now convert a backend ID into an index into the PGPROC array
simply by subtracting 1. We still use 0-based "pgprocnos" in various
places, for indexes into the PGPROC array, but the only difference now
is that backend IDs start at 1 while pgprocnos start at 0.
There is still a 'backendId' field in PGPROC, now part of 'vxid' which
encapsulates the backend ID and local transaction ID together. It's
needed for prepared xacts. For regular backends, the backendId is
always equal to pgprocno + 1, but for prepared xact PGPROC entries,
it's the ID of the original backend that processed the transaction.
Reviewed-by: Andres Freund
Discussion: https://www.postgresql.org/message-id/8171f1aa-496f-46a6-afc3-c46fe7a9b407@iki.fi
---
src/backend/access/transam/twophase.c | 37 ++--
src/backend/access/transam/xact.c | 12 +-
src/backend/catalog/namespace.c | 2 +-
src/backend/commands/sequence.c | 2 +-
src/backend/executor/functions.c | 4 +-
src/backend/postmaster/auxprocess.c | 12 +-
src/backend/storage/ipc/procarray.c | 73 ++++++-
src/backend/storage/ipc/procsignal.c | 27 ++-
src/backend/storage/ipc/sinvaladt.c | 200 ++++++--------------
src/backend/storage/ipc/standby.c | 1 +
src/backend/storage/lmgr/lock.c | 28 +--
src/backend/storage/lmgr/proc.c | 30 +--
src/backend/utils/activity/backend_status.c | 52 ++---
src/backend/utils/adt/lockfuncs.c | 2 +-
src/backend/utils/adt/mcxtfuncs.c | 14 +-
src/backend/utils/error/csvlog.c | 4 +-
src/backend/utils/error/elog.c | 6 +-
src/backend/utils/error/jsonlog.c | 6 +-
src/backend/utils/init/postinit.c | 10 +-
src/backend/utils/time/snapmgr.c | 5 +-
src/include/miscadmin.h | 2 -
src/include/storage/backendid.h | 12 +-
src/include/storage/lock.h | 9 +-
src/include/storage/proc.h | 32 +++-
src/include/storage/procarray.h | 4 +
src/include/storage/procsignal.h | 2 +-
src/include/storage/sinvaladt.h | 4 -
src/pl/plpgsql/src/pl_exec.c | 10 +-
28 files changed, 280 insertions(+), 322 deletions(-)
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 234c8d08ebc..5c282002900 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -151,7 +151,6 @@ typedef struct GlobalTransactionData
{
GlobalTransaction next; /* list link for free list */
int pgprocno; /* ID of associated dummy PGPROC */
- BackendId dummyBackendId; /* similar to backend id for backends */
TimestampTz prepared_at; /* time of preparation */
/*
@@ -285,20 +284,6 @@ TwoPhaseShmemInit(void)
/* associate it with a PGPROC assigned by InitProcGlobal */
gxacts[i].pgprocno = GetNumberFromPGProc(&PreparedXactProcs[i]);
-
- /*
- * Assign a unique ID for each dummy proc, so that the range of
- * dummy backend IDs immediately follows the range of normal
- * backend IDs. We don't dare to assign a real backend ID to dummy
- * procs, because prepared transactions don't take part in cache
- * invalidation like a real backend ID would imply, but having a
- * unique ID for them is nevertheless handy. This arrangement
- * allows you to allocate an array of size (MaxBackends +
- * max_prepared_xacts + 1), and have a slot for every backend and
- * prepared transaction. Currently multixact.c uses that
- * technique.
- */
- gxacts[i].dummyBackendId = MaxBackends + 1 + i;
}
}
else
@@ -457,24 +442,24 @@ MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid, const char *gid,
Assert(LWLockHeldByMeInMode(TwoPhaseStateLock, LW_EXCLUSIVE));
Assert(gxact != NULL);
- proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ proc = GetPGProcByNumber(gxact->pgprocno);
/* Initialize the PGPROC entry */
MemSet(proc, 0, sizeof(PGPROC));
dlist_node_init(&proc->links);
proc->waitStatus = PROC_WAIT_STATUS_OK;
- if (LocalTransactionIdIsValid(MyProc->lxid))
+ if (LocalTransactionIdIsValid(MyProc->vxid.lxid))
{
/* clone VXID, for TwoPhaseGetXidByVirtualXID() to find */
- proc->lxid = MyProc->lxid;
- proc->backendId = MyBackendId;
+ proc->vxid.lxid = MyProc->vxid.lxid;
+ proc->vxid.backendId = MyBackendId;
}
else
{
Assert(AmStartupProcess() || !IsPostmasterEnvironment);
/* GetLockConflicts() uses this to specify a wait on the XID */
- proc->lxid = xid;
- proc->backendId = InvalidBackendId;
+ proc->vxid.lxid = xid;
+ proc->vxid.backendId = InvalidBackendId;
}
proc->xid = xid;
Assert(proc->xmin == InvalidTransactionId);
@@ -522,7 +507,7 @@ static void
GXactLoadSubxactData(GlobalTransaction gxact, int nsubxacts,
TransactionId *children)
{
- PGPROC *proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ PGPROC *proc = GetPGProcByNumber(gxact->pgprocno);
/* We need no extra lock since the GXACT isn't valid yet */
if (nsubxacts > PGPROC_MAX_CACHED_SUBXIDS)
@@ -559,7 +544,7 @@ MarkAsPrepared(GlobalTransaction gxact, bool lock_held)
* Put it into the global ProcArray so TransactionIdIsInProgress considers
* the XID as still running.
*/
- ProcArrayAdd(&ProcGlobal->allProcs[gxact->pgprocno]);
+ ProcArrayAdd(GetPGProcByNumber(gxact->pgprocno));
}
/*
@@ -583,7 +568,7 @@ LockGXact(const char *gid, Oid user)
for (i = 0; i < TwoPhaseState->numPrepXacts; i++)
{
GlobalTransaction gxact = TwoPhaseState->prepXacts[i];
- PGPROC *proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ PGPROC *proc = GetPGProcByNumber(gxact->pgprocno);
/* Ignore not-yet-valid GIDs */
if (!gxact->valid)
@@ -884,7 +869,7 @@ TwoPhaseGetXidByVirtualXID(VirtualTransactionId vxid,
if (!gxact->valid)
continue;
- proc = &ProcGlobal->allProcs[gxact->pgprocno];
+ proc = GetPGProcByNumber(gxact->pgprocno);
GET_VXID_FROM_PGPROC(proc_vxid, *proc);
if (VirtualTransactionIdEquals(vxid, proc_vxid))
{
@@ -919,7 +904,7 @@ TwoPhaseGetDummyBackendId(TransactionId xid, bool lock_held)
{
GlobalTransaction gxact = TwoPhaseGetGXact(xid, lock_held);
- return gxact->dummyBackendId;
+ return gxact->pgprocno + 1;
}
/*
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 70ab6e27a13..4ac5b9ea834 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -600,9 +600,9 @@ GetStableLatestTransactionId(void)
static LocalTransactionId lxid = InvalidLocalTransactionId;
static TransactionId stablexid = InvalidTransactionId;
- if (lxid != MyProc->lxid)
+ if (lxid != MyProc->vxid.lxid)
{
- lxid = MyProc->lxid;
+ lxid = MyProc->vxid.lxid;
stablexid = GetTopTransactionIdIfAny();
if (!TransactionIdIsValid(stablexid))
stablexid = ReadNextTransactionId();
@@ -2099,8 +2099,8 @@ StartTransaction(void)
* Advertise it in the proc array. We assume assignment of
* localTransactionId is atomic, and the backendId should be set already.
*/
- Assert(MyProc->backendId == vxid.backendId);
- MyProc->lxid = vxid.localTransactionId;
+ Assert(MyProc->vxid.backendId == vxid.backendId);
+ MyProc->vxid.lxid = vxid.localTransactionId;
TRACE_POSTGRESQL_TRANSACTION_START(vxid.localTransactionId);
@@ -2289,7 +2289,7 @@ CommitTransaction(void)
ParallelWorkerReportLastRecEnd(XactLastRecEnd);
}
- TRACE_POSTGRESQL_TRANSACTION_COMMIT(MyProc->lxid);
+ TRACE_POSTGRESQL_TRANSACTION_COMMIT(MyProc->vxid.lxid);
/*
* Let others know about no transaction in progress by me. Note that this
@@ -2840,7 +2840,7 @@ AbortTransaction(void)
XLogSetAsyncXactLSN(XactLastRecEnd);
}
- TRACE_POSTGRESQL_TRANSACTION_ABORT(MyProc->lxid);
+ TRACE_POSTGRESQL_TRANSACTION_ABORT(MyProc->vxid.lxid);
/*
* Let others know about no transaction in progress by me. Note that this
diff --git a/src/backend/catalog/namespace.c b/src/backend/catalog/namespace.c
index 8df30b24401..620ce7e75d2 100644
--- a/src/backend/catalog/namespace.c
+++ b/src/backend/catalog/namespace.c
@@ -49,7 +49,7 @@
#include "parser/parse_func.h"
#include "storage/ipc.h"
#include "storage/lmgr.h"
-#include "storage/sinvaladt.h"
+#include "storage/procarray.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index f7803744a5b..f99dc48e8cf 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1076,7 +1076,7 @@ setval3_oid(PG_FUNCTION_ARGS)
static Relation
lock_and_open_sequence(SeqTable seq)
{
- LocalTransactionId thislxid = MyProc->lxid;
+ LocalTransactionId thislxid = MyProc->vxid.lxid;
/* Get the lock if not already held in this xact */
if (seq->lxid != thislxid)
diff --git a/src/backend/executor/functions.c b/src/backend/executor/functions.c
index 0f811fd2fc9..a4b6e1effdb 100644
--- a/src/backend/executor/functions.c
+++ b/src/backend/executor/functions.c
@@ -799,7 +799,7 @@ init_sql_fcache(FunctionCallInfo fcinfo, Oid collation, bool lazyEvalOK)
lazyEvalOK);
/* Mark fcache with time of creation to show it's valid */
- fcache->lxid = MyProc->lxid;
+ fcache->lxid = MyProc->vxid.lxid;
fcache->subxid = GetCurrentSubTransactionId();
ReleaseSysCache(procedureTuple);
@@ -1081,7 +1081,7 @@ fmgr_sql(PG_FUNCTION_ARGS)
if (fcache != NULL)
{
- if (fcache->lxid != MyProc->lxid ||
+ if (fcache->lxid != MyProc->vxid.lxid ||
!SubTransactionIsActive(fcache->subxid))
{
/* It's stale; unlink and delete */
diff --git a/src/backend/postmaster/auxprocess.c b/src/backend/postmaster/auxprocess.c
index ab86e802f21..39171fea06b 100644
--- a/src/backend/postmaster/auxprocess.c
+++ b/src/backend/postmaster/auxprocess.c
@@ -107,17 +107,7 @@ AuxiliaryProcessMain(AuxProcType auxtype)
BaseInit();
- /*
- * Assign the ProcSignalSlot for an auxiliary process. Since it doesn't
- * have a BackendId, the slot is statically allocated based on the
- * auxiliary process type (MyAuxProcType). Backends use slots indexed in
- * the range from 1 to MaxBackends (inclusive), so we use MaxBackends +
- * AuxProcType + 1 as the index of the slot for an auxiliary process.
- *
- * This will need rethinking if we ever want more than one of a particular
- * auxiliary process type.
- */
- ProcSignalInit(MaxBackends + MyAuxProcType + 1);
+ ProcSignalInit();
/*
* Auxiliary processes don't run transactions, but they may need a
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index dd329a86ef4..d96606ebba5 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -701,7 +701,7 @@ ProcArrayEndTransaction(PGPROC *proc, TransactionId latestXid)
Assert(proc->subxidStatus.count == 0);
Assert(!proc->subxidStatus.overflowed);
- proc->lxid = InvalidLocalTransactionId;
+ proc->vxid.lxid = InvalidLocalTransactionId;
proc->xmin = InvalidTransactionId;
/* be sure this is cleared in abort */
@@ -743,7 +743,7 @@ ProcArrayEndTransactionInternal(PGPROC *proc, TransactionId latestXid)
ProcGlobal->xids[pgxactoff] = InvalidTransactionId;
proc->xid = InvalidTransactionId;
- proc->lxid = InvalidLocalTransactionId;
+ proc->vxid.lxid = InvalidLocalTransactionId;
proc->xmin = InvalidTransactionId;
/* be sure this is cleared in abort */
@@ -930,7 +930,7 @@ ProcArrayClearTransaction(PGPROC *proc)
ProcGlobal->xids[pgxactoff] = InvalidTransactionId;
proc->xid = InvalidTransactionId;
- proc->lxid = InvalidLocalTransactionId;
+ proc->vxid.lxid = InvalidLocalTransactionId;
proc->xmin = InvalidTransactionId;
proc->recoveryConflictPending = false;
@@ -2536,6 +2536,11 @@ ProcArrayInstallImportedXmin(TransactionId xmin,
/* Get lock so source xact can't end while we're doing this */
LWLockAcquire(ProcArrayLock, LW_SHARED);
+ /*
+ * Find the PGPROC entry of the source transaction. (This could use
+ * GetPGProcByBackendId(), unless it's a prepared xact. But this isn't
+ * performance critical.)
+ */
for (index = 0; index < arrayP->numProcs; index++)
{
int pgprocno = arrayP->pgprocnos[index];
@@ -2548,9 +2553,9 @@ ProcArrayInstallImportedXmin(TransactionId xmin,
continue;
/* We are only interested in the specific virtual transaction. */
- if (proc->backendId != sourcevxid->backendId)
+ if (proc->vxid.backendId != sourcevxid->backendId)
continue;
- if (proc->lxid != sourcevxid->localTransactionId)
+ if (proc->vxid.lxid != sourcevxid->localTransactionId)
continue;
/*
@@ -3099,6 +3104,64 @@ HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids, int type)
return result;
}
+/*
+ * BackendIdGetProc -- get a backend's PGPROC given its backend ID
+ *
+ * The result may be out of date arbitrarily quickly, so the caller
+ * must be careful about how this information is used. NULL is
+ * returned if the backend is not active.
+ */
+PGPROC *
+BackendIdGetProc(int backendID)
+{
+ PGPROC *result;
+
+ if (backendID < 1 || backendID > ProcGlobal->allProcCount)
+ return NULL;
+ result = GetPGProcByBackendId(backendID);
+
+ if (result->pid == 0)
+ return NULL;
+
+ return result;
+}
+
+/*
+ * BackendIdGetTransactionIds -- get a backend's transaction status
+ *
+ * Get the xid, xmin, nsubxid and overflow status of the backend. The
+ * result may be out of date arbitrarily quickly, so the caller must be
+ * careful about how this information is used.
+ */
+void
+BackendIdGetTransactionIds(int backendID, TransactionId *xid,
+ TransactionId *xmin, int *nsubxid, bool *overflowed)
+{
+ PGPROC *proc;
+
+ *xid = InvalidTransactionId;
+ *xmin = InvalidTransactionId;
+ *nsubxid = 0;
+ *overflowed = false;
+
+ if (backendID < 1 || backendID > ProcGlobal->allProcCount)
+ return;
+ proc = GetPGProcByBackendId(backendID);
+
+ /* Need to lock out additions/removals of backends */
+ LWLockAcquire(ProcArrayLock, LW_SHARED);
+
+ if (proc->pid != 0)
+ {
+ *xid = proc->xid;
+ *xmin = proc->xmin;
+ *nsubxid = proc->subxidStatus.count;
+ *overflowed = proc->subxidStatus.overflowed;
+ }
+
+ LWLockRelease(ProcArrayLock);
+}
+
/*
* BackendPidGetProc -- get a backend's PGPROC given its PID
*
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 0f9f90d2c7b..199dd182253 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -87,7 +87,7 @@ typedef struct
* possible auxiliary process type. (This scheme assumes there is not
* more than one of any auxiliary process type at a time.)
*/
-#define NumProcSignalSlots (MaxBackends + NUM_AUXPROCTYPES)
+#define NumProcSignalSlots (MaxBackends + NUM_AUXILIARY_PROCS)
/* Check whether the relevant type bit is set in the flags. */
#define BARRIER_SHOULD_CHECK(flags, type) \
@@ -154,24 +154,23 @@ ProcSignalShmemInit(void)
/*
* ProcSignalInit
* Register the current process in the ProcSignal array
- *
- * The passed index should be my BackendId if the process has one,
- * or MaxBackends + aux process type if not.
*/
void
-ProcSignalInit(int pss_idx)
+ProcSignalInit(void)
{
ProcSignalSlot *slot;
uint64 barrier_generation;
- Assert(pss_idx >= 1 && pss_idx <= NumProcSignalSlots);
-
- slot = &ProcSignal->psh_slot[pss_idx - 1];
+ if (MyBackendId <= 0)
+ elog(ERROR, "MyBackendId not set");
+ if (MyBackendId > NumProcSignalSlots)
+ elog(ERROR, "unexpected MyBackendId %d in ProcSignalInit (max %d)", MyBackendId, NumProcSignalSlots);
+ slot = &ProcSignal->psh_slot[MyBackendId - 1];
/* sanity check */
if (slot->pss_pid != 0)
elog(LOG, "process %d taking over ProcSignal slot %d, but it's not empty",
- MyProcPid, pss_idx);
+ MyProcPid, MyBackendId - 1);
/* Clear out any leftover signal reasons */
MemSet(slot->pss_signalFlags, 0, NUM_PROCSIGNALS * sizeof(sig_atomic_t));
@@ -200,7 +199,7 @@ ProcSignalInit(int pss_idx)
MyProcSignalSlot = slot;
/* Set up to release the slot on process exit */
- on_shmem_exit(CleanupProcSignalState, Int32GetDatum(pss_idx));
+ on_shmem_exit(CleanupProcSignalState, (Datum) 0);
}
/*
@@ -212,11 +211,7 @@ ProcSignalInit(int pss_idx)
static void
CleanupProcSignalState(int status, Datum arg)
{
- int pss_idx = DatumGetInt32(arg);
- ProcSignalSlot *slot;
-
- slot = &ProcSignal->psh_slot[pss_idx - 1];
- Assert(slot == MyProcSignalSlot);
+ ProcSignalSlot *slot = MyProcSignalSlot;
/*
* Clear MyProcSignalSlot, so that a SIGUSR1 received after this point
@@ -233,7 +228,7 @@ CleanupProcSignalState(int status, Datum arg)
* infinite loop trying to exit
*/
elog(LOG, "process %d releasing ProcSignal slot %d, but it contains %d",
- MyProcPid, pss_idx, (int) slot->pss_pid);
+ MyProcPid, (int) (slot - ProcSignal->psh_slot), (int) slot->pss_pid);
return; /* XXX better to zero the slot anyway? */
}
diff --git a/src/backend/storage/ipc/sinvaladt.c b/src/backend/storage/ipc/sinvaladt.c
index 748a792a854..f624bfc7d78 100644
--- a/src/backend/storage/ipc/sinvaladt.c
+++ b/src/backend/storage/ipc/sinvaladt.c
@@ -139,7 +139,6 @@ typedef struct ProcState
{
/* procPid is zero in an inactive ProcState array entry. */
pid_t procPid; /* PID of backend, for signaling */
- PGPROC *proc; /* PGPROC of backend */
/* nextMsgNum is meaningless if procPid == 0 or resetState is true. */
int nextMsgNum; /* next message number to read */
bool resetState; /* backend needs to reset its state */
@@ -172,8 +171,6 @@ typedef struct SISeg
int minMsgNum; /* oldest message still needed */
int maxMsgNum; /* next message number to be assigned */
int nextThreshold; /* # of messages to call SICleanupQueue */
- int lastBackend; /* index of last active procState entry, +1 */
- int maxBackends; /* size of procState array */
slock_t msgnumLock; /* spinlock protecting maxMsgNum */
@@ -183,11 +180,29 @@ typedef struct SISeg
SharedInvalidationMessage buffer[MAXNUMMESSAGES];
/*
- * Per-backend invalidation state info (has MaxBackends entries).
+ * Per-backend invalidation state info.
+ *
+ * 'procState' has NumProcStateSlots entries, and is indexed by pgprocno.
+ * 'numProcs' is the number of slots currently in use, and 'pgprocnos' is
+ * a dense array of their indexes, to speed up scanning all in-use slots.
+ *
+ * 'pgprocnos' is largely redundant with ProcArrayStruct->pgprocnos, but
+ * having our separate copy avoids contention on ProcArrayLock, and allows
+ * us to track only the processes that participate in shared cache
+ * invalidations.
*/
+ int numProcs;
+ int *pgprocnos;
ProcState procState[FLEXIBLE_ARRAY_MEMBER];
} SISeg;
+/*
+ * We reserve a slot for each possible BackendId, plus one for each
+ * possible auxiliary process type. (This scheme assumes there is not
+ * more than one of any auxiliary process type at a time.)
+ */
+#define NumProcStateSlots (MaxBackends + NUM_AUXILIARY_PROCS)
+
static SISeg *shmInvalBuffer; /* pointer to the shared inval buffer */
@@ -205,16 +220,8 @@ SInvalShmemSize(void)
Size size;
size = offsetof(SISeg, procState);
-
- /*
- * In Hot Standby mode, the startup process requests a procState array
- * slot using InitRecoveryTransactionEnvironment(). Even though
- * MaxBackends doesn't account for the startup process, it is guaranteed
- * to get a free slot. This is because the autovacuum launcher and worker
- * processes, which are included in MaxBackends, are not started in Hot
- * Standby mode.
- */
- size = add_size(size, mul_size(sizeof(ProcState), MaxBackends));
+ size = add_size(size, mul_size(sizeof(ProcState), NumProcStateSlots)); /* procState */
+ size = add_size(size, mul_size(sizeof(int), NumProcStateSlots)); /* pgprocnos */
return size;
}
@@ -239,23 +246,22 @@ CreateSharedInvalidationState(void)
shmInvalBuffer->minMsgNum = 0;
shmInvalBuffer->maxMsgNum = 0;
shmInvalBuffer->nextThreshold = CLEANUP_MIN;
- shmInvalBuffer->lastBackend = 0;
- shmInvalBuffer->maxBackends = MaxBackends;
SpinLockInit(&shmInvalBuffer->msgnumLock);
/* The buffer[] array is initially all unused, so we need not fill it */
/* Mark all backends inactive, and initialize nextLXID */
- for (i = 0; i < shmInvalBuffer->maxBackends; i++)
+ for (i = 0; i < NumProcStateSlots; i++)
{
shmInvalBuffer->procState[i].procPid = 0; /* inactive */
- shmInvalBuffer->procState[i].proc = NULL;
shmInvalBuffer->procState[i].nextMsgNum = 0; /* meaningless */
shmInvalBuffer->procState[i].resetState = false;
shmInvalBuffer->procState[i].signaled = false;
shmInvalBuffer->procState[i].hasMessages = false;
shmInvalBuffer->procState[i].nextLXID = InvalidLocalTransactionId;
}
+ shmInvalBuffer->numProcs = 0;
+ shmInvalBuffer->pgprocnos = (int *) &shmInvalBuffer->procState[i];
}
/*
@@ -265,59 +271,41 @@ CreateSharedInvalidationState(void)
void
SharedInvalBackendInit(bool sendOnly)
{
- int index;
- ProcState *stateP = NULL;
+ ProcState *stateP;
+ pid_t oldPid;
SISeg *segP = shmInvalBuffer;
+ int pgprocno;
+
+ if (MyBackendId <= 0)
+ elog(ERROR, "MyBackendId not set");
+ if (MyBackendId > NumProcStateSlots)
+ elog(PANIC, "unexpected MyBackendId %d in SharedInvalBackendInit (max %d)",
+ MyBackendId, NumProcStateSlots);
+ pgprocno = MyBackendId - 1;
+ stateP = &segP->procState[pgprocno];
/*
* This can run in parallel with read operations, but not with write
- * operations, since SIInsertDataEntries relies on lastBackend to set
- * hasMessages appropriately.
+ * operations, since SIInsertDataEntries relies on the pgprocnos array to
+ * set hasMessages appropriately.
*/
LWLockAcquire(SInvalWriteLock, LW_EXCLUSIVE);
- /* Look for a free entry in the procState array */
- for (index = 0; index < segP->lastBackend; index++)
- {
- if (segP->procState[index].procPid == 0) /* inactive slot? */
- {
- stateP = &segP->procState[index];
- break;
- }
- }
-
- if (stateP == NULL)
+ oldPid = stateP->procPid;
+ if (oldPid != 0)
{
- if (segP->lastBackend < segP->maxBackends)
- {
- stateP = &segP->procState[segP->lastBackend];
- Assert(stateP->procPid == 0);
- segP->lastBackend++;
- }
- else
- {
- /*
- * out of procState slots: MaxBackends exceeded -- report normally
- */
- MyBackendId = InvalidBackendId;
- LWLockRelease(SInvalWriteLock);
- ereport(FATAL,
- (errcode(ERRCODE_TOO_MANY_CONNECTIONS),
- errmsg("sorry, too many clients already")));
- }
+ LWLockRelease(SInvalWriteLock);
+ elog(ERROR, "sinval slot for backend %d is already in use by process %d",
+ MyBackendId, (int) oldPid);
}
- MyBackendId = (stateP - &segP->procState[0]) + 1;
-
- /* Advertise assigned backend ID in MyProc */
- MyProc->backendId = MyBackendId;
+ shmInvalBuffer->pgprocnos[shmInvalBuffer->numProcs++] = pgprocno;
/* Fetch next local transaction ID into local memory */
nextLocalTransactionId = stateP->nextLXID;
/* mark myself active, with all extant messages already read */
stateP->procPid = MyProcPid;
- stateP->proc = MyProc;
stateP->nextMsgNum = segP->maxMsgNum;
stateP->resetState = false;
stateP->signaled = false;
@@ -328,8 +316,6 @@ SharedInvalBackendInit(bool sendOnly)
/* register exit routine to mark my entry inactive at exit */
on_shmem_exit(CleanupInvalidationState, PointerGetDatum(segP));
-
- elog(DEBUG4, "my backend ID is %d", MyBackendId);
}
/*
@@ -345,96 +331,36 @@ CleanupInvalidationState(int status, Datum arg)
{
SISeg *segP = (SISeg *) DatumGetPointer(arg);
ProcState *stateP;
+ int pgprocno = MyBackendId - 1;
int i;
Assert(PointerIsValid(segP));
LWLockAcquire(SInvalWriteLock, LW_EXCLUSIVE);
- stateP = &segP->procState[MyBackendId - 1];
+ stateP = &segP->procState[pgprocno];
/* Update next local transaction ID for next holder of this backendID */
stateP->nextLXID = nextLocalTransactionId;
/* Mark myself inactive */
stateP->procPid = 0;
- stateP->proc = NULL;
stateP->nextMsgNum = 0;
stateP->resetState = false;
stateP->signaled = false;
- /* Recompute index of last active backend */
- for (i = segP->lastBackend; i > 0; i--)
+ for (i = segP->numProcs - 1; i >= 0; i--)
{
- if (segP->procState[i - 1].procPid != 0)
- break;
- }
- segP->lastBackend = i;
-
- LWLockRelease(SInvalWriteLock);
-}
-
-/*
- * BackendIdGetProc
- * Get the PGPROC structure for a backend, given the backend ID.
- * The result may be out of date arbitrarily quickly, so the caller
- * must be careful about how this information is used. NULL is
- * returned if the backend is not active.
- */
-PGPROC *
-BackendIdGetProc(int backendID)
-{
- PGPROC *result = NULL;
- SISeg *segP = shmInvalBuffer;
-
- /* Need to lock out additions/removals of backends */
- LWLockAcquire(SInvalWriteLock, LW_SHARED);
-
- if (backendID > 0 && backendID <= segP->lastBackend)
- {
- ProcState *stateP = &segP->procState[backendID - 1];
-
- result = stateP->proc;
- }
-
- LWLockRelease(SInvalWriteLock);
-
- return result;
-}
-
-/*
- * BackendIdGetTransactionIds
- * Get the xid, xmin, nsubxid and overflow status of the backend. The
- * result may be out of date arbitrarily quickly, so the caller must be
- * careful about how this information is used.
- */
-void
-BackendIdGetTransactionIds(int backendID, TransactionId *xid,
- TransactionId *xmin, int *nsubxid, bool *overflowed)
-{
- SISeg *segP = shmInvalBuffer;
-
- *xid = InvalidTransactionId;
- *xmin = InvalidTransactionId;
- *nsubxid = 0;
- *overflowed = false;
-
- /* Need to lock out additions/removals of backends */
- LWLockAcquire(SInvalWriteLock, LW_SHARED);
-
- if (backendID > 0 && backendID <= segP->lastBackend)
- {
- ProcState *stateP = &segP->procState[backendID - 1];
- PGPROC *proc = stateP->proc;
-
- if (proc != NULL)
+ if (segP->pgprocnos[i] == pgprocno)
{
- *xid = proc->xid;
- *xmin = proc->xmin;
- *nsubxid = proc->subxidStatus.count;
- *overflowed = proc->subxidStatus.overflowed;
+ if (i != segP->numProcs - 1)
+ segP->pgprocnos[i] = segP->pgprocnos[segP->numProcs - 1];
+ break;
}
}
+ if (i < 0)
+ elog(PANIC, "could not find entry in sinval array");
+ segP->numProcs--;
LWLockRelease(SInvalWriteLock);
}
@@ -507,9 +433,9 @@ SIInsertDataEntries(const SharedInvalidationMessage *data, int n)
* these (unlocked) changes will be committed to memory before we exit
* the function.
*/
- for (i = 0; i < segP->lastBackend; i++)
+ for (i = 0; i < segP->numProcs; i++)
{
- ProcState *stateP = &segP->procState[i];
+ ProcState *stateP = &segP->procState[segP->pgprocnos[i]];
stateP->hasMessages = true;
}
@@ -677,13 +603,14 @@ SICleanupQueue(bool callerHasWriteLock, int minFree)
minsig = min - SIG_THRESHOLD;
lowbound = min - MAXNUMMESSAGES + minFree;
- for (i = 0; i < segP->lastBackend; i++)
+ for (i = 0; i < segP->numProcs; i++)
{
- ProcState *stateP = &segP->procState[i];
+ ProcState *stateP = &segP->procState[segP->pgprocnos[i]];
int n = stateP->nextMsgNum;
- /* Ignore if inactive or already in reset state */
- if (stateP->procPid == 0 || stateP->resetState || stateP->sendOnly)
+ /* Ignore if already in reset state */
+ Assert(stateP->procPid != 0);
+ if (stateP->resetState || stateP->sendOnly)
continue;
/*
@@ -719,11 +646,8 @@ SICleanupQueue(bool callerHasWriteLock, int minFree)
{
segP->minMsgNum -= MSGNUMWRAPAROUND;
segP->maxMsgNum -= MSGNUMWRAPAROUND;
- for (i = 0; i < segP->lastBackend; i++)
- {
- /* we don't bother skipping inactive entries here */
- segP->procState[i].nextMsgNum -= MSGNUMWRAPAROUND;
- }
+ for (i = 0; i < segP->numProcs; i++)
+ segP->procState[segP->pgprocnos[i]].nextMsgNum -= MSGNUMWRAPAROUND;
}
/*
diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c
index d8755a106d5..97d1ab65740 100644
--- a/src/backend/storage/ipc/standby.c
+++ b/src/backend/storage/ipc/standby.c
@@ -137,6 +137,7 @@ InitRecoveryTransactionEnvironment(void)
* are held by vxids and row level locks are held by xids. All queries
* hold AccessShareLocks so never block while we write or lock new rows.
*/
+ MyProc->vxid.backendId = MyBackendId;
vxid.backendId = MyBackendId;
vxid.localTransactionId = GetNextLocalTransactionId();
VirtualXactLockTableInsert(vxid);
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index c70a1adb9ad..e62968b4a86 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -3625,8 +3625,8 @@ GetLockStatusData(void)
proc->fpRelId[f]);
instance->holdMask = lockbits << FAST_PATH_LOCKNUMBER_OFFSET;
instance->waitLockMode = NoLock;
- instance->backend = proc->backendId;
- instance->lxid = proc->lxid;
+ instance->vxid.backendId = proc->vxid.backendId;
+ instance->vxid.localTransactionId = proc->vxid.lxid;
instance->pid = proc->pid;
instance->leaderPid = proc->pid;
instance->fastpath = true;
@@ -3652,15 +3652,15 @@ GetLockStatusData(void)
repalloc(data->locks, sizeof(LockInstanceData) * els);
}
- vxid.backendId = proc->backendId;
+ vxid.backendId = proc->vxid.backendId;
vxid.localTransactionId = proc->fpLocalTransactionId;
instance = &data->locks[el];
SET_LOCKTAG_VIRTUALTRANSACTION(instance->locktag, vxid);
instance->holdMask = LOCKBIT_ON(ExclusiveLock);
instance->waitLockMode = NoLock;
- instance->backend = proc->backendId;
- instance->lxid = proc->lxid;
+ instance->vxid.backendId = proc->vxid.backendId;
+ instance->vxid.localTransactionId = proc->vxid.lxid;
instance->pid = proc->pid;
instance->leaderPid = proc->pid;
instance->fastpath = true;
@@ -3712,8 +3712,8 @@ GetLockStatusData(void)
instance->waitLockMode = proc->waitLockMode;
else
instance->waitLockMode = NoLock;
- instance->backend = proc->backendId;
- instance->lxid = proc->lxid;
+ instance->vxid.backendId = proc->vxid.backendId;
+ instance->vxid.localTransactionId = proc->vxid.lxid;
instance->pid = proc->pid;
instance->leaderPid = proclock->groupLeader->pid;
instance->fastpath = false;
@@ -3888,8 +3888,8 @@ GetSingleProcBlockerStatusData(PGPROC *blocked_proc, BlockedProcsData *data)
instance->waitLockMode = proc->waitLockMode;
else
instance->waitLockMode = NoLock;
- instance->backend = proc->backendId;
- instance->lxid = proc->lxid;
+ instance->vxid.backendId = proc->vxid.backendId;
+ instance->vxid.localTransactionId = proc->vxid.lxid;
instance->pid = proc->pid;
instance->leaderPid = proclock->groupLeader->pid;
instance->fastpath = false;
@@ -4374,8 +4374,8 @@ lock_twophase_postabort(TransactionId xid, uint16 info,
* lockers, as we haven't advertised this vxid via the ProcArray yet.
*
* Since MyProc->fpLocalTransactionId will normally contain the same data
- * as MyProc->lxid, you might wonder if we really need both. The
- * difference is that MyProc->lxid is set and cleared unlocked, and
+ * as MyProc->vxid.lxid, you might wonder if we really need both. The
+ * difference is that MyProc->vxid.lxid is set and cleared unlocked, and
* examined by procarray.c, while fpLocalTransactionId is protected by
* fpInfoLock and is used only by the locking subsystem. Doing it this
* way makes it easier to verify that there are no funny race conditions.
@@ -4391,7 +4391,7 @@ VirtualXactLockTableInsert(VirtualTransactionId vxid)
LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE);
- Assert(MyProc->backendId == vxid.backendId);
+ Assert(MyProc->vxid.backendId == vxid.backendId);
Assert(MyProc->fpLocalTransactionId == InvalidLocalTransactionId);
Assert(MyProc->fpVXIDLock == false);
@@ -4413,7 +4413,7 @@ VirtualXactLockTableCleanup(void)
bool fastpath;
LocalTransactionId lxid;
- Assert(MyProc->backendId != InvalidBackendId);
+ Assert(MyProc->vxid.backendId != InvalidBackendId);
/*
* Clean up shared memory state.
@@ -4541,7 +4541,7 @@ VirtualXactLock(VirtualTransactionId vxid, bool wait)
*/
LWLockAcquire(&proc->fpInfoLock, LW_EXCLUSIVE);
- if (proc->backendId != vxid.backendId
+ if (proc->vxid.backendId != vxid.backendId
|| proc->fpLocalTransactionId != vxid.localTransactionId)
{
/* VXID ended */
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index 4aec4a3c5f4..cdcf4674e2e 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -241,25 +241,25 @@ InitProcGlobal(void)
if (i < MaxConnections)
{
/* PGPROC for normal backend, add to freeProcs list */
- dlist_push_head(&ProcGlobal->freeProcs, &proc->links);
+ dlist_push_tail(&ProcGlobal->freeProcs, &proc->links);
proc->procgloballist = &ProcGlobal->freeProcs;
}
else if (i < MaxConnections + autovacuum_max_workers + 1)
{
/* PGPROC for AV launcher/worker, add to autovacFreeProcs list */
- dlist_push_head(&ProcGlobal->autovacFreeProcs, &proc->links);
+ dlist_push_tail(&ProcGlobal->autovacFreeProcs, &proc->links);
proc->procgloballist = &ProcGlobal->autovacFreeProcs;
}
else if (i < MaxConnections + autovacuum_max_workers + 1 + max_worker_processes)
{
/* PGPROC for bgworker, add to bgworkerFreeProcs list */
- dlist_push_head(&ProcGlobal->bgworkerFreeProcs, &proc->links);
+ dlist_push_tail(&ProcGlobal->bgworkerFreeProcs, &proc->links);
proc->procgloballist = &ProcGlobal->bgworkerFreeProcs;
}
else if (i < MaxBackends)
{
/* PGPROC for walsender, add to walsenderFreeProcs list */
- dlist_push_head(&ProcGlobal->walsenderFreeProcs, &proc->links);
+ dlist_push_tail(&ProcGlobal->walsenderFreeProcs, &proc->links);
proc->procgloballist = &ProcGlobal->walsenderFreeProcs;
}
@@ -354,6 +354,7 @@ InitProcess(void)
errmsg("sorry, too many clients already")));
}
MyProcNumber = GetNumberFromPGProc(MyProc);
+ MyBackendId = GetBackendIdFromPGProc(MyProc);
/*
* Cross-check that the PGPROC is of the type we expect; if this were not
@@ -376,14 +377,14 @@ InitProcess(void)
*/
dlist_node_init(&MyProc->links);
MyProc->waitStatus = PROC_WAIT_STATUS_OK;
- MyProc->lxid = InvalidLocalTransactionId;
MyProc->fpVXIDLock = false;
MyProc->fpLocalTransactionId = InvalidLocalTransactionId;
MyProc->xid = InvalidTransactionId;
MyProc->xmin = InvalidTransactionId;
MyProc->pid = MyProcPid;
- /* backendId, databaseId and roleId will be filled in later */
- MyProc->backendId = InvalidBackendId;
+ MyProc->vxid.backendId = MyBackendId;
+ MyProc->vxid.lxid = InvalidLocalTransactionId;
+ /* databaseId and roleId will be filled in later */
MyProc->databaseId = InvalidOid;
MyProc->roleId = InvalidOid;
MyProc->tempNamespaceId = InvalidOid;
@@ -563,11 +564,11 @@ InitAuxiliaryProcess(void)
/* use volatile pointer to prevent code rearrangement */
((volatile PGPROC *) auxproc)->pid = MyProcPid;
- MyProc = auxproc;
-
SpinLockRelease(ProcStructLock);
+ MyProc = auxproc;
MyProcNumber = GetNumberFromPGProc(MyProc);
+ MyBackendId = GetBackendIdFromPGProc(MyProc);
/*
* Initialize all fields of MyProc, except for those previously
@@ -575,12 +576,12 @@ InitAuxiliaryProcess(void)
*/
dlist_node_init(&MyProc->links);
MyProc->waitStatus = PROC_WAIT_STATUS_OK;
- MyProc->lxid = InvalidLocalTransactionId;
MyProc->fpVXIDLock = false;
MyProc->fpLocalTransactionId = InvalidLocalTransactionId;
MyProc->xid = InvalidTransactionId;
MyProc->xmin = InvalidTransactionId;
- MyProc->backendId = InvalidBackendId;
+ MyProc->vxid.backendId = InvalidBackendId;
+ MyProc->vxid.lxid = InvalidLocalTransactionId;
MyProc->databaseId = InvalidOid;
MyProc->roleId = InvalidOid;
MyProc->tempNamespaceId = InvalidOid;
@@ -911,8 +912,14 @@ ProcKill(int code, Datum arg)
proc = MyProc;
MyProc = NULL;
MyProcNumber = INVALID_PGPROCNO;
+ MyBackendId = InvalidBackendId;
DisownLatch(&proc->procLatch);
+ /* Mark the proc no longer in use */
+ proc->pid = 0;
+ proc->vxid.backendId = InvalidBackendId;
+ proc->vxid.lxid = InvalidTransactionId;
+
procgloballist = proc->procgloballist;
SpinLockAcquire(ProcStructLock);
@@ -983,6 +990,7 @@ AuxiliaryProcKill(int code, Datum arg)
proc = MyProc;
MyProc = NULL;
MyProcNumber = INVALID_PGPROCNO;
+ MyBackendId = InvalidBackendId;
DisownLatch(&proc->procLatch);
SpinLockAcquire(ProcStructLock);
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 1a1050c8da1..3d3f7b06723 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -19,6 +19,7 @@
#include "port/atomics.h" /* for memory barriers */
#include "storage/ipc.h"
#include "storage/proc.h" /* for MyProc */
+#include "storage/procarray.h"
#include "storage/sinvaladt.h"
#include "utils/ascii.h"
#include "utils/backend_status.h"
@@ -29,13 +30,12 @@
/* ----------
* Total number of backends including auxiliary
*
- * We reserve a slot for each possible BackendId, plus one for each
- * possible auxiliary process type. (This scheme assumes there is not
- * more than one of any auxiliary process type at a time.) MaxBackends
- * includes autovacuum workers and background workers as well.
+ * We reserve a slot for each possible PGPROC entry, including aux processes.
+ * (But not including PGPROC entries reserved for prepared xacts; they are not
+ * real processes.)
* ----------
*/
-#define NumBackendStatSlots (MaxBackends + NUM_AUXPROCTYPES)
+#define NumBackendStatSlots (MaxBackends + NUM_AUXILIARY_PROCS)
/* ----------
@@ -238,10 +238,9 @@ CreateSharedBackendStatus(void)
/*
* Initialize pgstats backend activity state, and set up our on-proc-exit
- * hook. Called from InitPostgres and AuxiliaryProcessMain. For auxiliary
- * process, MyBackendId is invalid. Otherwise, MyBackendId must be set, but we
- * must not have started any transaction yet (since the exit hook must run
- * after the last transaction exit).
+ * hook. Called from InitPostgres and AuxiliaryProcessMain. MyBackendId must
+ * be set, but we must not have started any transaction yet (since the exit
+ * hook must run after the last transaction exit).
*
* NOTE: MyDatabaseId isn't set yet; so the shutdown hook has to be careful.
*/
@@ -249,26 +248,9 @@ void
pgstat_beinit(void)
{
/* Initialize MyBEEntry */
- if (MyBackendId != InvalidBackendId)
- {
- Assert(MyBackendId >= 1 && MyBackendId <= MaxBackends);
- MyBEEntry = &BackendStatusArray[MyBackendId - 1];
- }
- else
- {
- /* Must be an auxiliary process */
- Assert(MyAuxProcType != NotAnAuxProcess);
-
- /*
- * Assign the MyBEEntry for an auxiliary process. Since it doesn't
- * have a BackendId, the slot is statically allocated based on the
- * auxiliary process type (MyAuxProcType). Backends use slots indexed
- * in the range from 0 to MaxBackends (exclusive), so we use
- * MaxBackends + AuxProcType as the index of the slot for an auxiliary
- * process.
- */
- MyBEEntry = &BackendStatusArray[MaxBackends + MyAuxProcType];
- }
+ Assert(MyBackendId != InvalidBackendId);
+ Assert(MyBackendId >= 1 && MyBackendId <= NumBackendStatSlots);
+ MyBEEntry = &BackendStatusArray[MyBackendId - 1];
/* Set up a process-exit hook to clean up */
on_shmem_exit(pgstat_beshutdown_hook, 0);
@@ -281,12 +263,12 @@ pgstat_beinit(void)
* Initialize this backend's entry in the PgBackendStatus array.
* Called from InitPostgres.
*
- * Apart from auxiliary processes, MyBackendId, MyDatabaseId,
- * session userid, and application_name must be set for a
- * backend (hence, this cannot be combined with pgstat_beinit).
- * Note also that we must be inside a transaction if this isn't an aux
- * process, as we may need to do encoding conversion on some strings.
- * ----------
+ * Apart from auxiliary processes, MyDatabaseId, session userid, and
+ * application_name must already be set (hence, this cannot be combined
+ * with pgstat_beinit). Note also that we must be inside a transaction
+ * if this isn't an aux process, as we may need to do encoding conversion
+ * on some strings.
+ *----------
*/
void
pgstat_bestart(void)
diff --git a/src/backend/utils/adt/lockfuncs.c b/src/backend/utils/adt/lockfuncs.c
index 4b49f7fe3d8..bbe5cc0806e 100644
--- a/src/backend/utils/adt/lockfuncs.c
+++ b/src/backend/utils/adt/lockfuncs.c
@@ -353,7 +353,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
break;
}
- values[10] = VXIDGetDatum(instance->backend, instance->lxid);
+ values[10] = VXIDGetDatum(instance->vxid.backendId, instance->vxid.localTransactionId);
if (instance->pid != 0)
values[11] = Int32GetDatum(instance->pid);
else
diff --git a/src/backend/utils/adt/mcxtfuncs.c b/src/backend/utils/adt/mcxtfuncs.c
index 4708d73f5fa..a7267dc15d1 100644
--- a/src/backend/utils/adt/mcxtfuncs.c
+++ b/src/backend/utils/adt/mcxtfuncs.c
@@ -148,19 +148,11 @@ pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
PGPROC *proc;
BackendId backendId = InvalidBackendId;
- proc = BackendPidGetProc(pid);
-
/*
* See if the process with given pid is a backend or an auxiliary process.
- *
- * If the given process is a backend, use its backend id in
- * SendProcSignal() later to speed up the operation. Otherwise, don't do
- * that because auxiliary processes (except the startup process) don't
- * have a valid backend id.
*/
- if (proc != NULL)
- backendId = proc->backendId;
- else
+ proc = BackendPidGetProc(pid);
+ if (proc == NULL)
proc = AuxiliaryPidGetProc(pid);
/*
@@ -183,6 +175,8 @@ pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(false);
}
+ if (proc != NULL)
+ backendId = GetBackendIdFromPGProc(proc);
if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, backendId) < 0)
{
/* Again, just a warning to allow loops */
diff --git a/src/backend/utils/error/csvlog.c b/src/backend/utils/error/csvlog.c
index 1b62b07f231..1d44d8a6a31 100644
--- a/src/backend/utils/error/csvlog.c
+++ b/src/backend/utils/error/csvlog.c
@@ -152,8 +152,8 @@ write_csvlog(ErrorData *edata)
/* Virtual transaction id */
/* keep VXID format in sync with lockfuncs.c */
- if (MyProc != NULL && MyProc->backendId != InvalidBackendId)
- appendStringInfo(&buf, "%d/%u", MyProc->backendId, MyProc->lxid);
+ if (MyProc != NULL && MyProc->vxid.backendId != InvalidBackendId)
+ appendStringInfo(&buf, "%d/%u", MyProc->vxid.backendId, MyProc->vxid.lxid);
appendStringInfoChar(&buf, ',');
/* Transaction id */
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index bba00a0087f..8cf36cbce07 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -3078,18 +3078,18 @@ log_status_format(StringInfo buf, const char *format, ErrorData *edata)
break;
case 'v':
/* keep VXID format in sync with lockfuncs.c */
- if (MyProc != NULL && MyProc->backendId != InvalidBackendId)
+ if (MyProc != NULL && MyProc->vxid.backendId != InvalidBackendId)
{
if (padding != 0)
{
char strfbuf[128];
snprintf(strfbuf, sizeof(strfbuf) - 1, "%d/%u",
- MyProc->backendId, MyProc->lxid);
+ MyProc->vxid.backendId, MyProc->vxid.lxid);
appendStringInfo(buf, "%*s", padding, strfbuf);
}
else
- appendStringInfo(buf, "%d/%u", MyProc->backendId, MyProc->lxid);
+ appendStringInfo(buf, "%d/%u", MyProc->vxid.backendId, MyProc->vxid.lxid);
}
else if (padding != 0)
appendStringInfoSpaces(buf,
diff --git a/src/backend/utils/error/jsonlog.c b/src/backend/utils/error/jsonlog.c
index 2903561f1c4..067d9e30b16 100644
--- a/src/backend/utils/error/jsonlog.c
+++ b/src/backend/utils/error/jsonlog.c
@@ -197,9 +197,9 @@ write_jsonlog(ErrorData *edata)
/* Virtual transaction id */
/* keep VXID format in sync with lockfuncs.c */
- if (MyProc != NULL && MyProc->backendId != InvalidBackendId)
- appendJSONKeyValueFmt(&buf, "vxid", true, "%d/%u", MyProc->backendId,
- MyProc->lxid);
+ if (MyProc != NULL && MyProc->vxid.backendId != InvalidBackendId)
+ appendJSONKeyValueFmt(&buf, "vxid", true, "%d/%u",
+ MyProc->vxid.backendId, MyProc->vxid.lxid);
/* Transaction id */
appendJSONKeyValueFmt(&buf, "txid", false, "%u",
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index 7797876d008..2568e46df5f 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -741,18 +741,10 @@ InitPostgres(const char *in_dbname, Oid dboid,
/*
* Initialize my entry in the shared-invalidation manager's array of
* per-backend data.
- *
- * Sets up MyBackendId, a unique backend identifier.
*/
- MyBackendId = InvalidBackendId;
-
SharedInvalBackendInit(false);
- if (MyBackendId > MaxBackends || MyBackendId <= 0)
- elog(FATAL, "bad backend ID: %d", MyBackendId);
-
- /* Now that we have a BackendId, we can participate in ProcSignal */
- ProcSignalInit(MyBackendId);
+ ProcSignalInit();
/*
* Also set up timeout handlers needed for backend operation. We need
diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
index 675e81d82d7..a0916959b17 100644
--- a/src/backend/utils/time/snapmgr.c
+++ b/src/backend/utils/time/snapmgr.c
@@ -1154,7 +1154,8 @@ ExportSnapshot(Snapshot snapshot)
* inside the transaction from 1.
*/
snprintf(path, sizeof(path), SNAPSHOT_EXPORT_DIR "/%08X-%08X-%d",
- MyProc->backendId, MyProc->lxid, list_length(exportedSnapshots) + 1);
+ MyProc->vxid.backendId, MyProc->vxid.lxid,
+ list_length(exportedSnapshots) + 1);
/*
* Copy the snapshot into TopTransactionContext, add it to the
@@ -1181,7 +1182,7 @@ ExportSnapshot(Snapshot snapshot)
*/
initStringInfo(&buf);
- appendStringInfo(&buf, "vxid:%d/%u\n", MyProc->backendId, MyProc->lxid);
+ appendStringInfo(&buf, "vxid:%d/%u\n", MyProc->vxid.backendId, MyProc->vxid.lxid);
appendStringInfo(&buf, "pid:%d\n", MyProcPid);
appendStringInfo(&buf, "dbid:%u\n", MyDatabaseId);
appendStringInfo(&buf, "iso:%d\n", XactIsoLevel);
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 0445fbf61d7..19213693ea4 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -445,8 +445,6 @@ typedef enum
WalWriterProcess,
WalReceiverProcess,
WalSummarizerProcess,
-
- NUM_AUXPROCTYPES /* Must be last! */
} AuxProcType;
extern PGDLLIMPORT AuxProcType MyAuxProcType;
diff --git a/src/include/storage/backendid.h b/src/include/storage/backendid.h
index 50ac982da19..01387723f79 100644
--- a/src/include/storage/backendid.h
+++ b/src/include/storage/backendid.h
@@ -14,11 +14,15 @@
#ifndef BACKENDID_H
#define BACKENDID_H
-/* ----------------
- * -cim 8/17/90
- * ----------------
+/*
+ * BackendId uniquely identifies an active backend or auxiliary process. It's
+ * assigned at backend startup after authentication. Note that a backend ID
+ * can be reused for a different backend immediately after a backend exits.
+ *
+ * Backend IDs are assigned starting from 1. For historical reasons, BackendId
+ * 0 is unused, but InvalidBackendId is defined as -1.
*/
-typedef int BackendId; /* unique currently active backend identifier */
+typedef int BackendId;
#define InvalidBackendId (-1)
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index ed6071f3286..c5c84201378 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -74,9 +74,9 @@ typedef struct
#define SetInvalidVirtualTransactionId(vxid) \
((vxid).backendId = InvalidBackendId, \
(vxid).localTransactionId = InvalidLocalTransactionId)
-#define GET_VXID_FROM_PGPROC(vxid, proc) \
- ((vxid).backendId = (proc).backendId, \
- (vxid).localTransactionId = (proc).lxid)
+#define GET_VXID_FROM_PGPROC(vxid_dst, proc) \
+ ((vxid_dst).backendId = (proc).vxid.backendId, \
+ (vxid_dst).localTransactionId = (proc).vxid.lxid)
/* MAX_LOCKMODES cannot be larger than the # of bits in LOCKMASK */
#define MAX_LOCKMODES 10
@@ -454,8 +454,7 @@ typedef struct LockInstanceData
LOCKTAG locktag; /* tag for locked object */
LOCKMASK holdMask; /* locks held by this PGPROC */
LOCKMODE waitLockMode; /* lock awaited by this PGPROC, if any */
- BackendId backend; /* backend ID of this PGPROC */
- LocalTransactionId lxid; /* local transaction ID of this PGPROC */
+ VirtualTransactionId vxid; /* local transaction ID of this PGPROC */
TimestampTz waitStart; /* time at which this PGPROC started waiting
* for lock */
int pid; /* pid of this PGPROC */
diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h
index 4453c6df877..d29811ce16d 100644
--- a/src/include/storage/proc.h
+++ b/src/include/storage/proc.h
@@ -186,16 +186,31 @@ struct PGPROC
* vacuum must not remove tuples deleted by
* xid >= xmin ! */
- LocalTransactionId lxid; /* local id of top-level transaction currently
- * being executed by this proc, if running;
- * else InvalidLocalTransactionId */
int pid; /* Backend's process ID; 0 if prepared xact */
int pgxactoff; /* offset into various ProcGlobal->arrays with
* data mirrored from this PGPROC */
+ /*
+ * Currently running top-level transaction's virtual xid. Together these
+ * form a VirtualTransactionId, but we don't use that struct because this
+ * is not atomically assignable as whole, and we want to enforce code to
+ * consider both parts separately. See comments at VirtualTransactionId.
+ */
+ struct {
+ BackendId backendId; /* For regular backends, equal to
+ * GetBackendIdFromPGProc(proc). For
+ * prepared xacts, ID of the original
+ * backend that processed the
+ * transaction. For unused PGPROC entries,
+ * InvalidbackendID. */
+ LocalTransactionId lxid; /* local id of top-level transaction
+ * currently * being executed by this
+ * proc, if running; else
+ * InvalidLocaltransactionId */
+ } vxid;
+
/* These fields are zero while a backend is still starting up: */
- BackendId backendId; /* This backend's backend ID (if assigned) */
Oid databaseId; /* OID of database this backend is using */
Oid roleId; /* OID of role using this backend */
@@ -406,9 +421,16 @@ extern PGDLLIMPORT PROC_HDR *ProcGlobal;
extern PGDLLIMPORT PGPROC *PreparedXactProcs;
-/* Accessor for PGPROC given a pgprocno, and vice versa. */
+/*
+ * Accessors for getting PGPROC given a pgprocno or BackendId, and vice versa.
+ *
+ * For historical reasons, some code uses 0-based "proc numbers", while other
+ * code uses 1-based backend IDs.
+ */
#define GetPGProcByNumber(n) (&ProcGlobal->allProcs[(n)])
#define GetNumberFromPGProc(proc) ((proc) - &ProcGlobal->allProcs[0])
+#define GetPGProcByBackendId(n) (&ProcGlobal->allProcs[(n) - 1])
+#define GetBackendIdFromPGProc(proc) (GetNumberFromPGProc(proc) + 1)
/*
* We set aside some extra PGPROC structures for auxiliary processes,
diff --git a/src/include/storage/procarray.h b/src/include/storage/procarray.h
index f3eba9b7640..3af7577e8c6 100644
--- a/src/include/storage/procarray.h
+++ b/src/include/storage/procarray.h
@@ -64,6 +64,10 @@ extern VirtualTransactionId *GetVirtualXIDsDelayingChkpt(int *nvxids, int type);
extern bool HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids,
int nvxids, int type);
+extern PGPROC *BackendIdGetProc(int backendID);
+extern void BackendIdGetTransactionIds(int backendID, TransactionId *xid,
+ TransactionId *xmin, int *nsubxid,
+ bool *overflowed);
extern PGPROC *BackendPidGetProc(int pid);
extern PGPROC *BackendPidGetProcWithLock(int pid);
extern int BackendXidGetPid(TransactionId xid);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 52dcb4c2adf..febdda3611c 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -62,7 +62,7 @@ typedef enum
extern Size ProcSignalShmemSize(void);
extern void ProcSignalShmemInit(void);
-extern void ProcSignalInit(int pss_idx);
+extern void ProcSignalInit(void);
extern int SendProcSignal(pid_t pid, ProcSignalReason reason,
BackendId backendId);
diff --git a/src/include/storage/sinvaladt.h b/src/include/storage/sinvaladt.h
index aa3d203efca..c3c97b3f8b7 100644
--- a/src/include/storage/sinvaladt.h
+++ b/src/include/storage/sinvaladt.h
@@ -31,10 +31,6 @@
extern Size SInvalShmemSize(void);
extern void CreateSharedInvalidationState(void);
extern void SharedInvalBackendInit(bool sendOnly);
-extern PGPROC *BackendIdGetProc(int backendID);
-extern void BackendIdGetTransactionIds(int backendID, TransactionId *xid,
- TransactionId *xmin, int *nsubxid,
- bool *overflowed);
extern void SIInsertDataEntries(const SharedInvalidationMessage *data, int n);
extern int SIGetDataEntries(SharedInvalidationMessage *data, int datasize);
diff --git a/src/pl/plpgsql/src/pl_exec.c b/src/pl/plpgsql/src/pl_exec.c
index 6d1691340c5..ed51694428a 100644
--- a/src/pl/plpgsql/src/pl_exec.c
+++ b/src/pl/plpgsql/src/pl_exec.c
@@ -2211,7 +2211,7 @@ exec_stmt_call(PLpgSQL_execstate *estate, PLpgSQL_stmt_call *stmt)
paramLI = setup_param_list(estate, expr);
- before_lxid = MyProc->lxid;
+ before_lxid = MyProc->vxid.lxid;
/*
* If we have a procedure-lifespan resowner, use that to hold the refcount
@@ -2232,7 +2232,7 @@ exec_stmt_call(PLpgSQL_execstate *estate, PLpgSQL_stmt_call *stmt)
elog(ERROR, "SPI_execute_plan_extended failed executing query \"%s\": %s",
expr->query, SPI_result_code_string(rc));
- after_lxid = MyProc->lxid;
+ after_lxid = MyProc->vxid.lxid;
if (before_lxid != after_lxid)
{
@@ -6037,7 +6037,7 @@ exec_eval_simple_expr(PLpgSQL_execstate *estate,
int32 *rettypmod)
{
ExprContext *econtext = estate->eval_econtext;
- LocalTransactionId curlxid = MyProc->lxid;
+ LocalTransactionId curlxid = MyProc->vxid.lxid;
ParamListInfo paramLI;
void *save_setup_arg;
bool need_snapshot;
@@ -7943,7 +7943,7 @@ get_cast_hashentry(PLpgSQL_execstate *estate,
* functions do; DO blocks have private simple_eval_estates, and private
* cast hash tables to go with them.)
*/
- curlxid = MyProc->lxid;
+ curlxid = MyProc->vxid.lxid;
if (cast_entry->cast_lxid != curlxid || cast_entry->cast_in_use)
{
oldcontext = MemoryContextSwitchTo(estate->simple_eval_estate->es_query_cxt);
@@ -8070,7 +8070,7 @@ exec_simple_check_plan(PLpgSQL_execstate *estate, PLpgSQL_expr *expr)
/* Remember that we have the refcount */
expr->expr_simple_plansource = plansource;
expr->expr_simple_plan = cplan;
- expr->expr_simple_plan_lxid = MyProc->lxid;
+ expr->expr_simple_plan_lxid = MyProc->vxid.lxid;
/* Share the remaining work with the replan code path */
exec_save_simple_expr(expr, cplan);
--
2.39.2
v11-0002-Replace-BackendIds-with-0-based-ProcNumbers.patchtext/x-patch; charset=UTF-8; name=v11-0002-Replace-BackendIds-with-0-based-ProcNumbers.patchDownload
From 8c9fd4b2609d51e94b3a1a156a45d6c6f98f4162 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Thu, 22 Feb 2024 02:29:21 +0200
Subject: [PATCH v11 2/2] Replace BackendIds with 0-based ProcNumbers
Now that BackendId was just another index into the proc array, it was
redundant with the 0-based proc numbers used in other places. Replace
all usage of backend IDs with proc numbers.
The only place where the term "backend id" remains is in a few pgstat
functions that expose backend IDs at the SQL level. Those IDs are now
in fact 0-based ProcNumbers too, but the documentation still calls
them "backend ids". That term still seems appropriate to describe what
the numbers are, so I let it be.
One user-visible effect is that pg_temp_0 is now a valid temp schema
name, for backend with ProcNumber 0.
Discussion: https://www.postgresql.org/message-id/8171f1aa-496f-46a6-afc3-c46fe7a9b407@iki.fi
---
doc/src/sgml/config.sgml | 2 +-
doc/src/sgml/monitoring.sgml | 18 ++--
doc/src/sgml/storage.sgml | 2 +-
doc/src/sgml/xact.sgml | 4 +-
src/backend/access/transam/README | 10 +--
src/backend/access/transam/clog.c | 18 ++--
src/backend/access/transam/multixact.c | 64 +++++++------
src/backend/access/transam/parallel.c | 10 +--
src/backend/access/transam/twophase.c | 43 ++++-----
src/backend/access/transam/xact.c | 9 +-
src/backend/access/transam/xlogprefetcher.c | 2 +-
src/backend/access/transam/xlogutils.c | 6 +-
src/backend/backup/basebackup_incremental.c | 2 +-
src/backend/catalog/catalog.c | 14 +--
src/backend/catalog/namespace.c | 30 +++----
src/backend/catalog/storage.c | 30 +++----
src/backend/commands/async.c | 89 +++++++++----------
src/backend/commands/dbcommands.c | 2 +-
src/backend/commands/indexcmds.c | 2 +-
src/backend/commands/sequence.c | 2 +-
src/backend/libpq/pqmq.c | 10 +--
src/backend/postmaster/pgarch.c | 12 +--
src/backend/postmaster/walsummarizer.c | 16 ++--
.../replication/logical/applyparallelworker.c | 4 +-
src/backend/replication/slot.c | 2 +-
src/backend/replication/walsender.c | 2 +-
src/backend/storage/buffer/buf_init.c | 2 +-
src/backend/storage/buffer/bufmgr.c | 30 +++----
src/backend/storage/buffer/localbuf.c | 6 +-
src/backend/storage/ipc/procarray.c | 44 ++++-----
src/backend/storage/ipc/procsignal.c | 36 ++++----
src/backend/storage/ipc/sinvaladt.c | 45 +++++-----
src/backend/storage/ipc/standby.c | 6 +-
src/backend/storage/lmgr/lmgr.c | 2 +-
src/backend/storage/lmgr/lock.c | 26 +++---
src/backend/storage/lmgr/predicate.c | 10 +--
src/backend/storage/lmgr/proc.c | 37 ++++----
src/backend/storage/smgr/md.c | 4 +-
src/backend/storage/smgr/smgr.c | 2 +-
src/backend/utils/activity/backend_status.c | 50 +++++------
src/backend/utils/adt/dbsize.c | 14 +--
src/backend/utils/adt/lockfuncs.c | 13 +--
src/backend/utils/adt/mcxtfuncs.c | 7 +-
src/backend/utils/adt/pgstatfuncs.c | 50 +++++------
src/backend/utils/cache/inval.c | 4 +-
src/backend/utils/cache/relcache.c | 30 +++----
src/backend/utils/error/csvlog.c | 4 +-
src/backend/utils/error/elog.c | 6 +-
src/backend/utils/error/jsonlog.c | 4 +-
src/backend/utils/init/globals.c | 6 +-
src/backend/utils/time/snapmgr.c | 6 +-
src/common/relpath.c | 24 ++---
src/include/access/twophase.h | 2 +-
src/include/catalog/namespace.h | 3 +-
src/include/common/relpath.h | 4 +-
src/include/libpq/pqmq.h | 2 +-
src/include/miscadmin.h | 5 --
src/include/postmaster/postmaster.h | 2 +-
src/include/storage/backendid.h | 41 ---------
src/include/storage/lock.h | 18 ++--
src/include/storage/proc.h | 46 +++++-----
src/include/storage/procarray.h | 8 +-
src/include/storage/proclist.h | 38 ++++----
src/include/storage/proclist_types.h | 18 ++--
src/include/storage/procnumber.h | 43 +++++++++
src/include/storage/procsignal.h | 4 +-
src/include/storage/relfilelocator.h | 29 +++---
src/include/storage/sinval.h | 4 +-
src/include/storage/smgr.h | 2 +-
src/include/utils/backend_status.h | 14 ++-
src/include/utils/rel.h | 2 +-
71 files changed, 575 insertions(+), 583 deletions(-)
delete mode 100644 src/include/storage/backendid.h
create mode 100644 src/include/storage/procnumber.h
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index ffd711b7f21..16462b43fa9 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -7338,7 +7338,7 @@ local0.* /var/log/postgresql
</row>
<row>
<entry><literal>%v</literal></entry>
- <entry>Virtual transaction ID (backendID/localXID); see
+ <entry>Virtual transaction ID (procNumber/localXID); see
<xref linkend="transaction-id"/></entry>
<entry>no</entry>
</row>
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 5cf9363ac82..c39c37a9d80 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -4933,7 +4933,7 @@ description | Waiting for a newly initialized WAL file to reach durable storage
access functions can be used; these are shown in <xref
linkend="monitoring-stats-backend-funcs-table"/>.
These access functions use the session's backend ID number, which is a
- small positive integer that is distinct from the backend ID of any
+ small integer (>= 0) that is distinct from the backend ID of any
concurrent session, although a session's ID can be recycled as soon as
it exits. The backend ID is used, among other things, to identify the
session's temporary schema if it has one.
@@ -6806,7 +6806,7 @@ FROM pg_stat_get_backend_idset() AS backendid;
arg0 contains the fork to be extended. arg1, arg2, and arg3 contain the
tablespace, database, and relation OIDs identifying the relation. arg4
is the ID of the backend which created the temporary relation for a
- local buffer, or <symbol>InvalidBackendId</symbol> (-1) for a shared
+ local buffer, or <symbol>INVALID_PROC_NUMBER</symbol> (-1) for a shared
buffer. arg5 is the number of blocks the caller would like to extend
by.</entry>
</row>
@@ -6817,7 +6817,7 @@ FROM pg_stat_get_backend_idset() AS backendid;
arg0 contains the fork to be extended. arg1, arg2, and arg3 contain the
tablespace, database, and relation OIDs identifying the relation. arg4
is the ID of the backend which created the temporary relation for a
- local buffer, or <symbol>InvalidBackendId</symbol> (-1) for a shared
+ local buffer, or <symbol>INVALID_PROC_NUMBER</symbol> (-1) for a shared
buffer. arg5 is the number of blocks the relation was extended by, this
can be less than the number in the
<literal>buffer-extend-start</literal> due to resource
@@ -6832,7 +6832,7 @@ FROM pg_stat_get_backend_idset() AS backendid;
arg2, arg3, and arg4 contain the tablespace, database, and relation OIDs
identifying the relation.
arg5 is the ID of the backend which created the temporary relation for a
- local buffer, or <symbol>InvalidBackendId</symbol> (-1) for a shared buffer.
+ local buffer, or <symbol>INVALID_PROC_NUMBER</symbol> (-1) for a shared buffer.
</entry>
</row>
<row>
@@ -6843,7 +6843,7 @@ FROM pg_stat_get_backend_idset() AS backendid;
arg2, arg3, and arg4 contain the tablespace, database, and relation OIDs
identifying the relation.
arg5 is the ID of the backend which created the temporary relation for a
- local buffer, or <symbol>InvalidBackendId</symbol> (-1) for a shared buffer.
+ local buffer, or <symbol>INVALID_PROC_NUMBER</symbol> (-1) for a shared buffer.
arg6 is true if the buffer was found in the pool, false if not.</entry>
</row>
<row>
@@ -6896,7 +6896,7 @@ FROM pg_stat_get_backend_idset() AS backendid;
arg2, arg3, and arg4 contain the tablespace, database, and relation OIDs
identifying the relation.
arg5 is the ID of the backend which created the temporary relation for a
- local buffer, or <symbol>InvalidBackendId</symbol> (-1) for a shared buffer.</entry>
+ local buffer, or <symbol>INVALID_PROC_NUMBER</symbol> (-1) for a shared buffer.</entry>
</row>
<row>
<entry><literal>smgr-md-read-done</literal></entry>
@@ -6906,7 +6906,7 @@ FROM pg_stat_get_backend_idset() AS backendid;
arg2, arg3, and arg4 contain the tablespace, database, and relation OIDs
identifying the relation.
arg5 is the ID of the backend which created the temporary relation for a
- local buffer, or <symbol>InvalidBackendId</symbol> (-1) for a shared buffer.
+ local buffer, or <symbol>INVALID_PROC_NUMBER</symbol> (-1) for a shared buffer.
arg6 is the number of bytes actually read, while arg7 is the number
requested (if these are different it indicates a short read).</entry>
</row>
@@ -6918,7 +6918,7 @@ FROM pg_stat_get_backend_idset() AS backendid;
arg2, arg3, and arg4 contain the tablespace, database, and relation OIDs
identifying the relation.
arg5 is the ID of the backend which created the temporary relation for a
- local buffer, or <symbol>InvalidBackendId</symbol> (-1) for a shared buffer.</entry>
+ local buffer, or <symbol>INVALID_PROC_NUMBER</symbol> (-1) for a shared buffer.</entry>
</row>
<row>
<entry><literal>smgr-md-write-done</literal></entry>
@@ -6928,7 +6928,7 @@ FROM pg_stat_get_backend_idset() AS backendid;
arg2, arg3, and arg4 contain the tablespace, database, and relation OIDs
identifying the relation.
arg5 is the ID of the backend which created the temporary relation for a
- local buffer, or <symbol>InvalidBackendId</symbol> (-1) for a shared buffer.
+ local buffer, or <symbol>INVALID_PROC_NUMBER</symbol> (-1) for a shared buffer.
arg6 is the number of bytes actually written, while arg7 is the number
requested (if these are different it indicates a short write).</entry>
</row>
diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml
index 3ea4e5526df..652946db7d7 100644
--- a/doc/src/sgml/storage.sgml
+++ b/doc/src/sgml/storage.sgml
@@ -202,7 +202,7 @@ these files are named after the table or index's <firstterm>filenode</firstterm>
which can be found in <structname>pg_class</structname>.<structfield>relfilenode</structfield>. But
for temporary relations, the file name is of the form
<literal>t<replaceable>BBB</replaceable>_<replaceable>FFF</replaceable></literal>, where <replaceable>BBB</replaceable>
-is the backend ID of the backend which created the file, and <replaceable>FFF</replaceable>
+is the process number of the backend which created the file, and <replaceable>FFF</replaceable>
is the filenode number. In either case, in addition to the main file (a/k/a
main fork), each table and index has a <firstterm>free space map</firstterm> (see <xref
linkend="storage-fsm"/>), which stores information about free space available in
diff --git a/doc/src/sgml/xact.sgml b/doc/src/sgml/xact.sgml
index 07b94b11329..3aa7ee1383e 100644
--- a/doc/src/sgml/xact.sgml
+++ b/doc/src/sgml/xact.sgml
@@ -26,10 +26,10 @@
Every transaction is identified by a unique
<literal>VirtualTransactionId</literal> (also called
<literal>virtualXID</literal> or <literal>vxid</literal>), which
- is comprised of a backend ID (or <literal>backendID</literal>)
+ is comprised of a backend's process number (or <literal>procNumber</literal>)
and a sequentially-assigned number local to each backend, known as
<literal>localXID</literal>. For example, the virtual transaction
- ID <literal>4/12532</literal> has a <literal>backendID</literal>
+ ID <literal>4/12532</literal> has a <literal>procNumber</literal>
of <literal>4</literal> and a <literal>localXID</literal> of
<literal>12532</literal>.
</para>
diff --git a/src/backend/access/transam/README b/src/backend/access/transam/README
index 6de2378748a..28d196cf62b 100644
--- a/src/backend/access/transam/README
+++ b/src/backend/access/transam/README
@@ -203,12 +203,12 @@ pg_subtrans and PGPROC are done at the time it is assigned.
A transaction that has no XID still needs to be identified for various
purposes, notably holding locks. For this purpose we assign a "virtual
transaction ID" or VXID to each top-level transaction. VXIDs are formed from
-two fields, the backendID and a backend-local counter; this arrangement allows
-assignment of a new VXID at transaction start without any contention for
-shared memory. To ensure that a VXID isn't re-used too soon after backend
+two fields, the procNumber and a backend-local counter; this arrangement
+allows assignment of a new VXID at transaction start without any contention
+for shared memory. To ensure that a VXID isn't re-used too soon after backend
exit, we store the last local counter value into shared memory at backend
-exit, and initialize it from the previous value for the same backendID slot
-at backend start. All these counters go back to zero at shared memory
+exit, and initialize it from the previous value for the same PGPROC slot at
+backend start. All these counters go back to zero at shared memory
re-initialization, but that's OK because VXIDs never appear anywhere on-disk.
Internally, a backend needs a way to identify subtransactions whether or not
diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c
index 97f7434da34..06b5ecf6a12 100644
--- a/src/backend/access/transam/clog.c
+++ b/src/backend/access/transam/clog.c
@@ -458,7 +458,7 @@ TransactionGroupUpdateXidStatus(TransactionId xid, XidStatus status,
* updates. This isn't likely and will still work, just maybe a bit
* less efficiently.
*/
- if (nextidx != INVALID_PGPROCNO &&
+ if (nextidx != INVALID_PROC_NUMBER &&
GetPGProcByNumber(nextidx)->clogGroupMemberPage != proc->clogGroupMemberPage)
{
/*
@@ -466,7 +466,7 @@ TransactionGroupUpdateXidStatus(TransactionId xid, XidStatus status,
* needs an XID status update.
*/
proc->clogGroupMember = false;
- pg_atomic_write_u32(&proc->clogGroupNext, INVALID_PGPROCNO);
+ pg_atomic_write_u32(&proc->clogGroupNext, INVALID_PROC_NUMBER);
return false;
}
@@ -482,9 +482,9 @@ TransactionGroupUpdateXidStatus(TransactionId xid, XidStatus status,
* If the list was not empty, the leader will update the status of our
* XID. It is impossible to have followers without a leader because the
* first process that has added itself to the list will always have
- * nextidx as INVALID_PGPROCNO.
+ * nextidx as INVALID_PROC_NUMBER.
*/
- if (nextidx != INVALID_PGPROCNO)
+ if (nextidx != INVALID_PROC_NUMBER)
{
int extraWaits = 0;
@@ -500,7 +500,7 @@ TransactionGroupUpdateXidStatus(TransactionId xid, XidStatus status,
}
pgstat_report_wait_end();
- Assert(pg_atomic_read_u32(&proc->clogGroupNext) == INVALID_PGPROCNO);
+ Assert(pg_atomic_read_u32(&proc->clogGroupNext) == INVALID_PROC_NUMBER);
/* Fix semaphore count for any absorbed wakeups */
while (extraWaits-- > 0)
@@ -517,13 +517,13 @@ TransactionGroupUpdateXidStatus(TransactionId xid, XidStatus status,
* Trying to pop elements one at a time could lead to an ABA problem.
*/
nextidx = pg_atomic_exchange_u32(&procglobal->clogGroupFirst,
- INVALID_PGPROCNO);
+ INVALID_PROC_NUMBER);
/* Remember head of list so we can perform wakeups after dropping lock. */
wakeidx = nextidx;
/* Walk the list and update the status of all XIDs. */
- while (nextidx != INVALID_PGPROCNO)
+ while (nextidx != INVALID_PROC_NUMBER)
{
PGPROC *nextproc = &ProcGlobal->allProcs[nextidx];
@@ -552,12 +552,12 @@ TransactionGroupUpdateXidStatus(TransactionId xid, XidStatus status,
* don't do this under the lock so as to keep lock hold times to a
* minimum.
*/
- while (wakeidx != INVALID_PGPROCNO)
+ while (wakeidx != INVALID_PROC_NUMBER)
{
PGPROC *wakeproc = &ProcGlobal->allProcs[wakeidx];
wakeidx = pg_atomic_read_u32(&wakeproc->clogGroupNext);
- pg_atomic_write_u32(&wakeproc->clogGroupNext, INVALID_PGPROCNO);
+ pg_atomic_write_u32(&wakeproc->clogGroupNext, INVALID_PROC_NUMBER);
/* ensure all previous writes are visible before follower continues. */
pg_write_barrier();
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index febc429f724..10de02595f5 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -236,11 +236,11 @@ typedef struct MultiXactStateData
/*
* Per-backend data starts here. We have two arrays stored in the area
* immediately following the MultiXactStateData struct. Each is indexed by
- * BackendId.
+ * ProcNumber.
*
- * In both arrays, there's a slot for all normal backends (1..MaxBackends)
- * followed by a slot for max_prepared_xacts prepared transactions. Valid
- * BackendIds start from 1; element zero of each array is never used.
+ * In both arrays, there's a slot for all normal backends
+ * (0..MaxBackends-1) followed by a slot for max_prepared_xacts prepared
+ * transactions.
*
* OldestMemberMXactId[k] is the oldest MultiXactId each backend's current
* transaction(s) could possibly be a member of, or InvalidMultiXactId
@@ -284,8 +284,7 @@ typedef struct MultiXactStateData
} MultiXactStateData;
/*
- * Last element of OldestMemberMXactId and OldestVisibleMXactId arrays.
- * Valid elements are (1..MaxOldestSlot); element 0 is never used.
+ * Size of OldestMemberMXactId and OldestVisibleMXactId arrays.
*/
#define MaxOldestSlot (MaxBackends + max_prepared_xacts)
@@ -396,7 +395,7 @@ MultiXactIdCreate(TransactionId xid1, MultiXactStatus status1,
Assert(!TransactionIdEquals(xid1, xid2) || (status1 != status2));
/* MultiXactIdSetOldestMember() must have been called already. */
- Assert(MultiXactIdIsValid(OldestMemberMXactId[MyBackendId]));
+ Assert(MultiXactIdIsValid(OldestMemberMXactId[MyProcNumber]));
/*
* Note: unlike MultiXactIdExpand, we don't bother to check that both XIDs
@@ -450,7 +449,7 @@ MultiXactIdExpand(MultiXactId multi, TransactionId xid, MultiXactStatus status)
Assert(TransactionIdIsValid(xid));
/* MultiXactIdSetOldestMember() must have been called already. */
- Assert(MultiXactIdIsValid(OldestMemberMXactId[MyBackendId]));
+ Assert(MultiXactIdIsValid(OldestMemberMXactId[MyProcNumber]));
debug_elog5(DEBUG2, "Expand: received multi %u, xid %u status %s",
multi, xid, mxstatus_to_string(status));
@@ -625,7 +624,7 @@ MultiXactIdIsRunning(MultiXactId multi, bool isLockOnly)
void
MultiXactIdSetOldestMember(void)
{
- if (!MultiXactIdIsValid(OldestMemberMXactId[MyBackendId]))
+ if (!MultiXactIdIsValid(OldestMemberMXactId[MyProcNumber]))
{
MultiXactId nextMXact;
@@ -654,12 +653,12 @@ MultiXactIdSetOldestMember(void)
if (nextMXact < FirstMultiXactId)
nextMXact = FirstMultiXactId;
- OldestMemberMXactId[MyBackendId] = nextMXact;
+ OldestMemberMXactId[MyProcNumber] = nextMXact;
LWLockRelease(MultiXactGenLock);
debug_elog4(DEBUG2, "MultiXact: setting OldestMember[%d] = %u",
- MyBackendId, nextMXact);
+ MyProcNumber, nextMXact);
}
}
@@ -682,7 +681,7 @@ MultiXactIdSetOldestMember(void)
static void
MultiXactIdSetOldestVisible(void)
{
- if (!MultiXactIdIsValid(OldestVisibleMXactId[MyBackendId]))
+ if (!MultiXactIdIsValid(OldestVisibleMXactId[MyProcNumber]))
{
MultiXactId oldestMXact;
int i;
@@ -698,7 +697,7 @@ MultiXactIdSetOldestVisible(void)
if (oldestMXact < FirstMultiXactId)
oldestMXact = FirstMultiXactId;
- for (i = 1; i <= MaxOldestSlot; i++)
+ for (i = 0; i < MaxOldestSlot; i++)
{
MultiXactId thisoldest = OldestMemberMXactId[i];
@@ -707,12 +706,12 @@ MultiXactIdSetOldestVisible(void)
oldestMXact = thisoldest;
}
- OldestVisibleMXactId[MyBackendId] = oldestMXact;
+ OldestVisibleMXactId[MyProcNumber] = oldestMXact;
LWLockRelease(MultiXactGenLock);
debug_elog4(DEBUG2, "MultiXact: setting OldestVisible[%d] = %u",
- MyBackendId, oldestMXact);
+ MyProcNumber, oldestMXact);
}
}
@@ -1266,7 +1265,7 @@ GetMultiXactIdMembers(MultiXactId multi, MultiXactMember **members,
* multi. It cannot possibly still be running.
*/
if (isLockOnly &&
- MultiXactIdPrecedes(multi, OldestVisibleMXactId[MyBackendId]))
+ MultiXactIdPrecedes(multi, OldestVisibleMXactId[MyProcNumber]))
{
debug_elog2(DEBUG2, "GetMembers: a locker-only multi is too old");
*members = NULL;
@@ -1694,8 +1693,8 @@ AtEOXact_MultiXact(void)
* We assume that storing a MultiXactId is atomic and so we need not take
* MultiXactGenLock to do this.
*/
- OldestMemberMXactId[MyBackendId] = InvalidMultiXactId;
- OldestVisibleMXactId[MyBackendId] = InvalidMultiXactId;
+ OldestMemberMXactId[MyProcNumber] = InvalidMultiXactId;
+ OldestVisibleMXactId[MyProcNumber] = InvalidMultiXactId;
/*
* Discard the local MultiXactId cache. Since MXactContext was created as
@@ -1715,7 +1714,7 @@ AtEOXact_MultiXact(void)
void
AtPrepare_MultiXact(void)
{
- MultiXactId myOldestMember = OldestMemberMXactId[MyBackendId];
+ MultiXactId myOldestMember = OldestMemberMXactId[MyProcNumber];
if (MultiXactIdIsValid(myOldestMember))
RegisterTwoPhaseRecord(TWOPHASE_RM_MULTIXACT_ID, 0,
@@ -1735,10 +1734,10 @@ PostPrepare_MultiXact(TransactionId xid)
* Transfer our OldestMemberMXactId value to the slot reserved for the
* prepared transaction.
*/
- myOldestMember = OldestMemberMXactId[MyBackendId];
+ myOldestMember = OldestMemberMXactId[MyProcNumber];
if (MultiXactIdIsValid(myOldestMember))
{
- BackendId dummyBackendId = TwoPhaseGetDummyBackendId(xid, false);
+ ProcNumber dummyProcNumber = TwoPhaseGetDummyProcNumber(xid, false);
/*
* Even though storing MultiXactId is atomic, acquire lock to make
@@ -1748,8 +1747,8 @@ PostPrepare_MultiXact(TransactionId xid)
*/
LWLockAcquire(MultiXactGenLock, LW_EXCLUSIVE);
- OldestMemberMXactId[dummyBackendId] = myOldestMember;
- OldestMemberMXactId[MyBackendId] = InvalidMultiXactId;
+ OldestMemberMXactId[dummyProcNumber] = myOldestMember;
+ OldestMemberMXactId[MyProcNumber] = InvalidMultiXactId;
LWLockRelease(MultiXactGenLock);
}
@@ -1762,7 +1761,7 @@ PostPrepare_MultiXact(TransactionId xid)
* We assume that storing a MultiXactId is atomic and so we need not take
* MultiXactGenLock to do this.
*/
- OldestVisibleMXactId[MyBackendId] = InvalidMultiXactId;
+ OldestVisibleMXactId[MyProcNumber] = InvalidMultiXactId;
/*
* Discard the local MultiXactId cache like in AtEOXact_MultiXact.
@@ -1779,7 +1778,7 @@ void
multixact_twophase_recover(TransactionId xid, uint16 info,
void *recdata, uint32 len)
{
- BackendId dummyBackendId = TwoPhaseGetDummyBackendId(xid, false);
+ ProcNumber dummyProcNumber = TwoPhaseGetDummyProcNumber(xid, false);
MultiXactId oldestMember;
/*
@@ -1789,7 +1788,7 @@ multixact_twophase_recover(TransactionId xid, uint16 info,
Assert(len == sizeof(MultiXactId));
oldestMember = *((MultiXactId *) recdata);
- OldestMemberMXactId[dummyBackendId] = oldestMember;
+ OldestMemberMXactId[dummyProcNumber] = oldestMember;
}
/*
@@ -1800,11 +1799,11 @@ void
multixact_twophase_postcommit(TransactionId xid, uint16 info,
void *recdata, uint32 len)
{
- BackendId dummyBackendId = TwoPhaseGetDummyBackendId(xid, true);
+ ProcNumber dummyProcNumber = TwoPhaseGetDummyProcNumber(xid, true);
Assert(len == sizeof(MultiXactId));
- OldestMemberMXactId[dummyBackendId] = InvalidMultiXactId;
+ OldestMemberMXactId[dummyProcNumber] = InvalidMultiXactId;
}
/*
@@ -1828,9 +1827,9 @@ MultiXactShmemSize(void)
{
Size size;
- /* We need 2*MaxOldestSlot + 1 perBackendXactIds[] entries */
+ /* We need 2*MaxOldestSlot perBackendXactIds[] entries */
#define SHARED_MULTIXACT_STATE_SIZE \
- add_size(offsetof(MultiXactStateData, perBackendXactIds) + sizeof(MultiXactId), \
+ add_size(offsetof(MultiXactStateData, perBackendXactIds), \
mul_size(sizeof(MultiXactId) * 2, MaxOldestSlot))
size = SHARED_MULTIXACT_STATE_SIZE;
@@ -1880,8 +1879,7 @@ MultiXactShmemInit(void)
Assert(found);
/*
- * Set up array pointers. Note that perBackendXactIds[0] is wasted space
- * since we only use indexes 1..MaxOldestSlot in each array.
+ * Set up array pointers.
*/
OldestMemberMXactId = MultiXactState->perBackendXactIds;
OldestVisibleMXactId = OldestMemberMXactId + MaxOldestSlot;
@@ -2533,7 +2531,7 @@ GetOldestMultiXactId(void)
nextMXact = FirstMultiXactId;
oldestMXact = nextMXact;
- for (i = 1; i <= MaxOldestSlot; i++)
+ for (i = 0; i < MaxOldestSlot; i++)
{
MultiXactId thisoldest;
diff --git a/src/backend/access/transam/parallel.c b/src/backend/access/transam/parallel.c
index 849a03e4b65..cbfe6087da4 100644
--- a/src/backend/access/transam/parallel.c
+++ b/src/backend/access/transam/parallel.c
@@ -94,7 +94,7 @@ typedef struct FixedParallelState
bool is_superuser;
PGPROC *parallel_leader_pgproc;
pid_t parallel_leader_pid;
- BackendId parallel_leader_backend_id;
+ ProcNumber parallel_leader_proc_number;
TimestampTz xact_ts;
TimestampTz stmt_ts;
SerializableXactHandle serializable_xact_handle;
@@ -337,7 +337,7 @@ InitializeParallelDSM(ParallelContext *pcxt)
&fps->temp_toast_namespace_id);
fps->parallel_leader_pgproc = MyProc;
fps->parallel_leader_pid = MyProcPid;
- fps->parallel_leader_backend_id = MyBackendId;
+ fps->parallel_leader_proc_number = MyProcNumber;
fps->xact_ts = GetCurrentTransactionStartTimestamp();
fps->stmt_ts = GetCurrentStatementStartTimestamp();
fps->serializable_xact_handle = ShareSerializableXact();
@@ -1351,7 +1351,7 @@ ParallelWorkerMain(Datum main_arg)
/* Arrange to signal the leader if we exit. */
ParallelLeaderPid = fps->parallel_leader_pid;
- ParallelLeaderBackendId = fps->parallel_leader_backend_id;
+ ParallelLeaderProcNumber = fps->parallel_leader_proc_number;
before_shmem_exit(ParallelWorkerShutdown, PointerGetDatum(seg));
/*
@@ -1367,7 +1367,7 @@ ParallelWorkerMain(Datum main_arg)
mqh = shm_mq_attach(mq, seg, NULL);
pq_redirect_to_shm_mq(seg, mqh);
pq_set_parallel_leader(fps->parallel_leader_pid,
- fps->parallel_leader_backend_id);
+ fps->parallel_leader_proc_number);
/*
* Send a BackendKeyData message to the process that initiated parallelism
@@ -1594,7 +1594,7 @@ ParallelWorkerShutdown(int code, Datum arg)
{
SendProcSignal(ParallelLeaderPid,
PROCSIG_PARALLEL_MESSAGE,
- ParallelLeaderBackendId);
+ ParallelLeaderProcNumber);
dsm_detach((dsm_segment *) DatumGetPointer(arg));
}
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 5c282002900..a42af293437 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -132,7 +132,7 @@ int max_prepared_xacts = 0;
*
* 3. To begin COMMIT PREPARED or ROLLBACK PREPARED, check that the entry is
* valid and not locked, then mark the entry as locked by storing my current
- * backend ID into locking_backend. This prevents concurrent attempts to
+ * proc number into locking_backend. This prevents concurrent attempts to
* commit or rollback the same prepared xact.
*
* 4. On completion of COMMIT PREPARED or ROLLBACK PREPARED, remove the entry
@@ -165,7 +165,7 @@ typedef struct GlobalTransactionData
TransactionId xid; /* The GXACT id */
Oid owner; /* ID of user that executed the xact */
- BackendId locking_backend; /* backend currently working on the xact */
+ ProcNumber locking_backend; /* backend currently working on the xact */
bool valid; /* true if PGPROC entry is in proc array */
bool ondisk; /* true if prepare state file is on disk */
bool inredo; /* true if entry was added via xlog_redo */
@@ -333,7 +333,7 @@ AtAbort_Twophase(void)
if (!MyLockedGxact->valid)
RemoveGXact(MyLockedGxact);
else
- MyLockedGxact->locking_backend = InvalidBackendId;
+ MyLockedGxact->locking_backend = INVALID_PROC_NUMBER;
LWLockRelease(TwoPhaseStateLock);
MyLockedGxact = NULL;
@@ -347,7 +347,7 @@ void
PostPrepare_Twophase(void)
{
LWLockAcquire(TwoPhaseStateLock, LW_EXCLUSIVE);
- MyLockedGxact->locking_backend = InvalidBackendId;
+ MyLockedGxact->locking_backend = INVALID_PROC_NUMBER;
LWLockRelease(TwoPhaseStateLock);
MyLockedGxact = NULL;
@@ -452,14 +452,14 @@ MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid, const char *gid,
{
/* clone VXID, for TwoPhaseGetXidByVirtualXID() to find */
proc->vxid.lxid = MyProc->vxid.lxid;
- proc->vxid.backendId = MyBackendId;
+ proc->vxid.procNumber = MyProcNumber;
}
else
{
Assert(AmStartupProcess() || !IsPostmasterEnvironment);
/* GetLockConflicts() uses this to specify a wait on the XID */
proc->vxid.lxid = xid;
- proc->vxid.backendId = InvalidBackendId;
+ proc->vxid.procNumber = INVALID_PROC_NUMBER;
}
proc->xid = xid;
Assert(proc->xmin == InvalidTransactionId);
@@ -484,7 +484,7 @@ MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid, const char *gid,
gxact->prepared_at = prepared_at;
gxact->xid = xid;
gxact->owner = owner;
- gxact->locking_backend = MyBackendId;
+ gxact->locking_backend = MyProcNumber;
gxact->valid = false;
gxact->inredo = false;
strcpy(gxact->gid, gid);
@@ -577,7 +577,7 @@ LockGXact(const char *gid, Oid user)
continue;
/* Found it, but has someone else got it locked? */
- if (gxact->locking_backend != InvalidBackendId)
+ if (gxact->locking_backend != INVALID_PROC_NUMBER)
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("prepared transaction with identifier \"%s\" is busy",
@@ -602,7 +602,7 @@ LockGXact(const char *gid, Oid user)
errhint("Connect to the database where the transaction was prepared to finish it.")));
/* OK for me to lock it */
- gxact->locking_backend = MyBackendId;
+ gxact->locking_backend = MyProcNumber;
MyLockedGxact = gxact;
LWLockRelease(TwoPhaseStateLock);
@@ -849,7 +849,7 @@ TwoPhaseGetGXact(TransactionId xid, bool lock_held)
*
* (This won't find recovered xacts.) If more than one matches, return any
* and set "have_more" to true. To witness multiple matches, a single
- * BackendId must consume 2^32 LXIDs, with no intervening database restart.
+ * proc number must consume 2^32 LXIDs, with no intervening database restart.
*/
TransactionId
TwoPhaseGetXidByVirtualXID(VirtualTransactionId vxid,
@@ -873,7 +873,10 @@ TwoPhaseGetXidByVirtualXID(VirtualTransactionId vxid,
GET_VXID_FROM_PGPROC(proc_vxid, *proc);
if (VirtualTransactionIdEquals(vxid, proc_vxid))
{
- /* Startup process sets proc->backendId to InvalidBackendId. */
+ /*
+ * Startup process sets proc->vxid.procNumber to
+ * INVALID_PROC_NUMBER.
+ */
Assert(!gxact->inredo);
if (result != InvalidTransactionId)
@@ -891,20 +894,20 @@ TwoPhaseGetXidByVirtualXID(VirtualTransactionId vxid,
}
/*
- * TwoPhaseGetDummyBackendId
- * Get the dummy backend ID for prepared transaction specified by XID
+ * TwoPhaseGetDummyProcNumber
+ * Get the dummy proc number for prepared transaction specified by XID
*
- * Dummy backend IDs are similar to real backend IDs of real backends.
- * They start at MaxBackends + 1, and are unique across all currently active
- * real backends and prepared transactions. If lock_held is set to true,
+ * Dummy proc numbers are similar to proc numbers of real backends. They
+ * start at MaxBackends, and are unique across all currently active real
+ * backends and prepared transactions. If lock_held is set to true,
* TwoPhaseStateLock will not be taken, so the caller had better hold it.
*/
-BackendId
-TwoPhaseGetDummyBackendId(TransactionId xid, bool lock_held)
+ProcNumber
+TwoPhaseGetDummyProcNumber(TransactionId xid, bool lock_held)
{
GlobalTransaction gxact = TwoPhaseGetGXact(xid, lock_held);
- return gxact->pgprocno + 1;
+ return gxact->pgprocno;
}
/*
@@ -2549,7 +2552,7 @@ PrepareRedoAdd(char *buf, XLogRecPtr start_lsn,
gxact->prepare_end_lsn = end_lsn;
gxact->xid = hdr->xid;
gxact->owner = hdr->owner;
- gxact->locking_backend = InvalidBackendId;
+ gxact->locking_backend = INVALID_PROC_NUMBER;
gxact->valid = false;
gxact->ondisk = XLogRecPtrIsInvalid(start_lsn);
gxact->inredo = true; /* yes, added in redo */
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 4ac5b9ea834..e270007e28e 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -2084,10 +2084,10 @@ StartTransaction(void)
AtStart_ResourceOwner();
/*
- * Assign a new LocalTransactionId, and combine it with the backendId to
+ * Assign a new LocalTransactionId, and combine it with the proc number to
* form a virtual transaction id.
*/
- vxid.backendId = MyBackendId;
+ vxid.procNumber = MyProcNumber;
vxid.localTransactionId = GetNextLocalTransactionId();
/*
@@ -2097,9 +2097,10 @@ StartTransaction(void)
/*
* Advertise it in the proc array. We assume assignment of
- * localTransactionId is atomic, and the backendId should be set already.
+ * localTransactionId is atomic, and the proc number should be set
+ * already.
*/
- Assert(MyProc->vxid.backendId == vxid.backendId);
+ Assert(MyProc->vxid.procNumber == vxid.procNumber);
MyProc->vxid.lxid = vxid.localTransactionId;
TRACE_POSTGRESQL_TRANSACTION_START(vxid.localTransactionId);
diff --git a/src/backend/access/transam/xlogprefetcher.c b/src/backend/access/transam/xlogprefetcher.c
index 5885f5f643a..aa26ea2ef73 100644
--- a/src/backend/access/transam/xlogprefetcher.c
+++ b/src/backend/access/transam/xlogprefetcher.c
@@ -722,7 +722,7 @@ XLogPrefetcherNextBlock(uintptr_t pgsr_private, XLogRecPtr *lsn)
* same relation (with some scheme to handle invalidations
* safely), but for now we'll call smgropen() every time.
*/
- reln = smgropen(block->rlocator, InvalidBackendId);
+ reln = smgropen(block->rlocator, INVALID_PROC_NUMBER);
/*
* If the relation file doesn't exist on disk, for example because
diff --git a/src/backend/access/transam/xlogutils.c b/src/backend/access/transam/xlogutils.c
index ad93035d508..f65554eb8f7 100644
--- a/src/backend/access/transam/xlogutils.c
+++ b/src/backend/access/transam/xlogutils.c
@@ -491,7 +491,7 @@ XLogReadBufferExtended(RelFileLocator rlocator, ForkNumber forknum,
}
/* Open the relation at smgr level */
- smgr = smgropen(rlocator, InvalidBackendId);
+ smgr = smgropen(rlocator, INVALID_PROC_NUMBER);
/*
* Create the target file if it doesn't already exist. This lets us cope
@@ -598,7 +598,7 @@ CreateFakeRelcacheEntry(RelFileLocator rlocator)
* We will never be working with temp rels during recovery or while
* syncing WAL-skipped files.
*/
- rel->rd_backend = InvalidBackendId;
+ rel->rd_backend = INVALID_PROC_NUMBER;
/* It must be a permanent table here */
rel->rd_rel->relpersistence = RELPERSISTENCE_PERMANENT;
@@ -620,7 +620,7 @@ CreateFakeRelcacheEntry(RelFileLocator rlocator)
* Set up a non-pinned SMgrRelation reference, so that we don't need to
* worry about unpinning it on error.
*/
- rel->rd_smgr = smgropen(rlocator, InvalidBackendId);
+ rel->rd_smgr = smgropen(rlocator, INVALID_PROC_NUMBER);
return rel;
}
diff --git a/src/backend/backup/basebackup_incremental.c b/src/backend/backup/basebackup_incremental.c
index e994ee66bbf..1c2172b8bd0 100644
--- a/src/backend/backup/basebackup_incremental.c
+++ b/src/backend/backup/basebackup_incremental.c
@@ -672,7 +672,7 @@ GetIncrementalFilePath(Oid dboid, Oid spcoid, RelFileNumber relfilenumber,
char *lastslash;
char *ipath;
- path = GetRelationPath(dboid, spcoid, relfilenumber, InvalidBackendId,
+ path = GetRelationPath(dboid, spcoid, relfilenumber, INVALID_PROC_NUMBER,
forknum);
lastslash = strrchr(path, '/');
diff --git a/src/backend/catalog/catalog.c b/src/backend/catalog/catalog.c
index 1cf4e303ce6..e151dddab29 100644
--- a/src/backend/catalog/catalog.c
+++ b/src/backend/catalog/catalog.c
@@ -504,7 +504,7 @@ GetNewRelFileNumber(Oid reltablespace, Relation pg_class, char relpersistence)
RelFileLocatorBackend rlocator;
char *rpath;
bool collides;
- BackendId backend;
+ ProcNumber procNumber;
/*
* If we ever get here during pg_upgrade, there's something wrong; all
@@ -516,11 +516,11 @@ GetNewRelFileNumber(Oid reltablespace, Relation pg_class, char relpersistence)
switch (relpersistence)
{
case RELPERSISTENCE_TEMP:
- backend = BackendIdForTempRelations();
+ procNumber = ProcNumberForTempRelations();
break;
case RELPERSISTENCE_UNLOGGED:
case RELPERSISTENCE_PERMANENT:
- backend = InvalidBackendId;
+ procNumber = INVALID_PROC_NUMBER;
break;
default:
elog(ERROR, "invalid relpersistence: %c", relpersistence);
@@ -534,11 +534,11 @@ GetNewRelFileNumber(Oid reltablespace, Relation pg_class, char relpersistence)
InvalidOid : MyDatabaseId;
/*
- * The relpath will vary based on the backend ID, so we must initialize
- * that properly here to make sure that any collisions based on filename
- * are properly detected.
+ * The relpath will vary based on the backend number, so we must
+ * initialize that properly here to make sure that any collisions based on
+ * filename are properly detected.
*/
- rlocator.backend = backend;
+ rlocator.backend = procNumber;
do
{
diff --git a/src/backend/catalog/namespace.c b/src/backend/catalog/namespace.c
index 620ce7e75d2..5521f5bd137 100644
--- a/src/backend/catalog/namespace.c
+++ b/src/backend/catalog/namespace.c
@@ -3714,18 +3714,18 @@ TempNamespaceStatus
checkTempNamespaceStatus(Oid namespaceId)
{
PGPROC *proc;
- int backendId;
+ ProcNumber procNumber;
Assert(OidIsValid(MyDatabaseId));
- backendId = GetTempNamespaceBackendId(namespaceId);
+ procNumber = GetTempNamespaceProcNumber(namespaceId);
/* No such namespace, or its name shows it's not temp? */
- if (backendId == InvalidBackendId)
+ if (procNumber == INVALID_PROC_NUMBER)
return TEMP_NAMESPACE_NOT_TEMP;
/* Is the backend alive? */
- proc = BackendIdGetProc(backendId);
+ proc = ProcNumberGetProc(procNumber);
if (proc == NULL)
return TEMP_NAMESPACE_IDLE;
@@ -3742,13 +3742,13 @@ checkTempNamespaceStatus(Oid namespaceId)
}
/*
- * GetTempNamespaceBackendId - if the given namespace is a temporary-table
- * namespace (either my own, or another backend's), return the BackendId
+ * GetTempNamespaceProcNumber - if the given namespace is a temporary-table
+ * namespace (either my own, or another backend's), return the proc number
* that owns it. Temporary-toast-table namespaces are included, too.
- * If it isn't a temp namespace, return InvalidBackendId.
+ * If it isn't a temp namespace, return INVALID_PROC_NUMBER.
*/
-int
-GetTempNamespaceBackendId(Oid namespaceId)
+ProcNumber
+GetTempNamespaceProcNumber(Oid namespaceId)
{
int result;
char *nspname;
@@ -3756,13 +3756,13 @@ GetTempNamespaceBackendId(Oid namespaceId)
/* See if the namespace name starts with "pg_temp_" or "pg_toast_temp_" */
nspname = get_namespace_name(namespaceId);
if (!nspname)
- return InvalidBackendId; /* no such namespace? */
+ return INVALID_PROC_NUMBER; /* no such namespace? */
if (strncmp(nspname, "pg_temp_", 8) == 0)
result = atoi(nspname + 8);
else if (strncmp(nspname, "pg_toast_temp_", 14) == 0)
result = atoi(nspname + 14);
else
- result = InvalidBackendId;
+ result = INVALID_PROC_NUMBER;
pfree(nspname);
return result;
}
@@ -4400,8 +4400,8 @@ InitTempTableNamespace(void)
/*
* Do not allow a Hot Standby session to make temp tables. Aside from
* problems with modifying the system catalogs, there is a naming
- * conflict: pg_temp_N belongs to the session with BackendId N on the
- * primary, not to a hot standby session with the same BackendId. We
+ * conflict: pg_temp_N belongs to the session with proc number N on the
+ * primary, not to a hot standby session with the same proc number. We
* should not be able to get here anyway due to XactReadOnly checks, but
* let's just make real sure. Note that this also backstops various
* operations that allow XactReadOnly transactions to modify temp tables;
@@ -4418,7 +4418,7 @@ InitTempTableNamespace(void)
(errcode(ERRCODE_READ_ONLY_SQL_TRANSACTION),
errmsg("cannot create temporary tables during a parallel operation")));
- snprintf(namespaceName, sizeof(namespaceName), "pg_temp_%d", MyBackendId);
+ snprintf(namespaceName, sizeof(namespaceName), "pg_temp_%d", MyProcNumber);
namespaceId = get_namespace_oid(namespaceName, true);
if (!OidIsValid(namespaceId))
@@ -4451,7 +4451,7 @@ InitTempTableNamespace(void)
* dropping a parent table should make its toast table go away.)
*/
snprintf(namespaceName, sizeof(namespaceName), "pg_toast_temp_%d",
- MyBackendId);
+ MyProcNumber);
toastspaceId = get_namespace_oid(namespaceName, true);
if (!OidIsValid(toastspaceId))
diff --git a/src/backend/catalog/storage.c b/src/backend/catalog/storage.c
index b155c03386e..cb168cf0b4c 100644
--- a/src/backend/catalog/storage.c
+++ b/src/backend/catalog/storage.c
@@ -60,7 +60,7 @@ int wal_skip_threshold = 2048; /* in kilobytes */
typedef struct PendingRelDelete
{
RelFileLocator rlocator; /* relation that may need to be deleted */
- BackendId backend; /* InvalidBackendId if not a temp rel */
+ ProcNumber procNumber; /* INVALID_PROC_NUMBER if not a temp rel */
bool atCommit; /* T=delete at commit; F=delete at abort */
int nestLevel; /* xact nesting level of request */
struct PendingRelDelete *next; /* linked-list link */
@@ -121,7 +121,7 @@ RelationCreateStorage(RelFileLocator rlocator, char relpersistence,
bool register_delete)
{
SMgrRelation srel;
- BackendId backend;
+ ProcNumber procNumber;
bool needs_wal;
Assert(!IsInParallelMode()); /* couldn't update pendingSyncHash */
@@ -129,15 +129,15 @@ RelationCreateStorage(RelFileLocator rlocator, char relpersistence,
switch (relpersistence)
{
case RELPERSISTENCE_TEMP:
- backend = BackendIdForTempRelations();
+ procNumber = ProcNumberForTempRelations();
needs_wal = false;
break;
case RELPERSISTENCE_UNLOGGED:
- backend = InvalidBackendId;
+ procNumber = INVALID_PROC_NUMBER;
needs_wal = false;
break;
case RELPERSISTENCE_PERMANENT:
- backend = InvalidBackendId;
+ procNumber = INVALID_PROC_NUMBER;
needs_wal = true;
break;
default:
@@ -145,7 +145,7 @@ RelationCreateStorage(RelFileLocator rlocator, char relpersistence,
return NULL; /* placate compiler */
}
- srel = smgropen(rlocator, backend);
+ srel = smgropen(rlocator, procNumber);
smgrcreate(srel, MAIN_FORKNUM, false);
if (needs_wal)
@@ -162,7 +162,7 @@ RelationCreateStorage(RelFileLocator rlocator, char relpersistence,
pending = (PendingRelDelete *)
MemoryContextAlloc(TopMemoryContext, sizeof(PendingRelDelete));
pending->rlocator = rlocator;
- pending->backend = backend;
+ pending->procNumber = procNumber;
pending->atCommit = false; /* delete if abort */
pending->nestLevel = GetCurrentTransactionNestLevel();
pending->next = pendingDeletes;
@@ -171,7 +171,7 @@ RelationCreateStorage(RelFileLocator rlocator, char relpersistence,
if (relpersistence == RELPERSISTENCE_PERMANENT && !XLogIsNeeded())
{
- Assert(backend == InvalidBackendId);
+ Assert(procNumber == INVALID_PROC_NUMBER);
AddPendingSync(&rlocator);
}
@@ -210,7 +210,7 @@ RelationDropStorage(Relation rel)
pending = (PendingRelDelete *)
MemoryContextAlloc(TopMemoryContext, sizeof(PendingRelDelete));
pending->rlocator = rel->rd_locator;
- pending->backend = rel->rd_backend;
+ pending->procNumber = rel->rd_backend;
pending->atCommit = true; /* delete if commit */
pending->nestLevel = GetCurrentTransactionNestLevel();
pending->next = pendingDeletes;
@@ -678,7 +678,7 @@ smgrDoPendingDeletes(bool isCommit)
{
SMgrRelation srel;
- srel = smgropen(pending->rlocator, pending->backend);
+ srel = smgropen(pending->rlocator, pending->procNumber);
/* allocate the initial array, or extend it, if needed */
if (maxrels == 0)
@@ -759,7 +759,7 @@ smgrDoPendingSyncs(bool isCommit, bool isParallelWorker)
BlockNumber total_blocks = 0;
SMgrRelation srel;
- srel = smgropen(pendingsync->rlocator, InvalidBackendId);
+ srel = smgropen(pendingsync->rlocator, INVALID_PROC_NUMBER);
/*
* We emit newpage WAL records for smaller relations.
@@ -878,7 +878,7 @@ smgrGetPendingDeletes(bool forCommit, RelFileLocator **ptr)
for (pending = pendingDeletes; pending != NULL; pending = pending->next)
{
if (pending->nestLevel >= nestLevel && pending->atCommit == forCommit
- && pending->backend == InvalidBackendId)
+ && pending->procNumber == INVALID_PROC_NUMBER)
nrels++;
}
if (nrels == 0)
@@ -891,7 +891,7 @@ smgrGetPendingDeletes(bool forCommit, RelFileLocator **ptr)
for (pending = pendingDeletes; pending != NULL; pending = pending->next)
{
if (pending->nestLevel >= nestLevel && pending->atCommit == forCommit
- && pending->backend == InvalidBackendId)
+ && pending->procNumber == INVALID_PROC_NUMBER)
{
*rptr = pending->rlocator;
rptr++;
@@ -968,7 +968,7 @@ smgr_redo(XLogReaderState *record)
xl_smgr_create *xlrec = (xl_smgr_create *) XLogRecGetData(record);
SMgrRelation reln;
- reln = smgropen(xlrec->rlocator, InvalidBackendId);
+ reln = smgropen(xlrec->rlocator, INVALID_PROC_NUMBER);
smgrcreate(reln, xlrec->forkNum, true);
}
else if (info == XLOG_SMGR_TRUNCATE)
@@ -981,7 +981,7 @@ smgr_redo(XLogReaderState *record)
int nforks = 0;
bool need_fsm_vacuum = false;
- reln = smgropen(xlrec->rlocator, InvalidBackendId);
+ reln = smgropen(xlrec->rlocator, INVALID_PROC_NUMBER);
/*
* Forcibly create relation if it doesn't exist (which suggests that
diff --git a/src/backend/commands/async.c b/src/backend/commands/async.c
index 8b24b222931..f90463b369d 100644
--- a/src/backend/commands/async.c
+++ b/src/backend/commands/async.c
@@ -246,7 +246,7 @@ typedef struct QueueBackendStatus
{
int32 pid; /* either a PID or InvalidPid */
Oid dboid; /* backend's database OID, or InvalidOid */
- BackendId nextListener; /* id of next listener, or InvalidBackendId */
+ ProcNumber nextListener; /* id of next listener, or INVALID_PROC_NUMBER */
QueuePosition pos; /* backend has read queue up to here */
} QueueBackendStatus;
@@ -271,13 +271,12 @@ typedef struct QueueBackendStatus
* NotifyQueueTailLock, then NotifyQueueLock, and lastly NotifySLRULock.
*
* Each backend uses the backend[] array entry with index equal to its
- * BackendId (which can range from 1 to MaxBackends). We rely on this to make
- * SendProcSignal fast.
+ * ProcNumber. We rely on this to make SendProcSignal fast.
*
* The backend[] array entries for actively-listening backends are threaded
* together using firstListener and the nextListener links, so that we can
* scan them without having to iterate over inactive entries. We keep this
- * list in order by BackendId so that the scan is cache-friendly when there
+ * list in order by ProcNumber so that the scan is cache-friendly when there
* are many active entries.
*/
typedef struct AsyncQueueControl
@@ -287,10 +286,10 @@ typedef struct AsyncQueueControl
* listening backend */
int stopPage; /* oldest unrecycled page; must be <=
* tail.page */
- BackendId firstListener; /* id of first listener, or InvalidBackendId */
+ ProcNumber firstListener; /* id of first listener, or
+ * INVALID_PROC_NUMBER */
TimestampTz lastQueueFillWarn; /* time of last queue-full msg */
QueueBackendStatus backend[FLEXIBLE_ARRAY_MEMBER];
- /* backend[0] is not used; used entries are from [1] to [MaxBackends] */
} AsyncQueueControl;
static AsyncQueueControl *asyncQueueControl;
@@ -489,7 +488,7 @@ AsyncShmemSize(void)
Size size;
/* This had better match AsyncShmemInit */
- size = mul_size(MaxBackends + 1, sizeof(QueueBackendStatus));
+ size = mul_size(MaxBackends, sizeof(QueueBackendStatus));
size = add_size(size, offsetof(AsyncQueueControl, backend));
size = add_size(size, SimpleLruShmemSize(NUM_NOTIFY_BUFFERS, 0));
@@ -508,11 +507,8 @@ AsyncShmemInit(void)
/*
* Create or attach to the AsyncQueueControl structure.
- *
- * The used entries in the backend[] array run from 1 to MaxBackends; the
- * zero'th entry is unused but must be allocated.
*/
- size = mul_size(MaxBackends + 1, sizeof(QueueBackendStatus));
+ size = mul_size(MaxBackends, sizeof(QueueBackendStatus));
size = add_size(size, offsetof(AsyncQueueControl, backend));
asyncQueueControl = (AsyncQueueControl *)
@@ -524,14 +520,13 @@ AsyncShmemInit(void)
SET_QUEUE_POS(QUEUE_HEAD, 0, 0);
SET_QUEUE_POS(QUEUE_TAIL, 0, 0);
QUEUE_STOP_PAGE = 0;
- QUEUE_FIRST_LISTENER = InvalidBackendId;
+ QUEUE_FIRST_LISTENER = INVALID_PROC_NUMBER;
asyncQueueControl->lastQueueFillWarn = 0;
- /* zero'th entry won't be used, but let's initialize it anyway */
- for (int i = 0; i <= MaxBackends; i++)
+ for (int i = 0; i < MaxBackends; i++)
{
QUEUE_BACKEND_PID(i) = InvalidPid;
QUEUE_BACKEND_DBOID(i) = InvalidOid;
- QUEUE_NEXT_LISTENER(i) = InvalidBackendId;
+ QUEUE_NEXT_LISTENER(i) = INVALID_PROC_NUMBER;
SET_QUEUE_POS(QUEUE_BACKEND_POS(i), 0, 0);
}
}
@@ -1048,7 +1043,7 @@ Exec_ListenPreCommit(void)
{
QueuePosition head;
QueuePosition max;
- BackendId prevListener;
+ ProcNumber prevListener;
/*
* Nothing to do if we are already listening to something, nor if we
@@ -1093,28 +1088,28 @@ Exec_ListenPreCommit(void)
LWLockAcquire(NotifyQueueLock, LW_EXCLUSIVE);
head = QUEUE_HEAD;
max = QUEUE_TAIL;
- prevListener = InvalidBackendId;
- for (BackendId i = QUEUE_FIRST_LISTENER; i > 0; i = QUEUE_NEXT_LISTENER(i))
+ prevListener = INVALID_PROC_NUMBER;
+ for (ProcNumber i = QUEUE_FIRST_LISTENER; i != INVALID_PROC_NUMBER; i = QUEUE_NEXT_LISTENER(i))
{
if (QUEUE_BACKEND_DBOID(i) == MyDatabaseId)
max = QUEUE_POS_MAX(max, QUEUE_BACKEND_POS(i));
/* Also find last listening backend before this one */
- if (i < MyBackendId)
+ if (i < MyProcNumber)
prevListener = i;
}
- QUEUE_BACKEND_POS(MyBackendId) = max;
- QUEUE_BACKEND_PID(MyBackendId) = MyProcPid;
- QUEUE_BACKEND_DBOID(MyBackendId) = MyDatabaseId;
+ QUEUE_BACKEND_POS(MyProcNumber) = max;
+ QUEUE_BACKEND_PID(MyProcNumber) = MyProcPid;
+ QUEUE_BACKEND_DBOID(MyProcNumber) = MyDatabaseId;
/* Insert backend into list of listeners at correct position */
- if (prevListener > 0)
+ if (prevListener != INVALID_PROC_NUMBER)
{
- QUEUE_NEXT_LISTENER(MyBackendId) = QUEUE_NEXT_LISTENER(prevListener);
- QUEUE_NEXT_LISTENER(prevListener) = MyBackendId;
+ QUEUE_NEXT_LISTENER(MyProcNumber) = QUEUE_NEXT_LISTENER(prevListener);
+ QUEUE_NEXT_LISTENER(prevListener) = MyProcNumber;
}
else
{
- QUEUE_NEXT_LISTENER(MyBackendId) = QUEUE_FIRST_LISTENER;
- QUEUE_FIRST_LISTENER = MyBackendId;
+ QUEUE_NEXT_LISTENER(MyProcNumber) = QUEUE_FIRST_LISTENER;
+ QUEUE_FIRST_LISTENER = MyProcNumber;
}
LWLockRelease(NotifyQueueLock);
@@ -1246,23 +1241,23 @@ asyncQueueUnregister(void)
*/
LWLockAcquire(NotifyQueueLock, LW_EXCLUSIVE);
/* Mark our entry as invalid */
- QUEUE_BACKEND_PID(MyBackendId) = InvalidPid;
- QUEUE_BACKEND_DBOID(MyBackendId) = InvalidOid;
+ QUEUE_BACKEND_PID(MyProcNumber) = InvalidPid;
+ QUEUE_BACKEND_DBOID(MyProcNumber) = InvalidOid;
/* and remove it from the list */
- if (QUEUE_FIRST_LISTENER == MyBackendId)
- QUEUE_FIRST_LISTENER = QUEUE_NEXT_LISTENER(MyBackendId);
+ if (QUEUE_FIRST_LISTENER == MyProcNumber)
+ QUEUE_FIRST_LISTENER = QUEUE_NEXT_LISTENER(MyProcNumber);
else
{
- for (BackendId i = QUEUE_FIRST_LISTENER; i > 0; i = QUEUE_NEXT_LISTENER(i))
+ for (ProcNumber i = QUEUE_FIRST_LISTENER; i != INVALID_PROC_NUMBER; i = QUEUE_NEXT_LISTENER(i))
{
- if (QUEUE_NEXT_LISTENER(i) == MyBackendId)
+ if (QUEUE_NEXT_LISTENER(i) == MyProcNumber)
{
- QUEUE_NEXT_LISTENER(i) = QUEUE_NEXT_LISTENER(MyBackendId);
+ QUEUE_NEXT_LISTENER(i) = QUEUE_NEXT_LISTENER(MyProcNumber);
break;
}
}
}
- QUEUE_NEXT_LISTENER(MyBackendId) = InvalidBackendId;
+ QUEUE_NEXT_LISTENER(MyProcNumber) = INVALID_PROC_NUMBER;
LWLockRelease(NotifyQueueLock);
/* mark ourselves as no longer listed in the global array */
@@ -1533,7 +1528,7 @@ asyncQueueFillWarning(void)
QueuePosition min = QUEUE_HEAD;
int32 minPid = InvalidPid;
- for (BackendId i = QUEUE_FIRST_LISTENER; i > 0; i = QUEUE_NEXT_LISTENER(i))
+ for (ProcNumber i = QUEUE_FIRST_LISTENER; i != INVALID_PROC_NUMBER; i = QUEUE_NEXT_LISTENER(i))
{
Assert(QUEUE_BACKEND_PID(i) != InvalidPid);
min = QUEUE_POS_MIN(min, QUEUE_BACKEND_POS(i));
@@ -1564,7 +1559,7 @@ asyncQueueFillWarning(void)
* behind. Waken them anyway if they're far enough behind, so that they'll
* advance their queue position pointers, allowing the global tail to advance.
*
- * Since we know the BackendId and the Pid the signaling is quite cheap.
+ * Since we know the ProcNumber and the Pid the signaling is quite cheap.
*
* This is called during CommitTransaction(), so it's important for it
* to have very low probability of failure.
@@ -1573,7 +1568,7 @@ static void
SignalBackends(void)
{
int32 *pids;
- BackendId *ids;
+ ProcNumber *procnos;
int count;
/*
@@ -1585,11 +1580,11 @@ SignalBackends(void)
* preallocate the arrays? They're not that large, though.
*/
pids = (int32 *) palloc(MaxBackends * sizeof(int32));
- ids = (BackendId *) palloc(MaxBackends * sizeof(BackendId));
+ procnos = (ProcNumber *) palloc(MaxBackends * sizeof(ProcNumber));
count = 0;
LWLockAcquire(NotifyQueueLock, LW_EXCLUSIVE);
- for (BackendId i = QUEUE_FIRST_LISTENER; i > 0; i = QUEUE_NEXT_LISTENER(i))
+ for (ProcNumber i = QUEUE_FIRST_LISTENER; i != INVALID_PROC_NUMBER; i = QUEUE_NEXT_LISTENER(i))
{
int32 pid = QUEUE_BACKEND_PID(i);
QueuePosition pos;
@@ -1617,7 +1612,7 @@ SignalBackends(void)
}
/* OK, need to signal this one */
pids[count] = pid;
- ids[count] = i;
+ procnos[count] = i;
count++;
}
LWLockRelease(NotifyQueueLock);
@@ -1643,12 +1638,12 @@ SignalBackends(void)
* NotifyQueueLock; which is unlikely but certainly possible. So we
* just log a low-level debug message if it happens.
*/
- if (SendProcSignal(pid, PROCSIG_NOTIFY_INTERRUPT, ids[i]) < 0)
+ if (SendProcSignal(pid, PROCSIG_NOTIFY_INTERRUPT, procnos[i]) < 0)
elog(DEBUG3, "could not signal backend with PID %d: %m", pid);
}
pfree(pids);
- pfree(ids);
+ pfree(procnos);
}
/*
@@ -1856,8 +1851,8 @@ asyncQueueReadAllNotifications(void)
/* Fetch current state */
LWLockAcquire(NotifyQueueLock, LW_SHARED);
/* Assert checks that we have a valid state entry */
- Assert(MyProcPid == QUEUE_BACKEND_PID(MyBackendId));
- pos = QUEUE_BACKEND_POS(MyBackendId);
+ Assert(MyProcPid == QUEUE_BACKEND_PID(MyProcNumber));
+ pos = QUEUE_BACKEND_POS(MyProcNumber);
head = QUEUE_HEAD;
LWLockRelease(NotifyQueueLock);
@@ -1979,7 +1974,7 @@ asyncQueueReadAllNotifications(void)
{
/* Update shared state */
LWLockAcquire(NotifyQueueLock, LW_SHARED);
- QUEUE_BACKEND_POS(MyBackendId) = pos;
+ QUEUE_BACKEND_POS(MyProcNumber) = pos;
LWLockRelease(NotifyQueueLock);
}
PG_END_TRY();
@@ -2126,7 +2121,7 @@ asyncQueueAdvanceTail(void)
*/
LWLockAcquire(NotifyQueueLock, LW_EXCLUSIVE);
min = QUEUE_HEAD;
- for (BackendId i = QUEUE_FIRST_LISTENER; i > 0; i = QUEUE_NEXT_LISTENER(i))
+ for (ProcNumber i = QUEUE_FIRST_LISTENER; i != INVALID_PROC_NUMBER; i = QUEUE_NEXT_LISTENER(i))
{
Assert(QUEUE_BACKEND_PID(i) != InvalidPid);
min = QUEUE_POS_MIN(min, QUEUE_BACKEND_POS(i));
diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c
index b1327de71e8..78a158c3da4 100644
--- a/src/backend/commands/dbcommands.c
+++ b/src/backend/commands/dbcommands.c
@@ -276,7 +276,7 @@ ScanSourceDatabasePgClass(Oid tbid, Oid dbid, char *srcpath)
rlocator.dbOid = dbid;
rlocator.relNumber = relfilenumber;
- smgr = smgropen(rlocator, InvalidBackendId);
+ smgr = smgropen(rlocator, INVALID_PROC_NUMBER);
nblocks = smgrnblocks(smgr, MAIN_FORKNUM);
smgrclose(smgr);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 7a87626f5f0..06f90ff7522 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -476,7 +476,7 @@ WaitForOlderSnapshots(TransactionId limitXmin, bool progress)
/* If requested, publish who we're going to wait for. */
if (progress)
{
- PGPROC *holder = BackendIdGetProc(old_snapshots[i].backendId);
+ PGPROC *holder = ProcNumberGetProc(old_snapshots[i].procNumber);
if (holder)
pgstat_progress_update_param(PROGRESS_WAITFOR_CURRENT_PID,
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index f99dc48e8cf..7087c9f3e3f 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -342,7 +342,7 @@ fill_seq_with_data(Relation rel, HeapTuple tuple)
{
SMgrRelation srel;
- srel = smgropen(rel->rd_locator, InvalidBackendId);
+ srel = smgropen(rel->rd_locator, INVALID_PROC_NUMBER);
smgrcreate(srel, INIT_FORKNUM, false);
log_smgrcreate(&rel->rd_locator, INIT_FORKNUM);
fill_seq_fork_with_data(rel, tuple, INIT_FORKNUM);
diff --git a/src/backend/libpq/pqmq.c b/src/backend/libpq/pqmq.c
index 5764f8b2225..00a44ca803f 100644
--- a/src/backend/libpq/pqmq.c
+++ b/src/backend/libpq/pqmq.c
@@ -26,7 +26,7 @@
static shm_mq_handle *pq_mq_handle;
static bool pq_mq_busy = false;
static pid_t pq_mq_parallel_leader_pid = 0;
-static pid_t pq_mq_parallel_leader_backend_id = InvalidBackendId;
+static pid_t pq_mq_parallel_leader_proc_number = INVALID_PROC_NUMBER;
static void pq_cleanup_redirect_to_shm_mq(dsm_segment *seg, Datum arg);
static void mq_comm_reset(void);
@@ -75,11 +75,11 @@ pq_cleanup_redirect_to_shm_mq(dsm_segment *seg, Datum arg)
* message data via the shm_mq.
*/
void
-pq_set_parallel_leader(pid_t pid, BackendId backend_id)
+pq_set_parallel_leader(pid_t pid, ProcNumber procNumber)
{
Assert(PqCommMethods == &PqCommMqMethods);
pq_mq_parallel_leader_pid = pid;
- pq_mq_parallel_leader_backend_id = backend_id;
+ pq_mq_parallel_leader_proc_number = procNumber;
}
static void
@@ -168,13 +168,13 @@ mq_putmessage(char msgtype, const char *s, size_t len)
if (IsLogicalParallelApplyWorker())
SendProcSignal(pq_mq_parallel_leader_pid,
PROCSIG_PARALLEL_APPLY_MESSAGE,
- pq_mq_parallel_leader_backend_id);
+ pq_mq_parallel_leader_proc_number);
else
{
Assert(IsParallelWorker());
SendProcSignal(pq_mq_parallel_leader_pid,
PROCSIG_PARALLEL_MESSAGE,
- pq_mq_parallel_leader_backend_id);
+ pq_mq_parallel_leader_proc_number);
}
}
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 9c18e4b3efb..0a28b1360d6 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -80,7 +80,7 @@
/* Shared memory area for archiver process */
typedef struct PgArchData
{
- int pgprocno; /* pgprocno of archiver process */
+ int pgprocno; /* proc number of archiver process */
/*
* Forces a directory scan in pgarch_readyXlog(). Protected by arch_lck.
@@ -173,7 +173,7 @@ PgArchShmemInit(void)
{
/* First time through, so initialize */
MemSet(PgArch, 0, PgArchShmemSize());
- PgArch->pgprocno = INVALID_PGPROCNO;
+ PgArch->pgprocno = INVALID_PROC_NUMBER;
SpinLockInit(&PgArch->arch_lck);
}
}
@@ -239,8 +239,8 @@ PgArchiverMain(void)
on_shmem_exit(pgarch_die, 0);
/*
- * Advertise our pgprocno so that backends can use our latch to wake us up
- * while we're sleeping.
+ * Advertise our proc number so that backends can use our latch to wake us
+ * up while we're sleeping.
*/
PgArch->pgprocno = MyProcNumber;
@@ -274,7 +274,7 @@ PgArchWakeup(void)
* process' (or no process') latch. Even in that case the archiver will
* be relaunched shortly and will start archiving.
*/
- if (arch_pgprocno != INVALID_PGPROCNO)
+ if (arch_pgprocno != INVALID_PROC_NUMBER)
SetLatch(&ProcGlobal->allProcs[arch_pgprocno].procLatch);
}
@@ -752,7 +752,7 @@ pgarch_archiveDone(char *xlog)
static void
pgarch_die(int code, Datum arg)
{
- PgArch->pgprocno = INVALID_PGPROCNO;
+ PgArch->pgprocno = INVALID_PROC_NUMBER;
}
/*
diff --git a/src/backend/postmaster/walsummarizer.c b/src/backend/postmaster/walsummarizer.c
index f295eff32f4..8636d9424ec 100644
--- a/src/backend/postmaster/walsummarizer.c
+++ b/src/backend/postmaster/walsummarizer.c
@@ -71,8 +71,8 @@ typedef struct
* and so the LSN might point to the start of the next file even though
* that might happen to be in the middle of a WAL record.
*
- * summarizer_pgprocno is the pgprocno value for the summarizer process,
- * if one is running, or else INVALID_PGPROCNO.
+ * summarizer_pgprocno is the proc number of the summarizer process, if
+ * one is running, or else INVALID_PROC_NUMBER.
*
* pending_lsn is used by the summarizer to advertise the ending LSN of a
* record it has recently read. It shouldn't ever be less than
@@ -83,7 +83,7 @@ typedef struct
TimeLineID summarized_tli;
XLogRecPtr summarized_lsn;
bool lsn_is_exact;
- int summarizer_pgprocno;
+ ProcNumber summarizer_pgprocno;
XLogRecPtr pending_lsn;
/*
@@ -195,7 +195,7 @@ WalSummarizerShmemInit(void)
WalSummarizerCtl->summarized_tli = 0;
WalSummarizerCtl->summarized_lsn = InvalidXLogRecPtr;
WalSummarizerCtl->lsn_is_exact = false;
- WalSummarizerCtl->summarizer_pgprocno = INVALID_PGPROCNO;
+ WalSummarizerCtl->summarizer_pgprocno = INVALID_PROC_NUMBER;
WalSummarizerCtl->pending_lsn = InvalidXLogRecPtr;
ConditionVariableInit(&WalSummarizerCtl->summary_file_cv);
}
@@ -444,7 +444,7 @@ GetWalSummarizerState(TimeLineID *summarized_tli, XLogRecPtr *summarized_lsn,
*summarized_tli = WalSummarizerCtl->summarized_tli;
*summarized_lsn = WalSummarizerCtl->summarized_lsn;
- if (summarizer_pgprocno == INVALID_PGPROCNO)
+ if (summarizer_pgprocno == INVALID_PROC_NUMBER)
{
/*
* If the summarizer has exited, the fact that it had processed
@@ -613,7 +613,7 @@ GetOldestUnsummarizedLSN(TimeLineID *tli, bool *lsn_is_exact,
void
SetWalSummarizerLatch(void)
{
- int pgprocno;
+ ProcNumber pgprocno;
if (WalSummarizerCtl == NULL)
return;
@@ -622,7 +622,7 @@ SetWalSummarizerLatch(void)
pgprocno = WalSummarizerCtl->summarizer_pgprocno;
LWLockRelease(WALSummarizerLock);
- if (pgprocno != INVALID_PGPROCNO)
+ if (pgprocno != INVALID_PROC_NUMBER)
SetLatch(&ProcGlobal->allProcs[pgprocno].procLatch);
}
@@ -683,7 +683,7 @@ static void
WalSummarizerShutdown(int code, Datum arg)
{
LWLockAcquire(WALSummarizerLock, LW_EXCLUSIVE);
- WalSummarizerCtl->summarizer_pgprocno = INVALID_PGPROCNO;
+ WalSummarizerCtl->summarizer_pgprocno = INVALID_PROC_NUMBER;
LWLockRelease(WALSummarizerLock);
}
diff --git a/src/backend/replication/logical/applyparallelworker.c b/src/backend/replication/logical/applyparallelworker.c
index fb79ee1e7bf..e7f7d4c5e4b 100644
--- a/src/backend/replication/logical/applyparallelworker.c
+++ b/src/backend/replication/logical/applyparallelworker.c
@@ -845,7 +845,7 @@ pa_shutdown(int code, Datum arg)
{
SendProcSignal(MyLogicalRepWorker->leader_pid,
PROCSIG_PARALLEL_APPLY_MESSAGE,
- InvalidBackendId);
+ INVALID_PROC_NUMBER);
dsm_detach((dsm_segment *) DatumGetPointer(arg));
}
@@ -934,7 +934,7 @@ ParallelApplyWorkerMain(Datum main_arg)
pq_redirect_to_shm_mq(seg, error_mqh);
pq_set_parallel_leader(MyLogicalRepWorker->leader_pid,
- InvalidBackendId);
+ INVALID_PROC_NUMBER);
MyLogicalRepWorker->last_send_time = MyLogicalRepWorker->last_recv_time =
MyLogicalRepWorker->reply_time = 0;
diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c
index 233652b4799..7fd59923d0b 100644
--- a/src/backend/replication/slot.c
+++ b/src/backend/replication/slot.c
@@ -1637,7 +1637,7 @@ InvalidatePossiblyObsoleteSlot(ReplicationSlotInvalidationCause cause,
if (MyBackendType == B_STARTUP)
(void) SendProcSignal(active_pid,
PROCSIG_RECOVERY_CONFLICT_LOGICALSLOT,
- InvalidBackendId);
+ INVALID_PROC_NUMBER);
else
(void) kill(active_pid, SIGTERM);
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index 631d1e0c9fd..7b6f879b0cb 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -3646,7 +3646,7 @@ WalSndInitStopping(void)
if (pid == 0)
continue;
- SendProcSignal(pid, PROCSIG_WALSND_INIT_STOPPING, InvalidBackendId);
+ SendProcSignal(pid, PROCSIG_WALSND_INIT_STOPPING, INVALID_PROC_NUMBER);
}
}
diff --git a/src/backend/storage/buffer/buf_init.c b/src/backend/storage/buffer/buf_init.c
index 6a99372ca19..46116a1f64b 100644
--- a/src/backend/storage/buffer/buf_init.c
+++ b/src/backend/storage/buffer/buf_init.c
@@ -122,7 +122,7 @@ InitBufferPool(void)
ClearBufferTag(&buf->tag);
pg_atomic_init_u32(&buf->state, 0);
- buf->wait_backend_pgprocno = INVALID_PGPROCNO;
+ buf->wait_backend_pgprocno = INVALID_PROC_NUMBER;
buf->buf_id = i;
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index bdf89bbc4dc..c99730e0e5c 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -816,7 +816,7 @@ ReadBufferExtended(Relation reln, ForkNumber forkNum, BlockNumber blockNum,
* permanent = false for a RELPERSISTENCE_UNLOGGED relation. This function
* cannot be used for temporary relations (and making that work might be
* difficult, unless we only want to read temporary relations for our own
- * BackendId).
+ * ProcNumber).
*/
Buffer
ReadBufferWithoutRelcache(RelFileLocator rlocator, ForkNumber forkNum,
@@ -825,7 +825,7 @@ ReadBufferWithoutRelcache(RelFileLocator rlocator, ForkNumber forkNum,
{
bool hit;
- SMgrRelation smgr = smgropen(rlocator, InvalidBackendId);
+ SMgrRelation smgr = smgropen(rlocator, INVALID_PROC_NUMBER);
return ReadBuffer_common(smgr, permanent ? RELPERSISTENCE_PERMANENT :
RELPERSISTENCE_UNLOGGED, forkNum, blockNum,
@@ -3322,7 +3322,7 @@ DebugPrintBufferRefcount(Buffer buffer)
int32 loccount;
char *path;
char *result;
- BackendId backend;
+ ProcNumber backend;
uint32 buf_state;
Assert(BufferIsValid(buffer));
@@ -3330,13 +3330,13 @@ DebugPrintBufferRefcount(Buffer buffer)
{
buf = GetLocalBufferDescriptor(-buffer - 1);
loccount = LocalRefCount[-buffer - 1];
- backend = MyBackendId;
+ backend = MyProcNumber;
}
else
{
buf = GetBufferDescriptor(buffer - 1);
loccount = GetPrivateRefCount(buffer);
- backend = InvalidBackendId;
+ backend = INVALID_PROC_NUMBER;
}
/* theoretically we should lock the bufhdr here */
@@ -3461,7 +3461,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln, IOObject io_object,
/* Find smgr relation for buffer */
if (reln == NULL)
- reln = smgropen(BufTagGetRelFileLocator(&buf->tag), InvalidBackendId);
+ reln = smgropen(BufTagGetRelFileLocator(&buf->tag), INVALID_PROC_NUMBER);
TRACE_POSTGRESQL_BUFFER_FLUSH_START(BufTagGetForkNum(&buf->tag),
buf->tag.blockNum,
@@ -3696,7 +3696,7 @@ DropRelationBuffers(SMgrRelation smgr_reln, ForkNumber *forkNum,
/* If it's a local relation, it's localbuf.c's problem. */
if (RelFileLocatorBackendIsTemp(rlocator))
{
- if (rlocator.backend == MyBackendId)
+ if (rlocator.backend == MyProcNumber)
{
for (j = 0; j < nforks; j++)
DropRelationLocalBuffers(rlocator.locator, forkNum[j],
@@ -3826,7 +3826,7 @@ DropRelationsAllBuffers(SMgrRelation *smgr_reln, int nlocators)
{
if (RelFileLocatorBackendIsTemp(smgr_reln[i]->smgr_rlocator))
{
- if (smgr_reln[i]->smgr_rlocator.backend == MyBackendId)
+ if (smgr_reln[i]->smgr_rlocator.backend == MyProcNumber)
DropRelationAllLocalBuffers(smgr_reln[i]->smgr_rlocator.locator);
}
else
@@ -4090,7 +4090,7 @@ PrintBufferDescs(void)
"blockNum=%u, flags=0x%x, refcount=%u %d)",
i, buf->freeNext,
relpathbackend(BufTagGetRelFileLocator(&buf->tag),
- InvalidBackendId, BufTagGetForkNum(&buf->tag)),
+ INVALID_PROC_NUMBER, BufTagGetForkNum(&buf->tag)),
buf->tag.blockNum, buf->flags,
buf->refcount, GetPrivateRefCount(b));
}
@@ -4364,7 +4364,7 @@ RelationCopyStorageUsingBuffer(RelFileLocator srclocator,
use_wal = XLogIsNeeded() && (permanent || forkNum == INIT_FORKNUM);
/* Get number of blocks in the source relation. */
- nblocks = smgrnblocks(smgropen(srclocator, InvalidBackendId),
+ nblocks = smgrnblocks(smgropen(srclocator, INVALID_PROC_NUMBER),
forkNum);
/* Nothing to copy; just return. */
@@ -4376,7 +4376,7 @@ RelationCopyStorageUsingBuffer(RelFileLocator srclocator,
* relation before starting to copy block by block.
*/
memset(buf.data, 0, BLCKSZ);
- smgrextend(smgropen(dstlocator, InvalidBackendId), forkNum, nblocks - 1,
+ smgrextend(smgropen(dstlocator, INVALID_PROC_NUMBER), forkNum, nblocks - 1,
buf.data, true);
/* This is a bulk operation, so use buffer access strategies. */
@@ -4443,8 +4443,8 @@ CreateAndCopyRelationData(RelFileLocator src_rlocator,
relpersistence = permanent ?
RELPERSISTENCE_PERMANENT : RELPERSISTENCE_UNLOGGED;
- src_rel = smgropen(src_rlocator, InvalidBackendId);
- dst_rel = smgropen(dst_rlocator, InvalidBackendId);
+ src_rel = smgropen(src_rlocator, INVALID_PROC_NUMBER);
+ dst_rel = smgropen(dst_rlocator, INVALID_PROC_NUMBER);
/*
* Create and copy all forks of the relation. During create database we
@@ -5348,7 +5348,7 @@ local_buffer_write_error_callback(void *arg)
if (bufHdr != NULL)
{
char *path = relpathbackend(BufTagGetRelFileLocator(&bufHdr->tag),
- MyBackendId,
+ MyProcNumber,
BufTagGetForkNum(&bufHdr->tag));
errcontext("writing block %u of relation %s",
@@ -5653,7 +5653,7 @@ IssuePendingWritebacks(WritebackContext *wb_context, IOContext io_context)
i += ahead;
/* and finally tell the kernel to write the data to storage */
- reln = smgropen(currlocator, InvalidBackendId);
+ reln = smgropen(currlocator, INVALID_PROC_NUMBER);
smgrwriteback(reln, BufTagGetForkNum(&tag), tag.blockNum, nblocks);
}
diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c
index 1f02fed250e..f4b294476e5 100644
--- a/src/backend/storage/buffer/localbuf.c
+++ b/src/backend/storage/buffer/localbuf.c
@@ -242,7 +242,7 @@ GetLocalVictimBuffer(void)
Page localpage = (char *) LocalBufHdrGetBlock(bufHdr);
/* Find smgr relation for buffer */
- oreln = smgropen(BufTagGetRelFileLocator(&bufHdr->tag), MyBackendId);
+ oreln = smgropen(BufTagGetRelFileLocator(&bufHdr->tag), MyProcNumber);
PageSetChecksumInplace(localpage, bufHdr->tag.blockNum);
@@ -509,7 +509,7 @@ DropRelationLocalBuffers(RelFileLocator rlocator, ForkNumber forkNum,
elog(ERROR, "block %u of %s is still referenced (local %u)",
bufHdr->tag.blockNum,
relpathbackend(BufTagGetRelFileLocator(&bufHdr->tag),
- MyBackendId,
+ MyProcNumber,
BufTagGetForkNum(&bufHdr->tag)),
LocalRefCount[i]);
@@ -554,7 +554,7 @@ DropRelationAllLocalBuffers(RelFileLocator rlocator)
elog(ERROR, "block %u of %s is still referenced (local %u)",
bufHdr->tag.blockNum,
relpathbackend(BufTagGetRelFileLocator(&bufHdr->tag),
- MyBackendId,
+ MyProcNumber,
BufTagGetForkNum(&bufHdr->tag)),
LocalRefCount[i]);
/* Remove entry from hashtable */
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index d96606ebba5..04b50204b81 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -818,9 +818,9 @@ ProcArrayGroupClearXid(PGPROC *proc, TransactionId latestXid)
* If the list was not empty, the leader will clear our XID. It is
* impossible to have followers without a leader because the first process
* that has added itself to the list will always have nextidx as
- * INVALID_PGPROCNO.
+ * INVALID_PROC_NUMBER.
*/
- if (nextidx != INVALID_PGPROCNO)
+ if (nextidx != INVALID_PROC_NUMBER)
{
int extraWaits = 0;
@@ -836,7 +836,7 @@ ProcArrayGroupClearXid(PGPROC *proc, TransactionId latestXid)
}
pgstat_report_wait_end();
- Assert(pg_atomic_read_u32(&proc->procArrayGroupNext) == INVALID_PGPROCNO);
+ Assert(pg_atomic_read_u32(&proc->procArrayGroupNext) == INVALID_PROC_NUMBER);
/* Fix semaphore count for any absorbed wakeups */
while (extraWaits-- > 0)
@@ -853,13 +853,13 @@ ProcArrayGroupClearXid(PGPROC *proc, TransactionId latestXid)
* to pop elements one at a time could lead to an ABA problem.
*/
nextidx = pg_atomic_exchange_u32(&procglobal->procArrayGroupFirst,
- INVALID_PGPROCNO);
+ INVALID_PROC_NUMBER);
/* Remember head of list so we can perform wakeups after dropping lock. */
wakeidx = nextidx;
/* Walk the list and clear all XIDs. */
- while (nextidx != INVALID_PGPROCNO)
+ while (nextidx != INVALID_PROC_NUMBER)
{
PGPROC *nextproc = &allProcs[nextidx];
@@ -879,12 +879,12 @@ ProcArrayGroupClearXid(PGPROC *proc, TransactionId latestXid)
* up are probably much slower than the simple memory writes we did while
* holding the lock.
*/
- while (wakeidx != INVALID_PGPROCNO)
+ while (wakeidx != INVALID_PROC_NUMBER)
{
PGPROC *nextproc = &allProcs[wakeidx];
wakeidx = pg_atomic_read_u32(&nextproc->procArrayGroupNext);
- pg_atomic_write_u32(&nextproc->procArrayGroupNext, INVALID_PGPROCNO);
+ pg_atomic_write_u32(&nextproc->procArrayGroupNext, INVALID_PROC_NUMBER);
/* ensure all previous writes are visible before follower continues. */
pg_write_barrier();
@@ -2538,7 +2538,7 @@ ProcArrayInstallImportedXmin(TransactionId xmin,
/*
* Find the PGPROC entry of the source transaction. (This could use
- * GetPGProcByBackendId(), unless it's a prepared xact. But this isn't
+ * GetPGProcByNumber(), unless it's a prepared xact. But this isn't
* performance critical.)
*/
for (index = 0; index < arrayP->numProcs; index++)
@@ -2553,7 +2553,7 @@ ProcArrayInstallImportedXmin(TransactionId xmin,
continue;
/* We are only interested in the specific virtual transaction. */
- if (proc->vxid.backendId != sourcevxid->backendId)
+ if (proc->vxid.procNumber != sourcevxid->procNumber)
continue;
if (proc->vxid.lxid != sourcevxid->localTransactionId)
continue;
@@ -3105,20 +3105,20 @@ HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids, int type)
}
/*
- * BackendIdGetProc -- get a backend's PGPROC given its backend ID
+ * ProcNumberGetProc -- get a backend's PGPROC given its proc number
*
* The result may be out of date arbitrarily quickly, so the caller
* must be careful about how this information is used. NULL is
* returned if the backend is not active.
*/
PGPROC *
-BackendIdGetProc(int backendID)
+ProcNumberGetProc(ProcNumber procNumber)
{
PGPROC *result;
- if (backendID < 1 || backendID > ProcGlobal->allProcCount)
+ if (procNumber < 0 || procNumber >= ProcGlobal->allProcCount)
return NULL;
- result = GetPGProcByBackendId(backendID);
+ result = GetPGProcByNumber(procNumber);
if (result->pid == 0)
return NULL;
@@ -3127,15 +3127,15 @@ BackendIdGetProc(int backendID)
}
/*
- * BackendIdGetTransactionIds -- get a backend's transaction status
+ * ProcNumberGetTransactionIds -- get a backend's transaction status
*
* Get the xid, xmin, nsubxid and overflow status of the backend. The
* result may be out of date arbitrarily quickly, so the caller must be
* careful about how this information is used.
*/
void
-BackendIdGetTransactionIds(int backendID, TransactionId *xid,
- TransactionId *xmin, int *nsubxid, bool *overflowed)
+ProcNumberGetTransactionIds(ProcNumber procNumber, TransactionId *xid,
+ TransactionId *xmin, int *nsubxid, bool *overflowed)
{
PGPROC *proc;
@@ -3144,9 +3144,9 @@ BackendIdGetTransactionIds(int backendID, TransactionId *xid,
*nsubxid = 0;
*overflowed = false;
- if (backendID < 1 || backendID > ProcGlobal->allProcCount)
+ if (procNumber < 0 || procNumber >= ProcGlobal->allProcCount)
return;
- proc = GetPGProcByBackendId(backendID);
+ proc = GetPGProcByNumber(procNumber);
/* Need to lock out additions/removals of backends */
LWLockAcquire(ProcArrayLock, LW_SHARED);
@@ -3453,7 +3453,7 @@ GetConflictingVirtualXIDs(TransactionId limitXmin, Oid dbOid)
LWLockRelease(ProcArrayLock);
/* add the terminator */
- vxids[count].backendId = InvalidBackendId;
+ vxids[count].procNumber = INVALID_PROC_NUMBER;
vxids[count].localTransactionId = InvalidLocalTransactionId;
return vxids;
@@ -3488,7 +3488,7 @@ SignalVirtualTransaction(VirtualTransactionId vxid, ProcSignalReason sigmode,
GET_VXID_FROM_PGPROC(procvxid, *proc);
- if (procvxid.backendId == vxid.backendId &&
+ if (procvxid.procNumber == vxid.procNumber &&
procvxid.localTransactionId == vxid.localTransactionId)
{
proc->recoveryConflictPending = conflictPending;
@@ -3499,7 +3499,7 @@ SignalVirtualTransaction(VirtualTransactionId vxid, ProcSignalReason sigmode,
* Kill the pid if it's still here. If not, that's what we
* wanted so ignore any errors.
*/
- (void) SendProcSignal(pid, sigmode, vxid.backendId);
+ (void) SendProcSignal(pid, sigmode, vxid.procNumber);
}
break;
}
@@ -3662,7 +3662,7 @@ CancelDBBackends(Oid databaseid, ProcSignalReason sigmode, bool conflictPending)
* Kill the pid if it's still here. If not, that's what we
* wanted so ignore any errors.
*/
- (void) SendProcSignal(pid, sigmode, procvxid.backendId);
+ (void) SendProcSignal(pid, sigmode, procvxid.procNumber);
}
}
}
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 199dd182253..f7d9c9af511 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -43,10 +43,10 @@
* observe it only once.)
*
* Each process that wants to receive signals registers its process ID
- * in the ProcSignalSlots array. The array is indexed by backend ID to make
+ * in the ProcSignalSlots array. The array is indexed by ProcNumber to make
* slot allocation simple, and to avoid having to search the array when you
- * know the backend ID of the process you're signaling. (We do support
- * signaling without backend ID, but it's a bit less efficient.)
+ * know the ProcNumber of the process you're signaling. (We do support
+ * signaling without ProcNumber, but it's a bit less efficient.)
*
* The flags are actually declared as "volatile sig_atomic_t" for maximum
* portability. This should ensure that loads and stores of the flag
@@ -83,7 +83,7 @@ typedef struct
} ProcSignalHeader;
/*
- * We reserve a slot for each possible BackendId, plus one for each
+ * We reserve a slot for each possible ProcNumber, plus one for each
* possible auxiliary process type. (This scheme assumes there is not
* more than one of any auxiliary process type at a time.)
*/
@@ -161,16 +161,16 @@ ProcSignalInit(void)
ProcSignalSlot *slot;
uint64 barrier_generation;
- if (MyBackendId <= 0)
- elog(ERROR, "MyBackendId not set");
- if (MyBackendId > NumProcSignalSlots)
- elog(ERROR, "unexpected MyBackendId %d in ProcSignalInit (max %d)", MyBackendId, NumProcSignalSlots);
- slot = &ProcSignal->psh_slot[MyBackendId - 1];
+ if (MyProcNumber < 0)
+ elog(ERROR, "MyProcNumber not set");
+ if (MyProcNumber >= NumProcSignalSlots)
+ elog(ERROR, "unexpected MyProcNumber %d in ProcSignalInit (max %d)", MyProcNumber, NumProcSignalSlots);
+ slot = &ProcSignal->psh_slot[MyProcNumber];
/* sanity check */
if (slot->pss_pid != 0)
elog(LOG, "process %d taking over ProcSignal slot %d, but it's not empty",
- MyProcPid, MyBackendId - 1);
+ MyProcPid, MyProcNumber);
/* Clear out any leftover signal reasons */
MemSet(slot->pss_signalFlags, 0, NUM_PROCSIGNALS * sizeof(sig_atomic_t));
@@ -218,6 +218,7 @@ CleanupProcSignalState(int status, Datum arg)
* won't try to access it after it's no longer ours (and perhaps even
* after we've unmapped the shared memory segment).
*/
+ Assert(MyProcSignalSlot != NULL);
MyProcSignalSlot = NULL;
/* sanity check */
@@ -246,7 +247,7 @@ CleanupProcSignalState(int status, Datum arg)
* SendProcSignal
* Send a signal to a Postgres process
*
- * Providing backendId is optional, but it will speed up the operation.
+ * Providing procNumber is optional, but it will speed up the operation.
*
* On success (a signal was sent), zero is returned.
* On error, -1 is returned, and errno is set (typically to ESRCH or EPERM).
@@ -254,13 +255,13 @@ CleanupProcSignalState(int status, Datum arg)
* Not to be confused with ProcSendSignal
*/
int
-SendProcSignal(pid_t pid, ProcSignalReason reason, BackendId backendId)
+SendProcSignal(pid_t pid, ProcSignalReason reason, ProcNumber procNumber)
{
volatile ProcSignalSlot *slot;
- if (backendId != InvalidBackendId)
+ if (procNumber != INVALID_PROC_NUMBER)
{
- slot = &ProcSignal->psh_slot[backendId - 1];
+ slot = &ProcSignal->psh_slot[procNumber];
/*
* Note: Since there's no locking, it's possible that the target
@@ -281,10 +282,11 @@ SendProcSignal(pid_t pid, ProcSignalReason reason, BackendId backendId)
else
{
/*
- * BackendId not provided, so search the array using pid. We search
+ * Pronumber not provided, so search the array using pid. We search
* the array back to front so as to reduce search overhead. Passing
- * InvalidBackendId means that the target is most likely an auxiliary
- * process, which will have a slot near the end of the array.
+ * INVALID_PROC_NUMBER means that the target is most likely an
+ * auxiliary process, which will have a slot near the end of the
+ * array.
*/
int i;
diff --git a/src/backend/storage/ipc/sinvaladt.c b/src/backend/storage/ipc/sinvaladt.c
index f624bfc7d78..b486d8ddd1d 100644
--- a/src/backend/storage/ipc/sinvaladt.c
+++ b/src/backend/storage/ipc/sinvaladt.c
@@ -19,9 +19,9 @@
#include "access/transam.h"
#include "miscadmin.h"
-#include "storage/backendid.h"
#include "storage/ipc.h"
#include "storage/proc.h"
+#include "storage/procnumber.h"
#include "storage/procsignal.h"
#include "storage/shmem.h"
#include "storage/sinvaladt.h"
@@ -155,8 +155,8 @@ typedef struct ProcState
/*
* Next LocalTransactionId to use for each idle backend slot. We keep
- * this here because it is indexed by BackendId and it is convenient to
- * copy the value to and from local memory when MyBackendId is set. It's
+ * this here because it is indexed by ProcNumber and it is convenient to
+ * copy the value to and from local memory when MyProcNumber is set. It's
* meaningless in an active ProcState entry.
*/
LocalTransactionId nextLXID;
@@ -197,7 +197,7 @@ typedef struct SISeg
} SISeg;
/*
- * We reserve a slot for each possible BackendId, plus one for each
+ * We reserve a slot for each possible ProcNumber, plus one for each
* possible auxiliary process type. (This scheme assumes there is not
* more than one of any auxiliary process type at a time.)
*/
@@ -274,15 +274,13 @@ SharedInvalBackendInit(bool sendOnly)
ProcState *stateP;
pid_t oldPid;
SISeg *segP = shmInvalBuffer;
- int pgprocno;
- if (MyBackendId <= 0)
- elog(ERROR, "MyBackendId not set");
- if (MyBackendId > NumProcStateSlots)
- elog(PANIC, "unexpected MyBackendId %d in SharedInvalBackendInit (max %d)",
- MyBackendId, NumProcStateSlots);
- pgprocno = MyBackendId - 1;
- stateP = &segP->procState[pgprocno];
+ if (MyProcNumber < 0)
+ elog(ERROR, "MyProcNumber not set");
+ if (MyProcNumber >= NumProcStateSlots)
+ elog(PANIC, "unexpected MyProcNumber %d in SharedInvalBackendInit (max %d)",
+ MyProcNumber, NumProcStateSlots);
+ stateP = &segP->procState[MyProcNumber];
/*
* This can run in parallel with read operations, but not with write
@@ -296,10 +294,10 @@ SharedInvalBackendInit(bool sendOnly)
{
LWLockRelease(SInvalWriteLock);
elog(ERROR, "sinval slot for backend %d is already in use by process %d",
- MyBackendId, (int) oldPid);
+ MyProcNumber, (int) oldPid);
}
- shmInvalBuffer->pgprocnos[shmInvalBuffer->numProcs++] = pgprocno;
+ shmInvalBuffer->pgprocnos[shmInvalBuffer->numProcs++] = MyProcNumber;
/* Fetch next local transaction ID into local memory */
nextLocalTransactionId = stateP->nextLXID;
@@ -331,16 +329,15 @@ CleanupInvalidationState(int status, Datum arg)
{
SISeg *segP = (SISeg *) DatumGetPointer(arg);
ProcState *stateP;
- int pgprocno = MyBackendId - 1;
int i;
Assert(PointerIsValid(segP));
LWLockAcquire(SInvalWriteLock, LW_EXCLUSIVE);
- stateP = &segP->procState[pgprocno];
+ stateP = &segP->procState[MyProcNumber];
- /* Update next local transaction ID for next holder of this backendID */
+ /* Update next local transaction ID for next holder of this proc number */
stateP->nextLXID = nextLocalTransactionId;
/* Mark myself inactive */
@@ -351,7 +348,7 @@ CleanupInvalidationState(int status, Datum arg)
for (i = segP->numProcs - 1; i >= 0; i--)
{
- if (segP->pgprocnos[i] == pgprocno)
+ if (segP->pgprocnos[i] == MyProcNumber)
{
if (i != segP->numProcs - 1)
segP->pgprocnos[i] = segP->pgprocnos[segP->numProcs - 1];
@@ -481,7 +478,7 @@ SIGetDataEntries(SharedInvalidationMessage *data, int datasize)
int n;
segP = shmInvalBuffer;
- stateP = &segP->procState[MyBackendId - 1];
+ stateP = &segP->procState[MyProcNumber];
/*
* Before starting to take locks, do a quick, unlocked test to see whether
@@ -668,13 +665,13 @@ SICleanupQueue(bool callerHasWriteLock, int minFree)
if (needSig)
{
pid_t his_pid = needSig->procPid;
- BackendId his_backendId = (needSig - &segP->procState[0]) + 1;
+ ProcNumber his_procNumber = (needSig - &segP->procState[0]);
needSig->signaled = true;
LWLockRelease(SInvalReadLock);
LWLockRelease(SInvalWriteLock);
elog(DEBUG4, "sending sinval catchup signal to PID %d", (int) his_pid);
- SendProcSignal(his_pid, PROCSIG_CATCHUP_INTERRUPT, his_backendId);
+ SendProcSignal(his_pid, PROCSIG_CATCHUP_INTERRUPT, his_procNumber);
if (callerHasWriteLock)
LWLockAcquire(SInvalWriteLock, LW_EXCLUSIVE);
}
@@ -693,11 +690,11 @@ SICleanupQueue(bool callerHasWriteLock, int minFree)
* We split VirtualTransactionIds into two parts so that it is possible
* to allocate a new one without any contention for shared memory, except
* for a bit of additional overhead during backend startup/shutdown.
- * The high-order part of a VirtualTransactionId is a BackendId, and the
+ * The high-order part of a VirtualTransactionId is a ProcNumber, and the
* low-order part is a LocalTransactionId, which we assign from a local
* counter. To avoid the risk of a VirtualTransactionId being reused
- * within a short interval, successive procs occupying the same backend ID
- * slot should use a consecutive sequence of local IDs, which is implemented
+ * within a short interval, successive procs occupying the same PGPROC slot
+ * should use a consecutive sequence of local IDs, which is implemented
* by copying nextLocalTransactionId as seen above.
*/
LocalTransactionId
diff --git a/src/backend/storage/ipc/standby.c b/src/backend/storage/ipc/standby.c
index 97d1ab65740..23ab1cfde8d 100644
--- a/src/backend/storage/ipc/standby.c
+++ b/src/backend/storage/ipc/standby.c
@@ -137,8 +137,8 @@ InitRecoveryTransactionEnvironment(void)
* are held by vxids and row level locks are held by xids. All queries
* hold AccessShareLocks so never block while we write or lock new rows.
*/
- MyProc->vxid.backendId = MyBackendId;
- vxid.backendId = MyBackendId;
+ MyProc->vxid.procNumber = MyProcNumber;
+ vxid.procNumber = MyProcNumber;
vxid.localTransactionId = GetNextLocalTransactionId();
VirtualXactLockTableInsert(vxid);
@@ -300,7 +300,7 @@ LogRecoveryConflict(ProcSignalReason reason, TimestampTz wait_start,
vxids = wait_list;
while (VirtualTransactionIdIsValid(*vxids))
{
- PGPROC *proc = BackendIdGetProc(vxids->backendId);
+ PGPROC *proc = ProcNumberGetProc(vxids->procNumber);
/* proc can be NULL if the target backend is not active */
if (proc)
diff --git a/src/backend/storage/lmgr/lmgr.c b/src/backend/storage/lmgr/lmgr.c
index 4975d4b67d7..7d964f2dc07 100644
--- a/src/backend/storage/lmgr/lmgr.c
+++ b/src/backend/storage/lmgr/lmgr.c
@@ -947,7 +947,7 @@ WaitForLockersMultiple(List *locktags, LOCKMODE lockmode, bool progress)
/* If requested, publish who we're going to wait for. */
if (progress)
{
- PGPROC *holder = BackendIdGetProc(lockholders->backendId);
+ PGPROC *holder = ProcNumberGetProc(lockholders->procNumber);
if (holder)
pgstat_progress_update_param(PROGRESS_WAITFOR_CURRENT_PID,
diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c
index e62968b4a86..0d904ef6554 100644
--- a/src/backend/storage/lmgr/lock.c
+++ b/src/backend/storage/lmgr/lock.c
@@ -2995,7 +2995,7 @@ GetLockConflicts(const LOCKTAG *locktag, LOCKMODE lockmode, int *countp)
* on this lockable object.
*/
LWLockRelease(partitionLock);
- vxids[count].backendId = InvalidBackendId;
+ vxids[count].procNumber = INVALID_PROC_NUMBER;
vxids[count].localTransactionId = InvalidLocalTransactionId;
if (countp)
*countp = count;
@@ -3041,7 +3041,7 @@ GetLockConflicts(const LOCKTAG *locktag, LOCKMODE lockmode, int *countp)
if (count > MaxBackends + max_prepared_xacts) /* should never happen */
elog(PANIC, "too many conflicting locks found");
- vxids[count].backendId = InvalidBackendId;
+ vxids[count].procNumber = INVALID_PROC_NUMBER;
vxids[count].localTransactionId = InvalidLocalTransactionId;
if (countp)
*countp = count;
@@ -3625,7 +3625,7 @@ GetLockStatusData(void)
proc->fpRelId[f]);
instance->holdMask = lockbits << FAST_PATH_LOCKNUMBER_OFFSET;
instance->waitLockMode = NoLock;
- instance->vxid.backendId = proc->vxid.backendId;
+ instance->vxid.procNumber = proc->vxid.procNumber;
instance->vxid.localTransactionId = proc->vxid.lxid;
instance->pid = proc->pid;
instance->leaderPid = proc->pid;
@@ -3652,14 +3652,14 @@ GetLockStatusData(void)
repalloc(data->locks, sizeof(LockInstanceData) * els);
}
- vxid.backendId = proc->vxid.backendId;
+ vxid.procNumber = proc->vxid.procNumber;
vxid.localTransactionId = proc->fpLocalTransactionId;
instance = &data->locks[el];
SET_LOCKTAG_VIRTUALTRANSACTION(instance->locktag, vxid);
instance->holdMask = LOCKBIT_ON(ExclusiveLock);
instance->waitLockMode = NoLock;
- instance->vxid.backendId = proc->vxid.backendId;
+ instance->vxid.procNumber = proc->vxid.procNumber;
instance->vxid.localTransactionId = proc->vxid.lxid;
instance->pid = proc->pid;
instance->leaderPid = proc->pid;
@@ -3712,7 +3712,7 @@ GetLockStatusData(void)
instance->waitLockMode = proc->waitLockMode;
else
instance->waitLockMode = NoLock;
- instance->vxid.backendId = proc->vxid.backendId;
+ instance->vxid.procNumber = proc->vxid.procNumber;
instance->vxid.localTransactionId = proc->vxid.lxid;
instance->pid = proc->pid;
instance->leaderPid = proclock->groupLeader->pid;
@@ -3888,7 +3888,7 @@ GetSingleProcBlockerStatusData(PGPROC *blocked_proc, BlockedProcsData *data)
instance->waitLockMode = proc->waitLockMode;
else
instance->waitLockMode = NoLock;
- instance->vxid.backendId = proc->vxid.backendId;
+ instance->vxid.procNumber = proc->vxid.procNumber;
instance->vxid.localTransactionId = proc->vxid.lxid;
instance->pid = proc->pid;
instance->leaderPid = proclock->groupLeader->pid;
@@ -4391,7 +4391,7 @@ VirtualXactLockTableInsert(VirtualTransactionId vxid)
LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE);
- Assert(MyProc->vxid.backendId == vxid.backendId);
+ Assert(MyProc->vxid.procNumber == vxid.procNumber);
Assert(MyProc->fpLocalTransactionId == InvalidLocalTransactionId);
Assert(MyProc->fpVXIDLock == false);
@@ -4413,7 +4413,7 @@ VirtualXactLockTableCleanup(void)
bool fastpath;
LocalTransactionId lxid;
- Assert(MyProc->vxid.backendId != InvalidBackendId);
+ Assert(MyProc->vxid.procNumber != INVALID_PROC_NUMBER);
/*
* Clean up shared memory state.
@@ -4436,7 +4436,7 @@ VirtualXactLockTableCleanup(void)
VirtualTransactionId vxid;
LOCKTAG locktag;
- vxid.backendId = MyBackendId;
+ vxid.procNumber = MyProcNumber;
vxid.localTransactionId = lxid;
SET_LOCKTAG_VIRTUALTRANSACTION(locktag, vxid);
@@ -4530,18 +4530,18 @@ VirtualXactLock(VirtualTransactionId vxid, bool wait)
* relevant lxid is no longer running here, that's enough to prove that
* it's no longer running anywhere.
*/
- proc = BackendIdGetProc(vxid.backendId);
+ proc = ProcNumberGetProc(vxid.procNumber);
if (proc == NULL)
return XactLockForVirtualXact(vxid, InvalidTransactionId, wait);
/*
- * We must acquire this lock before checking the backendId and lxid
+ * We must acquire this lock before checking the procNumber and lxid
* against the ones we're waiting for. The target backend will only set
* or clear lxid while holding this lock.
*/
LWLockAcquire(&proc->fpInfoLock, LW_EXCLUSIVE);
- if (proc->vxid.backendId != vxid.backendId
+ if (proc->vxid.procNumber != vxid.procNumber
|| proc->fpLocalTransactionId != vxid.localTransactionId)
{
/* VXID ended */
diff --git a/src/backend/storage/lmgr/predicate.c b/src/backend/storage/lmgr/predicate.c
index d62060d58c8..54e343908ce 100644
--- a/src/backend/storage/lmgr/predicate.c
+++ b/src/backend/storage/lmgr/predicate.c
@@ -1248,7 +1248,7 @@ InitPredicateLocks(void)
PredXact->OldCommittedSxact->xmin = InvalidTransactionId;
PredXact->OldCommittedSxact->flags = SXACT_FLAG_COMMITTED;
PredXact->OldCommittedSxact->pid = 0;
- PredXact->OldCommittedSxact->pgprocno = INVALID_PGPROCNO;
+ PredXact->OldCommittedSxact->pgprocno = INVALID_PROC_NUMBER;
}
/* This never changes, so let's keep a local copy. */
OldCommittedSxact = PredXact->OldCommittedSxact;
@@ -4834,7 +4834,7 @@ PostPrepare_PredicateLocks(TransactionId xid)
Assert(SxactIsPrepared(MySerializableXact));
MySerializableXact->pid = 0;
- MySerializableXact->pgprocno = INVALID_PGPROCNO;
+ MySerializableXact->pgprocno = INVALID_PROC_NUMBER;
hash_destroy(LocalPredicateLockHash);
LocalPredicateLockHash = NULL;
@@ -4906,11 +4906,11 @@ predicatelock_twophase_recover(TransactionId xid, uint16 info,
(errcode(ERRCODE_OUT_OF_MEMORY),
errmsg("out of shared memory")));
- /* vxid for a prepared xact is InvalidBackendId/xid; no pid */
- sxact->vxid.backendId = InvalidBackendId;
+ /* vxid for a prepared xact is INVALID_PROC_NUMBER/xid; no pid */
+ sxact->vxid.procNumber = INVALID_PROC_NUMBER;
sxact->vxid.localTransactionId = (LocalTransactionId) xid;
sxact->pid = 0;
- sxact->pgprocno = INVALID_PGPROCNO;
+ sxact->pgprocno = INVALID_PROC_NUMBER;
/* a prepared xact hasn't committed yet */
sxact->prepareSeqNo = RecoverySerCommitSeqNo;
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index cdcf4674e2e..69eda840f08 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -65,7 +65,6 @@ bool log_lock_waits = false;
/* Pointer to this process's PGPROC struct, if any */
PGPROC *MyProc = NULL;
-int MyProcNumber = INVALID_PGPROCNO;
/*
* This spinlock protects the freelist of recycled PGPROC structures.
@@ -180,8 +179,8 @@ InitProcGlobal(void)
ProcGlobal->startupBufferPinWaitBufId = -1;
ProcGlobal->walwriterLatch = NULL;
ProcGlobal->checkpointerLatch = NULL;
- pg_atomic_init_u32(&ProcGlobal->procArrayGroupFirst, INVALID_PGPROCNO);
- pg_atomic_init_u32(&ProcGlobal->clogGroupFirst, INVALID_PGPROCNO);
+ pg_atomic_init_u32(&ProcGlobal->procArrayGroupFirst, INVALID_PROC_NUMBER);
+ pg_atomic_init_u32(&ProcGlobal->clogGroupFirst, INVALID_PROC_NUMBER);
/*
* Create and initialize all the PGPROC structures we'll need. There are
@@ -274,8 +273,8 @@ InitProcGlobal(void)
* Initialize the atomic variables, otherwise, it won't be safe to
* access them for backends that aren't currently in use.
*/
- pg_atomic_init_u32(&(proc->procArrayGroupNext), INVALID_PGPROCNO);
- pg_atomic_init_u32(&(proc->clogGroupNext), INVALID_PGPROCNO);
+ pg_atomic_init_u32(&(proc->procArrayGroupNext), INVALID_PROC_NUMBER);
+ pg_atomic_init_u32(&(proc->clogGroupNext), INVALID_PROC_NUMBER);
pg_atomic_init_u64(&(proc->waitStart), 0);
}
@@ -354,7 +353,6 @@ InitProcess(void)
errmsg("sorry, too many clients already")));
}
MyProcNumber = GetNumberFromPGProc(MyProc);
- MyBackendId = GetBackendIdFromPGProc(MyProc);
/*
* Cross-check that the PGPROC is of the type we expect; if this were not
@@ -382,7 +380,7 @@ InitProcess(void)
MyProc->xid = InvalidTransactionId;
MyProc->xmin = InvalidTransactionId;
MyProc->pid = MyProcPid;
- MyProc->vxid.backendId = MyBackendId;
+ MyProc->vxid.procNumber = MyProcNumber;
MyProc->vxid.lxid = InvalidLocalTransactionId;
/* databaseId and roleId will be filled in later */
MyProc->databaseId = InvalidOid;
@@ -418,7 +416,7 @@ InitProcess(void)
/* Initialize fields for group XID clearing. */
MyProc->procArrayGroupMember = false;
MyProc->procArrayGroupMemberXid = InvalidTransactionId;
- Assert(pg_atomic_read_u32(&MyProc->procArrayGroupNext) == INVALID_PGPROCNO);
+ Assert(pg_atomic_read_u32(&MyProc->procArrayGroupNext) == INVALID_PROC_NUMBER);
/* Check that group locking fields are in a proper initial state. */
Assert(MyProc->lockGroupLeader == NULL);
@@ -433,7 +431,7 @@ InitProcess(void)
MyProc->clogGroupMemberXidStatus = TRANSACTION_STATUS_IN_PROGRESS;
MyProc->clogGroupMemberPage = -1;
MyProc->clogGroupMemberLsn = InvalidXLogRecPtr;
- Assert(pg_atomic_read_u32(&MyProc->clogGroupNext) == INVALID_PGPROCNO);
+ Assert(pg_atomic_read_u32(&MyProc->clogGroupNext) == INVALID_PROC_NUMBER);
/*
* Acquire ownership of the PGPROC's latch, so that we can use WaitLatch
@@ -568,7 +566,6 @@ InitAuxiliaryProcess(void)
MyProc = auxproc;
MyProcNumber = GetNumberFromPGProc(MyProc);
- MyBackendId = GetBackendIdFromPGProc(MyProc);
/*
* Initialize all fields of MyProc, except for those previously
@@ -580,7 +577,7 @@ InitAuxiliaryProcess(void)
MyProc->fpLocalTransactionId = InvalidLocalTransactionId;
MyProc->xid = InvalidTransactionId;
MyProc->xmin = InvalidTransactionId;
- MyProc->vxid.backendId = InvalidBackendId;
+ MyProc->vxid.procNumber = INVALID_PROC_NUMBER;
MyProc->vxid.lxid = InvalidLocalTransactionId;
MyProc->databaseId = InvalidOid;
MyProc->roleId = InvalidOid;
@@ -911,13 +908,12 @@ ProcKill(int code, Datum arg)
proc = MyProc;
MyProc = NULL;
- MyProcNumber = INVALID_PGPROCNO;
- MyBackendId = InvalidBackendId;
+ MyProcNumber = INVALID_PROC_NUMBER;
DisownLatch(&proc->procLatch);
/* Mark the proc no longer in use */
proc->pid = 0;
- proc->vxid.backendId = InvalidBackendId;
+ proc->vxid.procNumber = INVALID_PROC_NUMBER;
proc->vxid.lxid = InvalidTransactionId;
procgloballist = proc->procgloballist;
@@ -989,8 +985,7 @@ AuxiliaryProcKill(int code, Datum arg)
proc = MyProc;
MyProc = NULL;
- MyProcNumber = INVALID_PGPROCNO;
- MyBackendId = InvalidBackendId;
+ MyProcNumber = INVALID_PROC_NUMBER;
DisownLatch(&proc->procLatch);
SpinLockAcquire(ProcStructLock);
@@ -1852,15 +1847,15 @@ ProcWaitForSignal(uint32 wait_event_info)
}
/*
- * ProcSendSignal - set the latch of a backend identified by pgprocno
+ * ProcSendSignal - set the latch of a backend identified by ProcNumber
*/
void
-ProcSendSignal(int pgprocno)
+ProcSendSignal(ProcNumber procNumber)
{
- if (pgprocno < 0 || pgprocno >= ProcGlobal->allProcCount)
- elog(ERROR, "pgprocno out of range");
+ if (procNumber < 0 || procNumber >= ProcGlobal->allProcCount)
+ elog(ERROR, "procNumber out of range");
- SetLatch(&ProcGlobal->allProcs[pgprocno].procLatch);
+ SetLatch(&ProcGlobal->allProcs[procNumber].procLatch);
}
/*
diff --git a/src/backend/storage/smgr/md.c b/src/backend/storage/smgr/md.c
index b1e9932a291..e474f9d834b 100644
--- a/src/backend/storage/smgr/md.c
+++ b/src/backend/storage/smgr/md.c
@@ -1411,7 +1411,7 @@ DropRelationFiles(RelFileLocator *delrels, int ndelrels, bool isRedo)
srels = palloc(sizeof(SMgrRelation) * ndelrels);
for (i = 0; i < ndelrels; i++)
{
- SMgrRelation srel = smgropen(delrels[i], InvalidBackendId);
+ SMgrRelation srel = smgropen(delrels[i], INVALID_PROC_NUMBER);
if (isRedo)
{
@@ -1695,7 +1695,7 @@ _mdnblocks(SMgrRelation reln, ForkNumber forknum, MdfdVec *seg)
int
mdsyncfiletag(const FileTag *ftag, char *path)
{
- SMgrRelation reln = smgropen(ftag->rlocator, InvalidBackendId);
+ SMgrRelation reln = smgropen(ftag->rlocator, INVALID_PROC_NUMBER);
File file;
instr_time io_start;
bool need_to_close;
diff --git a/src/backend/storage/smgr/smgr.c b/src/backend/storage/smgr/smgr.c
index c67ba9a51c8..22295b485f4 100644
--- a/src/backend/storage/smgr/smgr.c
+++ b/src/backend/storage/smgr/smgr.c
@@ -194,7 +194,7 @@ smgrshutdown(int code, Datum arg)
* This does not attempt to actually open the underlying files.
*/
SMgrRelation
-smgropen(RelFileLocator rlocator, BackendId backend)
+smgropen(RelFileLocator rlocator, ProcNumber backend)
{
RelFileLocatorBackend brlocator;
SMgrRelation reln;
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 3d3f7b06723..c91ed5a4b64 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -238,7 +238,7 @@ CreateSharedBackendStatus(void)
/*
* Initialize pgstats backend activity state, and set up our on-proc-exit
- * hook. Called from InitPostgres and AuxiliaryProcessMain. MyBackendId must
+ * hook. Called from InitPostgres and AuxiliaryProcessMain. MyProcNumber must
* be set, but we must not have started any transaction yet (since the exit
* hook must run after the last transaction exit).
*
@@ -248,9 +248,9 @@ void
pgstat_beinit(void)
{
/* Initialize MyBEEntry */
- Assert(MyBackendId != InvalidBackendId);
- Assert(MyBackendId >= 1 && MyBackendId <= NumBackendStatSlots);
- MyBEEntry = &BackendStatusArray[MyBackendId - 1];
+ Assert(MyProcNumber != INVALID_PROC_NUMBER);
+ Assert(MyProcNumber >= 0 && MyProcNumber < NumBackendStatSlots);
+ MyBEEntry = &BackendStatusArray[MyProcNumber];
/* Set up a process-exit hook to clean up */
on_shmem_exit(pgstat_beshutdown_hook, 0);
@@ -721,7 +721,7 @@ pgstat_read_current_status(void)
#ifdef ENABLE_GSS
PgBackendGSSStatus *localgssstatus;
#endif
- int i;
+ ProcNumber procNumber;
if (localBackendStatusTable)
return; /* already done */
@@ -764,7 +764,7 @@ pgstat_read_current_status(void)
beentry = BackendStatusArray;
localentry = localtable;
- for (i = 1; i <= NumBackendStatSlots; i++)
+ for (procNumber = 0; procNumber < NumBackendStatSlots; procNumber++)
{
/*
* Follow the protocol of retrying if st_changecount changes while we
@@ -830,17 +830,17 @@ pgstat_read_current_status(void)
if (localentry->backendStatus.st_procpid > 0)
{
/*
- * The BackendStatusArray index is exactly the BackendId of the
+ * The BackendStatusArray index is exactly the ProcNumber of the
* source backend. Note that this means localBackendStatusTable
- * is in order by backend_id. pgstat_get_beentry_by_backend_id()
+ * is in order by proc_number. pgstat_get_beentry_by_backend_id()
* depends on that.
*/
- localentry->backend_id = i;
- BackendIdGetTransactionIds(i,
- &localentry->backend_xid,
- &localentry->backend_xmin,
- &localentry->backend_subxact_count,
- &localentry->backend_subxact_overflowed);
+ localentry->proc_number = procNumber;
+ ProcNumberGetTransactionIds(procNumber,
+ &localentry->backend_xid,
+ &localentry->backend_xmin,
+ &localentry->backend_subxact_count,
+ &localentry->backend_subxact_overflowed);
localentry++;
localappname += NAMEDATALEN;
@@ -1043,7 +1043,7 @@ pgstat_get_my_query_id(void)
* cmp_lbestatus
*
* Comparison function for bsearch() on an array of LocalPgBackendStatus.
- * The backend_id field is used to compare the arguments.
+ * The proc_number field is used to compare the arguments.
* ----------
*/
static int
@@ -1052,17 +1052,17 @@ cmp_lbestatus(const void *a, const void *b)
const LocalPgBackendStatus *lbestatus1 = (const LocalPgBackendStatus *) a;
const LocalPgBackendStatus *lbestatus2 = (const LocalPgBackendStatus *) b;
- return lbestatus1->backend_id - lbestatus2->backend_id;
+ return lbestatus1->proc_number - lbestatus2->proc_number;
}
/* ----------
- * pgstat_get_beentry_by_backend_id() -
+ * pgstat_get_beentry_by_proc_number() -
*
* Support function for the SQL-callable pgstat* functions. Returns
* our local copy of the current-activity entry for one backend,
* or NULL if the given beid doesn't identify any known session.
*
- * The beid argument is the BackendId of the desired session
+ * The argument is the ProcNumber of the desired session
* (note that this is unlike pgstat_get_local_beentry_by_index()).
*
* NB: caller is responsible for a check if the user is permitted to see
@@ -1070,9 +1070,9 @@ cmp_lbestatus(const void *a, const void *b)
* ----------
*/
PgBackendStatus *
-pgstat_get_beentry_by_backend_id(BackendId beid)
+pgstat_get_beentry_by_proc_number(ProcNumber procNumber)
{
- LocalPgBackendStatus *ret = pgstat_get_local_beentry_by_backend_id(beid);
+ LocalPgBackendStatus *ret = pgstat_get_local_beentry_by_proc_number(procNumber);
if (ret)
return &ret->backendStatus;
@@ -1082,12 +1082,12 @@ pgstat_get_beentry_by_backend_id(BackendId beid)
/* ----------
- * pgstat_get_local_beentry_by_backend_id() -
+ * pgstat_get_local_beentry_by_proc_number() -
*
- * Like pgstat_get_beentry_by_backend_id() but with locally computed additions
+ * Like pgstat_get_beentry_by_proc_number() but with locally computed additions
* (like xid and xmin values of the backend)
*
- * The beid argument is the BackendId of the desired session
+ * The argument is the ProcNumber of the desired session
* (note that this is unlike pgstat_get_local_beentry_by_index()).
*
* NB: caller is responsible for checking if the user is permitted to see this
@@ -1095,7 +1095,7 @@ pgstat_get_beentry_by_backend_id(BackendId beid)
* ----------
*/
LocalPgBackendStatus *
-pgstat_get_local_beentry_by_backend_id(BackendId beid)
+pgstat_get_local_beentry_by_proc_number(ProcNumber procNumber)
{
LocalPgBackendStatus key;
@@ -1105,7 +1105,7 @@ pgstat_get_local_beentry_by_backend_id(BackendId beid)
* Since the localBackendStatusTable is in order by backend_id, we can use
* bsearch() to search it efficiently.
*/
- key.backend_id = beid;
+ key.proc_number = procNumber;
return bsearch(&key, localBackendStatusTable, localNumBackends,
sizeof(LocalPgBackendStatus), cmp_lbestatus);
}
diff --git a/src/backend/utils/adt/dbsize.c b/src/backend/utils/adt/dbsize.c
index 2df025d50aa..3e851032786 100644
--- a/src/backend/utils/adt/dbsize.c
+++ b/src/backend/utils/adt/dbsize.c
@@ -306,7 +306,7 @@ pg_tablespace_size_name(PG_FUNCTION_ARGS)
* is no check here or at the call sites for that.
*/
static int64
-calculate_relation_size(RelFileLocator *rfn, BackendId backend, ForkNumber forknum)
+calculate_relation_size(RelFileLocator *rfn, ProcNumber backend, ForkNumber forknum)
{
int64 totalsize = 0;
char *relationpath;
@@ -951,7 +951,7 @@ pg_relation_filepath(PG_FUNCTION_ARGS)
HeapTuple tuple;
Form_pg_class relform;
RelFileLocator rlocator;
- BackendId backend;
+ ProcNumber backend;
char *path;
tuple = SearchSysCache1(RELOID, ObjectIdGetDatum(relid));
@@ -996,21 +996,21 @@ pg_relation_filepath(PG_FUNCTION_ARGS)
{
case RELPERSISTENCE_UNLOGGED:
case RELPERSISTENCE_PERMANENT:
- backend = InvalidBackendId;
+ backend = INVALID_PROC_NUMBER;
break;
case RELPERSISTENCE_TEMP:
if (isTempOrTempToastNamespace(relform->relnamespace))
- backend = BackendIdForTempRelations();
+ backend = ProcNumberForTempRelations();
else
{
/* Do it the hard way. */
- backend = GetTempNamespaceBackendId(relform->relnamespace);
- Assert(backend != InvalidBackendId);
+ backend = GetTempNamespaceProcNumber(relform->relnamespace);
+ Assert(backend != INVALID_PROC_NUMBER);
}
break;
default:
elog(ERROR, "invalid relpersistence: %c", relform->relpersistence);
- backend = InvalidBackendId; /* placate compiler */
+ backend = INVALID_PROC_NUMBER; /* placate compiler */
break;
}
diff --git a/src/backend/utils/adt/lockfuncs.c b/src/backend/utils/adt/lockfuncs.c
index bbe5cc0806e..8624470b1d9 100644
--- a/src/backend/utils/adt/lockfuncs.c
+++ b/src/backend/utils/adt/lockfuncs.c
@@ -73,15 +73,16 @@ typedef struct
* This is currently only used in pg_lock_status, so we put it here.
*/
static Datum
-VXIDGetDatum(BackendId bid, LocalTransactionId lxid)
+VXIDGetDatum(ProcNumber procNumber, LocalTransactionId lxid)
{
/*
- * The representation is "<bid>/<lxid>", decimal and unsigned decimal
- * respectively. Note that elog.c also knows how to format a vxid.
+ * The representation is "<procNumber>/<lxid>", decimal and unsigned
+ * decimal respectively. Note that elog.c also knows how to format a
+ * vxid.
*/
char vxidstr[32];
- snprintf(vxidstr, sizeof(vxidstr), "%d/%u", bid, lxid);
+ snprintf(vxidstr, sizeof(vxidstr), "%d/%u", procNumber, lxid);
return CStringGetTextDatum(vxidstr);
}
@@ -353,7 +354,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
break;
}
- values[10] = VXIDGetDatum(instance->vxid.backendId, instance->vxid.localTransactionId);
+ values[10] = VXIDGetDatum(instance->vxid.procNumber, instance->vxid.localTransactionId);
if (instance->pid != 0)
values[11] = Int32GetDatum(instance->pid);
else
@@ -419,7 +420,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
nulls[9] = true; /* objsubid */
/* lock holder */
- values[10] = VXIDGetDatum(xact->vxid.backendId,
+ values[10] = VXIDGetDatum(xact->vxid.procNumber,
xact->vxid.localTransactionId);
if (xact->pid != 0)
values[11] = Int32GetDatum(xact->pid);
diff --git a/src/backend/utils/adt/mcxtfuncs.c b/src/backend/utils/adt/mcxtfuncs.c
index a7267dc15d1..c29320b3785 100644
--- a/src/backend/utils/adt/mcxtfuncs.c
+++ b/src/backend/utils/adt/mcxtfuncs.c
@@ -146,7 +146,7 @@ pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
{
int pid = PG_GETARG_INT32(0);
PGPROC *proc;
- BackendId backendId = InvalidBackendId;
+ ProcNumber procNumber = INVALID_PROC_NUMBER;
/*
* See if the process with given pid is a backend or an auxiliary process.
@@ -175,9 +175,8 @@ pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(false);
}
- if (proc != NULL)
- backendId = GetBackendIdFromPGProc(proc);
- if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, backendId) < 0)
+ procNumber = GetNumberFromPGProc(proc);
+ if (SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, procNumber) < 0)
{
/* Again, just a warning to allow loops */
ereport(WARNING,
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 30a20635057..d1e9aa8290f 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -213,7 +213,7 @@ pg_stat_get_backend_idset(PG_FUNCTION_ARGS)
/* do when there is more left to send */
LocalPgBackendStatus *local_beentry = pgstat_get_local_beentry_by_index(fctx[0]);
- SRF_RETURN_NEXT(funcctx, Int32GetDatum(local_beentry->backend_id));
+ SRF_RETURN_NEXT(funcctx, Int32GetDatum(local_beentry->proc_number));
}
else
{
@@ -669,10 +669,10 @@ pg_backend_pid(PG_FUNCTION_ARGS)
Datum
pg_stat_get_backend_pid(PG_FUNCTION_ARGS)
{
- int32 beid = PG_GETARG_INT32(0);
+ int32 procNumber = PG_GETARG_INT32(0);
PgBackendStatus *beentry;
- if ((beentry = pgstat_get_beentry_by_backend_id(beid)) == NULL)
+ if ((beentry = pgstat_get_beentry_by_proc_number(procNumber)) == NULL)
PG_RETURN_NULL();
PG_RETURN_INT32(beentry->st_procpid);
@@ -682,10 +682,10 @@ pg_stat_get_backend_pid(PG_FUNCTION_ARGS)
Datum
pg_stat_get_backend_dbid(PG_FUNCTION_ARGS)
{
- int32 beid = PG_GETARG_INT32(0);
+ int32 procNumber = PG_GETARG_INT32(0);
PgBackendStatus *beentry;
- if ((beentry = pgstat_get_beentry_by_backend_id(beid)) == NULL)
+ if ((beentry = pgstat_get_beentry_by_proc_number(procNumber)) == NULL)
PG_RETURN_NULL();
PG_RETURN_OID(beentry->st_databaseid);
@@ -695,10 +695,10 @@ pg_stat_get_backend_dbid(PG_FUNCTION_ARGS)
Datum
pg_stat_get_backend_userid(PG_FUNCTION_ARGS)
{
- int32 beid = PG_GETARG_INT32(0);
+ int32 procNumber = PG_GETARG_INT32(0);
PgBackendStatus *beentry;
- if ((beentry = pgstat_get_beentry_by_backend_id(beid)) == NULL)
+ if ((beentry = pgstat_get_beentry_by_proc_number(procNumber)) == NULL)
PG_RETURN_NULL();
PG_RETURN_OID(beentry->st_userid);
@@ -711,7 +711,7 @@ pg_stat_get_backend_subxact(PG_FUNCTION_ARGS)
TupleDesc tupdesc;
Datum values[PG_STAT_GET_SUBXACT_COLS] = {0};
bool nulls[PG_STAT_GET_SUBXACT_COLS] = {0};
- int32 beid = PG_GETARG_INT32(0);
+ int32 procNumber = PG_GETARG_INT32(0);
LocalPgBackendStatus *local_beentry;
/* Initialise attributes information in the tuple descriptor */
@@ -723,7 +723,7 @@ pg_stat_get_backend_subxact(PG_FUNCTION_ARGS)
BlessTupleDesc(tupdesc);
- if ((local_beentry = pgstat_get_local_beentry_by_backend_id(beid)) != NULL)
+ if ((local_beentry = pgstat_get_local_beentry_by_proc_number(procNumber)) != NULL)
{
/* Fill values and NULLs */
values[0] = Int32GetDatum(local_beentry->backend_subxact_count);
@@ -742,13 +742,13 @@ pg_stat_get_backend_subxact(PG_FUNCTION_ARGS)
Datum
pg_stat_get_backend_activity(PG_FUNCTION_ARGS)
{
- int32 beid = PG_GETARG_INT32(0);
+ int32 procNumber = PG_GETARG_INT32(0);
PgBackendStatus *beentry;
const char *activity;
char *clipped_activity;
text *ret;
- if ((beentry = pgstat_get_beentry_by_backend_id(beid)) == NULL)
+ if ((beentry = pgstat_get_beentry_by_proc_number(procNumber)) == NULL)
activity = "<backend information not available>";
else if (!HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
activity = "<insufficient privilege>";
@@ -767,12 +767,12 @@ pg_stat_get_backend_activity(PG_FUNCTION_ARGS)
Datum
pg_stat_get_backend_wait_event_type(PG_FUNCTION_ARGS)
{
- int32 beid = PG_GETARG_INT32(0);
+ int32 procNumber = PG_GETARG_INT32(0);
PgBackendStatus *beentry;
PGPROC *proc;
const char *wait_event_type = NULL;
- if ((beentry = pgstat_get_beentry_by_backend_id(beid)) == NULL)
+ if ((beentry = pgstat_get_beentry_by_proc_number(procNumber)) == NULL)
wait_event_type = "<backend information not available>";
else if (!HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
wait_event_type = "<insufficient privilege>";
@@ -788,12 +788,12 @@ pg_stat_get_backend_wait_event_type(PG_FUNCTION_ARGS)
Datum
pg_stat_get_backend_wait_event(PG_FUNCTION_ARGS)
{
- int32 beid = PG_GETARG_INT32(0);
+ int32 procNumber = PG_GETARG_INT32(0);
PgBackendStatus *beentry;
PGPROC *proc;
const char *wait_event = NULL;
- if ((beentry = pgstat_get_beentry_by_backend_id(beid)) == NULL)
+ if ((beentry = pgstat_get_beentry_by_proc_number(procNumber)) == NULL)
wait_event = "<backend information not available>";
else if (!HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
wait_event = "<insufficient privilege>";
@@ -810,11 +810,11 @@ pg_stat_get_backend_wait_event(PG_FUNCTION_ARGS)
Datum
pg_stat_get_backend_activity_start(PG_FUNCTION_ARGS)
{
- int32 beid = PG_GETARG_INT32(0);
+ int32 procNumber = PG_GETARG_INT32(0);
TimestampTz result;
PgBackendStatus *beentry;
- if ((beentry = pgstat_get_beentry_by_backend_id(beid)) == NULL)
+ if ((beentry = pgstat_get_beentry_by_proc_number(procNumber)) == NULL)
PG_RETURN_NULL();
else if (!HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
@@ -836,11 +836,11 @@ pg_stat_get_backend_activity_start(PG_FUNCTION_ARGS)
Datum
pg_stat_get_backend_xact_start(PG_FUNCTION_ARGS)
{
- int32 beid = PG_GETARG_INT32(0);
+ int32 procNumber = PG_GETARG_INT32(0);
TimestampTz result;
PgBackendStatus *beentry;
- if ((beentry = pgstat_get_beentry_by_backend_id(beid)) == NULL)
+ if ((beentry = pgstat_get_beentry_by_proc_number(procNumber)) == NULL)
PG_RETURN_NULL();
else if (!HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
@@ -858,11 +858,11 @@ pg_stat_get_backend_xact_start(PG_FUNCTION_ARGS)
Datum
pg_stat_get_backend_start(PG_FUNCTION_ARGS)
{
- int32 beid = PG_GETARG_INT32(0);
+ int32 procNumber = PG_GETARG_INT32(0);
TimestampTz result;
PgBackendStatus *beentry;
- if ((beentry = pgstat_get_beentry_by_backend_id(beid)) == NULL)
+ if ((beentry = pgstat_get_beentry_by_proc_number(procNumber)) == NULL)
PG_RETURN_NULL();
else if (!HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
@@ -880,13 +880,13 @@ pg_stat_get_backend_start(PG_FUNCTION_ARGS)
Datum
pg_stat_get_backend_client_addr(PG_FUNCTION_ARGS)
{
- int32 beid = PG_GETARG_INT32(0);
+ int32 procNumber = PG_GETARG_INT32(0);
PgBackendStatus *beentry;
SockAddr zero_clientaddr;
char remote_host[NI_MAXHOST];
int ret;
- if ((beentry = pgstat_get_beentry_by_backend_id(beid)) == NULL)
+ if ((beentry = pgstat_get_beentry_by_proc_number(procNumber)) == NULL)
PG_RETURN_NULL();
else if (!HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
@@ -925,13 +925,13 @@ pg_stat_get_backend_client_addr(PG_FUNCTION_ARGS)
Datum
pg_stat_get_backend_client_port(PG_FUNCTION_ARGS)
{
- int32 beid = PG_GETARG_INT32(0);
+ int32 procNumber = PG_GETARG_INT32(0);
PgBackendStatus *beentry;
SockAddr zero_clientaddr;
char remote_port[NI_MAXSERV];
int ret;
- if ((beentry = pgstat_get_beentry_by_backend_id(beid)) == NULL)
+ if ((beentry = pgstat_get_beentry_by_proc_number(procNumber)) == NULL)
PG_RETURN_NULL();
else if (!HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
diff --git a/src/backend/utils/cache/inval.c b/src/backend/utils/cache/inval.c
index f59b07a70f8..816b883b6da 100644
--- a/src/backend/utils/cache/inval.c
+++ b/src/backend/utils/cache/inval.c
@@ -1453,8 +1453,8 @@ CacheInvalidateRelcacheByRelid(Oid relid)
* replaying WAL as well as when creating it.
*
* Note: In order to avoid bloating SharedInvalidationMessage, we store only
- * three bytes of the backend ID using what would otherwise be padding space.
- * Thus, the maximum possible backend ID is 2^23-1.
+ * three bytes of the ProcNumber using what would otherwise be padding space.
+ * Thus, the maximum possible ProcNumber is 2^23-1.
*/
void
CacheInvalidateSmgr(RelFileLocatorBackend rlocator)
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 50acae45298..37c37df56c1 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -1144,13 +1144,13 @@ retry:
{
case RELPERSISTENCE_UNLOGGED:
case RELPERSISTENCE_PERMANENT:
- relation->rd_backend = InvalidBackendId;
+ relation->rd_backend = INVALID_PROC_NUMBER;
relation->rd_islocaltemp = false;
break;
case RELPERSISTENCE_TEMP:
if (isTempOrTempToastNamespace(relation->rd_rel->relnamespace))
{
- relation->rd_backend = BackendIdForTempRelations();
+ relation->rd_backend = ProcNumberForTempRelations();
relation->rd_islocaltemp = true;
}
else
@@ -1159,18 +1159,18 @@ retry:
* If it's a temp table, but not one of ours, we have to use
* the slow, grotty method to figure out the owning backend.
*
- * Note: it's possible that rd_backend gets set to MyBackendId
- * here, in case we are looking at a pg_class entry left over
- * from a crashed backend that coincidentally had the same
- * BackendId we're using. We should *not* consider such a
- * table to be "ours"; this is why we need the separate
- * rd_islocaltemp flag. The pg_class entry will get flushed
- * if/when we clean out the corresponding temp table namespace
- * in preparation for using it.
+ * Note: it's possible that rd_backend gets set to
+ * MyProcNumber here, in case we are looking at a pg_class
+ * entry left over from a crashed backend that coincidentally
+ * had the same ProcNumber we're using. We should *not*
+ * consider such a table to be "ours"; this is why we need the
+ * separate rd_islocaltemp flag. The pg_class entry will get
+ * flushed if/when we clean out the corresponding temp table
+ * namespace in preparation for using it.
*/
relation->rd_backend =
- GetTempNamespaceBackendId(relation->rd_rel->relnamespace);
- Assert(relation->rd_backend != InvalidBackendId);
+ GetTempNamespaceProcNumber(relation->rd_rel->relnamespace);
+ Assert(relation->rd_backend != INVALID_PROC_NUMBER);
relation->rd_islocaltemp = false;
}
break;
@@ -1896,7 +1896,7 @@ formrdesc(const char *relationName, Oid relationReltype,
relation->rd_newRelfilelocatorSubid = InvalidSubTransactionId;
relation->rd_firstRelfilelocatorSubid = InvalidSubTransactionId;
relation->rd_droppedSubid = InvalidSubTransactionId;
- relation->rd_backend = InvalidBackendId;
+ relation->rd_backend = INVALID_PROC_NUMBER;
relation->rd_islocaltemp = false;
/*
@@ -3611,12 +3611,12 @@ RelationBuildLocalRelation(const char *relname,
{
case RELPERSISTENCE_UNLOGGED:
case RELPERSISTENCE_PERMANENT:
- rel->rd_backend = InvalidBackendId;
+ rel->rd_backend = INVALID_PROC_NUMBER;
rel->rd_islocaltemp = false;
break;
case RELPERSISTENCE_TEMP:
Assert(isTempOrTempToastNamespace(relnamespace));
- rel->rd_backend = BackendIdForTempRelations();
+ rel->rd_backend = ProcNumberForTempRelations();
rel->rd_islocaltemp = true;
break;
default:
diff --git a/src/backend/utils/error/csvlog.c b/src/backend/utils/error/csvlog.c
index 1d44d8a6a31..99a3d862250 100644
--- a/src/backend/utils/error/csvlog.c
+++ b/src/backend/utils/error/csvlog.c
@@ -152,8 +152,8 @@ write_csvlog(ErrorData *edata)
/* Virtual transaction id */
/* keep VXID format in sync with lockfuncs.c */
- if (MyProc != NULL && MyProc->vxid.backendId != InvalidBackendId)
- appendStringInfo(&buf, "%d/%u", MyProc->vxid.backendId, MyProc->vxid.lxid);
+ if (MyProc != NULL && MyProc->vxid.procNumber != INVALID_PROC_NUMBER)
+ appendStringInfo(&buf, "%d/%u", MyProc->vxid.procNumber, MyProc->vxid.lxid);
appendStringInfoChar(&buf, ',');
/* Transaction id */
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 8cf36cbce07..10ccfe79eeb 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -3078,18 +3078,18 @@ log_status_format(StringInfo buf, const char *format, ErrorData *edata)
break;
case 'v':
/* keep VXID format in sync with lockfuncs.c */
- if (MyProc != NULL && MyProc->vxid.backendId != InvalidBackendId)
+ if (MyProc != NULL && MyProc->vxid.procNumber != INVALID_PROC_NUMBER)
{
if (padding != 0)
{
char strfbuf[128];
snprintf(strfbuf, sizeof(strfbuf) - 1, "%d/%u",
- MyProc->vxid.backendId, MyProc->vxid.lxid);
+ MyProc->vxid.procNumber, MyProc->vxid.lxid);
appendStringInfo(buf, "%*s", padding, strfbuf);
}
else
- appendStringInfo(buf, "%d/%u", MyProc->vxid.backendId, MyProc->vxid.lxid);
+ appendStringInfo(buf, "%d/%u", MyProc->vxid.procNumber, MyProc->vxid.lxid);
}
else if (padding != 0)
appendStringInfoSpaces(buf,
diff --git a/src/backend/utils/error/jsonlog.c b/src/backend/utils/error/jsonlog.c
index 067d9e30b16..d52c2143033 100644
--- a/src/backend/utils/error/jsonlog.c
+++ b/src/backend/utils/error/jsonlog.c
@@ -197,9 +197,9 @@ write_jsonlog(ErrorData *edata)
/* Virtual transaction id */
/* keep VXID format in sync with lockfuncs.c */
- if (MyProc != NULL && MyProc->vxid.backendId != InvalidBackendId)
+ if (MyProc != NULL && MyProc->vxid.procNumber != INVALID_PROC_NUMBER)
appendJSONKeyValueFmt(&buf, "vxid", true, "%d/%u",
- MyProc->vxid.backendId, MyProc->vxid.lxid);
+ MyProc->vxid.procNumber, MyProc->vxid.lxid);
/* Transaction id */
appendJSONKeyValueFmt(&buf, "txid", false, "%u",
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index f024b1a8497..f7672bab9db 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -22,7 +22,7 @@
#include "libpq/libpq-be.h"
#include "libpq/pqcomm.h"
#include "miscadmin.h"
-#include "storage/backendid.h"
+#include "storage/procnumber.h"
ProtocolVersion FrontendProtocol;
@@ -83,9 +83,9 @@ char postgres_exec_path[MAXPGPATH]; /* full path to backend */
/* note: currently this is not valid in backend processes */
#endif
-BackendId MyBackendId = InvalidBackendId;
+ProcNumber MyProcNumber = INVALID_PROC_NUMBER;
-BackendId ParallelLeaderBackendId = InvalidBackendId;
+ProcNumber ParallelLeaderProcNumber = INVALID_PROC_NUMBER;
Oid MyDatabaseId = InvalidOid;
diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
index a0916959b17..06735f66709 100644
--- a/src/backend/utils/time/snapmgr.c
+++ b/src/backend/utils/time/snapmgr.c
@@ -1154,7 +1154,7 @@ ExportSnapshot(Snapshot snapshot)
* inside the transaction from 1.
*/
snprintf(path, sizeof(path), SNAPSHOT_EXPORT_DIR "/%08X-%08X-%d",
- MyProc->vxid.backendId, MyProc->vxid.lxid,
+ MyProc->vxid.procNumber, MyProc->vxid.lxid,
list_length(exportedSnapshots) + 1);
/*
@@ -1182,7 +1182,7 @@ ExportSnapshot(Snapshot snapshot)
*/
initStringInfo(&buf);
- appendStringInfo(&buf, "vxid:%d/%u\n", MyProc->vxid.backendId, MyProc->vxid.lxid);
+ appendStringInfo(&buf, "vxid:%d/%u\n", MyProc->vxid.procNumber, MyProc->vxid.lxid);
appendStringInfo(&buf, "pid:%d\n", MyProcPid);
appendStringInfo(&buf, "dbid:%u\n", MyDatabaseId);
appendStringInfo(&buf, "iso:%d\n", XactIsoLevel);
@@ -1352,7 +1352,7 @@ parseVxidFromText(const char *prefix, char **s, const char *filename,
(errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
errmsg("invalid snapshot data in file \"%s\"", filename)));
ptr += prefixlen;
- if (sscanf(ptr, "%d/%u", &vxid->backendId, &vxid->localTransactionId) != 2)
+ if (sscanf(ptr, "%d/%u", &vxid->procNumber, &vxid->localTransactionId) != 2)
ereport(ERROR,
(errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
errmsg("invalid snapshot data in file \"%s\"", filename)));
diff --git a/src/common/relpath.c b/src/common/relpath.c
index b16fe19dea6..d6f45ffbf91 100644
--- a/src/common/relpath.c
+++ b/src/common/relpath.c
@@ -20,7 +20,7 @@
#include "catalog/pg_tablespace_d.h"
#include "common/relpath.h"
-#include "storage/backendid.h"
+#include "storage/procnumber.h"
/*
@@ -133,13 +133,13 @@ GetDatabasePath(Oid dbOid, Oid spcOid)
*
* Result is a palloc'd string.
*
- * Note: ideally, backendId would be declared as type BackendId, but relpath.h
- * would have to include a backend-only header to do that; doesn't seem worth
- * the trouble considering BackendId is just int anyway.
+ * Note: ideally, procNumber would be declared as type ProcNumber, but
+ * relpath.h would have to include a backend-only header to do that; doesn't
+ * seem worth the trouble considering ProcNumber is just int anyway.
*/
char *
GetRelationPath(Oid dbOid, Oid spcOid, RelFileNumber relNumber,
- int backendId, ForkNumber forkNumber)
+ int procNumber, ForkNumber forkNumber)
{
char *path;
@@ -147,7 +147,7 @@ GetRelationPath(Oid dbOid, Oid spcOid, RelFileNumber relNumber,
{
/* Shared system relations live in {datadir}/global */
Assert(dbOid == 0);
- Assert(backendId == InvalidBackendId);
+ Assert(procNumber == INVALID_PROC_NUMBER);
if (forkNumber != MAIN_FORKNUM)
path = psprintf("global/%u_%s",
relNumber, forkNames[forkNumber]);
@@ -157,7 +157,7 @@ GetRelationPath(Oid dbOid, Oid spcOid, RelFileNumber relNumber,
else if (spcOid == DEFAULTTABLESPACE_OID)
{
/* The default tablespace is {datadir}/base */
- if (backendId == InvalidBackendId)
+ if (procNumber == INVALID_PROC_NUMBER)
{
if (forkNumber != MAIN_FORKNUM)
path = psprintf("base/%u/%u_%s",
@@ -171,17 +171,17 @@ GetRelationPath(Oid dbOid, Oid spcOid, RelFileNumber relNumber,
{
if (forkNumber != MAIN_FORKNUM)
path = psprintf("base/%u/t%d_%u_%s",
- dbOid, backendId, relNumber,
+ dbOid, procNumber, relNumber,
forkNames[forkNumber]);
else
path = psprintf("base/%u/t%d_%u",
- dbOid, backendId, relNumber);
+ dbOid, procNumber, relNumber);
}
}
else
{
/* All other tablespaces are accessed via symlinks */
- if (backendId == InvalidBackendId)
+ if (procNumber == INVALID_PROC_NUMBER)
{
if (forkNumber != MAIN_FORKNUM)
path = psprintf("pg_tblspc/%u/%s/%u/%u_%s",
@@ -198,12 +198,12 @@ GetRelationPath(Oid dbOid, Oid spcOid, RelFileNumber relNumber,
if (forkNumber != MAIN_FORKNUM)
path = psprintf("pg_tblspc/%u/%s/%u/t%d_%u_%s",
spcOid, TABLESPACE_VERSION_DIRECTORY,
- dbOid, backendId, relNumber,
+ dbOid, procNumber, relNumber,
forkNames[forkNumber]);
else
path = psprintf("pg_tblspc/%u/%s/%u/t%d_%u",
spcOid, TABLESPACE_VERSION_DIRECTORY,
- dbOid, backendId, relNumber);
+ dbOid, procNumber, relNumber);
}
}
return path;
diff --git a/src/include/access/twophase.h b/src/include/access/twophase.h
index 0f6cf8e28b4..56248c00063 100644
--- a/src/include/access/twophase.h
+++ b/src/include/access/twophase.h
@@ -37,7 +37,7 @@ extern void PostPrepare_Twophase(void);
extern TransactionId TwoPhaseGetXidByVirtualXID(VirtualTransactionId vxid,
bool *have_more);
extern PGPROC *TwoPhaseGetDummyProc(TransactionId xid, bool lock_held);
-extern BackendId TwoPhaseGetDummyBackendId(TransactionId xid, bool lock_held);
+extern int TwoPhaseGetDummyProcNumber(TransactionId xid, bool lock_held);
extern GlobalTransaction MarkAsPreparing(TransactionId xid, const char *gid,
TimestampTz prepared_at,
diff --git a/src/include/catalog/namespace.h b/src/include/catalog/namespace.h
index 2599c7bdb14..8d434d48d57 100644
--- a/src/include/catalog/namespace.h
+++ b/src/include/catalog/namespace.h
@@ -16,6 +16,7 @@
#include "nodes/primnodes.h"
#include "storage/lock.h"
+#include "storage/procnumber.h"
/*
@@ -156,7 +157,7 @@ extern bool isTempOrTempToastNamespace(Oid namespaceId);
extern bool isAnyTempNamespace(Oid namespaceId);
extern bool isOtherTempNamespace(Oid namespaceId);
extern TempNamespaceStatus checkTempNamespaceStatus(Oid namespaceId);
-extern int GetTempNamespaceBackendId(Oid namespaceId);
+extern ProcNumber GetTempNamespaceProcNumber(Oid namespaceId);
extern Oid GetTempToastNamespace(void);
extern void GetTempNamespaceState(Oid *tempNamespaceId,
Oid *tempToastNamespaceId);
diff --git a/src/include/common/relpath.h b/src/include/common/relpath.h
index 1fab56b6d6a..6f006d5a938 100644
--- a/src/include/common/relpath.h
+++ b/src/include/common/relpath.h
@@ -74,7 +74,7 @@ extern int forkname_chars(const char *str, ForkNumber *fork);
extern char *GetDatabasePath(Oid dbOid, Oid spcOid);
extern char *GetRelationPath(Oid dbOid, Oid spcOid, RelFileNumber relNumber,
- int backendId, ForkNumber forkNumber);
+ int procNumber, ForkNumber forkNumber);
/*
* Wrapper macros for GetRelationPath. Beware of multiple
@@ -88,7 +88,7 @@ extern char *GetRelationPath(Oid dbOid, Oid spcOid, RelFileNumber relNumber,
/* First argument is a RelFileLocator */
#define relpathperm(rlocator, forknum) \
- relpathbackend(rlocator, InvalidBackendId, forknum)
+ relpathbackend(rlocator, INVALID_PROC_NUMBER, forknum)
/* First argument is a RelFileLocatorBackend */
#define relpath(rlocator, forknum) \
diff --git a/src/include/libpq/pqmq.h b/src/include/libpq/pqmq.h
index d6b2489c8cf..227df8976f4 100644
--- a/src/include/libpq/pqmq.h
+++ b/src/include/libpq/pqmq.h
@@ -17,7 +17,7 @@
#include "storage/shm_mq.h"
extern void pq_redirect_to_shm_mq(dsm_segment *seg, shm_mq_handle *mqh);
-extern void pq_set_parallel_leader(pid_t pid, BackendId backend_id);
+extern void pq_set_parallel_leader(pid_t pid, ProcNumber procNumber);
extern void pq_parse_errornotice(StringInfo msg, ErrorData *edata);
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 19213693ea4..f4983e892dd 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -195,11 +195,6 @@ extern PGDLLIMPORT char pkglib_path[];
extern PGDLLIMPORT char postgres_exec_path[];
#endif
-/*
- * done in storage/backendid.h for now.
- *
- * extern BackendId MyBackendId;
- */
extern PGDLLIMPORT Oid MyDatabaseId;
extern PGDLLIMPORT Oid MyDatabaseTableSpace;
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index 03f78b5b25f..46f95f7c8a6 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -72,7 +72,7 @@ extern void ShmemBackendArrayAllocation(void);
* by using a 64bit state; but it's unlikely to be worthwhile as 2^18-1
* backends exceed currently realistic configurations. Even if that limitation
* were removed, we still could not a) exceed 2^23-1 because inval.c stores
- * the backend ID as a 3-byte signed integer, b) INT_MAX/4 because some places
+ * the ProcNumber as a 3-byte signed integer, b) INT_MAX/4 because some places
* compute 4*MaxBackends without any overflow check. This is rechecked in the
* relevant GUC check hooks and in RegisterBackgroundWorker().
*/
diff --git a/src/include/storage/backendid.h b/src/include/storage/backendid.h
deleted file mode 100644
index 01387723f79..00000000000
--- a/src/include/storage/backendid.h
+++ /dev/null
@@ -1,41 +0,0 @@
-/*-------------------------------------------------------------------------
- *
- * backendid.h
- * POSTGRES backend id communication definitions
- *
- *
- * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
- * Portions Copyright (c) 1994, Regents of the University of California
- *
- * src/include/storage/backendid.h
- *
- *-------------------------------------------------------------------------
- */
-#ifndef BACKENDID_H
-#define BACKENDID_H
-
-/*
- * BackendId uniquely identifies an active backend or auxiliary process. It's
- * assigned at backend startup after authentication. Note that a backend ID
- * can be reused for a different backend immediately after a backend exits.
- *
- * Backend IDs are assigned starting from 1. For historical reasons, BackendId
- * 0 is unused, but InvalidBackendId is defined as -1.
- */
-typedef int BackendId;
-
-#define InvalidBackendId (-1)
-
-extern PGDLLIMPORT BackendId MyBackendId; /* backend id of this backend */
-
-/* backend id of our parallel session leader, or InvalidBackendId if none */
-extern PGDLLIMPORT BackendId ParallelLeaderBackendId;
-
-/*
- * The BackendId to use for our session's temp relations is normally our own,
- * but parallel workers should use their leader's ID.
- */
-#define BackendIdForTempRelations() \
- (ParallelLeaderBackendId == InvalidBackendId ? MyBackendId : ParallelLeaderBackendId)
-
-#endif /* BACKENDID_H */
diff --git a/src/include/storage/lock.h b/src/include/storage/lock.h
index c5c84201378..3b0961a288c 100644
--- a/src/include/storage/lock.h
+++ b/src/include/storage/lock.h
@@ -19,9 +19,9 @@
#endif
#include "lib/ilist.h"
-#include "storage/backendid.h"
#include "storage/lockdefs.h"
#include "storage/lwlock.h"
+#include "storage/procnumber.h"
#include "storage/shmem.h"
#include "utils/timestamp.h"
@@ -42,7 +42,7 @@ extern PGDLLIMPORT bool Debug_deadlocks;
/*
* Top-level transactions are identified by VirtualTransactionIDs comprising
- * PGPROC fields backendId and lxid. For recovered prepared transactions, the
+ * PGPROC fields procNumber and lxid. For recovered prepared transactions, the
* LocalTransactionId is an ordinary XID; LOCKTAG_VIRTUALTRANSACTION never
* refers to that kind. These are guaranteed unique over the short term, but
* will be reused after a database restart or XID wraparound; hence they
@@ -50,7 +50,7 @@ extern PGDLLIMPORT bool Debug_deadlocks;
*
* Note that struct VirtualTransactionId can not be assumed to be atomically
* assignable as a whole. However, type LocalTransactionId is assumed to
- * be atomically assignable, and the backend ID doesn't change often enough
+ * be atomically assignable, and the proc number doesn't change often enough
* to be a problem, so we can fetch or assign the two fields separately.
* We deliberately refrain from using the struct within PGPROC, to prevent
* coding errors from trying to use struct assignment with it; instead use
@@ -58,7 +58,7 @@ extern PGDLLIMPORT bool Debug_deadlocks;
*/
typedef struct
{
- BackendId backendId; /* backendId from PGPROC */
+ ProcNumber procNumber; /* proc number of the PGPROC */
LocalTransactionId localTransactionId; /* lxid from PGPROC */
} VirtualTransactionId;
@@ -67,15 +67,15 @@ typedef struct
#define VirtualTransactionIdIsValid(vxid) \
(LocalTransactionIdIsValid((vxid).localTransactionId))
#define VirtualTransactionIdIsRecoveredPreparedXact(vxid) \
- ((vxid).backendId == InvalidBackendId)
+ ((vxid).procNumber == INVALID_PROC_NUMBER)
#define VirtualTransactionIdEquals(vxid1, vxid2) \
- ((vxid1).backendId == (vxid2).backendId && \
+ ((vxid1).procNumber == (vxid2).procNumber && \
(vxid1).localTransactionId == (vxid2).localTransactionId)
#define SetInvalidVirtualTransactionId(vxid) \
- ((vxid).backendId = InvalidBackendId, \
+ ((vxid).procNumber = INVALID_PROC_NUMBER, \
(vxid).localTransactionId = InvalidLocalTransactionId)
#define GET_VXID_FROM_PGPROC(vxid_dst, proc) \
- ((vxid_dst).backendId = (proc).vxid.backendId, \
+ ((vxid_dst).procNumber = (proc).vxid.procNumber, \
(vxid_dst).localTransactionId = (proc).vxid.lxid)
/* MAX_LOCKMODES cannot be larger than the # of bits in LOCKMASK */
@@ -233,7 +233,7 @@ typedef struct LOCKTAG
/* ID info for a virtual transaction is its VirtualTransactionId */
#define SET_LOCKTAG_VIRTUALTRANSACTION(locktag,vxid) \
- ((locktag).locktag_field1 = (vxid).backendId, \
+ ((locktag).locktag_field1 = (vxid).procNumber, \
(locktag).locktag_field2 = (vxid).localTransactionId, \
(locktag).locktag_field3 = 0, \
(locktag).locktag_field4 = 0, \
diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h
index d29811ce16d..1095aefddfe 100644
--- a/src/include/storage/proc.h
+++ b/src/include/storage/proc.h
@@ -21,6 +21,7 @@
#include "storage/lock.h"
#include "storage/pg_sema.h"
#include "storage/proclist_types.h"
+#include "storage/procnumber.h"
/*
* Each backend advertises up to PGPROC_MAX_CACHED_SUBXIDS TransactionIds
@@ -84,12 +85,6 @@ struct XidCache
*/
#define FP_LOCK_SLOTS_PER_BACKEND 16
-/*
- * An invalid pgprocno. Must be larger than the maximum number of PGPROC
- * structures we could possibly have. See comments for MAX_BACKENDS.
- */
-#define INVALID_PGPROCNO PG_INT32_MAX
-
/*
* Flags for PGPROC.delayChkptFlags
*
@@ -197,18 +192,18 @@ struct PGPROC
* is not atomically assignable as whole, and we want to enforce code to
* consider both parts separately. See comments at VirtualTransactionId.
*/
- struct {
- BackendId backendId; /* For regular backends, equal to
- * GetBackendIdFromPGProc(proc). For
- * prepared xacts, ID of the original
- * backend that processed the
- * transaction. For unused PGPROC entries,
- * InvalidbackendID. */
+ struct
+ {
+ ProcNumber procNumber; /* For regular backends, equal to
+ * GetNumberFromPGProc(proc). For prepared
+ * xacts, ID of the original backend that
+ * processed the transaction. For unused
+ * PGPROC entries, INVALID_PROC_NUMBER. */
LocalTransactionId lxid; /* local id of top-level transaction
* currently * being executed by this
* proc, if running; else
* InvalidLocaltransactionId */
- } vxid;
+ } vxid;
/* These fields are zero while a backend is still starting up: */
Oid databaseId; /* OID of database this backend is using */
@@ -317,7 +312,19 @@ struct PGPROC
extern PGDLLIMPORT PGPROC *MyProc;
-extern PGDLLIMPORT int MyProcNumber; /* same as GetNumberFromPGProc(MyProc) */
+
+/* Proc number of this backend. Equal to GetNumberFromPGProc(MyProc). */
+extern PGDLLIMPORT ProcNumber MyProcNumber;
+
+/* Our parallel session leader, or INVALID_PROC_NUMBER if none */
+extern PGDLLIMPORT ProcNumber ParallelLeaderProcNumber;
+
+/*
+ * The proc number to use for our session's temp relations is normally our own,
+ * but parallel workers should use their leader's ID.
+ */
+#define ProcNumberForTempRelations() \
+ (ParallelLeaderProcNumber == INVALID_PROC_NUMBER ? MyProcNumber : ParallelLeaderProcNumber)
/*
* There is one ProcGlobal struct for the whole database cluster.
@@ -422,15 +429,10 @@ extern PGDLLIMPORT PROC_HDR *ProcGlobal;
extern PGDLLIMPORT PGPROC *PreparedXactProcs;
/*
- * Accessors for getting PGPROC given a pgprocno or BackendId, and vice versa.
- *
- * For historical reasons, some code uses 0-based "proc numbers", while other
- * code uses 1-based backend IDs.
+ * Accessors for getting PGPROC given a ProcNumber and vice versa.
*/
#define GetPGProcByNumber(n) (&ProcGlobal->allProcs[(n)])
#define GetNumberFromPGProc(proc) ((proc) - &ProcGlobal->allProcs[0])
-#define GetPGProcByBackendId(n) (&ProcGlobal->allProcs[(n) - 1])
-#define GetBackendIdFromPGProc(proc) (GetNumberFromPGProc(proc) + 1)
/*
* We set aside some extra PGPROC structures for auxiliary processes,
@@ -477,7 +479,7 @@ extern bool IsWaitingForLock(void);
extern void LockErrorCleanup(void);
extern void ProcWaitForSignal(uint32 wait_event_info);
-extern void ProcSendSignal(int pgprocno);
+extern void ProcSendSignal(ProcNumber procNumber);
extern PGPROC *AuxiliaryPidGetProc(int pid);
diff --git a/src/include/storage/procarray.h b/src/include/storage/procarray.h
index 3af7577e8c6..8ca60504622 100644
--- a/src/include/storage/procarray.h
+++ b/src/include/storage/procarray.h
@@ -64,10 +64,10 @@ extern VirtualTransactionId *GetVirtualXIDsDelayingChkpt(int *nvxids, int type);
extern bool HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids,
int nvxids, int type);
-extern PGPROC *BackendIdGetProc(int backendID);
-extern void BackendIdGetTransactionIds(int backendID, TransactionId *xid,
- TransactionId *xmin, int *nsubxid,
- bool *overflowed);
+extern PGPROC *ProcNumberGetProc(int procNumber);
+extern void ProcNumberGetTransactionIds(int procNumber, TransactionId *xid,
+ TransactionId *xmin, int *nsubxid,
+ bool *overflowed);
extern PGPROC *BackendPidGetProc(int pid);
extern PGPROC *BackendPidGetProcWithLock(int pid);
extern int BackendXidGetPid(TransactionId xid);
diff --git a/src/include/storage/proclist.h b/src/include/storage/proclist.h
index a1ca4cbd631..34a1b7dcab5 100644
--- a/src/include/storage/proclist.h
+++ b/src/include/storage/proclist.h
@@ -28,7 +28,7 @@
static inline void
proclist_init(proclist_head *list)
{
- list->head = list->tail = INVALID_PGPROCNO;
+ list->head = list->tail = INVALID_PROC_NUMBER;
}
/*
@@ -37,7 +37,7 @@ proclist_init(proclist_head *list)
static inline bool
proclist_is_empty(const proclist_head *list)
{
- return list->head == INVALID_PGPROCNO;
+ return list->head == INVALID_PROC_NUMBER;
}
/*
@@ -62,20 +62,20 @@ proclist_push_head_offset(proclist_head *list, int procno, size_t node_offset)
Assert(node->next == 0 && node->prev == 0);
- if (list->head == INVALID_PGPROCNO)
+ if (list->head == INVALID_PROC_NUMBER)
{
- Assert(list->tail == INVALID_PGPROCNO);
- node->next = node->prev = INVALID_PGPROCNO;
+ Assert(list->tail == INVALID_PROC_NUMBER);
+ node->next = node->prev = INVALID_PROC_NUMBER;
list->head = list->tail = procno;
}
else
{
- Assert(list->tail != INVALID_PGPROCNO);
+ Assert(list->tail != INVALID_PROC_NUMBER);
Assert(list->head != procno);
Assert(list->tail != procno);
node->next = list->head;
proclist_node_get(node->next, node_offset)->prev = procno;
- node->prev = INVALID_PGPROCNO;
+ node->prev = INVALID_PROC_NUMBER;
list->head = procno;
}
}
@@ -90,20 +90,20 @@ proclist_push_tail_offset(proclist_head *list, int procno, size_t node_offset)
Assert(node->next == 0 && node->prev == 0);
- if (list->tail == INVALID_PGPROCNO)
+ if (list->tail == INVALID_PROC_NUMBER)
{
- Assert(list->head == INVALID_PGPROCNO);
- node->next = node->prev = INVALID_PGPROCNO;
+ Assert(list->head == INVALID_PROC_NUMBER);
+ node->next = node->prev = INVALID_PROC_NUMBER;
list->head = list->tail = procno;
}
else
{
- Assert(list->head != INVALID_PGPROCNO);
+ Assert(list->head != INVALID_PROC_NUMBER);
Assert(list->head != procno);
Assert(list->tail != procno);
node->prev = list->tail;
proclist_node_get(node->prev, node_offset)->next = procno;
- node->next = INVALID_PGPROCNO;
+ node->next = INVALID_PROC_NUMBER;
list->tail = procno;
}
}
@@ -118,7 +118,7 @@ proclist_delete_offset(proclist_head *list, int procno, size_t node_offset)
Assert(node->next != 0 || node->prev != 0);
- if (node->prev == INVALID_PGPROCNO)
+ if (node->prev == INVALID_PROC_NUMBER)
{
Assert(list->head == procno);
list->head = node->next;
@@ -126,7 +126,7 @@ proclist_delete_offset(proclist_head *list, int procno, size_t node_offset)
else
proclist_node_get(node->prev, node_offset)->next = node->next;
- if (node->next == INVALID_PGPROCNO)
+ if (node->next == INVALID_PROC_NUMBER)
{
Assert(list->tail == procno);
list->tail = node->prev;
@@ -160,8 +160,8 @@ proclist_contains_offset(const proclist_head *list, int procno,
* tail, and that seems worth doing, since in practice that should often
* be enough to catch mistakes.
*/
- Assert(node->prev != INVALID_PGPROCNO || list->head == procno);
- Assert(node->next != INVALID_PGPROCNO || list->tail == procno);
+ Assert(node->prev != INVALID_PROC_NUMBER || list->head == procno);
+ Assert(node->next != INVALID_PROC_NUMBER || list->tail == procno);
return true;
}
@@ -207,12 +207,12 @@ proclist_pop_head_node_offset(proclist_head *list, size_t node_offset)
for (AssertVariableIsOfTypeMacro(iter, proclist_mutable_iter), \
AssertVariableIsOfTypeMacro(lhead, proclist_head *), \
(iter).cur = (lhead)->head, \
- (iter).next = (iter).cur == INVALID_PGPROCNO ? INVALID_PGPROCNO : \
+ (iter).next = (iter).cur == INVALID_PROC_NUMBER ? INVALID_PROC_NUMBER : \
proclist_node_get((iter).cur, \
offsetof(PGPROC, link_member))->next; \
- (iter).cur != INVALID_PGPROCNO; \
+ (iter).cur != INVALID_PROC_NUMBER; \
(iter).cur = (iter).next, \
- (iter).next = (iter).cur == INVALID_PGPROCNO ? INVALID_PGPROCNO : \
+ (iter).next = (iter).cur == INVALID_PROC_NUMBER ? INVALID_PROC_NUMBER : \
proclist_node_get((iter).cur, \
offsetof(PGPROC, link_member))->next)
diff --git a/src/include/storage/proclist_types.h b/src/include/storage/proclist_types.h
index 79c509f29a2..a4fb88c1d0d 100644
--- a/src/include/storage/proclist_types.h
+++ b/src/include/storage/proclist_types.h
@@ -15,28 +15,30 @@
#ifndef PROCLIST_TYPES_H
#define PROCLIST_TYPES_H
+#include "storage/procnumber.h"
+
/*
* A node in a doubly-linked list of processes. The link fields contain
* the 0-based PGPROC indexes of the next and previous process, or
- * INVALID_PGPROCNO in the next-link of the last node and the prev-link
+ * INVALID_PROC_NUMBER in the next-link of the last node and the prev-link
* of the first node. A node that is currently not in any list
* should have next == prev == 0; this is not a possible state for a node
* that is in a list, because we disallow circularity.
*/
typedef struct proclist_node
{
- int next; /* pgprocno of the next PGPROC */
- int prev; /* pgprocno of the prev PGPROC */
+ ProcNumber next; /* pgprocno of the next PGPROC */
+ ProcNumber prev; /* pgprocno of the prev PGPROC */
} proclist_node;
/*
* Header of a doubly-linked list of PGPROCs, identified by pgprocno.
- * An empty list is represented by head == tail == INVALID_PGPROCNO.
+ * An empty list is represented by head == tail == INVALID_PROC_NUMBER.
*/
typedef struct proclist_head
{
- int head; /* pgprocno of the head PGPROC */
- int tail; /* pgprocno of the tail PGPROC */
+ ProcNumber head; /* pgprocno of the head PGPROC */
+ ProcNumber tail; /* pgprocno of the tail PGPROC */
} proclist_head;
/*
@@ -44,8 +46,8 @@ typedef struct proclist_head
*/
typedef struct proclist_mutable_iter
{
- int cur; /* pgprocno of the current PGPROC */
- int next; /* pgprocno of the next PGPROC */
+ ProcNumber cur; /* pgprocno of the current PGPROC */
+ ProcNumber next; /* pgprocno of the next PGPROC */
} proclist_mutable_iter;
#endif /* PROCLIST_TYPES_H */
diff --git a/src/include/storage/procnumber.h b/src/include/storage/procnumber.h
new file mode 100644
index 00000000000..49eb0e215cc
--- /dev/null
+++ b/src/include/storage/procnumber.h
@@ -0,0 +1,43 @@
+/*-------------------------------------------------------------------------
+ *
+ * procnumber.h
+ * definition of process number
+ *
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/storage/procnumber.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PROCNUMBER_H
+#define PROCNUMBER_H
+
+/*
+ * ProcNumber uniquely identifies an active backend or auxiliary process.
+ * It's assigned at backend startup after authentication, when the process
+ * adds itself to the proc array. It is an index into the proc array,
+ * starting from 0. Note that a ProcNumber can be reused for a different
+ * backend immediately after a backend exits.
+ */
+typedef int ProcNumber;
+
+#define INVALID_PROC_NUMBER (-1)
+
+/*
+ * Proc number of this backend (same as GetNumberFromPGProc(MyProc))
+ */
+extern PGDLLIMPORT ProcNumber MyProcNumber;
+
+/* proc number of our parallel session leader, or INVALID_PROC_NUMBER if none */
+extern PGDLLIMPORT ProcNumber ParallelLeaderProcNumber;
+
+/*
+ * The ProcNumber to use for our session's temp relations is normally our own,
+ * but parallel workers should use their leader's proc number.
+ */
+#define ProcNumberForTempRelations() \
+ (ParallelLeaderProcNumber == INVALID_PROC_NUMBER ? MyProcNumber : ParallelLeaderProcNumber)
+
+#endif /* PROCNUMBER_H */
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index febdda3611c..7d290ea7d05 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -14,7 +14,7 @@
#ifndef PROCSIGNAL_H
#define PROCSIGNAL_H
-#include "storage/backendid.h"
+#include "storage/procnumber.h"
/*
@@ -64,7 +64,7 @@ extern void ProcSignalShmemInit(void);
extern void ProcSignalInit(void);
extern int SendProcSignal(pid_t pid, ProcSignalReason reason,
- BackendId backendId);
+ ProcNumber procNumber);
extern uint64 EmitProcSignalBarrier(ProcSignalBarrierType type);
extern void WaitForProcSignalBarrier(uint64 generation);
diff --git a/src/include/storage/relfilelocator.h b/src/include/storage/relfilelocator.h
index ef8c2b452b3..2f970a25c36 100644
--- a/src/include/storage/relfilelocator.h
+++ b/src/include/storage/relfilelocator.h
@@ -15,14 +15,15 @@
#define RELFILELOCATOR_H
#include "common/relpath.h"
-#include "storage/backendid.h"
+#include "storage/procnumber.h"
/*
* RelFileLocator must provide all that we need to know to physically access
- * a relation, with the exception of the backend ID, which can be provided
- * separately. Note, however, that a "physical" relation is comprised of
- * multiple files on the filesystem, as each fork is stored as a separate
- * file, and each fork can be divided into multiple segments. See md.c.
+ * a relation, with the exception of the backend's proc number, which can be
+ * provided separately. Note, however, that a "physical" relation is
+ * comprised of multiple files on the filesystem, as each fork is stored as
+ * a separate file, and each fork can be divided into multiple segments. See
+ * md.c.
*
* spcOid identifies the tablespace of the relation. It corresponds to
* pg_tablespace.oid.
@@ -62,28 +63,28 @@ typedef struct RelFileLocator
} RelFileLocator;
/*
- * Augmenting a relfilelocator with the backend ID provides all the information
- * we need to locate the physical storage. The backend ID is InvalidBackendId
- * for regular relations (those accessible to more than one backend), or the
- * owning backend's ID for backend-local relations. Backend-local relations
- * are always transient and removed in case of a database crash; they are
- * never WAL-logged or fsync'd.
+ * Augmenting a relfilelocator with the backend's proc number provides all the
+ * information we need to locate the physical storage. 'backend' is
+ * INVALID_PROC_NUMBER for regular relations (those accessible to more than
+ * one backend), or the owning backend's proc number for backend-local
+ * relations. Backend-local relations are always transient and removed in
+ * case of a database crash; they are never WAL-logged or fsync'd.
*/
typedef struct RelFileLocatorBackend
{
RelFileLocator locator;
- BackendId backend;
+ ProcNumber backend;
} RelFileLocatorBackend;
#define RelFileLocatorBackendIsTemp(rlocator) \
- ((rlocator).backend != InvalidBackendId)
+ ((rlocator).backend != INVALID_PROC_NUMBER)
/*
* Note: RelFileLocatorEquals and RelFileLocatorBackendEquals compare relNumber
* first since that is most likely to be different in two unequal
* RelFileLocators. It is probably redundant to compare spcOid if the other
* fields are found equal, but do it anyway to be sure. Likewise for checking
- * the backend ID in RelFileLocatorBackendEquals.
+ * the backend number in RelFileLocatorBackendEquals.
*/
#define RelFileLocatorEquals(locator1, locator2) \
((locator1).relNumber == (locator2).relNumber && \
diff --git a/src/include/storage/sinval.h b/src/include/storage/sinval.h
index 1bdd58945d2..8f5744b21bc 100644
--- a/src/include/storage/sinval.h
+++ b/src/include/storage/sinval.h
@@ -88,8 +88,8 @@ typedef struct
{
/* note: field layout chosen to pack into 16 bytes */
int8 id; /* type field --- must be first */
- int8 backend_hi; /* high bits of backend ID, if temprel */
- uint16 backend_lo; /* low bits of backend ID, if temprel */
+ int8 backend_hi; /* high bits of backend procno, if temprel */
+ uint16 backend_lo; /* low bits of backend procno, if temprel */
RelFileLocator rlocator; /* spcOid, dbOid, relNumber */
} SharedInvalSmgrMsg;
diff --git a/src/include/storage/smgr.h b/src/include/storage/smgr.h
index 8d90d1f8e22..e130aceea8a 100644
--- a/src/include/storage/smgr.h
+++ b/src/include/storage/smgr.h
@@ -74,7 +74,7 @@ typedef SMgrRelationData *SMgrRelation;
RelFileLocatorBackendIsTemp((smgr)->smgr_rlocator)
extern void smgrinit(void);
-extern SMgrRelation smgropen(RelFileLocator rlocator, BackendId backend);
+extern SMgrRelation smgropen(RelFileLocator rlocator, ProcNumber backend);
extern bool smgrexists(SMgrRelation reln, ForkNumber forknum);
extern void smgrpin(SMgrRelation reln);
extern void smgrunpin(SMgrRelation reln);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index 7b8a34f64f6..7b7f6f59d07 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -13,7 +13,7 @@
#include "datatype/timestamp.h"
#include "libpq/pqcomm.h"
#include "miscadmin.h" /* for BackendType */
-#include "storage/backendid.h"
+#include "storage/procnumber.h"
#include "utils/backend_progress.h"
@@ -87,7 +87,7 @@ typedef struct PgBackendGSSStatus
*
* Each live backend maintains a PgBackendStatus struct in shared memory
* showing its current activity. (The structs are allocated according to
- * BackendId, but that is not critical.) Note that this is unrelated to the
+ * ProcNumber, but that is not critical.) Note that this is unrelated to the
* cumulative stats system (i.e. pgstat.c et al).
*
* Each auxiliary process also maintains a PgBackendStatus struct in shared
@@ -250,11 +250,9 @@ typedef struct LocalPgBackendStatus
PgBackendStatus backendStatus;
/*
- * The backend ID. For auxiliary processes, this will be set to a value
- * greater than MaxBackends (since auxiliary processes do not have proper
- * backend IDs).
+ * The proc number.
*/
- BackendId backend_id;
+ ProcNumber proc_number;
/*
* The xid of the current transaction if available, InvalidTransactionId
@@ -333,8 +331,8 @@ extern uint64 pgstat_get_my_query_id(void);
* ----------
*/
extern int pgstat_fetch_stat_numbackends(void);
-extern PgBackendStatus *pgstat_get_beentry_by_backend_id(BackendId beid);
-extern LocalPgBackendStatus *pgstat_get_local_beentry_by_backend_id(BackendId beid);
+extern PgBackendStatus *pgstat_get_beentry_by_proc_number(ProcNumber procNumber);
+extern LocalPgBackendStatus *pgstat_get_local_beentry_by_proc_number(ProcNumber procNumber);
extern LocalPgBackendStatus *pgstat_get_local_beentry_by_index(int idx);
extern char *pgstat_clip_activity(const char *raw_activity);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index ab9fa4faf9f..b3ea2b20421 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -57,7 +57,7 @@ typedef struct RelationData
RelFileLocator rd_locator; /* relation physical identifier */
SMgrRelation rd_smgr; /* cached file handle, or NULL */
int rd_refcnt; /* reference count */
- BackendId rd_backend; /* owning backend id, if temporary relation */
+ ProcNumber rd_backend; /* owning backend's proc number, if temp rel */
bool rd_islocaltemp; /* rel is a temp rel of this session */
bool rd_isnailed; /* rel is nailed in cache */
bool rd_isvalid; /* relcache entry is valid */
--
2.39.2
Re: Refactoring backend fork+exec code
On 22/02/2024 02:37, Heikki Linnakangas wrote:
On 15/02/2024 07:09, Robert Haas wrote:
On Thu, Feb 15, 2024 at 3:07 AM Andres Freund <andres@anarazel.de> wrote:
I think the last remaining question here is about the 0- vs 1-based indexing
of BackendIds. Is it a good idea to switch to 0-based indexing? And if we do
it, should we reserve PGPROC 0. I'm on the fence on this one.I lean towards it being a good idea. Having two internal indexing schemes was
bad enough so far, but at least one would fairly quickly notice if one used
the wrong one. If they're just offset by 1, it might end up taking longer,
because that'll often also be a valid id.Yeah, I think making everything 0-based is probably the best way
forward long term. It might require more cleanup work to get there,
but it's just a lot simpler in the end, IMHO.Here's another patch version that does that. Yeah, I agree it's nicer in
the end.I'm pretty happy with this now. I'll read through these patches myself
again after sleeping over it and try to get this committed by the end of
the week, but another pair of eyes wouldn't hurt.
And pushed. Thanks for the reviews!
--
Heikki Linnakangas
Neon (https://neon.tech)
Re: Refactoring backend fork+exec code
I've now completed many of the side-quests, here are the patches that
remain.
The first three patches form a logical unit. They move the
initialization of the Port struct from postmaster to the backend
process. Currently, that work is split between the postmaster and the
backend process so that postmaster fills in the socket and some other
fields, and the backend process fills the rest after reading the startup
packet. With these patches, there is a new much smaller ClientSocket
struct that is passed from the postmaster to the child process, which
contains just the fields that postmaster initializes. The Port struct is
allocated in the child process. That makes the backend startup easier to
understand. I plan to commit those three patches next if there are no
objections.
That leaves the rest of the patches. I think they're in pretty good
shape too, and I've gotten some review on those earlier and have
addressed the comments I got so far, but would still appreciate another
round of review.
--
Heikki Linnakangas
Neon (https://neon.tech)
Attachments:
v12-0001-Pass-CAC-as-argument-to-backend-process.patchtext/x-patch; charset=UTF-8; name=v12-0001-Pass-CAC-as-argument-to-backend-process.patchDownload
From 7920cdf03b2fe9102d324eda7858a8a569feaf7b Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Thu, 30 Nov 2023 00:04:22 +0200
Subject: [PATCH v12 1/8] Pass CAC as argument to backend process
We used to smuggle it to the child process in the Port struct, but it
seems better to pass it down as a separate argument. This paves the
way for the next commits, in which we move the initialization of the
Port struct to the backend process, after forking.
Reviewed-by: Tristan Partin, Andres Freund
Discussion: https://www.postgresql.org/message-id/7a59b073-5b5b-151e-7ed3-8b01ff7ce9ef@iki.fi
---
src/backend/postmaster/postmaster.c | 44 +++++++++++++++++++++--------
src/include/libpq/libpq-be.h | 12 --------
2 files changed, 32 insertions(+), 24 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 59318067460..851828077b1 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -420,7 +420,19 @@ static void HandleChildCrash(int pid, int exitstatus, const char *procname);
static void LogChildExit(int lev, const char *procname,
int pid, int exitstatus);
static void PostmasterStateMachine(void);
-static void BackendInitialize(Port *port);
+
+/* Return value of canAcceptConnections() */
+typedef enum CAC_state
+{
+ CAC_OK,
+ CAC_STARTUP,
+ CAC_SHUTDOWN,
+ CAC_RECOVERY,
+ CAC_NOTCONSISTENT,
+ CAC_TOOMANY,
+} CAC_state;
+
+static void BackendInitialize(Port *port, CAC_state cac);
static void BackendRun(Port *port) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
@@ -479,7 +491,7 @@ typedef struct
} win32_deadchild_waitinfo;
#endif /* WIN32 */
-static pid_t backend_forkexec(Port *port);
+static pid_t backend_forkexec(Port *port, CAC_state cac);
static pid_t internal_forkexec(int argc, char *argv[], Port *port, BackgroundWorker *worker);
/* Type for a socket that can be inherited to a client process */
@@ -4089,6 +4101,7 @@ BackendStartup(Port *port)
{
Backend *bn; /* for backend cleanup */
pid_t pid;
+ CAC_state cac;
/*
* Create backend data structure. Better before the fork() so we can
@@ -4120,8 +4133,8 @@ BackendStartup(Port *port)
bn->cancel_key = MyCancelKey;
/* Pass down canAcceptConnections state */
- port->canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
- bn->dead_end = (port->canAcceptConnections != CAC_OK);
+ cac = canAcceptConnections(BACKEND_TYPE_NORMAL);
+ bn->dead_end = (cac != CAC_OK);
/*
* Unless it's a dead_end child, assign it a child slot number
@@ -4135,7 +4148,7 @@ BackendStartup(Port *port)
bn->bgworker_notify = false;
#ifdef EXEC_BACKEND
- pid = backend_forkexec(port);
+ pid = backend_forkexec(port, cac);
#else /* !EXEC_BACKEND */
pid = fork_process();
if (pid == 0) /* child */
@@ -4147,7 +4160,7 @@ BackendStartup(Port *port)
ClosePostmasterPorts(false);
/* Perform additional initialization and collect startup packet */
- BackendInitialize(port);
+ BackendInitialize(port, cac);
/* And run the backend */
BackendRun(port);
@@ -4234,7 +4247,7 @@ report_fork_failure_to_client(Port *port, int errnum)
* but have not yet set up most of our local pointers to shmem structures.
*/
static void
-BackendInitialize(Port *port)
+BackendInitialize(Port *port, CAC_state cac)
{
int status;
int ret;
@@ -4369,7 +4382,7 @@ BackendInitialize(Port *port)
*/
if (status == STATUS_OK)
{
- switch (port->canAcceptConnections)
+ switch (cac)
{
case CAC_STARTUP:
ereport(FATAL,
@@ -4508,15 +4521,19 @@ postmaster_forkexec(int argc, char *argv[])
* returns the pid of the fork/exec'd process, or -1 on failure
*/
static pid_t
-backend_forkexec(Port *port)
+backend_forkexec(Port *port, CAC_state cac)
{
- char *av[4];
+ char *av[5];
int ac = 0;
+ char cacbuf[10];
av[ac++] = "postgres";
av[ac++] = "--forkbackend";
av[ac++] = NULL; /* filled in by internal_forkexec */
+ snprintf(cacbuf, sizeof(cacbuf), "%d", (int) cac);
+ av[ac++] = cacbuf;
+
av[ac] = NULL;
Assert(ac < lengthof(av));
@@ -4923,7 +4940,10 @@ SubPostmasterMain(int argc, char *argv[])
/* Run backend or appropriate child */
if (strcmp(argv[1], "--forkbackend") == 0)
{
- Assert(argc == 3); /* shouldn't be any more args */
+ CAC_state cac;
+
+ Assert(argc == 4);
+ cac = (CAC_state) atoi(argv[3]);
/*
* Need to reinitialize the SSL library in the backend, since the
@@ -4957,7 +4977,7 @@ SubPostmasterMain(int argc, char *argv[])
* PGPROC slots, we have already initialized libpq and are able to
* report the error to the client.
*/
- BackendInitialize(port);
+ BackendInitialize(port, cac);
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 47d66d55241..59c2a1d874f 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -58,17 +58,6 @@ typedef struct
#include "libpq/pqcomm.h"
-typedef enum CAC_state
-{
- CAC_OK,
- CAC_STARTUP,
- CAC_SHUTDOWN,
- CAC_RECOVERY,
- CAC_NOTCONSISTENT,
- CAC_TOOMANY,
-} CAC_state;
-
-
/*
* GSSAPI specific state information
*/
@@ -156,7 +145,6 @@ typedef struct Port
int remote_hostname_resolv; /* see above */
int remote_hostname_errcode; /* see above */
char *remote_port; /* text rep of remote port */
- CAC_state canAcceptConnections; /* postmaster connection status */
/*
* Information that needs to be saved from the startup packet and passed
--
2.39.2
v12-0002-Remove-ConnCreate-and-ConnFree-and-allocate-Port.patchtext/x-patch; charset=UTF-8; name=v12-0002-Remove-ConnCreate-and-ConnFree-and-allocate-Port.patchDownload
From b3b74c2a992742d2bf8aaf6260bc18cf2a8523cb Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Fri, 8 Dec 2023 10:30:37 +0200
Subject: [PATCH v12 2/8] Remove ConnCreate and ConnFree, and allocate Port in
stack
By allocating Port in stack, we don't need to deal with out-of-memory
errors, which makes the code a little less verbose.
XXX: This should perhaps be squashed with the next commit. It changes
how the Port structure is allocated again. But maybe it's easier to
review separately.
Reviewed-by: Tristan Partin, Andres Freund
Discussion: https://www.postgresql.org/message-id/7a59b073-5b5b-151e-7ed3-8b01ff7ce9ef@iki.fi
---
src/backend/postmaster/postmaster.c | 68 +++++------------------------
src/backend/tcop/postgres.c | 6 +--
2 files changed, 13 insertions(+), 61 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 851828077b1..2cf5e228186 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -401,8 +401,6 @@ static void CloseServerPorts(int status, Datum arg);
static void unlink_external_pid_file(int status, Datum arg);
static void getInstallationPaths(const char *argv0);
static void checkControlFile(void);
-static Port *ConnCreate(int serverFd);
-static void ConnFree(Port *port);
static void handle_pm_pmsignal_signal(SIGNAL_ARGS);
static void handle_pm_child_exit_signal(SIGNAL_ARGS);
static void handle_pm_reload_request_signal(SIGNAL_ARGS);
@@ -1775,20 +1773,18 @@ ServerLoop(void)
if (events[i].events & WL_SOCKET_ACCEPT)
{
- Port *port;
+ Port port;
- port = ConnCreate(events[i].fd);
- if (port)
- {
- BackendStartup(port);
+ memset(&port, 0, sizeof(port));
+ if (StreamConnection(events[i].fd, &port) == STATUS_OK)
+ BackendStartup(&port);
- /*
- * We no longer need the open socket or port structure in
- * this process
- */
- StreamClose(port->sock);
- ConnFree(port);
- }
+ /*
+ * We no longer need the open socket or port structure in this
+ * process
+ */
+ if (port.sock != PGINVALID_SOCKET)
+ StreamClose(port.sock);
}
}
@@ -2456,50 +2452,6 @@ canAcceptConnections(int backend_type)
return result;
}
-
-/*
- * ConnCreate -- create a local connection data structure
- *
- * Returns NULL on failure, other than out-of-memory which is fatal.
- */
-static Port *
-ConnCreate(int serverFd)
-{
- Port *port;
-
- if (!(port = (Port *) calloc(1, sizeof(Port))))
- {
- ereport(LOG,
- (errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("out of memory")));
- ExitPostmaster(1);
- }
-
- if (StreamConnection(serverFd, port) != STATUS_OK)
- {
- if (port->sock != PGINVALID_SOCKET)
- StreamClose(port->sock);
- ConnFree(port);
- return NULL;
- }
-
- return port;
-}
-
-
-/*
- * ConnFree -- free a local connection data structure
- *
- * Caller has already closed the socket if any, so there's not much
- * to do here.
- */
-static void
-ConnFree(Port *port)
-{
- free(port);
-}
-
-
/*
* ClosePostmasterPorts -- close all the postmaster's open sockets
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index aec1b194424..43f05a35a2d 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -4234,9 +4234,9 @@ PostgresMain(const char *dbname, const char *username)
/*
* If the PostmasterContext is still around, recycle the space; we don't
* need it anymore after InitPostgres completes. Note this does not trash
- * *MyProcPort, because ConnCreate() allocated that space with malloc()
- * ... else we'd need to copy the Port data first. Also, subsidiary data
- * such as the username isn't lost either; see ProcessStartupPacket().
+ * *MyProcPort, because that space is allocated in stack ... else we'd
+ * need to copy the Port data first. Also, subsidiary data such as the
+ * username isn't lost either; see ProcessStartupPacket().
*/
if (PostmasterContext)
{
--
2.39.2
v12-0003-Move-initialization-of-Port-struct-to-child-proc.patchtext/x-patch; charset=UTF-8; name=v12-0003-Move-initialization-of-Port-struct-to-child-proc.patchDownload
From 3f889136a609c57a39813f28414b0a31ad1ecec8 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Mon, 4 Mar 2024 10:48:46 +0200
Subject: [PATCH v12 3/8] Move initialization of Port struct to child process
In postmaster, use a more lightweight ClientSocket struct that
encapsulates just the socket itself and the endpoint addresses that
you get from accept() call. ClientSocket is passed to the child
process, which initializes the bigger Port struct. This makes it more
clear what information postmaster initializes, and what is left to the
child process.
Rename the StreamServerPort, StreamConnection functions to make it
more clear what they do. Remove StreamClose, replacing it with plain
closesocket() calls.
Reviewed-by: Tristan Partin, Andres Freund
Discussion: https://www.postgresql.org/message-id/7a59b073-5b5b-151e-7ed3-8b01ff7ce9ef@iki.fi
---
src/backend/libpq/pqcomm.c | 95 ++++++++--------
src/backend/postmaster/postmaster.c | 165 ++++++++++++++++------------
src/backend/tcop/postgres.c | 5 +-
src/include/libpq/libpq-be.h | 21 +++-
src/include/libpq/libpq.h | 6 +-
src/tools/pgindent/typedefs.list | 1 +
6 files changed, 163 insertions(+), 130 deletions(-)
diff --git a/src/backend/libpq/pqcomm.c b/src/backend/libpq/pqcomm.c
index c606bf34473..0769995eef0 100644
--- a/src/backend/libpq/pqcomm.c
+++ b/src/backend/libpq/pqcomm.c
@@ -29,11 +29,11 @@
* INTERFACE ROUTINES
*
* setup/teardown:
- * StreamServerPort - Open postmaster's server port
- * StreamConnection - Create new connection with client
- * StreamClose - Close a client/backend connection
+ * ListenServerPort - Open postmaster's server port
+ * AcceptClientConnection - Accept new connection with client
+ * InitClientConnection - Initialize a client connection
* TouchSocketFiles - Protect socket files against /tmp cleaners
- * pq_init - initialize libpq at backend startup
+ * pq_init - initialize libpq at backend startup
* socket_comm_reset - reset libpq during error recovery
* socket_close - shutdown libpq at backend exit
*
@@ -304,7 +304,7 @@ socket_close(int code, Datum arg)
/*
- * StreamServerPort -- open a "listening" port to accept connections.
+ * ListenServerPort -- open a "listening" port to accept connections.
*
* family should be AF_UNIX or AF_UNSPEC; portNumber is the port number.
* For AF_UNIX ports, hostName should be NULL and unixSocketDir must be
@@ -319,7 +319,7 @@ socket_close(int code, Datum arg)
* RETURNS: STATUS_OK or STATUS_ERROR
*/
int
-StreamServerPort(int family, const char *hostName, unsigned short portNumber,
+ListenServerPort(int family, const char *hostName, unsigned short portNumber,
const char *unixSocketDir,
pgsocket ListenSockets[], int *NumListenSockets, int MaxListen)
{
@@ -685,8 +685,9 @@ Setup_AF_UNIX(const char *sock_path)
/*
- * StreamConnection -- create a new connection with client using
- * server port. Set port->sock to the FD of the new connection.
+ * AcceptClientConnection -- accept a new connection with client using
+ * server port. Fills *client_sock with the FD and endpoint info
+ * of the new connection.
*
* ASSUME: that this doesn't need to be non-blocking because
* the Postmaster waits for the socket to be ready to accept().
@@ -694,13 +695,13 @@ Setup_AF_UNIX(const char *sock_path)
* RETURNS: STATUS_OK or STATUS_ERROR
*/
int
-StreamConnection(pgsocket server_fd, Port *port)
+AcceptClientConnection(pgsocket server_fd, ClientSocket *client_sock)
{
/* accept connection and fill in the client (remote) address */
- port->raddr.salen = sizeof(port->raddr.addr);
- if ((port->sock = accept(server_fd,
- (struct sockaddr *) &port->raddr.addr,
- &port->raddr.salen)) == PGINVALID_SOCKET)
+ client_sock->raddr.salen = sizeof(client_sock->raddr.addr);
+ if ((client_sock->sock = accept(server_fd,
+ (struct sockaddr *) &client_sock->raddr.addr,
+ &client_sock->raddr.salen)) == PGINVALID_SOCKET)
{
ereport(LOG,
(errcode_for_socket_access(),
@@ -718,10 +719,10 @@ StreamConnection(pgsocket server_fd, Port *port)
}
/* fill in the server (local) address */
- port->laddr.salen = sizeof(port->laddr.addr);
- if (getsockname(port->sock,
- (struct sockaddr *) &port->laddr.addr,
- &port->laddr.salen) < 0)
+ client_sock->laddr.salen = sizeof(client_sock->laddr.addr);
+ if (getsockname(client_sock->sock,
+ (struct sockaddr *) &client_sock->laddr.addr,
+ &client_sock->laddr.salen) < 0)
{
ereport(LOG,
(errmsg("%s() failed: %m", "getsockname")));
@@ -729,7 +730,7 @@ StreamConnection(pgsocket server_fd, Port *port)
}
/* select NODELAY and KEEPALIVE options if it's a TCP connection */
- if (port->laddr.addr.ss_family != AF_UNIX)
+ if (client_sock->laddr.addr.ss_family != AF_UNIX)
{
int on;
#ifdef WIN32
@@ -740,7 +741,7 @@ StreamConnection(pgsocket server_fd, Port *port)
#ifdef TCP_NODELAY
on = 1;
- if (setsockopt(port->sock, IPPROTO_TCP, TCP_NODELAY,
+ if (setsockopt(client_sock->sock, IPPROTO_TCP, TCP_NODELAY,
(char *) &on, sizeof(on)) < 0)
{
ereport(LOG,
@@ -749,7 +750,7 @@ StreamConnection(pgsocket server_fd, Port *port)
}
#endif
on = 1;
- if (setsockopt(port->sock, SOL_SOCKET, SO_KEEPALIVE,
+ if (setsockopt(client_sock->sock, SOL_SOCKET, SO_KEEPALIVE,
(char *) &on, sizeof(on)) < 0)
{
ereport(LOG,
@@ -781,7 +782,7 @@ StreamConnection(pgsocket server_fd, Port *port)
* https://msdn.microsoft.com/en-us/library/bb736549%28v=vs.85%29.aspx
*/
optlen = sizeof(oldopt);
- if (getsockopt(port->sock, SOL_SOCKET, SO_SNDBUF, (char *) &oldopt,
+ if (getsockopt(client_sock->sock, SOL_SOCKET, SO_SNDBUF, (char *) &oldopt,
&optlen) < 0)
{
ereport(LOG,
@@ -791,7 +792,7 @@ StreamConnection(pgsocket server_fd, Port *port)
newopt = PQ_SEND_BUFFER_SIZE * 4;
if (oldopt < newopt)
{
- if (setsockopt(port->sock, SOL_SOCKET, SO_SNDBUF, (char *) &newopt,
+ if (setsockopt(client_sock->sock, SOL_SOCKET, SO_SNDBUF, (char *) &newopt,
sizeof(newopt)) < 0)
{
ereport(LOG,
@@ -800,13 +801,34 @@ StreamConnection(pgsocket server_fd, Port *port)
}
}
#endif
+ }
+ return STATUS_OK;
+}
+
+/*
+ * InitClientConnection -- create a new connection from the given socket.
+ *
+ * This runs in the backend process.
+ */
+Port *
+InitClientConnection(ClientSocket *client_sock)
+{
+ Port *port;
+
+ port = palloc0(sizeof(Port));
+ port->sock = client_sock->sock;
+ port->laddr = client_sock->laddr;
+ port->raddr = client_sock->raddr;
+ /* Apply the current keepalive parameters if it's a TCP connection */
+ if (port->laddr.addr.ss_family != AF_UNIX)
+ {
/*
- * Also apply the current keepalive parameters. If we fail to set a
- * parameter, don't error out, because these aren't universally
- * supported. (Note: you might think we need to reset the GUC
- * variables to 0 in such a case, but it's not necessary because the
- * show hooks for these variables report the truth anyway.)
+ * If we fail to set a parameter, don't error out, because these
+ * aren't universally supported. (Note: you might think we need to
+ * reset the GUC variables to 0 in such a case, but it's not necessary
+ * because the show hooks for these variables report the truth
+ * anyway.)
*/
(void) pq_setkeepalivesidle(tcp_keepalives_idle, port);
(void) pq_setkeepalivesinterval(tcp_keepalives_interval, port);
@@ -814,24 +836,7 @@ StreamConnection(pgsocket server_fd, Port *port)
(void) pq_settcpusertimeout(tcp_user_timeout, port);
}
- return STATUS_OK;
-}
-
-/*
- * StreamClose -- close a client/backend connection
- *
- * NOTE: this is NOT used to terminate a session; it is just used to release
- * the file descriptor in a process that should no longer have the socket
- * open. (For example, the postmaster calls this after passing ownership
- * of the connection to a child process.) It is expected that someone else
- * still has the socket open. So, we only want to close the descriptor,
- * we do NOT want to send anything to the far end.
- */
-void
-StreamClose(pgsocket sock)
-{
- if (closesocket(sock) != 0)
- elog(LOG, "could not close client or listen socket: %m");
+ return port;
}
/*
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 2cf5e228186..446dc738d97 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -430,15 +430,15 @@ typedef enum CAC_state
CAC_TOOMANY,
} CAC_state;
-static void BackendInitialize(Port *port, CAC_state cac);
-static void BackendRun(Port *port) pg_attribute_noreturn();
+static void BackendInitialize(ClientSocket *client_sock, CAC_state cac);
+static void BackendRun(void) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
-static int BackendStartup(Port *port);
+static int BackendStartup(ClientSocket *client_sock);
static int ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done);
static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
static void processCancelRequest(Port *port, void *pkt);
-static void report_fork_failure_to_client(Port *port, int errnum);
+static void report_fork_failure_to_client(ClientSocket *client_sock, int errnum);
static CAC_state canAcceptConnections(int backend_type);
static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
@@ -489,8 +489,8 @@ typedef struct
} win32_deadchild_waitinfo;
#endif /* WIN32 */
-static pid_t backend_forkexec(Port *port, CAC_state cac);
-static pid_t internal_forkexec(int argc, char *argv[], Port *port, BackgroundWorker *worker);
+static pid_t backend_forkexec(ClientSocket *client_sock, CAC_state cac);
+static pid_t internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker);
/* Type for a socket that can be inherited to a client process */
#ifdef WIN32
@@ -509,9 +509,9 @@ typedef int InheritableSocket;
*/
typedef struct
{
- bool has_port;
- Port port;
- InheritableSocket portsocket;
+ bool has_client_sock;
+ ClientSocket client_sock;
+ InheritableSocket inh_sock;
bool has_bgworker;
BackgroundWorker bgworker;
@@ -560,13 +560,13 @@ typedef struct
char pkglib_path[MAXPGPATH];
} BackendParameters;
-static void read_backend_variables(char *id, Port **port, BackgroundWorker **worker);
-static void restore_backend_variables(BackendParameters *param, Port **port, BackgroundWorker **worker);
+static void read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **worker);
+static void restore_backend_variables(BackendParameters *param, ClientSocket **client_sock, BackgroundWorker **worker);
#ifndef WIN32
-static bool save_backend_variables(BackendParameters *param, Port *port, BackgroundWorker *worker);
+static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker);
#else
-static bool save_backend_variables(BackendParameters *param, Port *port, BackgroundWorker *worker,
+static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker,
HANDLE childProcess, pid_t childPid);
#endif
@@ -1222,14 +1222,14 @@ PostmasterMain(int argc, char *argv[])
char *curhost = (char *) lfirst(l);
if (strcmp(curhost, "*") == 0)
- status = StreamServerPort(AF_UNSPEC, NULL,
+ status = ListenServerPort(AF_UNSPEC, NULL,
(unsigned short) PostPortNumber,
NULL,
ListenSockets,
&NumListenSockets,
MAXLISTEN);
else
- status = StreamServerPort(AF_UNSPEC, curhost,
+ status = ListenServerPort(AF_UNSPEC, curhost,
(unsigned short) PostPortNumber,
NULL,
ListenSockets,
@@ -1323,7 +1323,7 @@ PostmasterMain(int argc, char *argv[])
{
char *socketdir = (char *) lfirst(l);
- status = StreamServerPort(AF_UNIX, NULL,
+ status = ListenServerPort(AF_UNIX, NULL,
(unsigned short) PostPortNumber,
socketdir,
ListenSockets,
@@ -1503,7 +1503,10 @@ CloseServerPorts(int status, Datum arg)
* condition if a new postmaster wants to re-use the TCP port number.
*/
for (i = 0; i < NumListenSockets; i++)
- StreamClose(ListenSockets[i]);
+ {
+ if (closesocket(ListenSockets[i]) != 0)
+ elog(LOG, "could not close listen socket: %m");
+ }
NumListenSockets = 0;
/*
@@ -1773,18 +1776,20 @@ ServerLoop(void)
if (events[i].events & WL_SOCKET_ACCEPT)
{
- Port port;
+ ClientSocket s;
- memset(&port, 0, sizeof(port));
- if (StreamConnection(events[i].fd, &port) == STATUS_OK)
- BackendStartup(&port);
+ if (AcceptClientConnection(events[i].fd, &s) == STATUS_OK)
+ BackendStartup(&s);
/*
* We no longer need the open socket or port structure in this
* process
*/
- if (port.sock != PGINVALID_SOCKET)
- StreamClose(port.sock);
+ if (s.sock != PGINVALID_SOCKET)
+ {
+ if (closesocket(s.sock) != 0)
+ elog(LOG, "could not close client socket: %m");
+ }
}
}
@@ -2155,11 +2160,7 @@ retry1:
/*
* Now fetch parameters out of startup packet and save them into the Port
- * structure. All data structures attached to the Port struct must be
- * allocated in TopMemoryContext so that they will remain available in a
- * running backend (even after PostmasterContext is destroyed). We need
- * not worry about leaking this storage on failure, since we aren't in the
- * postmaster process anymore.
+ * structure.
*/
oldcontext = MemoryContextSwitchTo(TopMemoryContext);
@@ -2295,7 +2296,7 @@ retry1:
port->database_name[0] = '\0';
/*
- * Done putting stuff in TopMemoryContext.
+ * Done filling the Port structure
*/
MemoryContextSwitchTo(oldcontext);
@@ -2499,7 +2500,10 @@ ClosePostmasterPorts(bool am_syslogger)
if (ListenSockets)
{
for (int i = 0; i < NumListenSockets; i++)
- StreamClose(ListenSockets[i]);
+ {
+ if (closesocket(ListenSockets[i]) != 0)
+ elog(LOG, "could not close listen socket: %m");
+ }
pfree(ListenSockets);
}
NumListenSockets = 0;
@@ -4049,7 +4053,7 @@ TerminateChildren(int signal)
* Note: if you change this code, also consider StartAutovacuumWorker.
*/
static int
-BackendStartup(Port *port)
+BackendStartup(ClientSocket *client_sock)
{
Backend *bn; /* for backend cleanup */
pid_t pid;
@@ -4100,7 +4104,7 @@ BackendStartup(Port *port)
bn->bgworker_notify = false;
#ifdef EXEC_BACKEND
- pid = backend_forkexec(port, cac);
+ pid = backend_forkexec(client_sock, cac);
#else /* !EXEC_BACKEND */
pid = fork_process();
if (pid == 0) /* child */
@@ -4112,10 +4116,10 @@ BackendStartup(Port *port)
ClosePostmasterPorts(false);
/* Perform additional initialization and collect startup packet */
- BackendInitialize(port, cac);
+ BackendInitialize(client_sock, cac);
/* And run the backend */
- BackendRun(port);
+ BackendRun();
}
#endif /* EXEC_BACKEND */
@@ -4130,14 +4134,14 @@ BackendStartup(Port *port)
errno = save_errno;
ereport(LOG,
(errmsg("could not fork new process for connection: %m")));
- report_fork_failure_to_client(port, save_errno);
+ report_fork_failure_to_client(client_sock, save_errno);
return STATUS_ERROR;
}
/* in parent, successful fork */
ereport(DEBUG2,
(errmsg_internal("forked new backend, pid=%d socket=%d",
- (int) pid, (int) port->sock)));
+ (int) pid, (int) client_sock->sock)));
/*
* Everything's been successful, it's safe to add this backend to our list
@@ -4164,7 +4168,7 @@ BackendStartup(Port *port)
* it's not up and running.
*/
static void
-report_fork_failure_to_client(Port *port, int errnum)
+report_fork_failure_to_client(ClientSocket *client_sock, int errnum)
{
char buffer[1000];
int rc;
@@ -4175,13 +4179,13 @@ report_fork_failure_to_client(Port *port, int errnum)
strerror(errnum));
/* Set port to non-blocking. Don't do send() if this fails */
- if (!pg_set_noblock(port->sock))
+ if (!pg_set_noblock(client_sock->sock))
return;
/* We'll retry after EINTR, but ignore all other failures */
do
{
- rc = send(port->sock, buffer, strlen(buffer) + 1, 0);
+ rc = send(client_sock->sock, buffer, strlen(buffer) + 1, 0);
} while (rc < 0 && errno == EINTR);
}
@@ -4199,18 +4203,30 @@ report_fork_failure_to_client(Port *port, int errnum)
* but have not yet set up most of our local pointers to shmem structures.
*/
static void
-BackendInitialize(Port *port, CAC_state cac)
+BackendInitialize(ClientSocket *client_sock, CAC_state cac)
{
int status;
int ret;
+ Port *port;
char remote_host[NI_MAXHOST];
char remote_port[NI_MAXSERV];
StringInfoData ps_data;
+ MemoryContext oldcontext;
- /* Save port etc. for ps status */
+ /*
+ * Create the Port structure.
+ *
+ * The Port structure and all data structures attached to it are allocated
+ * in TopMemoryContext, so that they survive into PostgresMain execution.
+ * We need not worry about leaking this storage on failure, since we
+ * aren't in the postmaster process anymore.
+ */
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
+ port = InitClientConnection(client_sock);
MyProcPort = port;
+ MemoryContextSwitchTo(oldcontext);
- /* Tell fd.c about the long-lived FD associated with the port */
+ /* Tell fd.c about the long-lived FD associated with the client_sock */
ReserveExternalFD();
/*
@@ -4269,8 +4285,9 @@ BackendInitialize(Port *port, CAC_state cac)
* Save remote_host and remote_port in port structure (after this, they
* will appear in log_line_prefix data for log messages).
*/
- port->remote_host = strdup(remote_host);
- port->remote_port = strdup(remote_port);
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
+ port->remote_host = pstrdup(remote_host);
+ port->remote_port = pstrdup(remote_port);
/* And now we can issue the Log_connections message, if wanted */
if (Log_connections)
@@ -4301,7 +4318,10 @@ BackendInitialize(Port *port, CAC_state cac)
ret == 0 &&
strspn(remote_host, "0123456789.") < strlen(remote_host) &&
strspn(remote_host, "0123456789ABCDEFabcdef:") < strlen(remote_host))
- port->remote_hostname = strdup(remote_host);
+ {
+ port->remote_hostname = pstrdup(remote_host);
+ }
+ MemoryContextSwitchTo(oldcontext);
/*
* Ready to begin client interaction. We will give up and _exit(1) after
@@ -4425,7 +4445,7 @@ BackendInitialize(Port *port, CAC_state cac)
* Doesn't return at all.
*/
static void
-BackendRun(Port *port)
+BackendRun(void)
{
/*
* Create a per-backend PGPROC struct in shared memory. We must do this
@@ -4439,7 +4459,7 @@ BackendRun(Port *port)
*/
MemoryContextSwitchTo(TopMemoryContext);
- PostgresMain(port->database_name, port->user_name);
+ PostgresMain(MyProcPort->database_name, MyProcPort->user_name);
}
@@ -4473,7 +4493,7 @@ postmaster_forkexec(int argc, char *argv[])
* returns the pid of the fork/exec'd process, or -1 on failure
*/
static pid_t
-backend_forkexec(Port *port, CAC_state cac)
+backend_forkexec(ClientSocket *client_sock, CAC_state cac)
{
char *av[5];
int ac = 0;
@@ -4489,7 +4509,7 @@ backend_forkexec(Port *port, CAC_state cac)
av[ac] = NULL;
Assert(ac < lengthof(av));
- return internal_forkexec(ac, av, port, NULL);
+ return internal_forkexec(ac, av, client_sock, NULL);
}
#ifndef WIN32
@@ -4501,7 +4521,7 @@ backend_forkexec(Port *port, CAC_state cac)
* - fork():s, and then exec():s the child process
*/
static pid_t
-internal_forkexec(int argc, char *argv[], Port *port, BackgroundWorker *worker)
+internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker)
{
static unsigned long tmpBackendFileNum = 0;
pid_t pid;
@@ -4517,7 +4537,7 @@ internal_forkexec(int argc, char *argv[], Port *port, BackgroundWorker *worker)
*/
memset(¶m, 0, sizeof(BackendParameters));
- if (!save_backend_variables(¶m, port, worker))
+ if (!save_backend_variables(¶m, client_sock, worker))
return -1; /* log made by save_backend_variables */
/* Calculate name for temp file */
@@ -4815,7 +4835,7 @@ retry:
void
SubPostmasterMain(int argc, char *argv[])
{
- Port *port;
+ ClientSocket *client_sock;
BackgroundWorker *worker;
/* In EXEC_BACKEND case we will not have inherited these settings */
@@ -4830,7 +4850,7 @@ SubPostmasterMain(int argc, char *argv[])
elog(FATAL, "invalid subpostmaster invocation");
/* Read in the variables file */
- read_backend_variables(argv[2], &port, &worker);
+ read_backend_variables(argv[2], &client_sock, &worker);
/* Close the postmaster's sockets (as soon as we know them) */
ClosePostmasterPorts(strcmp(argv[1], "--forklog") == 0);
@@ -4929,13 +4949,14 @@ SubPostmasterMain(int argc, char *argv[])
* PGPROC slots, we have already initialized libpq and are able to
* report the error to the client.
*/
- BackendInitialize(port, cac);
+ BackendInitialize(client_sock, cac);
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
/* And run the backend */
- BackendRun(port); /* does not return */
+ BackendRun(); /* does not return */
+
}
if (strcmp(argv[1], "--forkaux") == 0)
{
@@ -6055,24 +6076,24 @@ static void read_inheritable_socket(SOCKET *dest, InheritableSocket *src);
/* Save critical backend variables into the BackendParameters struct */
#ifndef WIN32
static bool
-save_backend_variables(BackendParameters *param, Port *port, BackgroundWorker *worker)
+save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker)
#else
static bool
-save_backend_variables(BackendParameters *param, Port *port, BackgroundWorker *worker,
+save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker,
HANDLE childProcess, pid_t childPid)
#endif
{
- if (port)
+ if (client_sock)
{
- memcpy(¶m->port, port, sizeof(Port));
- if (!write_inheritable_socket(¶m->portsocket, port->sock, childPid))
+ memcpy(¶m->client_sock, client_sock, sizeof(ClientSocket));
+ if (!write_inheritable_socket(¶m->inh_sock, client_sock->sock, childPid))
return false;
- param->has_port = true;
+ param->has_client_sock = true;
}
else
{
- memset(¶m->port, 0, sizeof(Port));
- param->has_port = false;
+ memset(¶m->client_sock, 0, sizeof(ClientSocket));
+ param->has_client_sock = false;
}
if (worker)
@@ -6239,7 +6260,7 @@ read_inheritable_socket(SOCKET *dest, InheritableSocket *src)
#endif
static void
-read_backend_variables(char *id, Port **port, BackgroundWorker **worker)
+read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **worker)
{
BackendParameters param;
@@ -6306,21 +6327,21 @@ read_backend_variables(char *id, Port **port, BackgroundWorker **worker)
}
#endif
- restore_backend_variables(¶m, port, worker);
+ restore_backend_variables(¶m, client_sock, worker);
}
/* Restore critical backend variables from the BackendParameters struct */
static void
-restore_backend_variables(BackendParameters *param, Port **port, BackgroundWorker **worker)
+restore_backend_variables(BackendParameters *param, ClientSocket **client_sock, BackgroundWorker **worker)
{
- if (param->has_port)
+ if (param->has_client_sock)
{
- *port = (Port *) MemoryContextAlloc(TopMemoryContext, sizeof(Port));
- memcpy(*port, ¶m->port, sizeof(Port));
- read_inheritable_socket(&(*port)->sock, ¶m->portsocket);
+ *client_sock = (ClientSocket *) MemoryContextAlloc(TopMemoryContext, sizeof(ClientSocket));
+ memcpy(*client_sock, ¶m->client_sock, sizeof(ClientSocket));
+ read_inheritable_socket(&(*client_sock)->sock, ¶m->inh_sock);
}
else
- *port = NULL;
+ *client_sock = NULL;
if (param->has_bgworker)
{
@@ -6386,7 +6407,7 @@ restore_backend_variables(BackendParameters *param, Port **port, BackgroundWorke
/*
* We need to restore fd.c's counts of externally-opened FDs; to avoid
* confusion, be sure to do this after restoring max_safe_fds. (Note:
- * BackendInitialize will handle this for port->sock.)
+ * BackendInitialize will handle this for (*client_sock)->sock.)
*/
#ifndef WIN32
if (postmaster_alive_fds[0] >= 0)
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 43f05a35a2d..6b7903314ab 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -4233,10 +4233,7 @@ PostgresMain(const char *dbname, const char *username)
/*
* If the PostmasterContext is still around, recycle the space; we don't
- * need it anymore after InitPostgres completes. Note this does not trash
- * *MyProcPort, because that space is allocated in stack ... else we'd
- * need to copy the Port data first. Also, subsidiary data such as the
- * username isn't lost either; see ProcessStartupPacket().
+ * need it anymore after InitPostgres completes.
*/
if (PostmasterContext)
{
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 59c2a1d874f..c35c23001eb 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -110,12 +110,9 @@ typedef struct ClientConnectionInfo
} ClientConnectionInfo;
/*
- * This is used by the postmaster in its communication with frontends. It
- * contains all state information needed during this communication before the
- * backend is run. The Port structure is kept in malloc'd memory and is
- * still available when a backend is running (see MyProcPort). The data
- * it points to must also be malloc'd, or else palloc'd in TopMemoryContext,
- * so that it survives into PostgresMain execution!
+ * The Port structure holds state information about a client connection in a
+ * backend process. It is available in the global variable MyProcPort. The
+ * struct and all the data it points are kept in TopMemoryContext.
*
* remote_hostname is set if we did a successful reverse lookup of the
* client's IP address during connection setup.
@@ -217,6 +214,18 @@ typedef struct Port
#endif
} Port;
+/*
+ * ClientSocket holds a socket for an accepted connection, along with the
+ * information about the endpoints. This is passed from postmaster to
+ * the backend process.
+ */
+typedef struct ClientSocket
+{
+ pgsocket sock; /* File descriptor */
+ SockAddr laddr; /* local addr (postmaster) */
+ SockAddr raddr; /* remote addr (client) */
+} ClientSocket;
+
#ifdef USE_SSL
/*
* Hardcoded DH parameters, used in ephemeral DH keying. (See also
diff --git a/src/include/libpq/libpq.h b/src/include/libpq/libpq.h
index 6171a0d17a5..45c219844a4 100644
--- a/src/include/libpq/libpq.h
+++ b/src/include/libpq/libpq.h
@@ -64,11 +64,11 @@ extern PGDLLIMPORT WaitEventSet *FeBeWaitSet;
#define FeBeWaitSetLatchPos 1
#define FeBeWaitSetNEvents 3
-extern int StreamServerPort(int family, const char *hostName,
+extern int ListenServerPort(int family, const char *hostName,
unsigned short portNumber, const char *unixSocketDir,
pgsocket ListenSocket[], int *NumListenSockets, int MaxListen);
-extern int StreamConnection(pgsocket server_fd, Port *port);
-extern void StreamClose(pgsocket sock);
+extern int AcceptClientConnection(pgsocket server_fd, ClientSocket *client_sock);
+extern Port *InitClientConnection(ClientSocket *client_sock);
extern void TouchSocketFiles(void);
extern void RemoveSocketFiles(void);
extern void pq_init(void);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 95ae7845d86..da465602ea1 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -388,6 +388,7 @@ ClientCertMode
ClientCertName
ClientConnectionInfo
ClientData
+ClientSocket
ClonePtrType
ClosePortalStmt
ClosePtrType
--
2.39.2
v12-0004-Extract-registration-of-Win32-deadchild-callback.patchtext/x-patch; charset=UTF-8; name=v12-0004-Extract-registration-of-Win32-deadchild-callback.patchDownload
From 5eb58dda79879dc66f31e808d5cebdafae6f4e43 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Fri, 8 Dec 2023 10:31:49 +0200
Subject: [PATCH v12 4/8] Extract registration of Win32 deadchild callback to
separate function
The next commits will move the internal_forkexec() function to
different source file, but it makes sense to keep all the code related
to the win32 waitpid() emulation in postmaster.c. Split it off to a
separate function now, to make the commit that moves
internal_forkexec() more mechanical.
---
src/backend/postmaster/postmaster.c | 49 ++++++++++++++++++-----------
1 file changed, 30 insertions(+), 19 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 446dc738d97..f79bb3a526b 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -478,6 +478,7 @@ static void MaybeStartSlotSyncWorker(void);
static pid_t waitpid(pid_t pid, int *exitstatus, int options);
static void WINAPI pgwin32_deadchild_callback(PVOID lpParameter, BOOLEAN TimerOrWaitFired);
+static void pgwin32_register_deadchild_callback(HANDLE procHandle, DWORD procId);
static HANDLE win32ChildQueue;
@@ -4793,26 +4794,10 @@ retry:
return -1;
}
- /*
- * Queue a waiter to signal when this child dies. The wait will be handled
- * automatically by an operating system thread pool. The memory will be
- * freed by a later call to waitpid().
- */
- childinfo = palloc(sizeof(win32_deadchild_waitinfo));
- childinfo->procHandle = pi.hProcess;
- childinfo->procId = pi.dwProcessId;
-
- if (!RegisterWaitForSingleObject(&childinfo->waitHandle,
- pi.hProcess,
- pgwin32_deadchild_callback,
- childinfo,
- INFINITE,
- WT_EXECUTEONLYONCE | WT_EXECUTEINWAITTHREAD))
- ereport(FATAL,
- (errmsg_internal("could not register process for wait: error code %lu",
- GetLastError())));
+ /* Set up notification when the child process dies */
+ pgwin32_register_deadchild_callback(pi.hProcess, pi.dwProcessId);
- /* Don't close pi.hProcess here - waitpid() needs access to it */
+ /* Don't close pi.hProcess, it's owned by the deadchild callback now */
CloseHandle(pi.hThread);
@@ -6539,6 +6524,32 @@ pgwin32_deadchild_callback(PVOID lpParameter, BOOLEAN TimerOrWaitFired)
/* Queue SIGCHLD signal. */
pg_queue_signal(SIGCHLD);
}
+
+/*
+ * Queue a waiter to signal when this child dies. The wait will be handled
+ * automatically by an operating system thread pool. The memory and the
+ * process handle will be freed by a later call to waitpid().
+ */
+static void
+pgwin32_register_deadchild_callback(HANDLE procHandle, DWORD procId)
+{
+ win32_deadchild_waitinfo *childinfo;
+
+ childinfo = palloc(sizeof(win32_deadchild_waitinfo));
+ childinfo->procHandle = procHandle;
+ childinfo->procId = procId;
+
+ if (!RegisterWaitForSingleObject(&childinfo->waitHandle,
+ procHandle,
+ pgwin32_deadchild_callback,
+ childinfo,
+ INFINITE,
+ WT_EXECUTEONLYONCE | WT_EXECUTEINWAITTHREAD))
+ ereport(FATAL,
+ (errmsg_internal("could not register process for wait: error code %lu",
+ GetLastError())));
+}
+
#endif /* WIN32 */
/*
--
2.39.2
v12-0005-Move-some-functions-from-postmaster.c-to-new-sou.patchtext/x-patch; charset=UTF-8; name=v12-0005-Move-some-functions-from-postmaster.c-to-new-sou.patchDownload
From f797710f29099e286ec5486902224d83fe7b0b04 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Fri, 8 Dec 2023 10:31:57 +0200
Subject: [PATCH v12 5/8] Move some functions from postmaster.c to new source
file
This just moves the functions, with no other changes, to make the next
commits smaller and easier to review. The moved functions are related
to forking and execing a new backend in EXEC_BACKEND mode.
One noteworthy change is that the code to register win32 deadchild
callback is extracted to a separate function, so that it can be called
from the new file without exposing pgwin32_deadchild_callback().
---
src/backend/postmaster/Makefile | 1 +
src/backend/postmaster/launch_backend.c | 813 +++++++++++++++++++++
src/backend/postmaster/meson.build | 1 +
src/backend/postmaster/postmaster.c | 759 +------------------
src/backend/replication/logical/launcher.c | 1 -
src/include/postmaster/postmaster.h | 5 +
6 files changed, 830 insertions(+), 750 deletions(-)
create mode 100644 src/backend/postmaster/launch_backend.c
diff --git a/src/backend/postmaster/Makefile b/src/backend/postmaster/Makefile
index 367a46c6177..db08543d195 100644
--- a/src/backend/postmaster/Makefile
+++ b/src/backend/postmaster/Makefile
@@ -20,6 +20,7 @@ OBJS = \
checkpointer.o \
fork_process.o \
interrupt.o \
+ launch_backend.o \
pgarch.o \
postmaster.o \
startup.o \
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
new file mode 100644
index 00000000000..65f141a9f5a
--- /dev/null
+++ b/src/backend/postmaster/launch_backend.c
@@ -0,0 +1,813 @@
+/*-------------------------------------------------------------------------
+ *
+ * launch_backend.c
+ * Functions for launching backends and other postmaster child
+ * processes.
+ *
+ * On Unix systems, a new child process is launched with fork(). It inherits
+ * all the global variables and data structures that had been initialized in
+ * the postmaster. After forking, the child process closes the file
+ * descriptors that are not needed in the child process, and sets up the
+ * mechanism to detect death of the parent postmaster process, etc. After
+ * that, it calls the right Main function depending on the kind of child
+ * process.
+ *
+ * In EXEC_BACKEND mode, which is used on Windows but can be enabled on other
+ * platforms for testing, the child process is launched by fork() + exec() (or
+ * CreateProcess() on Windows). It does not inherit the state from the
+ * postmaster, so it needs to re-attach to the shared memory, re-initialize
+ * global variables, re-load the config file etc.
+ *
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/postmaster/launch_backend.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include <unistd.h>
+
+#include "access/xlog.h"
+#include "common/file_utils.h"
+#include "libpq/libpq-be.h"
+#include "libpq/pqsignal.h"
+#include "miscadmin.h"
+#include "nodes/queryjumble.h"
+#include "port.h"
+#include "postmaster/autovacuum.h"
+#include "postmaster/auxprocess.h"
+#include "postmaster/bgworker_internals.h"
+#include "postmaster/bgwriter.h"
+#include "postmaster/fork_process.h"
+#include "postmaster/pgarch.h"
+#include "postmaster/postmaster.h"
+#include "postmaster/startup.h"
+#include "postmaster/syslogger.h"
+#include "postmaster/walwriter.h"
+#include "replication/walreceiver.h"
+#include "storage/fd.h"
+#include "storage/ipc.h"
+#include "storage/pg_shmem.h"
+#include "storage/pmsignal.h"
+#include "storage/proc.h"
+#include "tcop/tcopprot.h"
+#include "utils/builtins.h"
+#include "utils/datetime.h"
+#include "utils/guc.h"
+#include "utils/memutils.h"
+
+#include "utils/timestamp.h"
+
+#ifdef EXEC_BACKEND
+#include "storage/spin.h"
+#endif
+
+
+#ifdef EXEC_BACKEND
+
+/* Type for a socket that can be inherited to a client process */
+#ifdef WIN32
+typedef struct
+{
+ SOCKET origsocket; /* Original socket value, or PGINVALID_SOCKET
+ * if not a socket */
+ WSAPROTOCOL_INFO wsainfo;
+} InheritableSocket;
+#else
+typedef int InheritableSocket;
+#endif
+
+/*
+ * Structure contains all variables passed to exec:ed backends
+ */
+typedef struct
+{
+ bool has_client_sock;
+ ClientSocket client_sock;
+ InheritableSocket inh_sock;
+
+ bool has_bgworker;
+ BackgroundWorker bgworker;
+
+ char DataDir[MAXPGPATH];
+ int32 MyCancelKey;
+ int MyPMChildSlot;
+#ifndef WIN32
+ unsigned long UsedShmemSegID;
+#else
+ void *ShmemProtectiveRegion;
+ HANDLE UsedShmemSegID;
+#endif
+ void *UsedShmemSegAddr;
+ slock_t *ShmemLock;
+ struct bkend *ShmemBackendArray;
+#ifndef HAVE_SPINLOCKS
+ PGSemaphore *SpinlockSemaArray;
+#endif
+ int NamedLWLockTrancheRequests;
+ NamedLWLockTranche *NamedLWLockTrancheArray;
+ LWLockPadded *MainLWLockArray;
+ slock_t *ProcStructLock;
+ PROC_HDR *ProcGlobal;
+ PGPROC *AuxiliaryProcs;
+ PGPROC *PreparedXactProcs;
+ PMSignalData *PMSignalState;
+ pid_t PostmasterPid;
+ TimestampTz PgStartTime;
+ TimestampTz PgReloadTime;
+ pg_time_t first_syslogger_file_time;
+ bool redirection_done;
+ bool IsBinaryUpgrade;
+ bool query_id_enabled;
+ int max_safe_fds;
+ int MaxBackends;
+#ifdef WIN32
+ HANDLE PostmasterHandle;
+ HANDLE initial_signal_pipe;
+ HANDLE syslogPipe[2];
+#else
+ int postmaster_alive_fds[2];
+ int syslogPipe[2];
+#endif
+ char my_exec_path[MAXPGPATH];
+ char pkglib_path[MAXPGPATH];
+} BackendParameters;
+
+#define SizeOfBackendParameters(startup_data_len) (offsetof(BackendParameters, startup_data) + startup_data_len)
+
+void read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **worker);
+static void restore_backend_variables(BackendParameters *param, ClientSocket **client_sock, BackgroundWorker **worker);
+
+#ifndef WIN32
+static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker);
+#else
+static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker,
+ HANDLE childProcess, pid_t childPid);
+#endif
+
+pid_t internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker);
+
+#ifndef WIN32
+
+/*
+ * internal_forkexec non-win32 implementation
+ *
+ * - writes out backend variables to the parameter file
+ * - fork():s, and then exec():s the child process
+ */
+pid_t
+internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker)
+{
+ static unsigned long tmpBackendFileNum = 0;
+ pid_t pid;
+ char tmpfilename[MAXPGPATH];
+ BackendParameters param;
+ FILE *fp;
+
+ /*
+ * Make sure padding bytes are initialized, to prevent Valgrind from
+ * complaining about writing uninitialized bytes to the file. This isn't
+ * performance critical, and the win32 implementation initializes the
+ * padding bytes to zeros, so do it even when not using Valgrind.
+ */
+ memset(¶m, 0, sizeof(BackendParameters));
+
+ if (!save_backend_variables(¶m, client_sock, worker))
+ return -1; /* log made by save_backend_variables */
+
+ /* Calculate name for temp file */
+ snprintf(tmpfilename, MAXPGPATH, "%s/%s.backend_var.%d.%lu",
+ PG_TEMP_FILES_DIR, PG_TEMP_FILE_PREFIX,
+ MyProcPid, ++tmpBackendFileNum);
+
+ /* Open file */
+ fp = AllocateFile(tmpfilename, PG_BINARY_W);
+ if (!fp)
+ {
+ /*
+ * As in OpenTemporaryFileInTablespace, try to make the temp-file
+ * directory, ignoring errors.
+ */
+ (void) MakePGDirectory(PG_TEMP_FILES_DIR);
+
+ fp = AllocateFile(tmpfilename, PG_BINARY_W);
+ if (!fp)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ tmpfilename)));
+ return -1;
+ }
+ }
+
+ if (fwrite(¶m, sizeof(param), 1, fp) != 1)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not write to file \"%s\": %m", tmpfilename)));
+ FreeFile(fp);
+ return -1;
+ }
+
+ /* Release file */
+ if (FreeFile(fp))
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not write to file \"%s\": %m", tmpfilename)));
+ return -1;
+ }
+
+ /* Make sure caller set up argv properly */
+ Assert(argc >= 3);
+ Assert(argv[argc] == NULL);
+ Assert(strncmp(argv[1], "--fork", 6) == 0);
+ Assert(argv[2] == NULL);
+
+ /* Insert temp file name after --fork argument */
+ argv[2] = tmpfilename;
+
+ /* Fire off execv in child */
+ if ((pid = fork_process()) == 0)
+ {
+ if (execv(postgres_exec_path, argv) < 0)
+ {
+ ereport(LOG,
+ (errmsg("could not execute server process \"%s\": %m",
+ postgres_exec_path)));
+ /* We're already in the child process here, can't return */
+ exit(1);
+ }
+ }
+
+ return pid; /* Parent returns pid, or -1 on fork failure */
+}
+#else /* WIN32 */
+
+/*
+ * internal_forkexec win32 implementation
+ *
+ * - starts backend using CreateProcess(), in suspended state
+ * - writes out backend variables to the parameter file
+ * - during this, duplicates handles and sockets required for
+ * inheritance into the new process
+ * - resumes execution of the new process once the backend parameter
+ * file is complete.
+ */
+pid_t
+internal_forkexec(int argc, char *argv[], Port *port, BackgroundWorker *worker)
+{
+ int retry_count = 0;
+ STARTUPINFO si;
+ PROCESS_INFORMATION pi;
+ int i;
+ int j;
+ char cmdLine[MAXPGPATH * 2];
+ HANDLE paramHandle;
+ BackendParameters *param;
+ SECURITY_ATTRIBUTES sa;
+ char paramHandleStr[32];
+ win32_deadchild_waitinfo *childinfo;
+
+ /* Make sure caller set up argv properly */
+ Assert(argc >= 3);
+ Assert(argv[argc] == NULL);
+ Assert(strncmp(argv[1], "--fork", 6) == 0);
+ Assert(argv[2] == NULL);
+
+ /* Resume here if we need to retry */
+retry:
+
+ /* Set up shared memory for parameter passing */
+ ZeroMemory(&sa, sizeof(sa));
+ sa.nLength = sizeof(sa);
+ sa.bInheritHandle = TRUE;
+ paramHandle = CreateFileMapping(INVALID_HANDLE_VALUE,
+ &sa,
+ PAGE_READWRITE,
+ 0,
+ sizeof(BackendParameters),
+ NULL);
+ if (paramHandle == INVALID_HANDLE_VALUE)
+ {
+ ereport(LOG,
+ (errmsg("could not create backend parameter file mapping: error code %lu",
+ GetLastError())));
+ return -1;
+ }
+
+ param = MapViewOfFile(paramHandle, FILE_MAP_WRITE, 0, 0, sizeof(BackendParameters));
+ if (!param)
+ {
+ ereport(LOG,
+ (errmsg("could not map backend parameter memory: error code %lu",
+ GetLastError())));
+ CloseHandle(paramHandle);
+ return -1;
+ }
+
+ /* Insert temp file name after --fork argument */
+#ifdef _WIN64
+ sprintf(paramHandleStr, "%llu", (LONG_PTR) paramHandle);
+#else
+ sprintf(paramHandleStr, "%lu", (DWORD) paramHandle);
+#endif
+ argv[2] = paramHandleStr;
+
+ /* Format the cmd line */
+ cmdLine[sizeof(cmdLine) - 1] = '\0';
+ cmdLine[sizeof(cmdLine) - 2] = '\0';
+ snprintf(cmdLine, sizeof(cmdLine) - 1, "\"%s\"", postgres_exec_path);
+ i = 0;
+ while (argv[++i] != NULL)
+ {
+ j = strlen(cmdLine);
+ snprintf(cmdLine + j, sizeof(cmdLine) - 1 - j, " \"%s\"", argv[i]);
+ }
+ if (cmdLine[sizeof(cmdLine) - 2] != '\0')
+ {
+ ereport(LOG,
+ (errmsg("subprocess command line too long")));
+ UnmapViewOfFile(param);
+ CloseHandle(paramHandle);
+ return -1;
+ }
+
+ memset(&pi, 0, sizeof(pi));
+ memset(&si, 0, sizeof(si));
+ si.cb = sizeof(si);
+
+ /*
+ * Create the subprocess in a suspended state. This will be resumed later,
+ * once we have written out the parameter file.
+ */
+ if (!CreateProcess(NULL, cmdLine, NULL, NULL, TRUE, CREATE_SUSPENDED,
+ NULL, NULL, &si, &pi))
+ {
+ ereport(LOG,
+ (errmsg("CreateProcess() call failed: %m (error code %lu)",
+ GetLastError())));
+ UnmapViewOfFile(param);
+ CloseHandle(paramHandle);
+ return -1;
+ }
+
+ if (!save_backend_variables(param, port, worker, pi.hProcess, pi.dwProcessId))
+ {
+ /*
+ * log made by save_backend_variables, but we have to clean up the
+ * mess with the half-started process
+ */
+ if (!TerminateProcess(pi.hProcess, 255))
+ ereport(LOG,
+ (errmsg_internal("could not terminate unstarted process: error code %lu",
+ GetLastError())));
+ CloseHandle(pi.hProcess);
+ CloseHandle(pi.hThread);
+ UnmapViewOfFile(param);
+ CloseHandle(paramHandle);
+ return -1; /* log made by save_backend_variables */
+ }
+
+ /* Drop the parameter shared memory that is now inherited to the backend */
+ if (!UnmapViewOfFile(param))
+ ereport(LOG,
+ (errmsg("could not unmap view of backend parameter file: error code %lu",
+ GetLastError())));
+ if (!CloseHandle(paramHandle))
+ ereport(LOG,
+ (errmsg("could not close handle to backend parameter file: error code %lu",
+ GetLastError())));
+
+ /*
+ * Reserve the memory region used by our main shared memory segment before
+ * we resume the child process. Normally this should succeed, but if ASLR
+ * is active then it might sometimes fail due to the stack or heap having
+ * gotten mapped into that range. In that case, just terminate the
+ * process and retry.
+ */
+ if (!pgwin32_ReserveSharedMemoryRegion(pi.hProcess))
+ {
+ /* pgwin32_ReserveSharedMemoryRegion already made a log entry */
+ if (!TerminateProcess(pi.hProcess, 255))
+ ereport(LOG,
+ (errmsg_internal("could not terminate process that failed to reserve memory: error code %lu",
+ GetLastError())));
+ CloseHandle(pi.hProcess);
+ CloseHandle(pi.hThread);
+ if (++retry_count < 100)
+ goto retry;
+ ereport(LOG,
+ (errmsg("giving up after too many tries to reserve shared memory"),
+ errhint("This might be caused by ASLR or antivirus software.")));
+ return -1;
+ }
+
+ /*
+ * Now that the backend variables are written out, we start the child
+ * thread so it can start initializing while we set up the rest of the
+ * parent state.
+ */
+ if (ResumeThread(pi.hThread) == -1)
+ {
+ if (!TerminateProcess(pi.hProcess, 255))
+ {
+ ereport(LOG,
+ (errmsg_internal("could not terminate unstartable process: error code %lu",
+ GetLastError())));
+ CloseHandle(pi.hProcess);
+ CloseHandle(pi.hThread);
+ return -1;
+ }
+ CloseHandle(pi.hProcess);
+ CloseHandle(pi.hThread);
+ ereport(LOG,
+ (errmsg_internal("could not resume thread of unstarted process: error code %lu",
+ GetLastError())));
+ return -1;
+ }
+
+ /* Set up notification when the child process dies */
+ pgwin32_register_deadchild_callback(pi.hProcess, pi.dwProcessId);
+
+ /* Don't close pi.hProcess, it's owned by the deadchild callback now */
+
+ CloseHandle(pi.hThread);
+
+ return pi.dwProcessId;
+}
+#endif /* WIN32 */
+
+/*
+ * The following need to be available to the save/restore_backend_variables
+ * functions. They are marked NON_EXEC_STATIC in their home modules.
+ */
+extern slock_t *ShmemLock;
+extern slock_t *ProcStructLock;
+extern PGPROC *AuxiliaryProcs;
+extern PMSignalData *PMSignalState;
+extern pg_time_t first_syslogger_file_time;
+extern struct bkend *ShmemBackendArray;
+extern bool redirection_done;
+
+/* Save critical backend variables into the BackendParameters struct */
+#ifndef WIN32
+#define write_inheritable_socket(dest, src, childpid) ((*(dest) = (src)), true)
+#define read_inheritable_socket(dest, src) (*(dest) = *(src))
+#else
+static bool write_duplicated_handle(HANDLE *dest, HANDLE src, HANDLE child);
+static bool write_inheritable_socket(InheritableSocket *dest, SOCKET src,
+ pid_t childPid);
+static void read_inheritable_socket(SOCKET *dest, InheritableSocket *src);
+#endif
+
+#ifndef WIN32
+static bool
+save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker)
+#else
+static bool
+save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker,
+ HANDLE childProcess, pid_t childPid)
+#endif
+{
+ if (client_sock)
+ {
+ memcpy(¶m->client_sock, client_sock, sizeof(ClientSocket));
+ if (!write_inheritable_socket(¶m->inh_sock, client_sock->sock, childPid))
+ return false;
+ param->has_client_sock = true;
+ }
+ else
+ {
+ memset(¶m->client_sock, 0, sizeof(ClientSocket));
+ param->has_client_sock = false;
+ }
+
+ if (worker)
+ {
+ memcpy(¶m->bgworker, worker, sizeof(BackgroundWorker));
+ param->has_bgworker = true;
+ }
+ else
+ {
+ memset(¶m->bgworker, 0, sizeof(BackgroundWorker));
+ param->has_bgworker = false;
+ }
+
+ strlcpy(param->DataDir, DataDir, MAXPGPATH);
+
+ param->MyCancelKey = MyCancelKey;
+ param->MyPMChildSlot = MyPMChildSlot;
+
+#ifdef WIN32
+ param->ShmemProtectiveRegion = ShmemProtectiveRegion;
+#endif
+ param->UsedShmemSegID = UsedShmemSegID;
+ param->UsedShmemSegAddr = UsedShmemSegAddr;
+
+ param->ShmemLock = ShmemLock;
+ param->ShmemBackendArray = ShmemBackendArray;
+
+#ifndef HAVE_SPINLOCKS
+ param->SpinlockSemaArray = SpinlockSemaArray;
+#endif
+ param->NamedLWLockTrancheRequests = NamedLWLockTrancheRequests;
+ param->NamedLWLockTrancheArray = NamedLWLockTrancheArray;
+ param->MainLWLockArray = MainLWLockArray;
+ param->ProcStructLock = ProcStructLock;
+ param->ProcGlobal = ProcGlobal;
+ param->AuxiliaryProcs = AuxiliaryProcs;
+ param->PreparedXactProcs = PreparedXactProcs;
+ param->PMSignalState = PMSignalState;
+
+ param->PostmasterPid = PostmasterPid;
+ param->PgStartTime = PgStartTime;
+ param->PgReloadTime = PgReloadTime;
+ param->first_syslogger_file_time = first_syslogger_file_time;
+
+ param->redirection_done = redirection_done;
+ param->IsBinaryUpgrade = IsBinaryUpgrade;
+ param->query_id_enabled = query_id_enabled;
+ param->max_safe_fds = max_safe_fds;
+
+ param->MaxBackends = MaxBackends;
+
+#ifdef WIN32
+ param->PostmasterHandle = PostmasterHandle;
+ if (!write_duplicated_handle(¶m->initial_signal_pipe,
+ pgwin32_create_signal_listener(childPid),
+ childProcess))
+ return false;
+#else
+ memcpy(¶m->postmaster_alive_fds, &postmaster_alive_fds,
+ sizeof(postmaster_alive_fds));
+#endif
+
+ memcpy(¶m->syslogPipe, &syslogPipe, sizeof(syslogPipe));
+
+ strlcpy(param->my_exec_path, my_exec_path, MAXPGPATH);
+
+ strlcpy(param->pkglib_path, pkglib_path, MAXPGPATH);
+
+ return true;
+}
+
+#ifdef WIN32
+/*
+ * Duplicate a handle for usage in a child process, and write the child
+ * process instance of the handle to the parameter file.
+ */
+static bool
+write_duplicated_handle(HANDLE *dest, HANDLE src, HANDLE childProcess)
+{
+ HANDLE hChild = INVALID_HANDLE_VALUE;
+
+ if (!DuplicateHandle(GetCurrentProcess(),
+ src,
+ childProcess,
+ &hChild,
+ 0,
+ TRUE,
+ DUPLICATE_CLOSE_SOURCE | DUPLICATE_SAME_ACCESS))
+ {
+ ereport(LOG,
+ (errmsg_internal("could not duplicate handle to be written to backend parameter file: error code %lu",
+ GetLastError())));
+ return false;
+ }
+
+ *dest = hChild;
+ return true;
+}
+
+/*
+ * Duplicate a socket for usage in a child process, and write the resulting
+ * structure to the parameter file.
+ * This is required because a number of LSPs (Layered Service Providers) very
+ * common on Windows (antivirus, firewalls, download managers etc) break
+ * straight socket inheritance.
+ */
+static bool
+write_inheritable_socket(InheritableSocket *dest, SOCKET src, pid_t childpid)
+{
+ dest->origsocket = src;
+ if (src != 0 && src != PGINVALID_SOCKET)
+ {
+ /* Actual socket */
+ if (WSADuplicateSocket(src, childpid, &dest->wsainfo) != 0)
+ {
+ ereport(LOG,
+ (errmsg("could not duplicate socket %d for use in backend: error code %d",
+ (int) src, WSAGetLastError())));
+ return false;
+ }
+ }
+ return true;
+}
+
+/*
+ * Read a duplicate socket structure back, and get the socket descriptor.
+ */
+static void
+read_inheritable_socket(SOCKET *dest, InheritableSocket *src)
+{
+ SOCKET s;
+
+ if (src->origsocket == PGINVALID_SOCKET || src->origsocket == 0)
+ {
+ /* Not a real socket! */
+ *dest = src->origsocket;
+ }
+ else
+ {
+ /* Actual socket, so create from structure */
+ s = WSASocket(FROM_PROTOCOL_INFO,
+ FROM_PROTOCOL_INFO,
+ FROM_PROTOCOL_INFO,
+ &src->wsainfo,
+ 0,
+ 0);
+ if (s == INVALID_SOCKET)
+ {
+ write_stderr("could not create inherited socket: error code %d\n",
+ WSAGetLastError());
+ exit(1);
+ }
+ *dest = s;
+
+ /*
+ * To make sure we don't get two references to the same socket, close
+ * the original one. (This would happen when inheritance actually
+ * works..
+ */
+ closesocket(src->origsocket);
+ }
+}
+#endif
+
+void
+read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **worker)
+{
+ BackendParameters param;
+
+#ifndef WIN32
+ /* Non-win32 implementation reads from file */
+ FILE *fp;
+
+ /* Open file */
+ fp = AllocateFile(id, PG_BINARY_R);
+ if (!fp)
+ {
+ write_stderr("could not open backend variables file \"%s\": %s\n",
+ id, strerror(errno));
+ exit(1);
+ }
+
+ if (fread(¶m, sizeof(param), 1, fp) != 1)
+ {
+ write_stderr("could not read from backend variables file \"%s\": %s\n",
+ id, strerror(errno));
+ exit(1);
+ }
+
+ /* Release file */
+ FreeFile(fp);
+ if (unlink(id) != 0)
+ {
+ write_stderr("could not remove file \"%s\": %s\n",
+ id, strerror(errno));
+ exit(1);
+ }
+#else
+ /* Win32 version uses mapped file */
+ HANDLE paramHandle;
+ BackendParameters *paramp;
+
+#ifdef _WIN64
+ paramHandle = (HANDLE) _atoi64(id);
+#else
+ paramHandle = (HANDLE) atol(id);
+#endif
+ paramp = MapViewOfFile(paramHandle, FILE_MAP_READ, 0, 0, 0);
+ if (!paramp)
+ {
+ write_stderr("could not map view of backend variables: error code %lu\n",
+ GetLastError());
+ exit(1);
+ }
+
+ memcpy(¶m, paramp, sizeof(BackendParameters));
+
+ if (!UnmapViewOfFile(paramp))
+ {
+ write_stderr("could not unmap view of backend variables: error code %lu\n",
+ GetLastError());
+ exit(1);
+ }
+
+ if (!CloseHandle(paramHandle))
+ {
+ write_stderr("could not close handle to backend parameter variables: error code %lu\n",
+ GetLastError());
+ exit(1);
+ }
+#endif
+
+ restore_backend_variables(¶m, client_sock, worker);
+}
+
+/* Restore critical backend variables from the BackendParameters struct */
+static void
+restore_backend_variables(BackendParameters *param, ClientSocket **client_sock, BackgroundWorker **worker)
+{
+ if (param->has_client_sock)
+ {
+ *client_sock = (ClientSocket *) MemoryContextAlloc(TopMemoryContext, sizeof(ClientSocket));
+ memcpy(*client_sock, ¶m->client_sock, sizeof(ClientSocket));
+ read_inheritable_socket(&(*client_sock)->sock, ¶m->inh_sock);
+ }
+ else
+ *client_sock = NULL;
+
+ if (param->has_bgworker)
+ {
+ *worker = (BackgroundWorker *)
+ MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
+ memcpy(*worker, ¶m->bgworker, sizeof(BackgroundWorker));
+ }
+ else
+ *worker = NULL;
+
+ SetDataDir(param->DataDir);
+
+ MyCancelKey = param->MyCancelKey;
+ MyPMChildSlot = param->MyPMChildSlot;
+
+#ifdef WIN32
+ ShmemProtectiveRegion = param->ShmemProtectiveRegion;
+#endif
+ UsedShmemSegID = param->UsedShmemSegID;
+ UsedShmemSegAddr = param->UsedShmemSegAddr;
+
+ ShmemLock = param->ShmemLock;
+ ShmemBackendArray = param->ShmemBackendArray;
+
+#ifndef HAVE_SPINLOCKS
+ SpinlockSemaArray = param->SpinlockSemaArray;
+#endif
+ NamedLWLockTrancheRequests = param->NamedLWLockTrancheRequests;
+ NamedLWLockTrancheArray = param->NamedLWLockTrancheArray;
+ MainLWLockArray = param->MainLWLockArray;
+ ProcStructLock = param->ProcStructLock;
+ ProcGlobal = param->ProcGlobal;
+ AuxiliaryProcs = param->AuxiliaryProcs;
+ PreparedXactProcs = param->PreparedXactProcs;
+ PMSignalState = param->PMSignalState;
+
+ PostmasterPid = param->PostmasterPid;
+ PgStartTime = param->PgStartTime;
+ PgReloadTime = param->PgReloadTime;
+ first_syslogger_file_time = param->first_syslogger_file_time;
+
+ redirection_done = param->redirection_done;
+ IsBinaryUpgrade = param->IsBinaryUpgrade;
+ query_id_enabled = param->query_id_enabled;
+ max_safe_fds = param->max_safe_fds;
+
+ MaxBackends = param->MaxBackends;
+
+#ifdef WIN32
+ PostmasterHandle = param->PostmasterHandle;
+ pgwin32_initial_signal_pipe = param->initial_signal_pipe;
+#else
+ memcpy(&postmaster_alive_fds, ¶m->postmaster_alive_fds,
+ sizeof(postmaster_alive_fds));
+#endif
+
+ memcpy(&syslogPipe, ¶m->syslogPipe, sizeof(syslogPipe));
+
+ strlcpy(my_exec_path, param->my_exec_path, MAXPGPATH);
+
+ strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+
+ /*
+ * We need to restore fd.c's counts of externally-opened FDs; to avoid
+ * confusion, be sure to do this after restoring max_safe_fds. (Note:
+ * BackendInitialize will handle this for (*client_sock)->sock.)
+ */
+#ifndef WIN32
+ if (postmaster_alive_fds[0] >= 0)
+ ReserveExternalFD();
+ if (postmaster_alive_fds[1] >= 0)
+ ReserveExternalFD();
+#endif
+}
+
+#endif /* EXEC_BACKEND */
diff --git a/src/backend/postmaster/meson.build b/src/backend/postmaster/meson.build
index 63a611b68be..0ea4bbe084e 100644
--- a/src/backend/postmaster/meson.build
+++ b/src/backend/postmaster/meson.build
@@ -8,6 +8,7 @@ backend_sources += files(
'checkpointer.c',
'fork_process.c',
'interrupt.c',
+ 'launch_backend.c',
'pgarch.c',
'postmaster.c',
'startup.c',
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index f79bb3a526b..132c6386d34 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -102,7 +102,6 @@
#include "libpq/libpq.h"
#include "libpq/pqformat.h"
#include "libpq/pqsignal.h"
-#include "nodes/queryjumble.h"
#include "pg_getopt.h"
#include "pgstat.h"
#include "port/pg_bswap.h"
@@ -132,10 +131,6 @@
#include "utils/timestamp.h"
#include "utils/varlena.h"
-#ifdef EXEC_BACKEND
-#include "storage/spin.h"
-#endif
-
/*
* Possible types of a backend. Beyond being the possible bkend_type values in
@@ -188,7 +183,7 @@ typedef struct bkend
static dlist_head BackendList = DLIST_STATIC_INIT(BackendList);
#ifdef EXEC_BACKEND
-static Backend *ShmemBackendArray;
+Backend *ShmemBackendArray;
#endif
BackgroundWorker *MyBgworkerEntry = NULL;
@@ -478,7 +473,6 @@ static void MaybeStartSlotSyncWorker(void);
static pid_t waitpid(pid_t pid, int *exitstatus, int options);
static void WINAPI pgwin32_deadchild_callback(PVOID lpParameter, BOOLEAN TimerOrWaitFired);
-static void pgwin32_register_deadchild_callback(HANDLE procHandle, DWORD procId);
static HANDLE win32ChildQueue;
@@ -491,85 +485,11 @@ typedef struct
#endif /* WIN32 */
static pid_t backend_forkexec(ClientSocket *client_sock, CAC_state cac);
-static pid_t internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker);
-
-/* Type for a socket that can be inherited to a client process */
-#ifdef WIN32
-typedef struct
-{
- SOCKET origsocket; /* Original socket value, or PGINVALID_SOCKET
- * if not a socket */
- WSAPROTOCOL_INFO wsainfo;
-} InheritableSocket;
-#else
-typedef int InheritableSocket;
-#endif
-
-/*
- * Structure contains all variables passed to exec:ed backends
- */
-typedef struct
-{
- bool has_client_sock;
- ClientSocket client_sock;
- InheritableSocket inh_sock;
-
- bool has_bgworker;
- BackgroundWorker bgworker;
-
- char DataDir[MAXPGPATH];
- int32 MyCancelKey;
- int MyPMChildSlot;
-#ifndef WIN32
- unsigned long UsedShmemSegID;
-#else
- void *ShmemProtectiveRegion;
- HANDLE UsedShmemSegID;
-#endif
- void *UsedShmemSegAddr;
- slock_t *ShmemLock;
- Backend *ShmemBackendArray;
-#ifndef HAVE_SPINLOCKS
- PGSemaphore *SpinlockSemaArray;
-#endif
- int NamedLWLockTrancheRequests;
- NamedLWLockTranche *NamedLWLockTrancheArray;
- LWLockPadded *MainLWLockArray;
- slock_t *ProcStructLock;
- PROC_HDR *ProcGlobal;
- PGPROC *AuxiliaryProcs;
- PGPROC *PreparedXactProcs;
- PMSignalData *PMSignalState;
- pid_t PostmasterPid;
- TimestampTz PgStartTime;
- TimestampTz PgReloadTime;
- pg_time_t first_syslogger_file_time;
- bool redirection_done;
- bool IsBinaryUpgrade;
- bool query_id_enabled;
- int max_safe_fds;
- int MaxBackends;
-#ifdef WIN32
- HANDLE PostmasterHandle;
- HANDLE initial_signal_pipe;
- HANDLE syslogPipe[2];
-#else
- int postmaster_alive_fds[2];
- int syslogPipe[2];
-#endif
- char my_exec_path[MAXPGPATH];
- char pkglib_path[MAXPGPATH];
-} BackendParameters;
-static void read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **worker);
-static void restore_backend_variables(BackendParameters *param, ClientSocket **client_sock, BackgroundWorker **worker);
-#ifndef WIN32
-static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker);
-#else
-static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker,
- HANDLE childProcess, pid_t childPid);
-#endif
+/* in launch_backend.c */
+extern pid_t internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker);
+extern void read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **worker);
static void ShmemBackendArrayAdd(Backend *bn);
static void ShmemBackendArrayRemove(Backend *bn);
@@ -1119,11 +1039,11 @@ PostmasterMain(int argc, char *argv[])
/*
* Clean out the temp directory used to transmit parameters to child
- * processes (see internal_forkexec, below). We must do this before
- * launching any child processes, else we have a race condition: we could
- * remove a parameter file before the child can read it. It should be
- * safe to do so now, because we verified earlier that there are no
- * conflicting Postgres processes in this data directory.
+ * processes (see internal_forkexec). We must do this before launching
+ * any child processes, else we have a race condition: we could remove a
+ * parameter file before the child can read it. It should be safe to do
+ * so now, because we verified earlier that there are no conflicting
+ * Postgres processes in this data directory.
*/
RemovePgTempFilesInDir(PG_TEMP_FILES_DIR, true, false);
#endif
@@ -4513,299 +4433,6 @@ backend_forkexec(ClientSocket *client_sock, CAC_state cac)
return internal_forkexec(ac, av, client_sock, NULL);
}
-#ifndef WIN32
-
-/*
- * internal_forkexec non-win32 implementation
- *
- * - writes out backend variables to the parameter file
- * - fork():s, and then exec():s the child process
- */
-static pid_t
-internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker)
-{
- static unsigned long tmpBackendFileNum = 0;
- pid_t pid;
- char tmpfilename[MAXPGPATH];
- BackendParameters param;
- FILE *fp;
-
- /*
- * Make sure padding bytes are initialized, to prevent Valgrind from
- * complaining about writing uninitialized bytes to the file. This isn't
- * performance critical, and the win32 implementation initializes the
- * padding bytes to zeros, so do it even when not using Valgrind.
- */
- memset(¶m, 0, sizeof(BackendParameters));
-
- if (!save_backend_variables(¶m, client_sock, worker))
- return -1; /* log made by save_backend_variables */
-
- /* Calculate name for temp file */
- snprintf(tmpfilename, MAXPGPATH, "%s/%s.backend_var.%d.%lu",
- PG_TEMP_FILES_DIR, PG_TEMP_FILE_PREFIX,
- MyProcPid, ++tmpBackendFileNum);
-
- /* Open file */
- fp = AllocateFile(tmpfilename, PG_BINARY_W);
- if (!fp)
- {
- /*
- * As in OpenTemporaryFileInTablespace, try to make the temp-file
- * directory, ignoring errors.
- */
- (void) MakePGDirectory(PG_TEMP_FILES_DIR);
-
- fp = AllocateFile(tmpfilename, PG_BINARY_W);
- if (!fp)
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not create file \"%s\": %m",
- tmpfilename)));
- return -1;
- }
- }
-
- if (fwrite(¶m, sizeof(param), 1, fp) != 1)
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not write to file \"%s\": %m", tmpfilename)));
- FreeFile(fp);
- return -1;
- }
-
- /* Release file */
- if (FreeFile(fp))
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not write to file \"%s\": %m", tmpfilename)));
- return -1;
- }
-
- /* Make sure caller set up argv properly */
- Assert(argc >= 3);
- Assert(argv[argc] == NULL);
- Assert(strncmp(argv[1], "--fork", 6) == 0);
- Assert(argv[2] == NULL);
-
- /* Insert temp file name after --fork argument */
- argv[2] = tmpfilename;
-
- /* Fire off execv in child */
- if ((pid = fork_process()) == 0)
- {
- if (execv(postgres_exec_path, argv) < 0)
- {
- ereport(LOG,
- (errmsg("could not execute server process \"%s\": %m",
- postgres_exec_path)));
- /* We're already in the child process here, can't return */
- exit(1);
- }
- }
-
- return pid; /* Parent returns pid, or -1 on fork failure */
-}
-#else /* WIN32 */
-
-/*
- * internal_forkexec win32 implementation
- *
- * - starts backend using CreateProcess(), in suspended state
- * - writes out backend variables to the parameter file
- * - during this, duplicates handles and sockets required for
- * inheritance into the new process
- * - resumes execution of the new process once the backend parameter
- * file is complete.
- */
-static pid_t
-internal_forkexec(int argc, char *argv[], Port *port, BackgroundWorker *worker)
-{
- int retry_count = 0;
- STARTUPINFO si;
- PROCESS_INFORMATION pi;
- int i;
- int j;
- char cmdLine[MAXPGPATH * 2];
- HANDLE paramHandle;
- BackendParameters *param;
- SECURITY_ATTRIBUTES sa;
- char paramHandleStr[32];
- win32_deadchild_waitinfo *childinfo;
-
- /* Make sure caller set up argv properly */
- Assert(argc >= 3);
- Assert(argv[argc] == NULL);
- Assert(strncmp(argv[1], "--fork", 6) == 0);
- Assert(argv[2] == NULL);
-
- /* Resume here if we need to retry */
-retry:
-
- /* Set up shared memory for parameter passing */
- ZeroMemory(&sa, sizeof(sa));
- sa.nLength = sizeof(sa);
- sa.bInheritHandle = TRUE;
- paramHandle = CreateFileMapping(INVALID_HANDLE_VALUE,
- &sa,
- PAGE_READWRITE,
- 0,
- sizeof(BackendParameters),
- NULL);
- if (paramHandle == INVALID_HANDLE_VALUE)
- {
- ereport(LOG,
- (errmsg("could not create backend parameter file mapping: error code %lu",
- GetLastError())));
- return -1;
- }
-
- param = MapViewOfFile(paramHandle, FILE_MAP_WRITE, 0, 0, sizeof(BackendParameters));
- if (!param)
- {
- ereport(LOG,
- (errmsg("could not map backend parameter memory: error code %lu",
- GetLastError())));
- CloseHandle(paramHandle);
- return -1;
- }
-
- /* Insert temp file name after --fork argument */
-#ifdef _WIN64
- sprintf(paramHandleStr, "%llu", (LONG_PTR) paramHandle);
-#else
- sprintf(paramHandleStr, "%lu", (DWORD) paramHandle);
-#endif
- argv[2] = paramHandleStr;
-
- /* Format the cmd line */
- cmdLine[sizeof(cmdLine) - 1] = '\0';
- cmdLine[sizeof(cmdLine) - 2] = '\0';
- snprintf(cmdLine, sizeof(cmdLine) - 1, "\"%s\"", postgres_exec_path);
- i = 0;
- while (argv[++i] != NULL)
- {
- j = strlen(cmdLine);
- snprintf(cmdLine + j, sizeof(cmdLine) - 1 - j, " \"%s\"", argv[i]);
- }
- if (cmdLine[sizeof(cmdLine) - 2] != '\0')
- {
- ereport(LOG,
- (errmsg("subprocess command line too long")));
- UnmapViewOfFile(param);
- CloseHandle(paramHandle);
- return -1;
- }
-
- memset(&pi, 0, sizeof(pi));
- memset(&si, 0, sizeof(si));
- si.cb = sizeof(si);
-
- /*
- * Create the subprocess in a suspended state. This will be resumed later,
- * once we have written out the parameter file.
- */
- if (!CreateProcess(NULL, cmdLine, NULL, NULL, TRUE, CREATE_SUSPENDED,
- NULL, NULL, &si, &pi))
- {
- ereport(LOG,
- (errmsg("CreateProcess() call failed: %m (error code %lu)",
- GetLastError())));
- UnmapViewOfFile(param);
- CloseHandle(paramHandle);
- return -1;
- }
-
- if (!save_backend_variables(param, port, worker, pi.hProcess, pi.dwProcessId))
- {
- /*
- * log made by save_backend_variables, but we have to clean up the
- * mess with the half-started process
- */
- if (!TerminateProcess(pi.hProcess, 255))
- ereport(LOG,
- (errmsg_internal("could not terminate unstarted process: error code %lu",
- GetLastError())));
- CloseHandle(pi.hProcess);
- CloseHandle(pi.hThread);
- UnmapViewOfFile(param);
- CloseHandle(paramHandle);
- return -1; /* log made by save_backend_variables */
- }
-
- /* Drop the parameter shared memory that is now inherited to the backend */
- if (!UnmapViewOfFile(param))
- ereport(LOG,
- (errmsg("could not unmap view of backend parameter file: error code %lu",
- GetLastError())));
- if (!CloseHandle(paramHandle))
- ereport(LOG,
- (errmsg("could not close handle to backend parameter file: error code %lu",
- GetLastError())));
-
- /*
- * Reserve the memory region used by our main shared memory segment before
- * we resume the child process. Normally this should succeed, but if ASLR
- * is active then it might sometimes fail due to the stack or heap having
- * gotten mapped into that range. In that case, just terminate the
- * process and retry.
- */
- if (!pgwin32_ReserveSharedMemoryRegion(pi.hProcess))
- {
- /* pgwin32_ReserveSharedMemoryRegion already made a log entry */
- if (!TerminateProcess(pi.hProcess, 255))
- ereport(LOG,
- (errmsg_internal("could not terminate process that failed to reserve memory: error code %lu",
- GetLastError())));
- CloseHandle(pi.hProcess);
- CloseHandle(pi.hThread);
- if (++retry_count < 100)
- goto retry;
- ereport(LOG,
- (errmsg("giving up after too many tries to reserve shared memory"),
- errhint("This might be caused by ASLR or antivirus software.")));
- return -1;
- }
-
- /*
- * Now that the backend variables are written out, we start the child
- * thread so it can start initializing while we set up the rest of the
- * parent state.
- */
- if (ResumeThread(pi.hThread) == -1)
- {
- if (!TerminateProcess(pi.hProcess, 255))
- {
- ereport(LOG,
- (errmsg_internal("could not terminate unstartable process: error code %lu",
- GetLastError())));
- CloseHandle(pi.hProcess);
- CloseHandle(pi.hThread);
- return -1;
- }
- CloseHandle(pi.hProcess);
- CloseHandle(pi.hThread);
- ereport(LOG,
- (errmsg_internal("could not resume thread of unstarted process: error code %lu",
- GetLastError())));
- return -1;
- }
-
- /* Set up notification when the child process dies */
- pgwin32_register_deadchild_callback(pi.hProcess, pi.dwProcessId);
-
- /* Don't close pi.hProcess, it's owned by the deadchild callback now */
-
- CloseHandle(pi.hThread);
-
- return pi.dwProcessId;
-}
-#endif /* WIN32 */
-
-
/*
* SubPostmasterMain -- Get the fork/exec'd process into a state equivalent
* to what it would be if we'd simply forked on Unix, and then
@@ -6037,372 +5664,6 @@ PostmasterMarkPIDForWorkerNotify(int pid)
#ifdef EXEC_BACKEND
-/*
- * The following need to be available to the save/restore_backend_variables
- * functions. They are marked NON_EXEC_STATIC in their home modules.
- */
-extern slock_t *ShmemLock;
-extern slock_t *ProcStructLock;
-extern PGPROC *AuxiliaryProcs;
-extern PMSignalData *PMSignalState;
-extern pg_time_t first_syslogger_file_time;
-
-#ifndef WIN32
-#define write_inheritable_socket(dest, src, childpid) ((*(dest) = (src)), true)
-#define read_inheritable_socket(dest, src) (*(dest) = *(src))
-#else
-static bool write_duplicated_handle(HANDLE *dest, HANDLE src, HANDLE child);
-static bool write_inheritable_socket(InheritableSocket *dest, SOCKET src,
- pid_t childPid);
-static void read_inheritable_socket(SOCKET *dest, InheritableSocket *src);
-#endif
-
-
-/* Save critical backend variables into the BackendParameters struct */
-#ifndef WIN32
-static bool
-save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker)
-#else
-static bool
-save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker,
- HANDLE childProcess, pid_t childPid)
-#endif
-{
- if (client_sock)
- {
- memcpy(¶m->client_sock, client_sock, sizeof(ClientSocket));
- if (!write_inheritable_socket(¶m->inh_sock, client_sock->sock, childPid))
- return false;
- param->has_client_sock = true;
- }
- else
- {
- memset(¶m->client_sock, 0, sizeof(ClientSocket));
- param->has_client_sock = false;
- }
-
- if (worker)
- {
- memcpy(¶m->bgworker, worker, sizeof(BackgroundWorker));
- param->has_bgworker = true;
- }
- else
- {
- memset(¶m->bgworker, 0, sizeof(BackgroundWorker));
- param->has_bgworker = false;
- }
-
- strlcpy(param->DataDir, DataDir, MAXPGPATH);
-
- param->MyCancelKey = MyCancelKey;
- param->MyPMChildSlot = MyPMChildSlot;
-
-#ifdef WIN32
- param->ShmemProtectiveRegion = ShmemProtectiveRegion;
-#endif
- param->UsedShmemSegID = UsedShmemSegID;
- param->UsedShmemSegAddr = UsedShmemSegAddr;
-
- param->ShmemLock = ShmemLock;
- param->ShmemBackendArray = ShmemBackendArray;
-
-#ifndef HAVE_SPINLOCKS
- param->SpinlockSemaArray = SpinlockSemaArray;
-#endif
- param->NamedLWLockTrancheRequests = NamedLWLockTrancheRequests;
- param->NamedLWLockTrancheArray = NamedLWLockTrancheArray;
- param->MainLWLockArray = MainLWLockArray;
- param->ProcStructLock = ProcStructLock;
- param->ProcGlobal = ProcGlobal;
- param->AuxiliaryProcs = AuxiliaryProcs;
- param->PreparedXactProcs = PreparedXactProcs;
- param->PMSignalState = PMSignalState;
-
- param->PostmasterPid = PostmasterPid;
- param->PgStartTime = PgStartTime;
- param->PgReloadTime = PgReloadTime;
- param->first_syslogger_file_time = first_syslogger_file_time;
-
- param->redirection_done = redirection_done;
- param->IsBinaryUpgrade = IsBinaryUpgrade;
- param->query_id_enabled = query_id_enabled;
- param->max_safe_fds = max_safe_fds;
-
- param->MaxBackends = MaxBackends;
-
-#ifdef WIN32
- param->PostmasterHandle = PostmasterHandle;
- if (!write_duplicated_handle(¶m->initial_signal_pipe,
- pgwin32_create_signal_listener(childPid),
- childProcess))
- return false;
-#else
- memcpy(¶m->postmaster_alive_fds, &postmaster_alive_fds,
- sizeof(postmaster_alive_fds));
-#endif
-
- memcpy(¶m->syslogPipe, &syslogPipe, sizeof(syslogPipe));
-
- strlcpy(param->my_exec_path, my_exec_path, MAXPGPATH);
-
- strlcpy(param->pkglib_path, pkglib_path, MAXPGPATH);
-
- return true;
-}
-
-
-#ifdef WIN32
-/*
- * Duplicate a handle for usage in a child process, and write the child
- * process instance of the handle to the parameter file.
- */
-static bool
-write_duplicated_handle(HANDLE *dest, HANDLE src, HANDLE childProcess)
-{
- HANDLE hChild = INVALID_HANDLE_VALUE;
-
- if (!DuplicateHandle(GetCurrentProcess(),
- src,
- childProcess,
- &hChild,
- 0,
- TRUE,
- DUPLICATE_CLOSE_SOURCE | DUPLICATE_SAME_ACCESS))
- {
- ereport(LOG,
- (errmsg_internal("could not duplicate handle to be written to backend parameter file: error code %lu",
- GetLastError())));
- return false;
- }
-
- *dest = hChild;
- return true;
-}
-
-/*
- * Duplicate a socket for usage in a child process, and write the resulting
- * structure to the parameter file.
- * This is required because a number of LSPs (Layered Service Providers) very
- * common on Windows (antivirus, firewalls, download managers etc) break
- * straight socket inheritance.
- */
-static bool
-write_inheritable_socket(InheritableSocket *dest, SOCKET src, pid_t childpid)
-{
- dest->origsocket = src;
- if (src != 0 && src != PGINVALID_SOCKET)
- {
- /* Actual socket */
- if (WSADuplicateSocket(src, childpid, &dest->wsainfo) != 0)
- {
- ereport(LOG,
- (errmsg("could not duplicate socket %d for use in backend: error code %d",
- (int) src, WSAGetLastError())));
- return false;
- }
- }
- return true;
-}
-
-/*
- * Read a duplicate socket structure back, and get the socket descriptor.
- */
-static void
-read_inheritable_socket(SOCKET *dest, InheritableSocket *src)
-{
- SOCKET s;
-
- if (src->origsocket == PGINVALID_SOCKET || src->origsocket == 0)
- {
- /* Not a real socket! */
- *dest = src->origsocket;
- }
- else
- {
- /* Actual socket, so create from structure */
- s = WSASocket(FROM_PROTOCOL_INFO,
- FROM_PROTOCOL_INFO,
- FROM_PROTOCOL_INFO,
- &src->wsainfo,
- 0,
- 0);
- if (s == INVALID_SOCKET)
- {
- write_stderr("could not create inherited socket: error code %d\n",
- WSAGetLastError());
- exit(1);
- }
- *dest = s;
-
- /*
- * To make sure we don't get two references to the same socket, close
- * the original one. (This would happen when inheritance actually
- * works..
- */
- closesocket(src->origsocket);
- }
-}
-#endif
-
-static void
-read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **worker)
-{
- BackendParameters param;
-
-#ifndef WIN32
- /* Non-win32 implementation reads from file */
- FILE *fp;
-
- /* Open file */
- fp = AllocateFile(id, PG_BINARY_R);
- if (!fp)
- {
- write_stderr("could not open backend variables file \"%s\": %s\n",
- id, strerror(errno));
- exit(1);
- }
-
- if (fread(¶m, sizeof(param), 1, fp) != 1)
- {
- write_stderr("could not read from backend variables file \"%s\": %s\n",
- id, strerror(errno));
- exit(1);
- }
-
- /* Release file */
- FreeFile(fp);
- if (unlink(id) != 0)
- {
- write_stderr("could not remove file \"%s\": %s\n",
- id, strerror(errno));
- exit(1);
- }
-#else
- /* Win32 version uses mapped file */
- HANDLE paramHandle;
- BackendParameters *paramp;
-
-#ifdef _WIN64
- paramHandle = (HANDLE) _atoi64(id);
-#else
- paramHandle = (HANDLE) atol(id);
-#endif
- paramp = MapViewOfFile(paramHandle, FILE_MAP_READ, 0, 0, 0);
- if (!paramp)
- {
- write_stderr("could not map view of backend variables: error code %lu\n",
- GetLastError());
- exit(1);
- }
-
- memcpy(¶m, paramp, sizeof(BackendParameters));
-
- if (!UnmapViewOfFile(paramp))
- {
- write_stderr("could not unmap view of backend variables: error code %lu\n",
- GetLastError());
- exit(1);
- }
-
- if (!CloseHandle(paramHandle))
- {
- write_stderr("could not close handle to backend parameter variables: error code %lu\n",
- GetLastError());
- exit(1);
- }
-#endif
-
- restore_backend_variables(¶m, client_sock, worker);
-}
-
-/* Restore critical backend variables from the BackendParameters struct */
-static void
-restore_backend_variables(BackendParameters *param, ClientSocket **client_sock, BackgroundWorker **worker)
-{
- if (param->has_client_sock)
- {
- *client_sock = (ClientSocket *) MemoryContextAlloc(TopMemoryContext, sizeof(ClientSocket));
- memcpy(*client_sock, ¶m->client_sock, sizeof(ClientSocket));
- read_inheritable_socket(&(*client_sock)->sock, ¶m->inh_sock);
- }
- else
- *client_sock = NULL;
-
- if (param->has_bgworker)
- {
- *worker = (BackgroundWorker *)
- MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
- memcpy(*worker, ¶m->bgworker, sizeof(BackgroundWorker));
- }
- else
- *worker = NULL;
-
- SetDataDir(param->DataDir);
-
- MyCancelKey = param->MyCancelKey;
- MyPMChildSlot = param->MyPMChildSlot;
-
-#ifdef WIN32
- ShmemProtectiveRegion = param->ShmemProtectiveRegion;
-#endif
- UsedShmemSegID = param->UsedShmemSegID;
- UsedShmemSegAddr = param->UsedShmemSegAddr;
-
- ShmemLock = param->ShmemLock;
- ShmemBackendArray = param->ShmemBackendArray;
-
-#ifndef HAVE_SPINLOCKS
- SpinlockSemaArray = param->SpinlockSemaArray;
-#endif
- NamedLWLockTrancheRequests = param->NamedLWLockTrancheRequests;
- NamedLWLockTrancheArray = param->NamedLWLockTrancheArray;
- MainLWLockArray = param->MainLWLockArray;
- ProcStructLock = param->ProcStructLock;
- ProcGlobal = param->ProcGlobal;
- AuxiliaryProcs = param->AuxiliaryProcs;
- PreparedXactProcs = param->PreparedXactProcs;
- PMSignalState = param->PMSignalState;
-
- PostmasterPid = param->PostmasterPid;
- PgStartTime = param->PgStartTime;
- PgReloadTime = param->PgReloadTime;
- first_syslogger_file_time = param->first_syslogger_file_time;
-
- redirection_done = param->redirection_done;
- IsBinaryUpgrade = param->IsBinaryUpgrade;
- query_id_enabled = param->query_id_enabled;
- max_safe_fds = param->max_safe_fds;
-
- MaxBackends = param->MaxBackends;
-
-#ifdef WIN32
- PostmasterHandle = param->PostmasterHandle;
- pgwin32_initial_signal_pipe = param->initial_signal_pipe;
-#else
- memcpy(&postmaster_alive_fds, ¶m->postmaster_alive_fds,
- sizeof(postmaster_alive_fds));
-#endif
-
- memcpy(&syslogPipe, ¶m->syslogPipe, sizeof(syslogPipe));
-
- strlcpy(my_exec_path, param->my_exec_path, MAXPGPATH);
-
- strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
-
- /*
- * We need to restore fd.c's counts of externally-opened FDs; to avoid
- * confusion, be sure to do this after restoring max_safe_fds. (Note:
- * BackendInitialize will handle this for (*client_sock)->sock.)
- */
-#ifndef WIN32
- if (postmaster_alive_fds[0] >= 0)
- ReserveExternalFD();
- if (postmaster_alive_fds[1] >= 0)
- ReserveExternalFD();
-#endif
-}
-
-
Size
ShmemBackendArraySize(void)
{
@@ -6530,7 +5791,7 @@ pgwin32_deadchild_callback(PVOID lpParameter, BOOLEAN TimerOrWaitFired)
* automatically by an operating system thread pool. The memory and the
* process handle will be freed by a later call to waitpid().
*/
-static void
+void
pgwin32_register_deadchild_callback(HANDLE procHandle, DWORD procId)
{
win32_deadchild_waitinfo *childinfo;
diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c
index 487f141a596..6e35fd3150f 100644
--- a/src/backend/replication/logical/launcher.c
+++ b/src/backend/replication/logical/launcher.c
@@ -30,7 +30,6 @@
#include "miscadmin.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "replication/logicallauncher.h"
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index 46f95f7c8a6..53f82bb5d43 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -59,11 +59,16 @@ extern int MaxLivePostmasterChildren(void);
extern bool PostmasterMarkPIDForWorkerNotify(int);
#ifdef EXEC_BACKEND
+
extern pid_t postmaster_forkexec(int argc, char *argv[]);
extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
extern Size ShmemBackendArraySize(void);
extern void ShmemBackendArrayAllocation(void);
+
+#ifdef WIN32
+extern void pgwin32_register_deadchild_callback(HANDLE procHandle, DWORD procId);
+#endif
#endif
/*
--
2.39.2
v12-0006-Refactor-AuxProcess-startup.patchtext/x-patch; charset=UTF-8; name=v12-0006-Refactor-AuxProcess-startup.patchDownload
From fb18820bc3938d685ef1ad40e3b392f4804c74f2 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Mon, 4 Dec 2023 13:24:01 +0200
Subject: [PATCH v12 6/8] Refactor AuxProcess startup
Old:
AuxProcessMain()
Initialize a bunch of stuff
<aux process specific Main function>()
New:
AuxProcessMain()
<aux process specific Main function>()
AuxiliaryProcessInit()
Initialize a bunch of stuff
This isn't too useful as is, but the next commits will remove the
AuxProcessMain() function and dispatch directly to the aux-process
specific Main function, like this:
<aux process specific Main function>()
AuxiliaryProcessInit()
Initialize a bunch of stuff
This commit makes that next commit smaller.
XXX: We now have functions called AuxiliaryProcessInit() and
InitAuxiliaryProcess(). Confusing.
---
src/backend/postmaster/auxprocess.c | 89 +++++++++++++++-----------
src/backend/postmaster/bgwriter.c | 3 +
src/backend/postmaster/checkpointer.c | 3 +
src/backend/postmaster/pgarch.c | 3 +
src/backend/postmaster/startup.c | 3 +
src/backend/postmaster/walsummarizer.c | 4 +-
src/backend/postmaster/walwriter.c | 3 +
src/backend/replication/walreceiver.c | 3 +
src/include/postmaster/auxprocess.h | 3 +-
9 files changed, 75 insertions(+), 39 deletions(-)
diff --git a/src/backend/postmaster/auxprocess.c b/src/backend/postmaster/auxprocess.c
index fc13cd76321..eb436d11d76 100644
--- a/src/backend/postmaster/auxprocess.c
+++ b/src/backend/postmaster/auxprocess.c
@@ -52,6 +52,59 @@ AuxiliaryProcessMain(BackendType auxtype)
Assert(IsUnderPostmaster);
MyBackendType = auxtype;
+ switch (MyBackendType)
+ {
+ case B_STARTUP:
+ StartupProcessMain();
+ proc_exit(1);
+
+ case B_ARCHIVER:
+ PgArchiverMain();
+ proc_exit(1);
+
+ case B_BG_WRITER:
+ BackgroundWriterMain();
+ proc_exit(1);
+
+ case B_CHECKPOINTER:
+ CheckpointerMain();
+ proc_exit(1);
+
+ case B_WAL_WRITER:
+ WalWriterMain();
+ proc_exit(1);
+
+ case B_WAL_RECEIVER:
+ WalReceiverMain();
+ proc_exit(1);
+
+ case B_WAL_SUMMARIZER:
+ WalSummarizerMain();
+ proc_exit(1);
+
+ default:
+ elog(PANIC, "unrecognized process type: %d", (int) MyBackendType);
+ proc_exit(1);
+ }
+}
+
+/*
+ * AuxiliaryProcessInit
+ *
+ * Common initialization code for auxiliary processes, such as the bgwriter,
+ * walwriter, walreceiver, bootstrapper and the shared memory checker code.
+ */
+void
+AuxiliaryProcessInit(void)
+{
+ /* Release postmaster's working memory context */
+ if (PostmasterContext)
+ {
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
+ }
+
+ Assert(IsUnderPostmaster);
init_ps_display(NULL);
@@ -80,7 +133,6 @@ AuxiliaryProcessMain(BackendType auxtype)
*/
CreateAuxProcessResourceOwner();
-
/* Initialize backend status information */
pgstat_beinit();
pgstat_bestart();
@@ -89,41 +141,6 @@ AuxiliaryProcessMain(BackendType auxtype)
before_shmem_exit(ShutdownAuxiliaryProcess, 0);
SetProcessingMode(NormalProcessing);
-
- switch (MyBackendType)
- {
- case B_STARTUP:
- StartupProcessMain();
- proc_exit(1);
-
- case B_ARCHIVER:
- PgArchiverMain();
- proc_exit(1);
-
- case B_BG_WRITER:
- BackgroundWriterMain();
- proc_exit(1);
-
- case B_CHECKPOINTER:
- CheckpointerMain();
- proc_exit(1);
-
- case B_WAL_WRITER:
- WalWriterMain();
- proc_exit(1);
-
- case B_WAL_RECEIVER:
- WalReceiverMain();
- proc_exit(1);
-
- case B_WAL_SUMMARIZER:
- WalSummarizerMain();
- proc_exit(1);
-
- default:
- elog(PANIC, "unrecognized process type: %d", (int) MyBackendType);
- proc_exit(1);
- }
}
/*
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index 6364b16261f..b3cf8ba02c7 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -36,6 +36,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/bgwriter.h"
#include "postmaster/interrupt.h"
#include "storage/buf_internals.h"
@@ -95,6 +96,8 @@ BackgroundWriterMain(void)
bool prev_hibernate;
WritebackContext wb_context;
+ AuxiliaryProcessInit();
+
/*
* Properly accept or ignore signals that might be sent to us.
*/
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 46197d56f86..a882d6f44c0 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -42,6 +42,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/bgwriter.h"
#include "postmaster/interrupt.h"
#include "replication/syncrep.h"
@@ -174,6 +175,8 @@ CheckpointerMain(void)
sigjmp_buf local_sigjmp_buf;
MemoryContext checkpointer_context;
+ AuxiliaryProcessInit();
+
CheckpointerShmem->checkpointer_pid = MyProcPid;
/*
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index bb0eb13a898..e830888d9a2 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -36,6 +36,7 @@
#include "lib/binaryheap.h"
#include "libpq/pqsignal.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/interrupt.h"
#include "postmaster/pgarch.h"
#include "storage/fd.h"
@@ -210,6 +211,8 @@ PgArchCanRestart(void)
void
PgArchiverMain(void)
{
+ AuxiliaryProcessInit();
+
/*
* Ignore all signals usually bound to some action in the postmaster,
* except for SIGHUP, SIGTERM, SIGUSR1, SIGUSR2, and SIGQUIT.
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index 44b6c5bb758..bfdc64e91fb 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -25,6 +25,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/startup.h"
#include "storage/ipc.h"
#include "storage/latch.h"
@@ -216,6 +217,8 @@ StartupProcExit(int code, Datum arg)
void
StartupProcessMain(void)
{
+ AuxiliaryProcessInit();
+
/* Arrange to clean up at startup process exit */
on_shmem_exit(StartupProcExit, 0);
diff --git a/src/backend/postmaster/walsummarizer.c b/src/backend/postmaster/walsummarizer.c
index 8636d9424ec..3addb9cd41b 100644
--- a/src/backend/postmaster/walsummarizer.c
+++ b/src/backend/postmaster/walsummarizer.c
@@ -32,7 +32,7 @@
#include "common/blkreftable.h"
#include "libpq/pqsignal.h"
#include "miscadmin.h"
-#include "postmaster/bgwriter.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/interrupt.h"
#include "postmaster/walsummarizer.h"
#include "replication/walreceiver.h"
@@ -227,6 +227,8 @@ WalSummarizerMain(void)
XLogRecPtr switch_lsn = InvalidXLogRecPtr;
TimeLineID switch_tli = 0;
+ AuxiliaryProcessInit();
+
ereport(DEBUG1,
(errmsg_internal("WAL summarizer started")));
diff --git a/src/backend/postmaster/walwriter.c b/src/backend/postmaster/walwriter.c
index e079dc65c88..f0046c53d71 100644
--- a/src/backend/postmaster/walwriter.c
+++ b/src/backend/postmaster/walwriter.c
@@ -48,6 +48,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/interrupt.h"
#include "postmaster/walwriter.h"
#include "storage/bufmgr.h"
@@ -92,6 +93,8 @@ WalWriterMain(void)
int left_till_hibernate;
bool hibernating;
+ AuxiliaryProcessInit();
+
/*
* Properly accept or ignore signals the postmaster might send us
*
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index b80447d15f1..d08751dca02 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -65,6 +65,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/interrupt.h"
#include "replication/walreceiver.h"
#include "replication/walsender.h"
@@ -199,6 +200,8 @@ WalReceiverMain(void)
char *sender_host = NULL;
int sender_port = 0;
+ AuxiliaryProcessInit();
+
/*
* WalRcv should be set up already (if we are a backend, we inherit this
* by fork() or EXEC_BACKEND mechanism from the postmaster).
diff --git a/src/include/postmaster/auxprocess.h b/src/include/postmaster/auxprocess.h
index 3e443edde70..8f8ffb25034 100644
--- a/src/include/postmaster/auxprocess.h
+++ b/src/include/postmaster/auxprocess.h
@@ -13,8 +13,7 @@
#ifndef AUXPROCESS_H
#define AUXPROCESS_H
-#include "miscadmin.h"
-
extern void AuxiliaryProcessMain(BackendType auxtype) pg_attribute_noreturn();
+extern void AuxiliaryProcessInit(void);
#endif /* AUXPROCESS_H */
--
2.39.2
v12-0007-Refactor-postmaster-child-process-launching.patchtext/x-patch; charset=UTF-8; name=v12-0007-Refactor-postmaster-child-process-launching.patchDownload
From c55f3661854a1659ae2645f861f6f70629da9e63 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Fri, 8 Dec 2023 13:31:35 +0200
Subject: [PATCH v12 7/8] Refactor postmaster child process launching
- Introduce new postmaster_child_launch() function that deals with the
differences between EXEC_BACKEND and fork mode.
- Refactor the mechanism of passing information from the parent to
child process. Instead of using different command-line arguments when
launching the child process in EXEC_BACKEND mode, pass a
variable-length blob of data along with all the global variables. The
contents of that blob depend on the kind of child process being
launched. In !EXEC_BACKEND mode, we use the same blob, but it's simply
inherited from the parent to child process.
Reviewed-by: Tristan Partin, Andres Freund
Discussion: https://www.postgresql.org/message-id/7a59b073-5b5b-151e-7ed3-8b01ff7ce9ef@iki.fi
---
src/backend/postmaster/autovacuum.c | 153 +-----
src/backend/postmaster/auxprocess.c | 50 --
src/backend/postmaster/bgworker.c | 14 +-
src/backend/postmaster/bgwriter.c | 5 +-
src/backend/postmaster/checkpointer.c | 5 +-
src/backend/postmaster/launch_backend.c | 417 ++++++++++++----
src/backend/postmaster/pgarch.c | 5 +-
src/backend/postmaster/postmaster.c | 515 +++-----------------
src/backend/postmaster/startup.c | 5 +-
src/backend/postmaster/syslogger.c | 275 +++++------
src/backend/postmaster/walsummarizer.c | 5 +-
src/backend/postmaster/walwriter.c | 5 +-
src/backend/replication/walreceiver.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/include/postmaster/autovacuum.h | 10 +-
src/include/postmaster/auxprocess.h | 1 -
src/include/postmaster/bgworker_internals.h | 2 +-
src/include/postmaster/bgwriter.h | 4 +-
src/include/postmaster/pgarch.h | 2 +-
src/include/postmaster/postmaster.h | 25 +-
src/include/postmaster/startup.h | 2 +-
src/include/postmaster/syslogger.h | 4 +-
src/include/postmaster/walsummarizer.h | 2 +-
src/include/postmaster/walwriter.h | 2 +-
src/include/replication/walreceiver.h | 2 +-
src/tools/pgindent/typedefs.list | 3 +
26 files changed, 592 insertions(+), 927 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 3ebe8e91a0b..bab43c9cd96 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -85,7 +85,6 @@
#include "nodes/makefuncs.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "storage/bufmgr.h"
@@ -312,13 +311,6 @@ static WorkerInfo MyWorkerInfo = NULL;
/* PID of launcher, valid only in worker while shutting down */
int AutovacuumLauncherPid = 0;
-#ifdef EXEC_BACKEND
-static pid_t avlauncher_forkexec(void);
-static pid_t avworker_forkexec(void);
-#endif
-NON_EXEC_STATIC void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
-NON_EXEC_STATIC void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
-
static Oid do_start_worker(void);
static void HandleAutoVacLauncherInterrupts(void);
static void AutoVacLauncherShutdown(void) pg_attribute_noreturn();
@@ -362,76 +354,21 @@ static void avl_sigusr2_handler(SIGNAL_ARGS);
* AUTOVACUUM LAUNCHER CODE
********************************************************************/
-#ifdef EXEC_BACKEND
/*
- * forkexec routine for the autovacuum launcher process.
- *
- * Format up the arglist, then fork and exec.
- */
-static pid_t
-avlauncher_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkavlauncher";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif
-
-/*
- * Main entry point for autovacuum launcher process, to be called from the
- * postmaster.
+ * Main loop for the autovacuum launcher process.
*/
-int
-StartAutoVacLauncher(void)
+void
+AutoVacLauncherMain(char *startup_data, size_t startup_data_len)
{
- pid_t AutoVacPID;
+ sigjmp_buf local_sigjmp_buf;
-#ifdef EXEC_BACKEND
- switch ((AutoVacPID = avlauncher_forkexec()))
-#else
- switch ((AutoVacPID = fork_process()))
-#endif
+ /* Release postmaster's working memory context */
+ if (PostmasterContext)
{
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum launcher process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacLauncherMain(0, NULL);
- break;
-#endif
- default:
- return (int) AutoVacPID;
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
}
- /* shouldn't get here */
- return 0;
-}
-
-/*
- * Main loop for the autovacuum launcher process.
- */
-NON_EXEC_STATIC void
-AutoVacLauncherMain(int argc, char *argv[])
-{
- sigjmp_buf local_sigjmp_buf;
-
MyBackendType = B_AUTOVAC_LAUNCHER;
init_ps_display(NULL);
@@ -1413,78 +1350,22 @@ avl_sigusr2_handler(SIGNAL_ARGS)
* AUTOVACUUM WORKER CODE
********************************************************************/
-#ifdef EXEC_BACKEND
-/*
- * forkexec routines for the autovacuum worker.
- *
- * Format up the arglist, then fork and exec.
- */
-static pid_t
-avworker_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkavworker";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif
-
-/*
- * Main entry point for autovacuum worker process.
- *
- * This code is heavily based on pgarch.c, q.v.
- */
-int
-StartAutoVacWorker(void)
-{
- pid_t worker_pid;
-
-#ifdef EXEC_BACKEND
- switch ((worker_pid = avworker_forkexec()))
-#else
- switch ((worker_pid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum worker process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacWorkerMain(0, NULL);
- break;
-#endif
- default:
- return (int) worker_pid;
- }
-
- /* shouldn't get here */
- return 0;
-}
-
/*
* AutoVacWorkerMain
*/
-NON_EXEC_STATIC void
-AutoVacWorkerMain(int argc, char *argv[])
+void
+AutoVacWorkerMain(char *startup_data, size_t startup_data_len)
{
sigjmp_buf local_sigjmp_buf;
Oid dbid;
+ /* Release postmaster's working memory context */
+ if (PostmasterContext)
+ {
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
+ }
+
MyBackendType = B_AUTOVAC_WORKER;
init_ps_display(NULL);
diff --git a/src/backend/postmaster/auxprocess.c b/src/backend/postmaster/auxprocess.c
index eb436d11d76..5a35658fc34 100644
--- a/src/backend/postmaster/auxprocess.c
+++ b/src/backend/postmaster/auxprocess.c
@@ -38,56 +38,6 @@
static void ShutdownAuxiliaryProcess(int code, Datum arg);
-/*
- * AuxiliaryProcessMain
- *
- * The main entry point for auxiliary processes, such as the bgwriter,
- * walwriter, walreceiver, bootstrapper and the shared memory checker code.
- *
- * This code is here just because of historical reasons.
- */
-void
-AuxiliaryProcessMain(BackendType auxtype)
-{
- Assert(IsUnderPostmaster);
-
- MyBackendType = auxtype;
- switch (MyBackendType)
- {
- case B_STARTUP:
- StartupProcessMain();
- proc_exit(1);
-
- case B_ARCHIVER:
- PgArchiverMain();
- proc_exit(1);
-
- case B_BG_WRITER:
- BackgroundWriterMain();
- proc_exit(1);
-
- case B_CHECKPOINTER:
- CheckpointerMain();
- proc_exit(1);
-
- case B_WAL_WRITER:
- WalWriterMain();
- proc_exit(1);
-
- case B_WAL_RECEIVER:
- WalReceiverMain();
- proc_exit(1);
-
- case B_WAL_SUMMARIZER:
- WalSummarizerMain();
- proc_exit(1);
-
- default:
- elog(PANIC, "unrecognized process type: %d", (int) MyBackendType);
- proc_exit(1);
- }
-}
-
/*
* AuxiliaryProcessInit
*
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index d8e89de7494..314051c616a 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -722,15 +722,25 @@ bgworker_die(SIGNAL_ARGS)
* Main entry point for background worker processes.
*/
void
-BackgroundWorkerMain(void)
+BackgroundWorkerMain(char *startup_data, size_t startup_data_len)
{
sigjmp_buf local_sigjmp_buf;
- BackgroundWorker *worker = MyBgworkerEntry;
+ BackgroundWorker *worker;
bgworker_main_type entrypt;
+ /* Release postmaster's working memory context */
+ if (PostmasterContext)
+ {
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
+ }
+
+ Assert(startup_data_len == sizeof(BackgroundWorker));
+ worker = (BackgroundWorker *) startup_data;
if (worker == NULL)
elog(FATAL, "unable to find bgworker entry");
+ MyBgworkerEntry = worker;
MyBackendType = B_BG_WORKER;
init_ps_display(worker->bgw_name);
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index b3cf8ba02c7..0bc50987839 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -89,13 +89,16 @@ static XLogRecPtr last_snapshot_lsn = InvalidXLogRecPtr;
* basic execution environment, but not enabled signals yet.
*/
void
-BackgroundWriterMain(void)
+BackgroundWriterMain(char *startup_data, size_t startup_data_len)
{
sigjmp_buf local_sigjmp_buf;
MemoryContext bgwriter_context;
bool prev_hibernate;
WritebackContext wb_context;
+ Assert(startup_data_len == 0);
+
+ MyBackendType = B_BG_WRITER;
AuxiliaryProcessInit();
/*
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index a882d6f44c0..8204a518b70 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -170,11 +170,14 @@ static void ReqCheckpointHandler(SIGNAL_ARGS);
* basic execution environment, but not enabled signals yet.
*/
void
-CheckpointerMain(void)
+CheckpointerMain(char *startup_data, size_t startup_data_len)
{
sigjmp_buf local_sigjmp_buf;
MemoryContext checkpointer_context;
+ Assert(startup_data_len == 0);
+
+ MyBackendType = B_CHECKPOINTER;
AuxiliaryProcessInit();
CheckpointerShmem->checkpointer_pid = MyProcPid;
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 65f141a9f5a..09115803a6b 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -49,6 +49,7 @@
#include "postmaster/postmaster.h"
#include "postmaster/startup.h"
#include "postmaster/syslogger.h"
+#include "postmaster/walsummarizer.h"
#include "postmaster/walwriter.h"
#include "replication/walreceiver.h"
#include "storage/fd.h"
@@ -84,17 +85,10 @@ typedef int InheritableSocket;
#endif
/*
- * Structure contains all variables passed to exec:ed backends
+ * Structure contains all global variables passed to exec:ed backends
*/
typedef struct
{
- bool has_client_sock;
- ClientSocket client_sock;
- InheritableSocket inh_sock;
-
- bool has_bgworker;
- BackgroundWorker bgworker;
-
char DataDir[MAXPGPATH];
int32 MyCancelKey;
int MyPMChildSlot;
@@ -137,22 +131,134 @@ typedef struct
#endif
char my_exec_path[MAXPGPATH];
char pkglib_path[MAXPGPATH];
+
+ /*
+ * These are only used by backend processes, but it's here because passing
+ * a socket needs some special handling on Windows. 'client_sock' is an
+ * explicit argument to postmaster_child_launch, but is stored in
+ * MyClientSocket in the child process.
+ */
+ ClientSocket client_sock;
+ InheritableSocket inh_sock;
+
+ size_t startup_data_len;
+ /* startup data follows */
+ char startup_data[FLEXIBLE_ARRAY_MEMBER];
} BackendParameters;
#define SizeOfBackendParameters(startup_data_len) (offsetof(BackendParameters, startup_data) + startup_data_len)
-void read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **worker);
-static void restore_backend_variables(BackendParameters *param, ClientSocket **client_sock, BackgroundWorker **worker);
+static void read_backend_variables(char *id, char **startup_data, size_t *startup_data_len);
+static void restore_backend_variables(BackendParameters *param);
-#ifndef WIN32
-static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker);
-#else
-static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker,
- HANDLE childProcess, pid_t childPid);
+static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock,
+#ifdef WIN32
+ HANDLE childProcess, pid_t childPid,
#endif
+ char *startup_data, size_t startup_data_len);
+
+static pid_t internal_forkexec(const char *child_kind, char *startup_data, size_t startup_data_len, ClientSocket *client_sock);
+
+#endif /* EXEC_BACKEND */
+
+/*
+ * Information needed to launch different kinds of child processes.
+ */
+typedef struct
+{
+ const char *name;
+ void (*main_fn) (char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
+ bool shmem_attach;
+} child_process_kind;
+
+child_process_kind child_process_kinds[] = {
+ [B_BACKEND] = {"backend", BackendMain, true},
+ [B_AUTOVAC_LAUNCHER] = {"autovacuum launcher", AutoVacLauncherMain, true},
+ [B_AUTOVAC_WORKER] = {"autovacuum worker", AutoVacWorkerMain, true},
+ [B_BG_WORKER] = {"bgworker", BackgroundWorkerMain, true},
+ [B_WAL_SENDER] = {"wal sender", NULL, false},
+ [B_SLOTSYNC_WORKER] = {"slot sync worker", NULL, true},
+
+ [B_STANDALONE_BACKEND] = {"standalone backend", NULL, false},
+
+ [B_ARCHIVER] = {"archiver", PgArchiverMain, true},
+ [B_BG_WRITER] = {"bgwriter", BackgroundWriterMain, true},
+ [B_CHECKPOINTER] = {"checkpointer", CheckpointerMain, true},
+ [B_STARTUP] = {"startup", StartupProcessMain, true},
+ [B_WAL_RECEIVER] = {"wal_receiver", WalReceiverMain, true},
+ [B_WAL_SUMMARIZER] = {"wal_summarizer", WalSummarizerMain, true},
+ [B_WAL_WRITER] = {"wal_writer", WalWriterMain, true},
+
+ [B_LOGGER] = {"syslogger", SysLoggerMain, false},
+};
+
+const char *
+PostmasterChildName(BackendType child_type)
+{
+ Assert(child_type >= 0 && child_type < lengthof(child_process_kinds));
+ return child_process_kinds[child_type].name;
+}
+
+/*
+ * Start a new postmaster child process.
+ *
+ * The child process will be restored to roughly the same state, whether
+ * EXEC_BACKEND is used or not: it will be attached to shared memory, and fds
+ * and other resources that we've inherited from postmaster that are not
+ * needed in a child process have been closed.
+ *
+ * 'startup_data' is an optional contiguous chunk of data that is passed to
+ * the child process.
+ */
+pid_t
+postmaster_child_launch(BackendType child_type, char *startup_data, size_t startup_data_len, ClientSocket *client_sock)
+{
+ pid_t pid;
+
+ Assert(child_type >= 0 && child_type < lengthof(child_process_kinds));
+ Assert(IsPostmasterEnvironment && !IsUnderPostmaster);
+
+#ifdef EXEC_BACKEND
+ pid = internal_forkexec(child_process_kinds[child_type].name,
+ startup_data, startup_data_len, client_sock);
+ /* the child process will arrive in SubPostmasterMain */
+#else /* !EXEC_BACKEND */
+ pid = fork_process();
+ if (pid == 0) /* child */
+ {
+ /* Close the postmaster's sockets */
+ ClosePostmasterPorts(child_type == B_LOGGER);
-pid_t internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker);
+ /* Detangle from postmaster */
+ InitPostmasterChild();
+ /*
+ * Before blowing away PostmasterContext (in the Main function), save
+ * the startup data.
+ */
+ MemoryContextSwitchTo(TopMemoryContext);
+ if (startup_data != NULL)
+ {
+ char *cp = palloc(startup_data_len);
+
+ memcpy(cp, startup_data, startup_data_len);
+ startup_data = cp;
+ }
+
+ if (client_sock)
+ {
+ MyClientSocket = palloc(sizeof(ClientSocket));
+ memcpy(MyClientSocket, client_sock, sizeof(ClientSocket));
+ }
+
+ child_process_kinds[child_type].main_fn(startup_data, startup_data_len);
+ pg_unreachable(); /* main_fn never returns */
+ }
+#endif /* EXEC_BACKEND */
+ return pid;
+}
+
+#ifdef EXEC_BACKEND
#ifndef WIN32
/*
@@ -161,25 +267,25 @@ pid_t internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, Back
* - writes out backend variables to the parameter file
* - fork():s, and then exec():s the child process
*/
-pid_t
-internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker)
+static pid_t
+internal_forkexec(const char *child_kind, char *startup_data, size_t startup_data_len, ClientSocket *client_sock)
{
static unsigned long tmpBackendFileNum = 0;
pid_t pid;
char tmpfilename[MAXPGPATH];
- BackendParameters param;
+ size_t paramsz;
+ BackendParameters *param;
FILE *fp;
+ char *argv[4];
+ char forkav[MAXPGPATH];
- /*
- * Make sure padding bytes are initialized, to prevent Valgrind from
- * complaining about writing uninitialized bytes to the file. This isn't
- * performance critical, and the win32 implementation initializes the
- * padding bytes to zeros, so do it even when not using Valgrind.
- */
- memset(¶m, 0, sizeof(BackendParameters));
-
- if (!save_backend_variables(¶m, client_sock, worker))
+ paramsz = SizeOfBackendParameters(startup_data_len);
+ param = palloc(paramsz);
+ if (!save_backend_variables(param, client_sock, startup_data, startup_data_len))
+ {
+ pfree(param);
return -1; /* log made by save_backend_variables */
+ }
/* Calculate name for temp file */
snprintf(tmpfilename, MAXPGPATH, "%s/%s.backend_var.%d.%lu",
@@ -207,7 +313,7 @@ internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundW
}
}
- if (fwrite(¶m, sizeof(param), 1, fp) != 1)
+ if (fwrite(param, paramsz, 1, fp) != 1)
{
ereport(LOG,
(errcode_for_file_access(),
@@ -225,14 +331,13 @@ internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundW
return -1;
}
- /* Make sure caller set up argv properly */
- Assert(argc >= 3);
- Assert(argv[argc] == NULL);
- Assert(strncmp(argv[1], "--fork", 6) == 0);
- Assert(argv[2] == NULL);
-
- /* Insert temp file name after --fork argument */
+ /* set up argv properly */
+ argv[0] = "postgres";
+ snprintf(forkav, MAXPGPATH, "--forkchild=%s", child_kind);
+ argv[1] = forkav;
+ /* Insert temp file name after --forkchild argument */
argv[2] = tmpfilename;
+ argv[3] = NULL;
/* Fire off execv in child */
if ((pid = fork_process()) == 0)
@@ -261,26 +366,21 @@ internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundW
* - resumes execution of the new process once the backend parameter
* file is complete.
*/
-pid_t
-internal_forkexec(int argc, char *argv[], Port *port, BackgroundWorker *worker)
+static pid_t
+internal_forkexec(const char *child_kind, char *startup_data, size_t startup_data_len, ClientSocket *client_sock)
{
int retry_count = 0;
STARTUPINFO si;
PROCESS_INFORMATION pi;
- int i;
- int j;
char cmdLine[MAXPGPATH * 2];
HANDLE paramHandle;
BackendParameters *param;
SECURITY_ATTRIBUTES sa;
+ size_t paramsz;
char paramHandleStr[32];
- win32_deadchild_waitinfo *childinfo;
+ int l;
- /* Make sure caller set up argv properly */
- Assert(argc >= 3);
- Assert(argv[argc] == NULL);
- Assert(strncmp(argv[1], "--fork", 6) == 0);
- Assert(argv[2] == NULL);
+ paramsz = SizeOfBackendParameters(startup_data_len);
/* Resume here if we need to retry */
retry:
@@ -293,7 +393,7 @@ retry:
&sa,
PAGE_READWRITE,
0,
- sizeof(BackendParameters),
+ paramsz,
NULL);
if (paramHandle == INVALID_HANDLE_VALUE)
{
@@ -302,8 +402,7 @@ retry:
GetLastError())));
return -1;
}
-
- param = MapViewOfFile(paramHandle, FILE_MAP_WRITE, 0, 0, sizeof(BackendParameters));
+ param = MapViewOfFile(paramHandle, FILE_MAP_WRITE, 0, 0, paramsz);
if (!param)
{
ereport(LOG,
@@ -313,25 +412,15 @@ retry:
return -1;
}
- /* Insert temp file name after --fork argument */
+ /* Format the cmd line */
#ifdef _WIN64
sprintf(paramHandleStr, "%llu", (LONG_PTR) paramHandle);
#else
sprintf(paramHandleStr, "%lu", (DWORD) paramHandle);
#endif
- argv[2] = paramHandleStr;
-
- /* Format the cmd line */
- cmdLine[sizeof(cmdLine) - 1] = '\0';
- cmdLine[sizeof(cmdLine) - 2] = '\0';
- snprintf(cmdLine, sizeof(cmdLine) - 1, "\"%s\"", postgres_exec_path);
- i = 0;
- while (argv[++i] != NULL)
- {
- j = strlen(cmdLine);
- snprintf(cmdLine + j, sizeof(cmdLine) - 1 - j, " \"%s\"", argv[i]);
- }
- if (cmdLine[sizeof(cmdLine) - 2] != '\0')
+ l = snprintf(cmdLine, sizeof(cmdLine) - 1, "\"%s\" --forkchild=\"%s\" %s",
+ postgres_exec_path, child_kind, paramHandleStr);
+ if (l >= sizeof(cmdLine))
{
ereport(LOG,
(errmsg("subprocess command line too long")));
@@ -359,7 +448,7 @@ retry:
return -1;
}
- if (!save_backend_variables(param, port, worker, pi.hProcess, pi.dwProcessId))
+ if (!save_backend_variables(param, client_sock, pi.hProcess, pi.dwProcessId, startup_data, startup_data_len))
{
/*
* log made by save_backend_variables, but we have to clean up the
@@ -445,6 +534,117 @@ retry:
}
#endif /* WIN32 */
+/*
+ * SubPostmasterMain -- Get the fork/exec'd process into a state equivalent
+ * to what it would be if we'd simply forked on Unix, and then
+ * dispatch to the appropriate place.
+ *
+ * The first two command line arguments are expected to be "--forkchild=<name>",
+ * where <name> indicates which postmaster child we are to become, and
+ * the name of a variables file that we can read to load data that would
+ * have been inherited by fork() on Unix.
+ */
+void
+SubPostmasterMain(int argc, char *argv[])
+{
+ char *startup_data;
+ size_t startup_data_len;
+ char *child_kind;
+ PostmasterChildType child_type;
+ bool found = false;
+
+ /* In EXEC_BACKEND case we will not have inherited these settings */
+ IsPostmasterEnvironment = true;
+ whereToSendOutput = DestNone;
+
+ /* Setup essential subsystems (to ensure elog() behaves sanely) */
+ InitializeGUCOptions();
+
+ /* Check we got appropriate args */
+ if (argc != 3)
+ elog(FATAL, "invalid subpostmaster invocation");
+
+ /* Find the entry in child_process_kinds */
+ if (strncmp(argv[1], "--forkchild=", 12) != 0)
+ elog(FATAL, "invalid subpostmaster invocation (--forkchild argument missing)");
+ child_kind = argv[1] + 12;
+ found = false;
+ for (int idx = 0; idx < lengthof(child_process_kinds); idx++)
+ {
+ if (strcmp(child_process_kinds[idx].name, child_kind) == 0)
+ {
+ child_type = idx;
+ found = true;
+ break;
+ }
+ }
+ if (!found)
+ elog(ERROR, "unknown child kind %s", child_kind);
+
+ /* Read in the variables file */
+ read_backend_variables(argv[2], &startup_data, &startup_data_len);
+
+ /* Close the postmaster's sockets (as soon as we know them) */
+ ClosePostmasterPorts(child_type == PMC_SYSLOGGER);
+
+ /* Setup as postmaster child */
+ InitPostmasterChild();
+
+ /*
+ * If appropriate, physically re-attach to shared memory segment. We want
+ * to do this before going any further to ensure that we can attach at the
+ * same address the postmaster used. On the other hand, if we choose not
+ * to re-attach, we may have other cleanup to do.
+ *
+ * If testing EXEC_BACKEND on Linux, you should run this as root before
+ * starting the postmaster:
+ *
+ * sysctl -w kernel.randomize_va_space=0
+ *
+ * This prevents using randomized stack and code addresses that cause the
+ * child process's memory map to be different from the parent's, making it
+ * sometimes impossible to attach to shared memory at the desired address.
+ * Return the setting to its old value (usually '1' or '2') when finished.
+ */
+ if (child_process_kinds[child_type].shmem_attach)
+ PGSharedMemoryReAttach();
+ else
+ PGSharedMemoryNoReAttach();
+
+ /* Read in remaining GUC variables */
+ read_nondefault_variables();
+
+ /*
+ * Check that the data directory looks valid, which will also check the
+ * privileges on the data directory and update our umask and file/group
+ * variables for creating files later. Note: this should really be done
+ * before we create any files or directories.
+ */
+ checkDataDir();
+
+ /*
+ * (re-)read control file, as it contains config. The postmaster will
+ * already have read this, but this process doesn't know about that.
+ */
+ LocalProcessControlFile(false);
+
+ /*
+ * Reload any libraries that were preloaded by the postmaster. Since we
+ * exec'd this process, those libraries didn't come along with us; but we
+ * should load them into all child processes to be consistent with the
+ * non-EXEC_BACKEND behavior.
+ */
+ process_shared_preload_libraries();
+
+ /* Restore basic shared memory pointers */
+ if (UsedShmemSegAddr != NULL)
+ InitShmemAccess(UsedShmemSegAddr);
+
+ /* Run backend or appropriate child */
+ child_process_kinds[child_type].main_fn(startup_data, startup_data_len);
+ pg_unreachable(); /* main_fn never returns */
+}
+
/*
* The following need to be available to the save/restore_backend_variables
* functions. They are marked NON_EXEC_STATIC in their home modules.
@@ -468,38 +668,22 @@ static bool write_inheritable_socket(InheritableSocket *dest, SOCKET src,
static void read_inheritable_socket(SOCKET *dest, InheritableSocket *src);
#endif
-#ifndef WIN32
-static bool
-save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker)
-#else
+
static bool
-save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker,
- HANDLE childProcess, pid_t childPid)
+save_backend_variables(BackendParameters *param, ClientSocket *client_sock,
+#ifdef WIN32
+ HANDLE childProcess, pid_t childPid,
#endif
+ char *startup_data, size_t startup_data_len)
{
if (client_sock)
- {
memcpy(¶m->client_sock, client_sock, sizeof(ClientSocket));
- if (!write_inheritable_socket(¶m->inh_sock, client_sock->sock, childPid))
- return false;
- param->has_client_sock = true;
- }
else
- {
memset(¶m->client_sock, 0, sizeof(ClientSocket));
- param->has_client_sock = false;
- }
-
- if (worker)
- {
- memcpy(¶m->bgworker, worker, sizeof(BackgroundWorker));
- param->has_bgworker = true;
- }
- else
- {
- memset(¶m->bgworker, 0, sizeof(BackgroundWorker));
- param->has_bgworker = false;
- }
+ if (!write_inheritable_socket(¶m->inh_sock,
+ client_sock ? client_sock->sock : PGINVALID_SOCKET,
+ childPid))
+ return false;
strlcpy(param->DataDir, DataDir, MAXPGPATH);
@@ -556,6 +740,9 @@ save_backend_variables(BackendParameters *param, ClientSocket *client_sock, Back
strlcpy(param->pkglib_path, pkglib_path, MAXPGPATH);
+ param->startup_data_len = startup_data_len;
+ memcpy(param->startup_data, startup_data, startup_data_len);
+
return true;
}
@@ -652,8 +839,8 @@ read_inheritable_socket(SOCKET *dest, InheritableSocket *src)
}
#endif
-void
-read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **worker)
+static void
+read_backend_variables(char *id, char **startup_data, size_t *startup_data_len)
{
BackendParameters param;
@@ -677,6 +864,21 @@ read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **
exit(1);
}
+ /* read startup data */
+ *startup_data_len = param.startup_data_len;
+ if (param.startup_data_len > 0)
+ {
+ *startup_data = palloc(*startup_data_len);
+ if (fread(*startup_data, *startup_data_len, 1, fp) != 1)
+ {
+ write_stderr("could not read startup data from backend variables file \"%s\": %s\n",
+ id, strerror(errno));
+ exit(1);
+ }
+ }
+ else
+ *startup_data = NULL;
+
/* Release file */
FreeFile(fp);
if (unlink(id) != 0)
@@ -705,6 +907,16 @@ read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **
memcpy(¶m, paramp, sizeof(BackendParameters));
+ /* read startup data */
+ *startup_data_len = param.startup_data_len;
+ if (param.startup_data_len > 0)
+ {
+ *startup_data = palloc(paramp->startup_data_len);
+ memcpy(*startup_data, paramp->startup_data, param.startup_data_len);
+ }
+ else
+ *startup_data = NULL;
+
if (!UnmapViewOfFile(paramp))
{
write_stderr("could not unmap view of backend variables: error code %lu\n",
@@ -720,30 +932,19 @@ read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **
}
#endif
- restore_backend_variables(¶m, client_sock, worker);
+ restore_backend_variables(¶m);
}
/* Restore critical backend variables from the BackendParameters struct */
static void
-restore_backend_variables(BackendParameters *param, ClientSocket **client_sock, BackgroundWorker **worker)
+restore_backend_variables(BackendParameters *param)
{
- if (param->has_client_sock)
- {
- *client_sock = (ClientSocket *) MemoryContextAlloc(TopMemoryContext, sizeof(ClientSocket));
- memcpy(*client_sock, ¶m->client_sock, sizeof(ClientSocket));
- read_inheritable_socket(&(*client_sock)->sock, ¶m->inh_sock);
- }
- else
- *client_sock = NULL;
-
- if (param->has_bgworker)
+ if (param->client_sock.sock != PGINVALID_SOCKET)
{
- *worker = (BackgroundWorker *)
- MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
- memcpy(*worker, ¶m->bgworker, sizeof(BackgroundWorker));
+ MyClientSocket = MemoryContextAlloc(TopMemoryContext, sizeof(ClientSocket));
+ memcpy(MyClientSocket, ¶m->client_sock, sizeof(ClientSocket));
+ read_inheritable_socket(&MyClientSocket->sock, ¶m->inh_sock);
}
- else
- *worker = NULL;
SetDataDir(param->DataDir);
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index e830888d9a2..e3ca9ba1752 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -209,8 +209,11 @@ PgArchCanRestart(void)
/* Main entry point for archiver process */
void
-PgArchiverMain(void)
+PgArchiverMain(char *startup_data, size_t startup_data_len)
{
+ Assert(startup_data_len == 0);
+
+ MyBackendType = B_ARCHIVER;
AuxiliaryProcessInit();
/*
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 132c6386d34..76aca6e48e1 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -2,9 +2,9 @@
*
* postmaster.c
* This program acts as a clearing house for requests to the
- * POSTGRES system. Frontend programs send a startup message
- * to the Postmaster and the postmaster uses the info in the
- * message to setup a backend process.
+ * POSTGRES system. Frontend programs connect to the Postmaster,
+ * and postmaster forks a new backend process to handle the
+ * connection.
*
* The postmaster also manages system-wide operations such as
* startup and shutdown. The postmaster itself doesn't do those
@@ -108,7 +108,6 @@
#include "postmaster/autovacuum.h"
#include "postmaster/auxprocess.h"
#include "postmaster/bgworker_internals.h"
-#include "postmaster/fork_process.h"
#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
#include "postmaster/syslogger.h"
@@ -118,7 +117,6 @@
#include "replication/walsender.h"
#include "storage/fd.h"
#include "storage/ipc.h"
-#include "storage/pg_shmem.h"
#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "tcop/tcopprot.h"
@@ -426,7 +424,6 @@ typedef enum CAC_state
} CAC_state;
static void BackendInitialize(ClientSocket *client_sock, CAC_state cac);
-static void BackendRun(void) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
static int BackendStartup(ClientSocket *client_sock);
@@ -484,13 +481,6 @@ typedef struct
} win32_deadchild_waitinfo;
#endif /* WIN32 */
-static pid_t backend_forkexec(ClientSocket *client_sock, CAC_state cac);
-
-
-/* in launch_backend.c */
-extern pid_t internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker);
-extern void read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **worker);
-
static void ShmemBackendArrayAdd(Backend *bn);
static void ShmemBackendArrayRemove(Backend *bn);
#endif /* EXEC_BACKEND */
@@ -1750,7 +1740,7 @@ ServerLoop(void)
(AutoVacuumingActive() || start_autovac_launcher) &&
pmState == PM_RUN)
{
- AutoVacPID = StartAutoVacLauncher();
+ AutoVacPID = StartChildProcess(B_AUTOVAC_LAUNCHER);
if (AutoVacPID != 0)
start_autovac_launcher = false; /* signal processed */
}
@@ -2904,7 +2894,7 @@ process_pm_child_exit(void)
* situation, some of them may be alive already.
*/
if (!IsBinaryUpgrade && AutoVacuumingActive() && AutoVacPID == 0)
- AutoVacPID = StartAutoVacLauncher();
+ AutoVacPID = StartChildProcess(B_AUTOVAC_LAUNCHER);
if (PgArchStartupAllowed() && PgArchPID == 0)
PgArchPID = StartChildProcess(B_ARCHIVER);
MaybeStartSlotSyncWorker();
@@ -3966,6 +3956,12 @@ TerminateChildren(int signal)
signal_child(SlotSyncWorkerPID, signal);
}
+/* Information passed from postmaster to backend process */
+typedef struct BackendStartupInfo
+{
+ CAC_state canAcceptConnections;
+} BackendStartupInfo;
+
/*
* BackendStartup -- start backend process
*
@@ -3978,7 +3974,7 @@ BackendStartup(ClientSocket *client_sock)
{
Backend *bn; /* for backend cleanup */
pid_t pid;
- CAC_state cac;
+ BackendStartupInfo info;
/*
* Create backend data structure. Better before the fork() so we can
@@ -4007,11 +4003,10 @@ BackendStartup(ClientSocket *client_sock)
return STATUS_ERROR;
}
- bn->cancel_key = MyCancelKey;
-
/* Pass down canAcceptConnections state */
- cac = canAcceptConnections(BACKEND_TYPE_NORMAL);
- bn->dead_end = (cac != CAC_OK);
+ info.canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
+ bn->dead_end = (info.canAcceptConnections != CAC_OK);
+ bn->cancel_key = MyCancelKey;
/*
* Unless it's a dead_end child, assign it a child slot number
@@ -4024,26 +4019,7 @@ BackendStartup(ClientSocket *client_sock)
/* Hasn't asked to be notified about any bgworkers yet */
bn->bgworker_notify = false;
-#ifdef EXEC_BACKEND
- pid = backend_forkexec(client_sock, cac);
-#else /* !EXEC_BACKEND */
- pid = fork_process();
- if (pid == 0) /* child */
- {
- /* Detangle from postmaster */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Perform additional initialization and collect startup packet */
- BackendInitialize(client_sock, cac);
-
- /* And run the backend */
- BackendRun();
- }
-#endif /* EXEC_BACKEND */
-
+ pid = postmaster_child_launch(B_BACKEND, (char *) &info, sizeof(info), client_sock);
if (pid < 0)
{
/* in parent, fork failed */
@@ -4358,269 +4334,57 @@ BackendInitialize(ClientSocket *client_sock, CAC_state cac)
set_ps_display("initializing");
}
-
-/*
- * BackendRun -- set up the backend's argument list and invoke PostgresMain()
- *
- * returns:
- * Doesn't return at all.
- */
-static void
-BackendRun(void)
-{
- /*
- * Create a per-backend PGPROC struct in shared memory. We must do this
- * before we can use LWLocks or access any shared memory.
- */
- InitProcess();
-
- /*
- * Make sure we aren't in PostmasterContext anymore. (We can't delete it
- * just yet, though, because InitPostgres will need the HBA data.)
- */
- MemoryContextSwitchTo(TopMemoryContext);
-
- PostgresMain(MyProcPort->database_name, MyProcPort->user_name);
-}
-
-
-#ifdef EXEC_BACKEND
-
-/*
- * postmaster_forkexec -- fork and exec a postmaster subprocess
- *
- * The caller must have set up the argv array already, except for argv[2]
- * which will be filled with the name of the temp variable file.
- *
- * Returns the child process PID, or -1 on fork failure (a suitable error
- * message has been logged on failure).
- *
- * All uses of this routine will dispatch to SubPostmasterMain in the
- * child process.
- */
-pid_t
-postmaster_forkexec(int argc, char *argv[])
-{
- return internal_forkexec(argc, argv, NULL, NULL);
-}
-
-/*
- * backend_forkexec -- fork/exec off a backend process
- *
- * Some operating systems (WIN32) don't have fork() so we have to simulate
- * it by storing parameters that need to be passed to the child and
- * then create a new child process.
- *
- * returns the pid of the fork/exec'd process, or -1 on failure
- */
-static pid_t
-backend_forkexec(ClientSocket *client_sock, CAC_state cac)
-{
- char *av[5];
- int ac = 0;
- char cacbuf[10];
-
- av[ac++] = "postgres";
- av[ac++] = "--forkbackend";
- av[ac++] = NULL; /* filled in by internal_forkexec */
-
- snprintf(cacbuf, sizeof(cacbuf), "%d", (int) cac);
- av[ac++] = cacbuf;
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return internal_forkexec(ac, av, client_sock, NULL);
-}
-
-/*
- * SubPostmasterMain -- Get the fork/exec'd process into a state equivalent
- * to what it would be if we'd simply forked on Unix, and then
- * dispatch to the appropriate place.
- *
- * The first two command line arguments are expected to be "--forkFOO"
- * (where FOO indicates which postmaster child we are to become), and
- * the name of a variables file that we can read to load data that would
- * have been inherited by fork() on Unix. Remaining arguments go to the
- * subprocess FooMain() routine.
- */
void
-SubPostmasterMain(int argc, char *argv[])
+BackendMain(char *startup_data, size_t startup_data_len)
{
- ClientSocket *client_sock;
- BackgroundWorker *worker;
-
- /* In EXEC_BACKEND case we will not have inherited these settings */
- IsPostmasterEnvironment = true;
- whereToSendOutput = DestNone;
-
- /* Setup essential subsystems (to ensure elog() behaves sanely) */
- InitializeGUCOptions();
-
- /* Check we got appropriate args */
- if (argc < 3)
- elog(FATAL, "invalid subpostmaster invocation");
-
- /* Read in the variables file */
- read_backend_variables(argv[2], &client_sock, &worker);
+ BackendStartupInfo *info = (BackendStartupInfo *) startup_data;
- /* Close the postmaster's sockets (as soon as we know them) */
- ClosePostmasterPorts(strcmp(argv[1], "--forklog") == 0);
+ Assert(startup_data_len == sizeof(BackendStartupInfo));
+ Assert(MyClientSocket != NULL);
- /* Setup as postmaster child */
- InitPostmasterChild();
+#ifdef EXEC_BACKEND
/*
- * If appropriate, physically re-attach to shared memory segment. We want
- * to do this before going any further to ensure that we can attach at the
- * same address the postmaster used. On the other hand, if we choose not
- * to re-attach, we may have other cleanup to do.
- *
- * If testing EXEC_BACKEND on Linux, you should run this as root before
- * starting the postmaster:
+ * Need to reinitialize the SSL library in the backend, since the context
+ * structures contain function pointers and cannot be passed through the
+ * parameter file.
*
- * sysctl -w kernel.randomize_va_space=0
+ * If for some reason reload fails (maybe the user installed broken key
+ * files), soldier on without SSL; that's better than all connections
+ * becoming impossible.
*
- * This prevents using randomized stack and code addresses that cause the
- * child process's memory map to be different from the parent's, making it
- * sometimes impossible to attach to shared memory at the desired address.
- * Return the setting to its old value (usually '1' or '2') when finished.
+ * XXX should we do this in all child processes? For the moment it's
+ * enough to do it in backend children.
*/
- if (strcmp(argv[1], "--forkbackend") == 0 ||
- strcmp(argv[1], "--forkavlauncher") == 0 ||
- strcmp(argv[1], "--forkssworker") == 0 ||
- strcmp(argv[1], "--forkavworker") == 0 ||
- strcmp(argv[1], "--forkaux") == 0 ||
- strcmp(argv[1], "--forkbgworker") == 0)
- PGSharedMemoryReAttach();
- else
- PGSharedMemoryNoReAttach();
+#ifdef USE_SSL
+ if (EnableSSL)
+ {
+ if (secure_initialize(false) == 0)
+ LoadedSSL = true;
+ else
+ ereport(LOG,
+ (errmsg("SSL configuration could not be loaded in child process")));
+ }
+#endif
+#endif
- /* Read in remaining GUC variables */
- read_nondefault_variables();
+ /* Perform additional initialization and collect startup packet */
+ BackendInitialize(MyClientSocket, info->canAcceptConnections);
/*
- * Check that the data directory looks valid, which will also check the
- * privileges on the data directory and update our umask and file/group
- * variables for creating files later. Note: this should really be done
- * before we create any files or directories.
- */
- checkDataDir();
-
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
+ * Create a per-backend PGPROC struct in shared memory. We must do this
+ * before we can use LWLocks or access any shared memory.
*/
- LocalProcessControlFile(false);
+ InitProcess();
/*
- * Reload any libraries that were preloaded by the postmaster. Since we
- * exec'd this process, those libraries didn't come along with us; but we
- * should load them into all child processes to be consistent with the
- * non-EXEC_BACKEND behavior.
+ * Make sure we aren't in PostmasterContext anymore. (We can't delete it
+ * just yet, though, because InitPostgres will need the HBA data.)
*/
- process_shared_preload_libraries();
-
- /* Run backend or appropriate child */
- if (strcmp(argv[1], "--forkbackend") == 0)
- {
- CAC_state cac;
-
- Assert(argc == 4);
- cac = (CAC_state) atoi(argv[3]);
-
- /*
- * Need to reinitialize the SSL library in the backend, since the
- * context structures contain function pointers and cannot be passed
- * through the parameter file.
- *
- * If for some reason reload fails (maybe the user installed broken
- * key files), soldier on without SSL; that's better than all
- * connections becoming impossible.
- *
- * XXX should we do this in all child processes? For the moment it's
- * enough to do it in backend children.
- */
-#ifdef USE_SSL
- if (EnableSSL)
- {
- if (secure_initialize(false) == 0)
- LoadedSSL = true;
- else
- ereport(LOG,
- (errmsg("SSL configuration could not be loaded in child process")));
- }
-#endif
-
- /*
- * Perform additional initialization and collect startup packet.
- *
- * We want to do this before InitProcess() for a couple of reasons: 1.
- * so that we aren't eating up a PGPROC slot while waiting on the
- * client. 2. so that if InitProcess() fails due to being out of
- * PGPROC slots, we have already initialized libpq and are able to
- * report the error to the client.
- */
- BackendInitialize(client_sock, cac);
-
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* And run the backend */
- BackendRun(); /* does not return */
-
- }
- if (strcmp(argv[1], "--forkaux") == 0)
- {
- BackendType auxtype;
-
- Assert(argc == 4);
-
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- auxtype = atoi(argv[3]);
- AuxiliaryProcessMain(auxtype); /* does not return */
- }
- if (strcmp(argv[1], "--forkavlauncher") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- AutoVacLauncherMain(argc - 2, argv + 2); /* does not return */
- }
- if (strcmp(argv[1], "--forkavworker") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
- }
- if (strcmp(argv[1], "--forkssworker") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- ReplSlotSyncWorkerMain(argc - 2, argv + 2); /* does not return */
- }
- if (strcmp(argv[1], "--forkbgworker") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- MyBgworkerEntry = worker;
- BackgroundWorkerMain();
- }
- if (strcmp(argv[1], "--forklog") == 0)
- {
- /* Do not want to attach to shared memory */
-
- SysLoggerMain(argc, argv); /* does not return */
- }
+ MemoryContextSwitchTo(TopMemoryContext);
- abort(); /* shouldn't get here */
+ PostgresMain(MyProcPort->database_name, MyProcPort->user_name);
}
-#endif /* EXEC_BACKEND */
/*
@@ -4919,87 +4683,13 @@ StartChildProcess(BackendType type)
{
pid_t pid;
-#ifdef EXEC_BACKEND
- {
- char *av[10];
- int ac = 0;
- char typebuf[32];
-
- /*
- * Set up command-line arguments for subprocess
- */
- av[ac++] = "postgres";
- av[ac++] = "--forkaux";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- snprintf(typebuf, sizeof(typebuf), "%d", type);
- av[ac++] = typebuf;
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- pid = postmaster_forkexec(ac, av);
- }
-#else /* !EXEC_BACKEND */
- pid = fork_process();
-
- if (pid == 0) /* child */
- {
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Release postmaster's working memory context */
- MemoryContextSwitchTo(TopMemoryContext);
- MemoryContextDelete(PostmasterContext);
- PostmasterContext = NULL;
-
- AuxiliaryProcessMain(type); /* does not return */
- }
-#endif /* EXEC_BACKEND */
-
+ pid = postmaster_child_launch(type, NULL, 0, NULL);
if (pid < 0)
{
/* in parent, fork failed */
- int save_errno = errno;
-
- errno = save_errno;
- switch (type)
- {
- case B_STARTUP:
- ereport(LOG,
- (errmsg("could not fork startup process: %m")));
- break;
- case B_ARCHIVER:
- ereport(LOG,
- (errmsg("could not fork archiver process: %m")));
- break;
- case B_BG_WRITER:
- ereport(LOG,
- (errmsg("could not fork background writer process: %m")));
- break;
- case B_CHECKPOINTER:
- ereport(LOG,
- (errmsg("could not fork checkpointer process: %m")));
- break;
- case B_WAL_WRITER:
- ereport(LOG,
- (errmsg("could not fork WAL writer process: %m")));
- break;
- case B_WAL_RECEIVER:
- ereport(LOG,
- (errmsg("could not fork WAL receiver process: %m")));
- break;
- case B_WAL_SUMMARIZER:
- ereport(LOG,
- (errmsg("could not fork WAL summarizer process: %m")));
- break;
- default:
- ereport(LOG,
- (errmsg("could not fork process: %m")));
- break;
- }
+ /* XXX: translation? */
+ ereport(LOG,
+ (errmsg("could not fork %s process: %m", PostmasterChildName(type))));
/*
* fork failure is fatal during startup, but there's no need to choke
@@ -5063,7 +4753,7 @@ StartAutovacuumWorker(void)
bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
bn->bgworker_notify = false;
- bn->pid = StartAutoVacWorker();
+ bn->pid = StartChildProcess(B_AUTOVAC_WORKER);
if (bn->pid > 0)
{
bn->bkend_type = BACKEND_TYPE_AUTOVAC;
@@ -5077,7 +4767,7 @@ StartAutovacuumWorker(void)
/*
* fork failed, fall through to report -- actual error message was
- * logged by StartAutoVacWorker
+ * logged by StartChildProcess
*/
(void) ReleasePostmasterChildSlot(bn->child_slot);
pfree(bn);
@@ -5300,24 +4990,6 @@ BackgroundWorkerUnblockSignals(void)
sigprocmask(SIG_SETMASK, &UnBlockSig, NULL);
}
-#ifdef EXEC_BACKEND
-static pid_t
-bgworker_forkexec(BackgroundWorker *worker)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkbgworker";
- av[ac++] = NULL; /* filled in by internal_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return internal_forkexec(ac, av, NULL, worker);
-}
-#endif
-
/*
* Start a new bgworker.
* Starting time conditions must have been checked already.
@@ -5354,65 +5026,32 @@ do_start_bgworker(RegisteredBgWorker *rw)
(errmsg_internal("starting background worker process \"%s\"",
rw->rw_worker.bgw_name)));
-#ifdef EXEC_BACKEND
- switch ((worker_pid = bgworker_forkexec(&rw->rw_worker)))
-#else
- switch ((worker_pid = fork_process()))
-#endif
+ worker_pid = postmaster_child_launch(B_BG_WORKER, (char *) &rw->rw_worker, sizeof(BackgroundWorker), NULL);
+ if (worker_pid == -1)
{
- case -1:
- /* in postmaster, fork failed ... */
- ereport(LOG,
- (errmsg("could not fork worker process: %m")));
- /* undo what assign_backendlist_entry did */
- ReleasePostmasterChildSlot(rw->rw_child_slot);
- rw->rw_child_slot = 0;
- pfree(rw->rw_backend);
- rw->rw_backend = NULL;
- /* mark entry as crashed, so we'll try again later */
- rw->rw_crashed_at = GetCurrentTimestamp();
- break;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /*
- * Before blowing away PostmasterContext, save this bgworker's
- * data where it can find it.
- */
- MyBgworkerEntry = (BackgroundWorker *)
- MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
- memcpy(MyBgworkerEntry, &rw->rw_worker, sizeof(BackgroundWorker));
-
- /* Release postmaster's working memory context */
- MemoryContextSwitchTo(TopMemoryContext);
- MemoryContextDelete(PostmasterContext);
- PostmasterContext = NULL;
-
- BackgroundWorkerMain();
+ /* in postmaster, fork failed ... */
+ ereport(LOG,
+ (errmsg("could not fork worker process: %m")));
+ /* undo what assign_backendlist_entry did */
+ ReleasePostmasterChildSlot(rw->rw_child_slot);
+ rw->rw_child_slot = 0;
+ pfree(rw->rw_backend);
+ rw->rw_backend = NULL;
+ /* mark entry as crashed, so we'll try again later */
+ rw->rw_crashed_at = GetCurrentTimestamp();
+ return false;
+ }
- exit(1); /* should not get here */
- break;
-#endif
- default:
- /* in postmaster, fork successful ... */
- rw->rw_pid = worker_pid;
- rw->rw_backend->pid = rw->rw_pid;
- ReportBackgroundWorkerPID(rw);
- /* add new worker to lists of backends */
- dlist_push_head(&BackendList, &rw->rw_backend->elem);
+ /* in postmaster, fork successful ... */
+ rw->rw_pid = worker_pid;
+ rw->rw_backend->pid = rw->rw_pid;
+ ReportBackgroundWorkerPID(rw);
+ /* add new worker to lists of backends */
+ dlist_push_head(&BackendList, &rw->rw_backend->elem);
#ifdef EXEC_BACKEND
- ShmemBackendArrayAdd(rw->rw_backend);
+ ShmemBackendArrayAdd(rw->rw_backend);
#endif
- return true;
- }
-
- return false;
+ return true;
}
/*
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index bfdc64e91fb..60e4cc839a4 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -215,8 +215,11 @@ StartupProcExit(int code, Datum arg)
* ----------------------------------
*/
void
-StartupProcessMain(void)
+StartupProcessMain(char *startup_data, size_t startup_data_len)
{
+ Assert(startup_data_len == 0);
+
+ MyBackendType = B_STARTUP;
AuxiliaryProcessInit();
/* Arrange to clean up at startup process exit */
diff --git a/src/backend/postmaster/syslogger.c b/src/backend/postmaster/syslogger.c
index 6db280e483e..89f380f5f66 100644
--- a/src/backend/postmaster/syslogger.c
+++ b/src/backend/postmaster/syslogger.c
@@ -39,7 +39,6 @@
#include "pgstat.h"
#include "pgtime.h"
#include "port/pg_bitutils.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "postmaster/syslogger.h"
@@ -50,6 +49,7 @@
#include "storage/pg_shmem.h"
#include "tcop/tcopprot.h"
#include "utils/guc.h"
+#include "utils/memutils.h"
#include "utils/ps_status.h"
#include "utils/timestamp.h"
@@ -134,10 +134,7 @@ static volatile sig_atomic_t rotation_requested = false;
#ifdef EXEC_BACKEND
static int syslogger_fdget(FILE *file);
static FILE *syslogger_fdopen(int fd);
-static pid_t syslogger_forkexec(void);
-static void syslogger_parseArgs(int argc, char *argv[]);
#endif
-NON_EXEC_STATIC void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
static void process_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static void flush_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static FILE *logfile_open(const char *filename, const char *mode,
@@ -156,13 +153,19 @@ static void set_next_rotation_time(void);
static void sigUsr1Handler(SIGNAL_ARGS);
static void update_metainfo_datafile(void);
+typedef struct
+{
+ int syslogFile;
+ int csvlogFile;
+ int jsonlogFile;
+} syslogger_startup_data;
/*
* Main entry point for syslogger process
* argc/argv parameters are valid only in EXEC_BACKEND case.
*/
-NON_EXEC_STATIC void
-SysLoggerMain(int argc, char *argv[])
+void
+SysLoggerMain(char *startup_data, size_t startup_data_len)
{
#ifndef WIN32
char logbuffer[READ_BUF_SIZE];
@@ -174,11 +177,34 @@ SysLoggerMain(int argc, char *argv[])
pg_time_t now;
WaitEventSet *wes;
- now = MyStartTime;
+ /* Release postmaster's working memory context */
+ if (PostmasterContext)
+ {
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
+ }
+ /*
+ * Re-open the error output files that were opened by SysLogger_Start().
+ *
+ * We expect this will always succeed, which is too optimistic, but if it
+ * fails there's not a lot we can do to report the problem anyway. As
+ * coded, we'll just crash on a null pointer dereference after failure...
+ */
#ifdef EXEC_BACKEND
- syslogger_parseArgs(argc, argv);
-#endif /* EXEC_BACKEND */
+ {
+ syslogger_startup_data *info = (syslogger_startup_data *) startup_data;
+
+ Assert(startup_data_len == sizeof(*info));
+ syslogFile = syslogger_fdopen(info->syslogFile);
+ csvlogFile = syslogger_fdopen(info->csvlogFile);
+ jsonlogFile = syslogger_fdopen(info->jsonlogFile);
+ }
+#else
+ Assert(startup_data_len == 0);
+#endif
+
+ now = MyStartTime;
MyBackendType = B_LOGGER;
init_ps_display(NULL);
@@ -568,6 +594,9 @@ SysLogger_Start(void)
{
pid_t sysloggerPid;
char *filename;
+#ifdef EXEC_BACKEND
+ syslogger_startup_data startup_data;
+#endif /* EXEC_BACKEND */
if (!Logging_collector)
return 0;
@@ -667,112 +696,95 @@ SysLogger_Start(void)
}
#ifdef EXEC_BACKEND
- switch ((sysloggerPid = syslogger_forkexec()))
+ startup_data.syslogFile = syslogger_fdget(syslogFile);
+ startup_data.csvlogFile = syslogger_fdget(csvlogFile);
+ startup_data.jsonlogFile = syslogger_fdget(jsonlogFile);
+ sysloggerPid = postmaster_child_launch(B_LOGGER, (char *) &startup_data, sizeof(startup_data), NULL);
#else
- switch ((sysloggerPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork system logger: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(true);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
+ sysloggerPid = postmaster_child_launch(B_LOGGER, NULL, 0, NULL);
+#endif /* EXEC_BACKEND */
- /* do the work */
- SysLoggerMain(0, NULL);
- break;
-#endif
+ if (sysloggerPid == -1)
+ {
+ ereport(LOG,
+ (errmsg("could not fork system logger: %m")));
+ return 0;
+ }
- default:
- /* success, in postmaster */
+ /* success, in postmaster */
- /* now we redirect stderr, if not done already */
- if (!redirection_done)
- {
+ /* now we redirect stderr, if not done already */
+ if (!redirection_done)
+ {
#ifdef WIN32
- int fd;
+ int fd;
#endif
- /*
- * Leave a breadcrumb trail when redirecting, in case the user
- * forgets that redirection is active and looks only at the
- * original stderr target file.
- */
- ereport(LOG,
- (errmsg("redirecting log output to logging collector process"),
- errhint("Future log output will appear in directory \"%s\".",
- Log_directory)));
+ /*
+ * Leave a breadcrumb trail when redirecting, in case the user forgets
+ * that redirection is active and looks only at the original stderr
+ * target file.
+ */
+ ereport(LOG,
+ (errmsg("redirecting log output to logging collector process"),
+ errhint("Future log output will appear in directory \"%s\".",
+ Log_directory)));
#ifndef WIN32
- fflush(stdout);
- if (dup2(syslogPipe[1], STDOUT_FILENO) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stdout: %m")));
- fflush(stderr);
- if (dup2(syslogPipe[1], STDERR_FILENO) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stderr: %m")));
- /* Now we are done with the write end of the pipe. */
- close(syslogPipe[1]);
- syslogPipe[1] = -1;
+ fflush(stdout);
+ if (dup2(syslogPipe[1], STDOUT_FILENO) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stdout: %m")));
+ fflush(stderr);
+ if (dup2(syslogPipe[1], STDERR_FILENO) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stderr: %m")));
+ /* Now we are done with the write end of the pipe. */
+ close(syslogPipe[1]);
+ syslogPipe[1] = -1;
#else
- /*
- * open the pipe in binary mode and make sure stderr is binary
- * after it's been dup'ed into, to avoid disturbing the pipe
- * chunking protocol.
- */
- fflush(stderr);
- fd = _open_osfhandle((intptr_t) syslogPipe[1],
- _O_APPEND | _O_BINARY);
- if (dup2(fd, STDERR_FILENO) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stderr: %m")));
- close(fd);
- _setmode(STDERR_FILENO, _O_BINARY);
+ /*
+ * open the pipe in binary mode and make sure stderr is binary after
+ * it's been dup'ed into, to avoid disturbing the pipe chunking
+ * protocol.
+ */
+ fflush(stderr);
+ fd = _open_osfhandle((intptr_t) syslogPipe[1],
+ _O_APPEND | _O_BINARY);
+ if (dup2(fd, STDERR_FILENO) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stderr: %m")));
+ close(fd);
+ _setmode(STDERR_FILENO, _O_BINARY);
- /*
- * Now we are done with the write end of the pipe.
- * CloseHandle() must not be called because the preceding
- * close() closes the underlying handle.
- */
- syslogPipe[1] = 0;
+ /*
+ * Now we are done with the write end of the pipe. CloseHandle() must
+ * not be called because the preceding close() closes the underlying
+ * handle.
+ */
+ syslogPipe[1] = 0;
#endif
- redirection_done = true;
- }
-
- /* postmaster will never write the file(s); close 'em */
- fclose(syslogFile);
- syslogFile = NULL;
- if (csvlogFile != NULL)
- {
- fclose(csvlogFile);
- csvlogFile = NULL;
- }
- if (jsonlogFile != NULL)
- {
- fclose(jsonlogFile);
- jsonlogFile = NULL;
- }
- return (int) sysloggerPid;
+ redirection_done = true;
}
- /* we should never reach here */
- return 0;
+ /* postmaster will never write the file(s); close 'em */
+ fclose(syslogFile);
+ syslogFile = NULL;
+ if (csvlogFile != NULL)
+ {
+ fclose(csvlogFile);
+ csvlogFile = NULL;
+ }
+ if (jsonlogFile != NULL)
+ {
+ fclose(jsonlogFile);
+ jsonlogFile = NULL;
+ }
+ return (int) sysloggerPid;
}
@@ -831,69 +843,6 @@ syslogger_fdopen(int fd)
return file;
}
-
-/*
- * syslogger_forkexec() -
- *
- * Format up the arglist for, then fork and exec, a syslogger process
- */
-static pid_t
-syslogger_forkexec(void)
-{
- char *av[10];
- int ac = 0;
- char filenobuf[32];
- char csvfilenobuf[32];
- char jsonfilenobuf[32];
-
- av[ac++] = "postgres";
- av[ac++] = "--forklog";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- /* static variables (those not passed by write_backend_variables) */
- snprintf(filenobuf, sizeof(filenobuf), "%d",
- syslogger_fdget(syslogFile));
- av[ac++] = filenobuf;
- snprintf(csvfilenobuf, sizeof(csvfilenobuf), "%d",
- syslogger_fdget(csvlogFile));
- av[ac++] = csvfilenobuf;
- snprintf(jsonfilenobuf, sizeof(jsonfilenobuf), "%d",
- syslogger_fdget(jsonlogFile));
- av[ac++] = jsonfilenobuf;
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-
-/*
- * syslogger_parseArgs() -
- *
- * Extract data from the arglist for exec'ed syslogger process
- */
-static void
-syslogger_parseArgs(int argc, char *argv[])
-{
- int fd;
-
- Assert(argc == 6);
- argv += 3;
-
- /*
- * Re-open the error output files that were opened by SysLogger_Start().
- *
- * We expect this will always succeed, which is too optimistic, but if it
- * fails there's not a lot we can do to report the problem anyway. As
- * coded, we'll just crash on a null pointer dereference after failure...
- */
- fd = atoi(*argv++);
- syslogFile = syslogger_fdopen(fd);
- fd = atoi(*argv++);
- csvlogFile = syslogger_fdopen(fd);
- fd = atoi(*argv++);
- jsonlogFile = syslogger_fdopen(fd);
-}
#endif /* EXEC_BACKEND */
diff --git a/src/backend/postmaster/walsummarizer.c b/src/backend/postmaster/walsummarizer.c
index 3addb9cd41b..d5ae73c219b 100644
--- a/src/backend/postmaster/walsummarizer.c
+++ b/src/backend/postmaster/walsummarizer.c
@@ -205,7 +205,7 @@ WalSummarizerShmemInit(void)
* Entry point for walsummarizer process.
*/
void
-WalSummarizerMain(void)
+WalSummarizerMain(char *startup_data, size_t startup_data_len)
{
sigjmp_buf local_sigjmp_buf;
MemoryContext context;
@@ -227,6 +227,9 @@ WalSummarizerMain(void)
XLogRecPtr switch_lsn = InvalidXLogRecPtr;
TimeLineID switch_tli = 0;
+ Assert(startup_data_len == 0);
+
+ MyBackendType = B_WAL_SUMMARIZER;
AuxiliaryProcessInit();
ereport(DEBUG1,
diff --git a/src/backend/postmaster/walwriter.c b/src/backend/postmaster/walwriter.c
index f0046c53d71..aecacc6595b 100644
--- a/src/backend/postmaster/walwriter.c
+++ b/src/backend/postmaster/walwriter.c
@@ -86,13 +86,16 @@ int WalWriterFlushAfter = DEFAULT_WAL_WRITER_FLUSH_AFTER;
* basic execution environment, but not enabled signals yet.
*/
void
-WalWriterMain(void)
+WalWriterMain(char *startup_data, size_t startup_data_len)
{
sigjmp_buf local_sigjmp_buf;
MemoryContext walwriter_context;
int left_till_hibernate;
bool hibernating;
+ Assert(startup_data_len == 0);
+
+ MyBackendType = B_WAL_WRITER;
AuxiliaryProcessInit();
/*
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index d08751dca02..a1566bc629f 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -184,7 +184,7 @@ ProcessWalRcvInterrupts(void)
/* Main entry point for walreceiver process */
void
-WalReceiverMain(void)
+WalReceiverMain(char *startup_data, size_t startup_data_len)
{
char conninfo[MAXCONNINFO];
char *tmp_conninfo;
@@ -200,6 +200,9 @@ WalReceiverMain(void)
char *sender_host = NULL;
int sender_port = 0;
+ Assert(startup_data_len == 0);
+
+ MyBackendType = B_WAL_RECEIVER;
AuxiliaryProcessInit();
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 5b536ac50d1..3e38bb1311d 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -45,6 +45,7 @@ volatile uint32 CritSectionCount = 0;
int MyProcPid;
pg_time_t MyStartTime;
TimestampTz MyStartTimestamp;
+struct ClientSocket *MyClientSocket;
struct Port *MyProcPort;
int32 MyCancelKey;
int MyPMChildSlot;
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 80cf4cdd969..cae1e8b3294 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -50,18 +50,14 @@ extern PGDLLIMPORT int Log_autovacuum_min_duration;
/* Status inquiry functions */
extern bool AutoVacuumingActive(void);
-/* Functions to start autovacuum process, called from postmaster */
+/* called from postmaster at server startup */
extern void autovac_init(void);
-extern int StartAutoVacLauncher(void);
-extern int StartAutoVacWorker(void);
/* called from postmaster when a worker could not be forked */
extern void AutoVacWorkerFailed(void);
-#ifdef EXEC_BACKEND
-extern void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
-extern void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
+extern void AutoVacLauncherMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
+extern void AutoVacWorkerMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern bool AutoVacuumRequestWork(AutoVacuumWorkItemType type,
Oid relationId, BlockNumber blkno);
diff --git a/src/include/postmaster/auxprocess.h b/src/include/postmaster/auxprocess.h
index 8f8ffb25034..caad9f5abcc 100644
--- a/src/include/postmaster/auxprocess.h
+++ b/src/include/postmaster/auxprocess.h
@@ -13,7 +13,6 @@
#ifndef AUXPROCESS_H
#define AUXPROCESS_H
-extern void AuxiliaryProcessMain(BackendType auxtype) pg_attribute_noreturn();
extern void AuxiliaryProcessInit(void);
#endif /* AUXPROCESS_H */
diff --git a/src/include/postmaster/bgworker_internals.h b/src/include/postmaster/bgworker_internals.h
index 5e30525364b..9106a0ef3f0 100644
--- a/src/include/postmaster/bgworker_internals.h
+++ b/src/include/postmaster/bgworker_internals.h
@@ -55,6 +55,6 @@ extern void ForgetUnstartedBackgroundWorkers(void);
extern void ResetBackgroundWorkerCrashTimes(void);
/* Entry point for background worker processes */
-extern void BackgroundWorkerMain(void) pg_attribute_noreturn();
+extern void BackgroundWorkerMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
#endif /* BGWORKER_INTERNALS_H */
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index b52dc19ef0b..407f26e5302 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -27,8 +27,8 @@ extern PGDLLIMPORT int CheckPointTimeout;
extern PGDLLIMPORT int CheckPointWarning;
extern PGDLLIMPORT double CheckPointCompletionTarget;
-extern void BackgroundWriterMain(void) pg_attribute_noreturn();
-extern void CheckpointerMain(void) pg_attribute_noreturn();
+extern void BackgroundWriterMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
+extern void CheckpointerMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern void RequestCheckpoint(int flags);
extern void CheckpointWriteDelay(int flags, double progress);
diff --git a/src/include/postmaster/pgarch.h b/src/include/postmaster/pgarch.h
index 6135a9718d6..a7a417226b0 100644
--- a/src/include/postmaster/pgarch.h
+++ b/src/include/postmaster/pgarch.h
@@ -29,7 +29,7 @@
extern Size PgArchShmemSize(void);
extern void PgArchShmemInit(void);
extern bool PgArchCanRestart(void);
-extern void PgArchiverMain(void) pg_attribute_noreturn();
+extern void PgArchiverMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern void PgArchWakeup(void);
extern void PgArchForceDirScan(void);
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index 53f82bb5d43..76280cdf453 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -13,6 +13,8 @@
#ifndef _POSTMASTER_H
#define _POSTMASTER_H
+#include "miscadmin.h"
+
/* GUC options */
extern PGDLLIMPORT bool EnableSSL;
extern PGDLLIMPORT int SuperuserReservedConnections;
@@ -58,11 +60,9 @@ extern int MaxLivePostmasterChildren(void);
extern bool PostmasterMarkPIDForWorkerNotify(int);
-#ifdef EXEC_BACKEND
-
-extern pid_t postmaster_forkexec(int argc, char *argv[]);
-extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
+extern void BackendMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
+#ifdef EXEC_BACKEND
extern Size ShmemBackendArraySize(void);
extern void ShmemBackendArrayAllocation(void);
@@ -71,6 +71,23 @@ extern void pgwin32_register_deadchild_callback(HANDLE procHandle, DWORD procId)
#endif
#endif
+/* defined in globals.c */
+extern struct ClientSocket *MyClientSocket;
+
+/* in launch_backend.c */
+
+extern pid_t postmaster_child_launch(BackendType child_type, char *startup_data, size_t startup_data_len, struct ClientSocket *sock);
+
+#ifdef EXEC_BACKEND
+extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
+#endif
+
+#ifdef WIN32
+extern void pgwin32_register_deadchild_callback(HANDLE procHandle, DWORD procId);
+#endif
+
+const char *PostmasterChildName(BackendType child_type);
+
/*
* Note: MAX_BACKENDS is limited to 2^18-1 because that's the width reserved
* for buffer references in buf_internals.h. This limitation could be lifted
diff --git a/src/include/postmaster/startup.h b/src/include/postmaster/startup.h
index cf7a43e38cc..dde7ebde881 100644
--- a/src/include/postmaster/startup.h
+++ b/src/include/postmaster/startup.h
@@ -26,7 +26,7 @@
extern PGDLLIMPORT int log_startup_progress_interval;
extern void HandleStartupProcInterrupts(void);
-extern void StartupProcessMain(void) pg_attribute_noreturn();
+extern void StartupProcessMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern void PreRestoreCommand(void);
extern void PostRestoreCommand(void);
extern bool IsPromoteSignaled(void);
diff --git a/src/include/postmaster/syslogger.h b/src/include/postmaster/syslogger.h
index b26a2f29473..0f28ebcba55 100644
--- a/src/include/postmaster/syslogger.h
+++ b/src/include/postmaster/syslogger.h
@@ -86,9 +86,7 @@ extern int SysLogger_Start(void);
extern void write_syslogger_file(const char *buffer, int count, int destination);
-#ifdef EXEC_BACKEND
-extern void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
+extern void SysLoggerMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern bool CheckLogrotateSignal(void);
extern void RemoveLogrotateSignalFiles(void);
diff --git a/src/include/postmaster/walsummarizer.h b/src/include/postmaster/walsummarizer.h
index ecb4ea37fba..ad346d0c119 100644
--- a/src/include/postmaster/walsummarizer.h
+++ b/src/include/postmaster/walsummarizer.h
@@ -21,7 +21,7 @@ extern PGDLLIMPORT int wal_summary_keep_time;
extern Size WalSummarizerShmemSize(void);
extern void WalSummarizerShmemInit(void);
-extern void WalSummarizerMain(void) pg_attribute_noreturn();
+extern void WalSummarizerMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern void GetWalSummarizerState(TimeLineID *summarized_tli,
XLogRecPtr *summarized_lsn,
diff --git a/src/include/postmaster/walwriter.h b/src/include/postmaster/walwriter.h
index 60f23618cbd..5884d69fed1 100644
--- a/src/include/postmaster/walwriter.h
+++ b/src/include/postmaster/walwriter.h
@@ -18,6 +18,6 @@
extern PGDLLIMPORT int WalWriterDelay;
extern PGDLLIMPORT int WalWriterFlushAfter;
-extern void WalWriterMain(void) pg_attribute_noreturn();
+extern void WalWriterMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
#endif /* _WALWRITER_H */
diff --git a/src/include/replication/walreceiver.h b/src/include/replication/walreceiver.h
index b906bb5ce83..12f71fa99b0 100644
--- a/src/include/replication/walreceiver.h
+++ b/src/include/replication/walreceiver.h
@@ -483,7 +483,7 @@ walrcv_clear_result(WalRcvExecResult *walres)
}
/* prototypes for functions in walreceiver.c */
-extern void WalReceiverMain(void) pg_attribute_noreturn();
+extern void WalReceiverMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern void ProcessWalRcvInterrupts(void);
extern void WalRcvForceReply(void);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index da465602ea1..af7e4c7fa58 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -231,6 +231,7 @@ BY_HANDLE_FILE_INFORMATION
Backend
BackendId
BackendParameters
+BackendStartupInfo
BackendState
BackendType
BackgroundWorker
@@ -2133,6 +2134,7 @@ PortalStrategy
PostParseColumnRefHook
PostgresPollingStatusType
PostingItem
+PostmasterChildType
PreParseColumnRefHook
PredClass
PredIterInfo
@@ -3813,6 +3815,7 @@ substitute_actual_parameters_context
substitute_actual_srf_parameters_context
substitute_phv_relids_context
symbol
+syslogger_startup_data
tablespaceinfo
teSection
temp_tablespaces_extra
--
2.39.2
v12-0008-Move-code-for-backend-startup-to-separate-file.patchtext/x-patch; charset=UTF-8; name=v12-0008-Move-code-for-backend-startup-to-separate-file.patchDownload
From 805e846aa0274a14433076139d19eb514f9f915e Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Sun, 3 Mar 2024 20:38:44 +0200
Subject: [PATCH v12 8/8] Move code for backend startup to separate file
This is code that runs in the backend process after forking.
---
src/backend/postmaster/postmaster.c | 769 +--------------------------
src/backend/tcop/Makefile | 1 +
src/backend/tcop/backend_startup.c | 782 ++++++++++++++++++++++++++++
src/backend/tcop/meson.build | 1 +
src/include/postmaster/postmaster.h | 4 +
src/include/tcop/backend_startup.h | 41 ++
6 files changed, 834 insertions(+), 764 deletions(-)
create mode 100644 src/backend/tcop/backend_startup.c
create mode 100644 src/include/tcop/backend_startup.h
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 76aca6e48e1..a33a60b0d96 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -96,11 +96,9 @@
#include "common/file_utils.h"
#include "common/ip.h"
#include "common/pg_prng.h"
-#include "common/string.h"
#include "lib/ilist.h"
#include "libpq/auth.h"
#include "libpq/libpq.h"
-#include "libpq/pqformat.h"
#include "libpq/pqsignal.h"
#include "pg_getopt.h"
#include "pgstat.h"
@@ -119,13 +117,11 @@
#include "storage/ipc.h"
#include "storage/pmsignal.h"
#include "storage/proc.h"
+#include "tcop/backend_startup.h"
#include "tcop/tcopprot.h"
-#include "utils/builtins.h"
#include "utils/datetime.h"
#include "utils/memutils.h"
#include "utils/pidfile.h"
-#include "utils/ps_status.h"
-#include "utils/timeout.h"
#include "utils/timestamp.h"
#include "utils/varlena.h"
@@ -380,7 +376,7 @@ static WaitEventSet *pm_wait_set;
#ifdef USE_SSL
/* Set when and if SSL has been initialized properly */
-static bool LoadedSSL = false;
+bool LoadedSSL = false;
#endif
#ifdef USE_BONJOUR
@@ -402,9 +398,7 @@ static void process_pm_pmsignal(void);
static void process_pm_child_exit(void);
static void process_pm_reload_request(void);
static void process_pm_shutdown_request(void);
-static void process_startup_packet_die(SIGNAL_ARGS);
static void dummy_handler(SIGNAL_ARGS);
-static void StartupPacketTimeoutHandler(void);
static void CleanupBackend(int pid, int exitstatus);
static bool CleanupBackgroundWorker(int pid, int exitstatus);
static void HandleChildCrash(int pid, int exitstatus, const char *procname);
@@ -412,24 +406,9 @@ static void LogChildExit(int lev, const char *procname,
int pid, int exitstatus);
static void PostmasterStateMachine(void);
-/* Return value of canAcceptConnections() */
-typedef enum CAC_state
-{
- CAC_OK,
- CAC_STARTUP,
- CAC_SHUTDOWN,
- CAC_RECOVERY,
- CAC_NOTCONSISTENT,
- CAC_TOOMANY,
-} CAC_state;
-
-static void BackendInitialize(ClientSocket *client_sock, CAC_state cac);
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
-static int BackendStartup(ClientSocket *client_sock);
-static int ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done);
-static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
-static void processCancelRequest(Port *port, void *pkt);
+static int BackendStartup(ClientSocket *port);
static void report_fork_failure_to_client(ClientSocket *client_sock, int errnum);
static CAC_state canAcceptConnections(int backend_type);
static bool RandomCancelKey(int32 *cancel_key);
@@ -1848,412 +1827,14 @@ ServerLoop(void)
}
}
-/*
- * Read a client's startup packet and do something according to it.
- *
- * Returns STATUS_OK or STATUS_ERROR, or might call ereport(FATAL) and
- * not return at all.
- *
- * (Note that ereport(FATAL) stuff is sent to the client, so only use it
- * if that's what you want. Return STATUS_ERROR if you don't want to
- * send anything to the client, which would typically be appropriate
- * if we detect a communications failure.)
- *
- * Set ssl_done and/or gss_done when negotiation of an encrypted layer
- * (currently, TLS or GSSAPI) is completed. A successful negotiation of either
- * encryption layer sets both flags, but a rejected negotiation sets only the
- * flag for that layer, since the client may wish to try the other one. We
- * should make no assumption here about the order in which the client may make
- * requests.
- */
-static int
-ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
-{
- int32 len;
- char *buf;
- ProtocolVersion proto;
- MemoryContext oldcontext;
-
- pq_startmsgread();
-
- /*
- * Grab the first byte of the length word separately, so that we can tell
- * whether we have no data at all or an incomplete packet. (This might
- * sound inefficient, but it's not really, because of buffering in
- * pqcomm.c.)
- */
- if (pq_getbytes((char *) &len, 1) == EOF)
- {
- /*
- * If we get no data at all, don't clutter the log with a complaint;
- * such cases often occur for legitimate reasons. An example is that
- * we might be here after responding to NEGOTIATE_SSL_CODE, and if the
- * client didn't like our response, it'll probably just drop the
- * connection. Service-monitoring software also often just opens and
- * closes a connection without sending anything. (So do port
- * scanners, which may be less benign, but it's not really our job to
- * notice those.)
- */
- return STATUS_ERROR;
- }
-
- if (pq_getbytes(((char *) &len) + 1, 3) == EOF)
- {
- /* Got a partial length word, so bleat about that */
- if (!ssl_done && !gss_done)
- ereport(COMMERROR,
- (errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("incomplete startup packet")));
- return STATUS_ERROR;
- }
-
- len = pg_ntoh32(len);
- len -= 4;
-
- if (len < (int32) sizeof(ProtocolVersion) ||
- len > MAX_STARTUP_PACKET_LENGTH)
- {
- ereport(COMMERROR,
- (errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("invalid length of startup packet")));
- return STATUS_ERROR;
- }
-
- /*
- * Allocate space to hold the startup packet, plus one extra byte that's
- * initialized to be zero. This ensures we will have null termination of
- * all strings inside the packet.
- */
- buf = palloc(len + 1);
- buf[len] = '\0';
-
- if (pq_getbytes(buf, len) == EOF)
- {
- ereport(COMMERROR,
- (errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("incomplete startup packet")));
- return STATUS_ERROR;
- }
- pq_endmsgread();
-
- /*
- * The first field is either a protocol version number or a special
- * request code.
- */
- port->proto = proto = pg_ntoh32(*((ProtocolVersion *) buf));
-
- if (proto == CANCEL_REQUEST_CODE)
- {
- if (len != sizeof(CancelRequestPacket))
- {
- ereport(COMMERROR,
- (errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("invalid length of startup packet")));
- return STATUS_ERROR;
- }
- processCancelRequest(port, buf);
- /* Not really an error, but we don't want to proceed further */
- return STATUS_ERROR;
- }
-
- if (proto == NEGOTIATE_SSL_CODE && !ssl_done)
- {
- char SSLok;
-
-#ifdef USE_SSL
- /* No SSL when disabled or on Unix sockets */
- if (!LoadedSSL || port->laddr.addr.ss_family == AF_UNIX)
- SSLok = 'N';
- else
- SSLok = 'S'; /* Support for SSL */
-#else
- SSLok = 'N'; /* No support for SSL */
-#endif
-
-retry1:
- if (send(port->sock, &SSLok, 1, 0) != 1)
- {
- if (errno == EINTR)
- goto retry1; /* if interrupted, just retry */
- ereport(COMMERROR,
- (errcode_for_socket_access(),
- errmsg("failed to send SSL negotiation response: %m")));
- return STATUS_ERROR; /* close the connection */
- }
-
-#ifdef USE_SSL
- if (SSLok == 'S' && secure_open_server(port) == -1)
- return STATUS_ERROR;
-#endif
-
- /*
- * At this point we should have no data already buffered. If we do,
- * it was received before we performed the SSL handshake, so it wasn't
- * encrypted and indeed may have been injected by a man-in-the-middle.
- * We report this case to the client.
- */
- if (pq_buffer_has_data())
- ereport(FATAL,
- (errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("received unencrypted data after SSL request"),
- errdetail("This could be either a client-software bug or evidence of an attempted man-in-the-middle attack.")));
-
- /*
- * regular startup packet, cancel, etc packet should follow, but not
- * another SSL negotiation request, and a GSS request should only
- * follow if SSL was rejected (client may negotiate in either order)
- */
- return ProcessStartupPacket(port, true, SSLok == 'S');
- }
- else if (proto == NEGOTIATE_GSS_CODE && !gss_done)
- {
- char GSSok = 'N';
-
-#ifdef ENABLE_GSS
- /* No GSSAPI encryption when on Unix socket */
- if (port->laddr.addr.ss_family != AF_UNIX)
- GSSok = 'G';
-#endif
-
- while (send(port->sock, &GSSok, 1, 0) != 1)
- {
- if (errno == EINTR)
- continue;
- ereport(COMMERROR,
- (errcode_for_socket_access(),
- errmsg("failed to send GSSAPI negotiation response: %m")));
- return STATUS_ERROR; /* close the connection */
- }
-
-#ifdef ENABLE_GSS
- if (GSSok == 'G' && secure_open_gssapi(port) == -1)
- return STATUS_ERROR;
-#endif
-
- /*
- * At this point we should have no data already buffered. If we do,
- * it was received before we performed the GSS handshake, so it wasn't
- * encrypted and indeed may have been injected by a man-in-the-middle.
- * We report this case to the client.
- */
- if (pq_buffer_has_data())
- ereport(FATAL,
- (errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("received unencrypted data after GSSAPI encryption request"),
- errdetail("This could be either a client-software bug or evidence of an attempted man-in-the-middle attack.")));
-
- /*
- * regular startup packet, cancel, etc packet should follow, but not
- * another GSS negotiation request, and an SSL request should only
- * follow if GSS was rejected (client may negotiate in either order)
- */
- return ProcessStartupPacket(port, GSSok == 'G', true);
- }
-
- /* Could add additional special packet types here */
-
- /*
- * Set FrontendProtocol now so that ereport() knows what format to send if
- * we fail during startup.
- */
- FrontendProtocol = proto;
-
- /* Check that the major protocol version is in range. */
- if (PG_PROTOCOL_MAJOR(proto) < PG_PROTOCOL_MAJOR(PG_PROTOCOL_EARLIEST) ||
- PG_PROTOCOL_MAJOR(proto) > PG_PROTOCOL_MAJOR(PG_PROTOCOL_LATEST))
- ereport(FATAL,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("unsupported frontend protocol %u.%u: server supports %u.0 to %u.%u",
- PG_PROTOCOL_MAJOR(proto), PG_PROTOCOL_MINOR(proto),
- PG_PROTOCOL_MAJOR(PG_PROTOCOL_EARLIEST),
- PG_PROTOCOL_MAJOR(PG_PROTOCOL_LATEST),
- PG_PROTOCOL_MINOR(PG_PROTOCOL_LATEST))));
-
- /*
- * Now fetch parameters out of startup packet and save them into the Port
- * structure.
- */
- oldcontext = MemoryContextSwitchTo(TopMemoryContext);
-
- /* Handle protocol version 3 startup packet */
- {
- int32 offset = sizeof(ProtocolVersion);
- List *unrecognized_protocol_options = NIL;
-
- /*
- * Scan packet body for name/option pairs. We can assume any string
- * beginning within the packet body is null-terminated, thanks to
- * zeroing extra byte above.
- */
- port->guc_options = NIL;
-
- while (offset < len)
- {
- char *nameptr = buf + offset;
- int32 valoffset;
- char *valptr;
-
- if (*nameptr == '\0')
- break; /* found packet terminator */
- valoffset = offset + strlen(nameptr) + 1;
- if (valoffset >= len)
- break; /* missing value, will complain below */
- valptr = buf + valoffset;
-
- if (strcmp(nameptr, "database") == 0)
- port->database_name = pstrdup(valptr);
- else if (strcmp(nameptr, "user") == 0)
- port->user_name = pstrdup(valptr);
- else if (strcmp(nameptr, "options") == 0)
- port->cmdline_options = pstrdup(valptr);
- else if (strcmp(nameptr, "replication") == 0)
- {
- /*
- * Due to backward compatibility concerns the replication
- * parameter is a hybrid beast which allows the value to be
- * either boolean or the string 'database'. The latter
- * connects to a specific database which is e.g. required for
- * logical decoding while.
- */
- if (strcmp(valptr, "database") == 0)
- {
- am_walsender = true;
- am_db_walsender = true;
- }
- else if (!parse_bool(valptr, &am_walsender))
- ereport(FATAL,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("invalid value for parameter \"%s\": \"%s\"",
- "replication",
- valptr),
- errhint("Valid values are: \"false\", 0, \"true\", 1, \"database\".")));
- }
- else if (strncmp(nameptr, "_pq_.", 5) == 0)
- {
- /*
- * Any option beginning with _pq_. is reserved for use as a
- * protocol-level option, but at present no such options are
- * defined.
- */
- unrecognized_protocol_options =
- lappend(unrecognized_protocol_options, pstrdup(nameptr));
- }
- else
- {
- /* Assume it's a generic GUC option */
- port->guc_options = lappend(port->guc_options,
- pstrdup(nameptr));
- port->guc_options = lappend(port->guc_options,
- pstrdup(valptr));
-
- /*
- * Copy application_name to port if we come across it. This
- * is done so we can log the application_name in the
- * connection authorization message. Note that the GUC would
- * be used but we haven't gone through GUC setup yet.
- */
- if (strcmp(nameptr, "application_name") == 0)
- {
- port->application_name = pg_clean_ascii(valptr, 0);
- }
- }
- offset = valoffset + strlen(valptr) + 1;
- }
-
- /*
- * If we didn't find a packet terminator exactly at the end of the
- * given packet length, complain.
- */
- if (offset != len - 1)
- ereport(FATAL,
- (errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("invalid startup packet layout: expected terminator as last byte")));
-
- /*
- * If the client requested a newer protocol version or if the client
- * requested any protocol options we didn't recognize, let them know
- * the newest minor protocol version we do support and the names of
- * any unrecognized options.
- */
- if (PG_PROTOCOL_MINOR(proto) > PG_PROTOCOL_MINOR(PG_PROTOCOL_LATEST) ||
- unrecognized_protocol_options != NIL)
- SendNegotiateProtocolVersion(unrecognized_protocol_options);
- }
-
- /* Check a user name was given. */
- if (port->user_name == NULL || port->user_name[0] == '\0')
- ereport(FATAL,
- (errcode(ERRCODE_INVALID_AUTHORIZATION_SPECIFICATION),
- errmsg("no PostgreSQL user name specified in startup packet")));
-
- /* The database defaults to the user name. */
- if (port->database_name == NULL || port->database_name[0] == '\0')
- port->database_name = pstrdup(port->user_name);
-
- if (am_walsender)
- MyBackendType = B_WAL_SENDER;
- else
- MyBackendType = B_BACKEND;
-
- /*
- * Normal walsender backends, e.g. for streaming replication, are not
- * connected to a particular database. But walsenders used for logical
- * replication need to connect to a specific database. We allow streaming
- * replication commands to be issued even if connected to a database as it
- * can make sense to first make a basebackup and then stream changes
- * starting from that.
- */
- if (am_walsender && !am_db_walsender)
- port->database_name[0] = '\0';
-
- /*
- * Done filling the Port structure
- */
- MemoryContextSwitchTo(oldcontext);
-
- return STATUS_OK;
-}
-
-/*
- * Send a NegotiateProtocolVersion to the client. This lets the client know
- * that they have requested a newer minor protocol version than we are able
- * to speak. We'll speak the highest version we know about; the client can,
- * of course, abandon the connection if that's a problem.
- *
- * We also include in the response a list of protocol options we didn't
- * understand. This allows clients to include optional parameters that might
- * be present either in newer protocol versions or third-party protocol
- * extensions without fear of having to reconnect if those options are not
- * understood, while at the same time making certain that the client is aware
- * of which options were actually accepted.
- */
-static void
-SendNegotiateProtocolVersion(List *unrecognized_protocol_options)
-{
- StringInfoData buf;
- ListCell *lc;
-
- pq_beginmessage(&buf, PqMsg_NegotiateProtocolVersion);
- pq_sendint32(&buf, PG_PROTOCOL_LATEST);
- pq_sendint32(&buf, list_length(unrecognized_protocol_options));
- foreach(lc, unrecognized_protocol_options)
- pq_sendstring(&buf, lfirst(lc));
- pq_endmessage(&buf);
-
- /* no need to flush, some other message will follow */
-}
-
/*
* The client has sent a cancel request packet, not a normal
* start-a-new-connection packet. Perform the necessary processing.
* Nothing is sent back to the client.
*/
-static void
-processCancelRequest(Port *port, void *pkt)
+void
+processCancelRequest(int backendPID, int32 cancelAuthCode)
{
- CancelRequestPacket *canc = (CancelRequestPacket *) pkt;
- int backendPID;
- int32 cancelAuthCode;
Backend *bp;
#ifndef EXEC_BACKEND
@@ -2262,9 +1843,6 @@ processCancelRequest(Port *port, void *pkt)
int i;
#endif
- backendPID = (int) pg_ntoh32(canc->backendPID);
- cancelAuthCode = (int32) pg_ntoh32(canc->cancelAuthCode);
-
/*
* See if we have a matching backend. In the EXEC_BACKEND case, we can no
* longer access the postmaster's own backend list, and must rely on the
@@ -3956,12 +3534,6 @@ TerminateChildren(int signal)
signal_child(SlotSyncWorkerPID, signal);
}
-/* Information passed from postmaster to backend process */
-typedef struct BackendStartupInfo
-{
- CAC_state canAcceptConnections;
-} BackendStartupInfo;
-
/*
* BackendStartup -- start backend process
*
@@ -4086,307 +3658,6 @@ report_fork_failure_to_client(ClientSocket *client_sock, int errnum)
} while (rc < 0 && errno == EINTR);
}
-
-/*
- * BackendInitialize -- initialize an interactive (postmaster-child)
- * backend process, and collect the client's startup packet.
- *
- * returns: nothing. Will not return at all if there's any failure.
- *
- * Note: this code does not depend on having any access to shared memory.
- * Indeed, our approach to SIGTERM/timeout handling *requires* that
- * shared memory not have been touched yet; see comments within.
- * In the EXEC_BACKEND case, we are physically attached to shared memory
- * but have not yet set up most of our local pointers to shmem structures.
- */
-static void
-BackendInitialize(ClientSocket *client_sock, CAC_state cac)
-{
- int status;
- int ret;
- Port *port;
- char remote_host[NI_MAXHOST];
- char remote_port[NI_MAXSERV];
- StringInfoData ps_data;
- MemoryContext oldcontext;
-
- /*
- * Create the Port structure.
- *
- * The Port structure and all data structures attached to it are allocated
- * in TopMemoryContext, so that they survive into PostgresMain execution.
- * We need not worry about leaking this storage on failure, since we
- * aren't in the postmaster process anymore.
- */
- oldcontext = MemoryContextSwitchTo(TopMemoryContext);
- port = InitClientConnection(client_sock);
- MyProcPort = port;
- MemoryContextSwitchTo(oldcontext);
-
- /* Tell fd.c about the long-lived FD associated with the client_sock */
- ReserveExternalFD();
-
- /*
- * PreAuthDelay is a debugging aid for investigating problems in the
- * authentication cycle: it can be set in postgresql.conf to allow time to
- * attach to the newly-forked backend with a debugger. (See also
- * PostAuthDelay, which we allow clients to pass through PGOPTIONS, but it
- * is not honored until after authentication.)
- */
- if (PreAuthDelay > 0)
- pg_usleep(PreAuthDelay * 1000000L);
-
- /* This flag will remain set until InitPostgres finishes authentication */
- ClientAuthInProgress = true; /* limit visibility of log messages */
-
- /* set these to empty in case they are needed before we set them up */
- port->remote_host = "";
- port->remote_port = "";
-
- /*
- * Initialize libpq and enable reporting of ereport errors to the client.
- * Must do this now because authentication uses libpq to send messages.
- */
- pq_init(); /* initialize libpq to talk to client */
- whereToSendOutput = DestRemote; /* now safe to ereport to client */
-
- /*
- * We arrange to do _exit(1) if we receive SIGTERM or timeout while trying
- * to collect the startup packet; while SIGQUIT results in _exit(2).
- * Otherwise the postmaster cannot shutdown the database FAST or IMMED
- * cleanly if a buggy client fails to send the packet promptly.
- *
- * Exiting with _exit(1) is only possible because we have not yet touched
- * shared memory; therefore no outside-the-process state needs to get
- * cleaned up.
- */
- pqsignal(SIGTERM, process_startup_packet_die);
- /* SIGQUIT handler was already set up by InitPostmasterChild */
- InitializeTimeouts(); /* establishes SIGALRM handler */
- sigprocmask(SIG_SETMASK, &StartupBlockSig, NULL);
-
- /*
- * Get the remote host name and port for logging and status display.
- */
- remote_host[0] = '\0';
- remote_port[0] = '\0';
- if ((ret = pg_getnameinfo_all(&port->raddr.addr, port->raddr.salen,
- remote_host, sizeof(remote_host),
- remote_port, sizeof(remote_port),
- (log_hostname ? 0 : NI_NUMERICHOST) | NI_NUMERICSERV)) != 0)
- ereport(WARNING,
- (errmsg_internal("pg_getnameinfo_all() failed: %s",
- gai_strerror(ret))));
-
- /*
- * Save remote_host and remote_port in port structure (after this, they
- * will appear in log_line_prefix data for log messages).
- */
- oldcontext = MemoryContextSwitchTo(TopMemoryContext);
- port->remote_host = pstrdup(remote_host);
- port->remote_port = pstrdup(remote_port);
-
- /* And now we can issue the Log_connections message, if wanted */
- if (Log_connections)
- {
- if (remote_port[0])
- ereport(LOG,
- (errmsg("connection received: host=%s port=%s",
- remote_host,
- remote_port)));
- else
- ereport(LOG,
- (errmsg("connection received: host=%s",
- remote_host)));
- }
-
- /*
- * If we did a reverse lookup to name, we might as well save the results
- * rather than possibly repeating the lookup during authentication.
- *
- * Note that we don't want to specify NI_NAMEREQD above, because then we'd
- * get nothing useful for a client without an rDNS entry. Therefore, we
- * must check whether we got a numeric IPv4 or IPv6 address, and not save
- * it into remote_hostname if so. (This test is conservative and might
- * sometimes classify a hostname as numeric, but an error in that
- * direction is safe; it only results in a possible extra lookup.)
- */
- if (log_hostname &&
- ret == 0 &&
- strspn(remote_host, "0123456789.") < strlen(remote_host) &&
- strspn(remote_host, "0123456789ABCDEFabcdef:") < strlen(remote_host))
- {
- port->remote_hostname = pstrdup(remote_host);
- }
- MemoryContextSwitchTo(oldcontext);
-
- /*
- * Ready to begin client interaction. We will give up and _exit(1) after
- * a time delay, so that a broken client can't hog a connection
- * indefinitely. PreAuthDelay and any DNS interactions above don't count
- * against the time limit.
- *
- * Note: AuthenticationTimeout is applied here while waiting for the
- * startup packet, and then again in InitPostgres for the duration of any
- * authentication operations. So a hostile client could tie up the
- * process for nearly twice AuthenticationTimeout before we kick him off.
- *
- * Note: because PostgresMain will call InitializeTimeouts again, the
- * registration of STARTUP_PACKET_TIMEOUT will be lost. This is okay
- * since we never use it again after this function.
- */
- RegisterTimeout(STARTUP_PACKET_TIMEOUT, StartupPacketTimeoutHandler);
- enable_timeout_after(STARTUP_PACKET_TIMEOUT, AuthenticationTimeout * 1000);
-
- /*
- * Receive the startup packet (which might turn out to be a cancel request
- * packet).
- */
- status = ProcessStartupPacket(port, false, false);
-
- /*
- * If we're going to reject the connection due to database state, say so
- * now instead of wasting cycles on an authentication exchange. (This also
- * allows a pg_ping utility to be written.)
- */
- if (status == STATUS_OK)
- {
- switch (cac)
- {
- case CAC_STARTUP:
- ereport(FATAL,
- (errcode(ERRCODE_CANNOT_CONNECT_NOW),
- errmsg("the database system is starting up")));
- break;
- case CAC_NOTCONSISTENT:
- if (EnableHotStandby)
- ereport(FATAL,
- (errcode(ERRCODE_CANNOT_CONNECT_NOW),
- errmsg("the database system is not yet accepting connections"),
- errdetail("Consistent recovery state has not been yet reached.")));
- else
- ereport(FATAL,
- (errcode(ERRCODE_CANNOT_CONNECT_NOW),
- errmsg("the database system is not accepting connections"),
- errdetail("Hot standby mode is disabled.")));
- break;
- case CAC_SHUTDOWN:
- ereport(FATAL,
- (errcode(ERRCODE_CANNOT_CONNECT_NOW),
- errmsg("the database system is shutting down")));
- break;
- case CAC_RECOVERY:
- ereport(FATAL,
- (errcode(ERRCODE_CANNOT_CONNECT_NOW),
- errmsg("the database system is in recovery mode")));
- break;
- case CAC_TOOMANY:
- ereport(FATAL,
- (errcode(ERRCODE_TOO_MANY_CONNECTIONS),
- errmsg("sorry, too many clients already")));
- break;
- case CAC_OK:
- break;
- }
- }
-
- /*
- * Disable the timeout, and prevent SIGTERM again.
- */
- disable_timeout(STARTUP_PACKET_TIMEOUT, false);
- sigprocmask(SIG_SETMASK, &BlockSig, NULL);
-
- /*
- * As a safety check that nothing in startup has yet performed
- * shared-memory modifications that would need to be undone if we had
- * exited through SIGTERM or timeout above, check that no on_shmem_exit
- * handlers have been registered yet. (This isn't terribly bulletproof,
- * since someone might misuse an on_proc_exit handler for shmem cleanup,
- * but it's a cheap and helpful check. We cannot disallow on_proc_exit
- * handlers unfortunately, since pq_init() already registered one.)
- */
- check_on_shmem_exit_lists_are_empty();
-
- /*
- * Stop here if it was bad or a cancel packet. ProcessStartupPacket
- * already did any appropriate error reporting.
- */
- if (status != STATUS_OK)
- proc_exit(0);
-
- /*
- * Now that we have the user and database name, we can set the process
- * title for ps. It's good to do this as early as possible in startup.
- */
- initStringInfo(&ps_data);
- if (am_walsender)
- appendStringInfo(&ps_data, "%s ", GetBackendTypeDesc(B_WAL_SENDER));
- appendStringInfo(&ps_data, "%s ", port->user_name);
- if (port->database_name[0] != '\0')
- appendStringInfo(&ps_data, "%s ", port->database_name);
- appendStringInfoString(&ps_data, port->remote_host);
- if (port->remote_port[0] != '\0')
- appendStringInfo(&ps_data, "(%s)", port->remote_port);
-
- init_ps_display(ps_data.data);
- pfree(ps_data.data);
-
- set_ps_display("initializing");
-}
-
-void
-BackendMain(char *startup_data, size_t startup_data_len)
-{
- BackendStartupInfo *info = (BackendStartupInfo *) startup_data;
-
- Assert(startup_data_len == sizeof(BackendStartupInfo));
- Assert(MyClientSocket != NULL);
-
-#ifdef EXEC_BACKEND
-
- /*
- * Need to reinitialize the SSL library in the backend, since the context
- * structures contain function pointers and cannot be passed through the
- * parameter file.
- *
- * If for some reason reload fails (maybe the user installed broken key
- * files), soldier on without SSL; that's better than all connections
- * becoming impossible.
- *
- * XXX should we do this in all child processes? For the moment it's
- * enough to do it in backend children.
- */
-#ifdef USE_SSL
- if (EnableSSL)
- {
- if (secure_initialize(false) == 0)
- LoadedSSL = true;
- else
- ereport(LOG,
- (errmsg("SSL configuration could not be loaded in child process")));
- }
-#endif
-#endif
-
- /* Perform additional initialization and collect startup packet */
- BackendInitialize(MyClientSocket, info->canAcceptConnections);
-
- /*
- * Create a per-backend PGPROC struct in shared memory. We must do this
- * before we can use LWLocks or access any shared memory.
- */
- InitProcess();
-
- /*
- * Make sure we aren't in PostmasterContext anymore. (We can't delete it
- * just yet, though, because InitPostgres will need the HBA data.)
- */
- MemoryContextSwitchTo(TopMemoryContext);
-
- PostgresMain(MyProcPort->database_name, MyProcPort->user_name);
-}
-
-
/*
* ExitPostmaster -- cleanup
*
@@ -4575,25 +3846,6 @@ process_pm_pmsignal(void)
}
}
-/*
- * SIGTERM while processing startup packet.
- *
- * Running proc_exit() from a signal handler would be quite unsafe.
- * However, since we have not yet touched shared memory, we can just
- * pull the plug and exit without running any atexit handlers.
- *
- * One might be tempted to try to send a message, or log one, indicating
- * why we are disconnecting. However, that would be quite unsafe in itself.
- * Also, it seems undesirable to provide clues about the database's state
- * to a client that has not yet completed authentication, or even sent us
- * a startup packet.
- */
-static void
-process_startup_packet_die(SIGNAL_ARGS)
-{
- _exit(1);
-}
-
/*
* Dummy signal handler
*
@@ -4608,17 +3860,6 @@ dummy_handler(SIGNAL_ARGS)
{
}
-/*
- * Timeout while processing startup packet.
- * As for process_startup_packet_die(), we exit via _exit(1).
- */
-static void
-StartupPacketTimeoutHandler(void)
-{
- _exit(1);
-}
-
-
/*
* Generate a random cancel key.
*/
diff --git a/src/backend/tcop/Makefile b/src/backend/tcop/Makefile
index f662a7dd1cf..9119667345a 100644
--- a/src/backend/tcop/Makefile
+++ b/src/backend/tcop/Makefile
@@ -13,6 +13,7 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
OBJS = \
+ backend_startup.o \
cmdtag.o \
dest.o \
fastpath.o \
diff --git a/src/backend/tcop/backend_startup.c b/src/backend/tcop/backend_startup.c
new file mode 100644
index 00000000000..cacff79c236
--- /dev/null
+++ b/src/backend/tcop/backend_startup.c
@@ -0,0 +1,782 @@
+/*-------------------------------------------------------------------------
+ *
+ * backend_startup.c
+ * Backend startup code
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/tcop/backend_startup.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include <unistd.h>
+
+#include "access/xlog.h"
+#include "common/ip.h"
+#include "common/string.h"
+#include "libpq/libpq.h"
+#include "libpq/libpq-be.h"
+#include "libpq/pqformat.h"
+#include "libpq/pqsignal.h"
+#include "miscadmin.h"
+#include "postmaster/postmaster.h"
+#include "replication/walsender.h"
+#include "storage/fd.h"
+#include "storage/ipc.h"
+#include "storage/proc.h"
+#include "tcop/backend_startup.h"
+#include "tcop/tcopprot.h"
+#include "utils/builtins.h"
+#include "utils/memutils.h"
+#include "utils/ps_status.h"
+#include "utils/timeout.h"
+
+static void BackendInitialize(ClientSocket *client_sock, CAC_state cac);
+static int ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done);
+static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
+static void process_startup_packet_die(SIGNAL_ARGS);
+static void StartupPacketTimeoutHandler(void);
+
+/*
+ * Entry point for a new backend process.
+ *
+ * Initialize the connection, read the startup packet.
+ */
+void
+BackendMain(char *startup_data, size_t startup_data_len)
+{
+ BackendStartupInfo *info = (BackendStartupInfo *) startup_data;
+
+ Assert(startup_data_len == sizeof(BackendStartupInfo));
+ Assert(MyClientSocket != NULL);
+
+#ifdef EXEC_BACKEND
+
+ /*
+ * Need to reinitialize the SSL library in the backend, since the context
+ * structures contain function pointers and cannot be passed through the
+ * parameter file.
+ *
+ * If for some reason reload fails (maybe the user installed broken key
+ * files), soldier on without SSL; that's better than all connections
+ * becoming impossible.
+ *
+ * XXX should we do this in all child processes? For the moment it's
+ * enough to do it in backend children. XXX good question indeed
+ */
+#ifdef USE_SSL
+ if (EnableSSL)
+ {
+ if (secure_initialize(false) == 0)
+ LoadedSSL = true;
+ else
+ ereport(LOG,
+ (errmsg("SSL configuration could not be loaded in child process")));
+ }
+#endif
+#endif
+
+ /* Perform additional initialization and collect startup packet */
+ BackendInitialize(MyClientSocket, info->canAcceptConnections);
+
+ /*
+ * Create a per-backend PGPROC struct in shared memory. We must do this
+ * before we can use LWLocks or access any shared memory.
+ */
+ InitProcess();
+
+ /*
+ * Make sure we aren't in PostmasterContext anymore. (We can't delete it
+ * just yet, though, because InitPostgres will need the HBA data.)
+ */
+ MemoryContextSwitchTo(TopMemoryContext);
+
+ PostgresMain(MyProcPort->database_name, MyProcPort->user_name);
+}
+
+
+/*
+ * BackendInitialize -- initialize an interactive (postmaster-child)
+ * backend process, and collect the client's startup packet.
+ *
+ * returns: nothing. Will not return at all if there's any failure.
+ *
+ * Note: this code does not depend on having any access to shared memory.
+ * Indeed, our approach to SIGTERM/timeout handling *requires* that
+ * shared memory not have been touched yet; see comments within.
+ * In the EXEC_BACKEND case, we are physically attached to shared memory
+ * but have not yet set up most of our local pointers to shmem structures.
+ */
+static void
+BackendInitialize(ClientSocket *client_sock, CAC_state cac)
+{
+ int status;
+ int ret;
+ Port *port;
+ char remote_host[NI_MAXHOST];
+ char remote_port[NI_MAXSERV];
+ StringInfoData ps_data;
+ MemoryContext oldcontext;
+
+ /*
+ * Create the Port structure.
+ *
+ * The Port structure and all data structures attached to it are allocated
+ * in TopMemoryContext, so that they survive into PostgresMain execution.
+ * We need not worry about leaking this storage on failure, since we
+ * aren't in the postmaster process anymore.
+ */
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
+ port = InitClientConnection(client_sock);
+ MyProcPort = port;
+ MemoryContextSwitchTo(oldcontext);
+
+ /* Tell fd.c about the long-lived FD associated with the client_sock */
+ ReserveExternalFD();
+
+ /*
+ * PreAuthDelay is a debugging aid for investigating problems in the
+ * authentication cycle: it can be set in postgresql.conf to allow time to
+ * attach to the newly-forked backend with a debugger. (See also
+ * PostAuthDelay, which we allow clients to pass through PGOPTIONS, but it
+ * is not honored until after authentication.)
+ */
+ if (PreAuthDelay > 0)
+ pg_usleep(PreAuthDelay * 1000000L);
+
+ /* This flag will remain set until InitPostgres finishes authentication */
+ ClientAuthInProgress = true; /* limit visibility of log messages */
+
+ /* set these to empty in case they are needed before we set them up */
+ port->remote_host = "";
+ port->remote_port = "";
+
+ /*
+ * Initialize libpq and enable reporting of ereport errors to the client.
+ * Must do this now because authentication uses libpq to send messages.
+ */
+ pq_init(); /* initialize libpq to talk to client */
+ whereToSendOutput = DestRemote; /* now safe to ereport to client */
+
+ /*
+ * We arrange to do _exit(1) if we receive SIGTERM or timeout while trying
+ * to collect the startup packet; while SIGQUIT results in _exit(2).
+ * Otherwise the postmaster cannot shutdown the database FAST or IMMED
+ * cleanly if a buggy client fails to send the packet promptly.
+ *
+ * Exiting with _exit(1) is only possible because we have not yet touched
+ * shared memory; therefore no outside-the-process state needs to get
+ * cleaned up.
+ */
+ pqsignal(SIGTERM, process_startup_packet_die);
+ /* SIGQUIT handler was already set up by InitPostmasterChild */
+ InitializeTimeouts(); /* establishes SIGALRM handler */
+ sigprocmask(SIG_SETMASK, &StartupBlockSig, NULL);
+
+ /*
+ * Get the remote host name and port for logging and status display.
+ */
+ remote_host[0] = '\0';
+ remote_port[0] = '\0';
+ if ((ret = pg_getnameinfo_all(&port->raddr.addr, port->raddr.salen,
+ remote_host, sizeof(remote_host),
+ remote_port, sizeof(remote_port),
+ (log_hostname ? 0 : NI_NUMERICHOST) | NI_NUMERICSERV)) != 0)
+ ereport(WARNING,
+ (errmsg_internal("pg_getnameinfo_all() failed: %s",
+ gai_strerror(ret))));
+
+ /*
+ * Save remote_host and remote_port in port structure (after this, they
+ * will appear in log_line_prefix data for log messages).
+ */
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
+ port->remote_host = pstrdup(remote_host);
+ port->remote_port = pstrdup(remote_port);
+
+ /* And now we can issue the Log_connections message, if wanted */
+ if (Log_connections)
+ {
+ if (remote_port[0])
+ ereport(LOG,
+ (errmsg("connection received: host=%s port=%s",
+ remote_host,
+ remote_port)));
+ else
+ ereport(LOG,
+ (errmsg("connection received: host=%s",
+ remote_host)));
+ }
+
+ /*
+ * If we did a reverse lookup to name, we might as well save the results
+ * rather than possibly repeating the lookup during authentication.
+ *
+ * Note that we don't want to specify NI_NAMEREQD above, because then we'd
+ * get nothing useful for a client without an rDNS entry. Therefore, we
+ * must check whether we got a numeric IPv4 or IPv6 address, and not save
+ * it into remote_hostname if so. (This test is conservative and might
+ * sometimes classify a hostname as numeric, but an error in that
+ * direction is safe; it only results in a possible extra lookup.)
+ */
+ if (log_hostname &&
+ ret == 0 &&
+ strspn(remote_host, "0123456789.") < strlen(remote_host) &&
+ strspn(remote_host, "0123456789ABCDEFabcdef:") < strlen(remote_host))
+ {
+ port->remote_hostname = pstrdup(remote_host);
+ }
+ MemoryContextSwitchTo(oldcontext);
+
+ /*
+ * Ready to begin client interaction. We will give up and _exit(1) after
+ * a time delay, so that a broken client can't hog a connection
+ * indefinitely. PreAuthDelay and any DNS interactions above don't count
+ * against the time limit.
+ *
+ * Note: AuthenticationTimeout is applied here while waiting for the
+ * startup packet, and then again in InitPostgres for the duration of any
+ * authentication operations. So a hostile client could tie up the
+ * process for nearly twice AuthenticationTimeout before we kick him off.
+ *
+ * Note: because PostgresMain will call InitializeTimeouts again, the
+ * registration of STARTUP_PACKET_TIMEOUT will be lost. This is okay
+ * since we never use it again after this function.
+ */
+ RegisterTimeout(STARTUP_PACKET_TIMEOUT, StartupPacketTimeoutHandler);
+ enable_timeout_after(STARTUP_PACKET_TIMEOUT, AuthenticationTimeout * 1000);
+
+ /*
+ * Receive the startup packet (which might turn out to be a cancel request
+ * packet).
+ */
+ status = ProcessStartupPacket(port, false, false);
+
+ /*
+ * If we're going to reject the connection due to database state, say so
+ * now instead of wasting cycles on an authentication exchange. (This also
+ * allows a pg_ping utility to be written.)
+ */
+ if (status == STATUS_OK)
+ {
+ switch (cac)
+ {
+ case CAC_STARTUP:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is starting up")));
+ break;
+ case CAC_NOTCONSISTENT:
+ if (EnableHotStandby)
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is not yet accepting connections"),
+ errdetail("Consistent recovery state has not been yet reached.")));
+ else
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is not accepting connections"),
+ errdetail("Hot standby mode is disabled.")));
+ break;
+ case CAC_SHUTDOWN:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is shutting down")));
+ break;
+ case CAC_RECOVERY:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is in recovery mode")));
+ break;
+ case CAC_TOOMANY:
+ ereport(FATAL,
+ (errcode(ERRCODE_TOO_MANY_CONNECTIONS),
+ errmsg("sorry, too many clients already")));
+ break;
+ case CAC_OK:
+ break;
+ }
+ }
+
+ /*
+ * Disable the timeout, and prevent SIGTERM again.
+ */
+ disable_timeout(STARTUP_PACKET_TIMEOUT, false);
+ sigprocmask(SIG_SETMASK, &BlockSig, NULL);
+
+ /*
+ * As a safety check that nothing in startup has yet performed
+ * shared-memory modifications that would need to be undone if we had
+ * exited through SIGTERM or timeout above, check that no on_shmem_exit
+ * handlers have been registered yet. (This isn't terribly bulletproof,
+ * since someone might misuse an on_proc_exit handler for shmem cleanup,
+ * but it's a cheap and helpful check. We cannot disallow on_proc_exit
+ * handlers unfortunately, since pq_init() already registered one.)
+ */
+ check_on_shmem_exit_lists_are_empty();
+
+ /*
+ * Stop here if it was bad or a cancel packet. ProcessStartupPacket
+ * already did any appropriate error reporting.
+ */
+ if (status != STATUS_OK)
+ proc_exit(0);
+
+ /*
+ * Now that we have the user and database name, we can set the process
+ * title for ps. It's good to do this as early as possible in startup.
+ */
+ initStringInfo(&ps_data);
+ if (am_walsender)
+ appendStringInfo(&ps_data, "%s ", GetBackendTypeDesc(B_WAL_SENDER));
+ appendStringInfo(&ps_data, "%s ", port->user_name);
+ if (port->database_name[0] != '\0')
+ appendStringInfo(&ps_data, "%s ", port->database_name);
+ appendStringInfoString(&ps_data, port->remote_host);
+ if (port->remote_port[0] != '\0')
+ appendStringInfo(&ps_data, "(%s)", port->remote_port);
+
+ init_ps_display(ps_data.data);
+ pfree(ps_data.data);
+
+ set_ps_display("initializing");
+}
+
+/*
+ * Read a client's startup packet and do something according to it.
+ *
+ * Returns STATUS_OK or STATUS_ERROR, or might call ereport(FATAL) and
+ * not return at all.
+ *
+ * (Note that ereport(FATAL) stuff is sent to the client, so only use it
+ * if that's what you want. Return STATUS_ERROR if you don't want to
+ * send anything to the client, which would typically be appropriate
+ * if we detect a communications failure.)
+ *
+ * Set ssl_done and/or gss_done when negotiation of an encrypted layer
+ * (currently, TLS or GSSAPI) is completed. A successful negotiation of either
+ * encryption layer sets both flags, but a rejected negotiation sets only the
+ * flag for that layer, since the client may wish to try the other one. We
+ * should make no assumption here about the order in which the client may make
+ * requests.
+ */
+static int
+ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
+{
+ int32 len;
+ char *buf;
+ ProtocolVersion proto;
+ MemoryContext oldcontext;
+
+ pq_startmsgread();
+
+ /*
+ * Grab the first byte of the length word separately, so that we can tell
+ * whether we have no data at all or an incomplete packet. (This might
+ * sound inefficient, but it's not really, because of buffering in
+ * pqcomm.c.)
+ */
+ if (pq_getbytes((char *) &len, 1) == EOF)
+ {
+ /*
+ * If we get no data at all, don't clutter the log with a complaint;
+ * such cases often occur for legitimate reasons. An example is that
+ * we might be here after responding to NEGOTIATE_SSL_CODE, and if the
+ * client didn't like our response, it'll probably just drop the
+ * connection. Service-monitoring software also often just opens and
+ * closes a connection without sending anything. (So do port
+ * scanners, which may be less benign, but it's not really our job to
+ * notice those.)
+ */
+ return STATUS_ERROR;
+ }
+
+ if (pq_getbytes(((char *) &len) + 1, 3) == EOF)
+ {
+ /* Got a partial length word, so bleat about that */
+ if (!ssl_done && !gss_done)
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("incomplete startup packet")));
+ return STATUS_ERROR;
+ }
+
+ len = pg_ntoh32(len);
+ len -= 4;
+
+ if (len < (int32) sizeof(ProtocolVersion) ||
+ len > MAX_STARTUP_PACKET_LENGTH)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("invalid length of startup packet")));
+ return STATUS_ERROR;
+ }
+
+ /*
+ * Allocate space to hold the startup packet, plus one extra byte that's
+ * initialized to be zero. This ensures we will have null termination of
+ * all strings inside the packet.
+ */
+ buf = palloc(len + 1);
+ buf[len] = '\0';
+
+ if (pq_getbytes(buf, len) == EOF)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("incomplete startup packet")));
+ return STATUS_ERROR;
+ }
+ pq_endmsgread();
+
+ /*
+ * The first field is either a protocol version number or a special
+ * request code.
+ */
+ port->proto = proto = pg_ntoh32(*((ProtocolVersion *) buf));
+
+ if (proto == CANCEL_REQUEST_CODE)
+ {
+ CancelRequestPacket *canc;
+ int backendPID;
+ int32 cancelAuthCode;
+
+ if (len != sizeof(CancelRequestPacket))
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("invalid length of startup packet")));
+ return STATUS_ERROR;
+ }
+ canc = (CancelRequestPacket *) buf;
+ backendPID = (int) pg_ntoh32(canc->backendPID);
+ cancelAuthCode = (int32) pg_ntoh32(canc->cancelAuthCode);
+
+ processCancelRequest(backendPID, cancelAuthCode);
+ /* Not really an error, but we don't want to proceed further */
+ return STATUS_ERROR;
+ }
+
+ if (proto == NEGOTIATE_SSL_CODE && !ssl_done)
+ {
+ char SSLok;
+
+#ifdef USE_SSL
+ /* No SSL when disabled or on Unix sockets */
+ if (!LoadedSSL || port->laddr.addr.ss_family == AF_UNIX)
+ SSLok = 'N';
+ else
+ SSLok = 'S'; /* Support for SSL */
+#else
+ SSLok = 'N'; /* No support for SSL */
+#endif
+
+retry1:
+ if (send(port->sock, &SSLok, 1, 0) != 1)
+ {
+ if (errno == EINTR)
+ goto retry1; /* if interrupted, just retry */
+ ereport(COMMERROR,
+ (errcode_for_socket_access(),
+ errmsg("failed to send SSL negotiation response: %m")));
+ return STATUS_ERROR; /* close the connection */
+ }
+
+#ifdef USE_SSL
+ if (SSLok == 'S' && secure_open_server(port) == -1)
+ return STATUS_ERROR;
+#endif
+
+ /*
+ * At this point we should have no data already buffered. If we do,
+ * it was received before we performed the SSL handshake, so it wasn't
+ * encrypted and indeed may have been injected by a man-in-the-middle.
+ * We report this case to the client.
+ */
+ if (pq_buffer_has_data())
+ ereport(FATAL,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("received unencrypted data after SSL request"),
+ errdetail("This could be either a client-software bug or evidence of an attempted man-in-the-middle attack.")));
+
+ /*
+ * regular startup packet, cancel, etc packet should follow, but not
+ * another SSL negotiation request, and a GSS request should only
+ * follow if SSL was rejected (client may negotiate in either order)
+ */
+ return ProcessStartupPacket(port, true, SSLok == 'S');
+ }
+ else if (proto == NEGOTIATE_GSS_CODE && !gss_done)
+ {
+ char GSSok = 'N';
+
+#ifdef ENABLE_GSS
+ /* No GSSAPI encryption when on Unix socket */
+ if (port->laddr.addr.ss_family != AF_UNIX)
+ GSSok = 'G';
+#endif
+
+ while (send(port->sock, &GSSok, 1, 0) != 1)
+ {
+ if (errno == EINTR)
+ continue;
+ ereport(COMMERROR,
+ (errcode_for_socket_access(),
+ errmsg("failed to send GSSAPI negotiation response: %m")));
+ return STATUS_ERROR; /* close the connection */
+ }
+
+#ifdef ENABLE_GSS
+ if (GSSok == 'G' && secure_open_gssapi(port) == -1)
+ return STATUS_ERROR;
+#endif
+
+ /*
+ * At this point we should have no data already buffered. If we do,
+ * it was received before we performed the GSS handshake, so it wasn't
+ * encrypted and indeed may have been injected by a man-in-the-middle.
+ * We report this case to the client.
+ */
+ if (pq_buffer_has_data())
+ ereport(FATAL,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("received unencrypted data after GSSAPI encryption request"),
+ errdetail("This could be either a client-software bug or evidence of an attempted man-in-the-middle attack.")));
+
+ /*
+ * regular startup packet, cancel, etc packet should follow, but not
+ * another GSS negotiation request, and an SSL request should only
+ * follow if GSS was rejected (client may negotiate in either order)
+ */
+ return ProcessStartupPacket(port, GSSok == 'G', true);
+ }
+
+ /* Could add additional special packet types here */
+
+ /*
+ * Set FrontendProtocol now so that ereport() knows what format to send if
+ * we fail during startup.
+ */
+ FrontendProtocol = proto;
+
+ /* Check that the major protocol version is in range. */
+ if (PG_PROTOCOL_MAJOR(proto) < PG_PROTOCOL_MAJOR(PG_PROTOCOL_EARLIEST) ||
+ PG_PROTOCOL_MAJOR(proto) > PG_PROTOCOL_MAJOR(PG_PROTOCOL_LATEST))
+ ereport(FATAL,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("unsupported frontend protocol %u.%u: server supports %u.0 to %u.%u",
+ PG_PROTOCOL_MAJOR(proto), PG_PROTOCOL_MINOR(proto),
+ PG_PROTOCOL_MAJOR(PG_PROTOCOL_EARLIEST),
+ PG_PROTOCOL_MAJOR(PG_PROTOCOL_LATEST),
+ PG_PROTOCOL_MINOR(PG_PROTOCOL_LATEST))));
+
+ /*
+ * Now fetch parameters out of startup packet and save them into the Port
+ * structure.
+ */
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
+
+ /* Handle protocol version 3 startup packet */
+ {
+ int32 offset = sizeof(ProtocolVersion);
+ List *unrecognized_protocol_options = NIL;
+
+ /*
+ * Scan packet body for name/option pairs. We can assume any string
+ * beginning within the packet body is null-terminated, thanks to
+ * zeroing extra byte above.
+ */
+ port->guc_options = NIL;
+
+ while (offset < len)
+ {
+ char *nameptr = buf + offset;
+ int32 valoffset;
+ char *valptr;
+
+ if (*nameptr == '\0')
+ break; /* found packet terminator */
+ valoffset = offset + strlen(nameptr) + 1;
+ if (valoffset >= len)
+ break; /* missing value, will complain below */
+ valptr = buf + valoffset;
+
+ if (strcmp(nameptr, "database") == 0)
+ port->database_name = pstrdup(valptr);
+ else if (strcmp(nameptr, "user") == 0)
+ port->user_name = pstrdup(valptr);
+ else if (strcmp(nameptr, "options") == 0)
+ port->cmdline_options = pstrdup(valptr);
+ else if (strcmp(nameptr, "replication") == 0)
+ {
+ /*
+ * Due to backward compatibility concerns the replication
+ * parameter is a hybrid beast which allows the value to be
+ * either boolean or the string 'database'. The latter
+ * connects to a specific database which is e.g. required for
+ * logical decoding while.
+ */
+ if (strcmp(valptr, "database") == 0)
+ {
+ am_walsender = true;
+ am_db_walsender = true;
+ }
+ else if (!parse_bool(valptr, &am_walsender))
+ ereport(FATAL,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid value for parameter \"%s\": \"%s\"",
+ "replication",
+ valptr),
+ errhint("Valid values are: \"false\", 0, \"true\", 1, \"database\".")));
+ }
+ else if (strncmp(nameptr, "_pq_.", 5) == 0)
+ {
+ /*
+ * Any option beginning with _pq_. is reserved for use as a
+ * protocol-level option, but at present no such options are
+ * defined.
+ */
+ unrecognized_protocol_options =
+ lappend(unrecognized_protocol_options, pstrdup(nameptr));
+ }
+ else
+ {
+ /* Assume it's a generic GUC option */
+ port->guc_options = lappend(port->guc_options,
+ pstrdup(nameptr));
+ port->guc_options = lappend(port->guc_options,
+ pstrdup(valptr));
+
+ /*
+ * Copy application_name to port if we come across it. This
+ * is done so we can log the application_name in the
+ * connection authorization message. Note that the GUC would
+ * be used but we haven't gone through GUC setup yet.
+ */
+ if (strcmp(nameptr, "application_name") == 0)
+ {
+ port->application_name = pg_clean_ascii(valptr, 0);
+ }
+ }
+ offset = valoffset + strlen(valptr) + 1;
+ }
+
+ /*
+ * If we didn't find a packet terminator exactly at the end of the
+ * given packet length, complain.
+ */
+ if (offset != len - 1)
+ ereport(FATAL,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("invalid startup packet layout: expected terminator as last byte")));
+
+ /*
+ * If the client requested a newer protocol version or if the client
+ * requested any protocol options we didn't recognize, let them know
+ * the newest minor protocol version we do support and the names of
+ * any unrecognized options.
+ */
+ if (PG_PROTOCOL_MINOR(proto) > PG_PROTOCOL_MINOR(PG_PROTOCOL_LATEST) ||
+ unrecognized_protocol_options != NIL)
+ SendNegotiateProtocolVersion(unrecognized_protocol_options);
+ }
+
+ /* Check a user name was given. */
+ if (port->user_name == NULL || port->user_name[0] == '\0')
+ ereport(FATAL,
+ (errcode(ERRCODE_INVALID_AUTHORIZATION_SPECIFICATION),
+ errmsg("no PostgreSQL user name specified in startup packet")));
+
+ /* The database defaults to the user name. */
+ if (port->database_name == NULL || port->database_name[0] == '\0')
+ port->database_name = pstrdup(port->user_name);
+
+ if (am_walsender)
+ MyBackendType = B_WAL_SENDER;
+ else
+ MyBackendType = B_BACKEND;
+
+ /*
+ * Normal walsender backends, e.g. for streaming replication, are not
+ * connected to a particular database. But walsenders used for logical
+ * replication need to connect to a specific database. We allow streaming
+ * replication commands to be issued even if connected to a database as it
+ * can make sense to first make a basebackup and then stream changes
+ * starting from that.
+ */
+ if (am_walsender && !am_db_walsender)
+ port->database_name[0] = '\0';
+
+ /*
+ * Done filling the Port structure
+ */
+ MemoryContextSwitchTo(oldcontext);
+
+ return STATUS_OK;
+}
+
+/*
+ * Send a NegotiateProtocolVersion to the client. This lets the client know
+ * that they have requested a newer minor protocol version than we are able
+ * to speak. We'll speak the highest version we know about; the client can,
+ * of course, abandon the connection if that's a problem.
+ *
+ * We also include in the response a list of protocol options we didn't
+ * understand. This allows clients to include optional parameters that might
+ * be present either in newer protocol versions or third-party protocol
+ * extensions without fear of having to reconnect if those options are not
+ * understood, while at the same time making certain that the client is aware
+ * of which options were actually accepted.
+ */
+static void
+SendNegotiateProtocolVersion(List *unrecognized_protocol_options)
+{
+ StringInfoData buf;
+ ListCell *lc;
+
+ pq_beginmessage(&buf, PqMsg_NegotiateProtocolVersion);
+ pq_sendint32(&buf, PG_PROTOCOL_LATEST);
+ pq_sendint32(&buf, list_length(unrecognized_protocol_options));
+ foreach(lc, unrecognized_protocol_options)
+ pq_sendstring(&buf, lfirst(lc));
+ pq_endmessage(&buf);
+
+ /* no need to flush, some other message will follow */
+}
+
+
+/*
+ * SIGTERM while processing startup packet.
+ *
+ * Running proc_exit() from a signal handler would be quite unsafe.
+ * However, since we have not yet touched shared memory, we can just
+ * pull the plug and exit without running any atexit handlers.
+ *
+ * One might be tempted to try to send a message, or log one, indicating
+ * why we are disconnecting. However, that would be quite unsafe in itself.
+ * Also, it seems undesirable to provide clues about the database's state
+ * to a client that has not yet completed authentication, or even sent us
+ * a startup packet.
+ */
+static void
+process_startup_packet_die(SIGNAL_ARGS)
+{
+ _exit(1);
+}
+
+/*
+ * Timeout while processing startup packet.
+ * As for process_startup_packet_die(), we exit via _exit(1).
+ */
+static void
+StartupPacketTimeoutHandler(void)
+{
+ _exit(1);
+}
diff --git a/src/backend/tcop/meson.build b/src/backend/tcop/meson.build
index 6104f746f9e..19a97bbf55e 100644
--- a/src/backend/tcop/meson.build
+++ b/src/backend/tcop/meson.build
@@ -1,6 +1,7 @@
# Copyright (c) 2022-2024, PostgreSQL Global Development Group
backend_sources += files(
+ 'backend_startup.c',
'cmdtag.c',
'dest.c',
'fastpath.c',
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index 76280cdf453..c4c96d8d61b 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -52,6 +52,9 @@ extern PGDLLIMPORT int postmaster_alive_fds[2];
extern PGDLLIMPORT const char *progname;
+/* XXX: where does this belong? */
+extern bool LoadedSSL;
+
extern void PostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
extern void ClosePostmasterPorts(bool am_syslogger);
extern void InitProcessGlobals(void);
@@ -61,6 +64,7 @@ extern int MaxLivePostmasterChildren(void);
extern bool PostmasterMarkPIDForWorkerNotify(int);
extern void BackendMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
+extern void processCancelRequest(int backendPID, int32 cancelAuthCode);
#ifdef EXEC_BACKEND
extern Size ShmemBackendArraySize(void);
diff --git a/src/include/tcop/backend_startup.h b/src/include/tcop/backend_startup.h
new file mode 100644
index 00000000000..643aa7150b7
--- /dev/null
+++ b/src/include/tcop/backend_startup.h
@@ -0,0 +1,41 @@
+/*-------------------------------------------------------------------------
+ *
+ * backend_startup.h
+ * prototypes for backend_startup.c.
+ *
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/tcop/backend_startup.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef BACKEND_STARTUP_H
+#define BACKEND_STARTUP_H
+
+/*
+ * CAC_state is passed from postmaster to the backend process, to indicate
+ * whether the connection should be accepted, or if the process should just
+ * send an error to the client and close the cnnection. Note that the
+ * connection can fail for various reasons even if postmaster passed CAC_OK.
+ */
+typedef enum CAC_state
+{
+ CAC_OK,
+ CAC_STARTUP,
+ CAC_SHUTDOWN,
+ CAC_RECOVERY,
+ CAC_NOTCONSISTENT,
+ CAC_TOOMANY,
+} CAC_state;
+
+/* Information passed from postmaster to backend process in 'startup_data' */
+typedef struct BackendStartupInfo
+{
+ CAC_state canAcceptConnections;
+} BackendStartupInfo;
+
+extern void BackendMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
+
+#endif /* BACKEND_STARTUP_H */
--
2.39.2
Re: Refactoring backend fork+exec code
On Mon, Mar 4, 2024 at 1:40 AM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
On 22/02/2024 02:37, Heikki Linnakangas wrote:
Here's another patch version that does that. Yeah, I agree it's nicer in
the end.I'm pretty happy with this now. I'll read through these patches myself
again after sleeping over it and try to get this committed by the end of
the week, but another pair of eyes wouldn't hurt.And pushed. Thanks for the reviews!
I noticed that there are still three places in backend_status.c where
pgstat_get_beentry_by_backend_id() is referenced. I think we should
replace them with pgstat_get_beentry_by_proc_number().
Thanks
Richard
Re: Refactoring backend fork+exec code
On 05/03/2024 11:44, Richard Guo wrote:
I noticed that there are still three places in backend_status.c where
pgstat_get_beentry_by_backend_id() is referenced. I think we should
replace them with pgstat_get_beentry_by_proc_number().
Fixed, thanks!
--
Heikki Linnakangas
Neon (https://neon.tech)
Re: Refactoring backend fork+exec code
On Mon Mar 4, 2024 at 3:05 AM CST, Heikki Linnakangas wrote:
I've now completed many of the side-quests, here are the patches that
remain.The first three patches form a logical unit. They move the
initialization of the Port struct from postmaster to the backend
process. Currently, that work is split between the postmaster and the
backend process so that postmaster fills in the socket and some other
fields, and the backend process fills the rest after reading the startup
packet. With these patches, there is a new much smaller ClientSocket
struct that is passed from the postmaster to the child process, which
contains just the fields that postmaster initializes. The Port struct is
allocated in the child process. That makes the backend startup easier to
understand. I plan to commit those three patches next if there are no
objections.That leaves the rest of the patches. I think they're in pretty good
shape too, and I've gotten some review on those earlier and have
addressed the comments I got so far, but would still appreciate another
round of review.
- * *MyProcPort, because ConnCreate() allocated that space with malloc() - * ... else we'd need to copy the Port data first. Also, subsidiary data - * such as the username isn't lost either; see ProcessStartupPacket(). + * *MyProcPort, because that space is allocated in stack ... else we'd + * need to copy the Port data first. Also, subsidiary data such as the + * username isn't lost either; see ProcessStartupPacket().
s/allocated in/allocated on the
The first 3 patches seem good to go, in my opinion.
@@ -225,14 +331,13 @@ internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundW
return -1;
}- /* Make sure caller set up argv properly */ - Assert(argc >= 3); - Assert(argv[argc] == NULL); - Assert(strncmp(argv[1], "--fork", 6) == 0); - Assert(argv[2] == NULL); - - /* Insert temp file name after --fork argument */ + /* set up argv properly */ + argv[0] = "postgres"; + snprintf(forkav, MAXPGPATH, "--forkchild=%s", child_kind); + argv[1] = forkav; + /* Insert temp file name after --forkchild argument */ argv[2] = tmpfilename; + argv[3] = NULL;
Should we use postgres_exec_path instead of the naked "postgres" here?
+ /* in postmaster, fork failed ... */ + ereport(LOG, + (errmsg("could not fork worker process: %m"))); + /* undo what assign_backendlist_entry did */ + ReleasePostmasterChildSlot(rw->rw_child_slot); + rw->rw_child_slot = 0; + pfree(rw->rw_backend); + rw->rw_backend = NULL; + /* mark entry as crashed, so we'll try again later */ + rw->rw_crashed_at = GetCurrentTimestamp(); + return false;
I think the error message should include the word "background." It would
be more consistent with the log message above it.
+typedef struct +{ + int syslogFile; + int csvlogFile; + int jsonlogFile; +} syslogger_startup_data;
It would be nice if all of these startup data structs were named
similarly. For instance, a previous one was BackendStartupInfo. It would
help with greppability.
I noticed there were a few XXX comments left that you created. I'll
highlight them here for more visibility.
+/* XXX: where does this belong? */
+extern bool LoadedSSL;
Perhaps near the My* variables or maybe in the Port struct?
+#ifdef EXEC_BACKEND + + /* + * Need to reinitialize the SSL library in the backend, since the context + * structures contain function pointers and cannot be passed through the + * parameter file. + * + * If for some reason reload fails (maybe the user installed broken key + * files), soldier on without SSL; that's better than all connections + * becoming impossible. + * + * XXX should we do this in all child processes? For the moment it's + * enough to do it in backend children. XXX good question indeed + */ +#ifdef USE_SSL + if (EnableSSL) + { + if (secure_initialize(false) == 0) + LoadedSSL = true; + else + ereport(LOG, + (errmsg("SSL configuration could not be loaded in child process"))); + } +#endif +#endif
Here you added the "good question indeed." I am not sure what the best
answer is either! :)
+ /* XXX: translation? */ + ereport(LOG, + (errmsg("could not fork %s process: %m", PostmasterChildName(type))));
I assume you are referring to the child name here?
XXX: We now have functions called AuxiliaryProcessInit() and
InitAuxiliaryProcess(). Confusing.
Based on my analysis, the *Init() is called in the Main functions, while
Init*() is called before the Main functions. Maybe
AuxiliaryProcessInit() could be renamed to AuxiliaryProcessStartup()?
Rename the other to AuxiliaryProcessInit().
--
Tristan Partin
Neon (https://neon.tech)
Re: Refactoring backend fork+exec code
On 06/03/2024 01:02, Tristan Partin wrote:
The first 3 patches seem good to go, in my opinion.
Committed these first patches, with a few more changes. Notably, I
realized that we should move the logic that I originally put in the new
InitClientConnection function to the existing pq_init() function. It
servers the same purpose, initialization of the socket in the child
process. Thanks for the review!
@@ -225,14 +331,13 @@ internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundW
return -1;
}- /* Make sure caller set up argv properly */ - Assert(argc >= 3); - Assert(argv[argc] == NULL); - Assert(strncmp(argv[1], "--fork", 6) == 0); - Assert(argv[2] == NULL); - - /* Insert temp file name after --fork argument */ + /* set up argv properly */ + argv[0] = "postgres"; + snprintf(forkav, MAXPGPATH, "--forkchild=%s", child_kind); + argv[1] = forkav; + /* Insert temp file name after --forkchild argument */ argv[2] = tmpfilename; + argv[3] = NULL;Should we use postgres_exec_path instead of the naked "postgres" here?
I don't know, but it's the same as on 'master' currently. The code just
got moved around.
+ /* in postmaster, fork failed ... */ + ereport(LOG, + (errmsg("could not fork worker process: %m"))); + /* undo what assign_backendlist_entry did */ + ReleasePostmasterChildSlot(rw->rw_child_slot); + rw->rw_child_slot = 0; + pfree(rw->rw_backend); + rw->rw_backend = NULL; + /* mark entry as crashed, so we'll try again later */ + rw->rw_crashed_at = GetCurrentTimestamp(); + return false;I think the error message should include the word "background." It would
be more consistent with the log message above it.
This is also a pre-existing message I just moved around. But yeah, I
agree, so changed.
+typedef struct +{ + int syslogFile; + int csvlogFile; + int jsonlogFile; +} syslogger_startup_data;It would be nice if all of these startup data structs were named
similarly. For instance, a previous one was BackendStartupInfo. It would
help with greppability.
Renamed them to SysloggerStartupData and BackendStartupData. Background
worker startup still passes a struct called BackgroundWorker, however. I
left that as it is, because the struct is used for other purposes too.
I noticed there were a few XXX comments left that you created. I'll
highlight them here for more visibility.+/* XXX: where does this belong? */
+extern bool LoadedSSL;Perhaps near the My* variables or maybe in the Port struct?
It is valid in the postmaster, too, though. The My* variables and Port
struct only make sense in the child process.
I think this is the best place after all, so I just removed the XXX comment.
+#ifdef EXEC_BACKEND + + /* + * Need to reinitialize the SSL library in the backend, since the context + * structures contain function pointers and cannot be passed through the + * parameter file. + * + * If for some reason reload fails (maybe the user installed broken key + * files), soldier on without SSL; that's better than all connections + * becoming impossible. + * + * XXX should we do this in all child processes? For the moment it's + * enough to do it in backend children. XXX good question indeed + */ +#ifdef USE_SSL + if (EnableSSL) + { + if (secure_initialize(false) == 0) + LoadedSSL = true; + else + ereport(LOG, + (errmsg("SSL configuration could not be loaded in child process"))); + } +#endif +#endifHere you added the "good question indeed." I am not sure what the best
answer is either! :)
I just removed the extra XXX comment. It's still a valid question, but
this patch just moves it around, we don't need to answer it here.
+ /* XXX: translation? */ + ereport(LOG, + (errmsg("could not fork %s process: %m", PostmasterChildName(type))));I assume you are referring to the child name here?
Correct. Does the process name need to be translated? And this way of
constructing sentences is not translation-friendly anyway. In some
languages, the word 'process' might need to be inflected differently
depending on the child name, for example.
I put the process name in quotes, and didn't mark the process name for
translation.
XXX: We now have functions called AuxiliaryProcessInit() and
InitAuxiliaryProcess(). Confusing.Based on my analysis, the *Init() is called in the Main functions, while
Init*() is called before the Main functions. Maybe
AuxiliaryProcessInit() could be renamed to AuxiliaryProcessStartup()?
Rename the other to AuxiliaryProcessInit().
Hmm. There's also BackendStartup() function in postmaster.c, which is
very different: it runs in the postmaster process and launches the
backend process. So the Startup suffix is not great either.
I renamed AuxiliaryProcessInit() to AuxiliaryProcessMainCommon(). As in
"the common parts of the main functions of all the aux processes".
(We should perhaps merge InitProcess() and InitAuxiliaryProcess() into
one function. There's a lot of duplicated code between them. And the
parts that differ should perhaps be refactored to be more similar
anyway. I don't want to take on that refactoring right now though.)
Attached is a new version of the remaining patches.
--
Heikki Linnakangas
Neon (https://neon.tech)
Attachments:
v13-0001-Improve-log-messages-referring-to-background-wor.patchtext/x-patch; charset=UTF-8; name=v13-0001-Improve-log-messages-referring-to-background-wor.patchDownload
From 409a68572943d83db3603367e49b6541dc9da483 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Tue, 12 Mar 2024 13:48:27 +0200
Subject: [PATCH v13 1/6] Improve log messages referring to background worker
processes
Per Tristan Partin's suggestion.
---
src/backend/postmaster/postmaster.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 49fc6f979e9..a8793fcb0bc 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -5741,7 +5741,7 @@ do_start_bgworker(RegisteredBgWorker *rw)
case -1:
/* in postmaster, fork failed ... */
ereport(LOG,
- (errmsg("could not fork worker process: %m")));
+ (errmsg("could not fork background worker process: %m")));
/* undo what assign_backendlist_entry did */
ReleasePostmasterChildSlot(rw->rw_child_slot);
rw->rw_child_slot = 0;
@@ -5853,7 +5853,7 @@ assign_backendlist_entry(RegisteredBgWorker *rw)
{
ereport(LOG,
(errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
- errmsg("no slot available for new worker process")));
+ errmsg("no slot available for new background worker process")));
return false;
}
--
2.39.2
v13-0002-Extract-registration-of-Win32-deadchild-callback.patchtext/x-patch; charset=UTF-8; name=v13-0002-Extract-registration-of-Win32-deadchild-callback.patchDownload
From ccfb8bf9288ef9d50169174683b093c74b41cb52 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Fri, 8 Dec 2023 10:31:49 +0200
Subject: [PATCH v13 2/6] Extract registration of Win32 deadchild callback to
separate function
The next commits will move the internal_forkexec() function to a
different source file, but it makes sense to keep all the code related
to the win32 waitpid() emulation in postmaster.c. Split it off to a
separate function now, to make the commit that moves
internal_forkexec() more mechanical.
---
src/backend/postmaster/postmaster.c | 49 ++++++++++++++++++-----------
1 file changed, 30 insertions(+), 19 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index a8793fcb0bc..9cdca024887 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -476,6 +476,7 @@ static void MaybeStartSlotSyncWorker(void);
static pid_t waitpid(pid_t pid, int *exitstatus, int options);
static void WINAPI pgwin32_deadchild_callback(PVOID lpParameter, BOOLEAN TimerOrWaitFired);
+static void pgwin32_register_deadchild_callback(HANDLE procHandle, DWORD procId);
static HANDLE win32ChildQueue;
@@ -4783,26 +4784,10 @@ retry:
return -1;
}
- /*
- * Queue a waiter to signal when this child dies. The wait will be handled
- * automatically by an operating system thread pool. The memory will be
- * freed by a later call to waitpid().
- */
- childinfo = palloc(sizeof(win32_deadchild_waitinfo));
- childinfo->procHandle = pi.hProcess;
- childinfo->procId = pi.dwProcessId;
-
- if (!RegisterWaitForSingleObject(&childinfo->waitHandle,
- pi.hProcess,
- pgwin32_deadchild_callback,
- childinfo,
- INFINITE,
- WT_EXECUTEONLYONCE | WT_EXECUTEINWAITTHREAD))
- ereport(FATAL,
- (errmsg_internal("could not register process for wait: error code %lu",
- GetLastError())));
+ /* Set up notification when the child process dies */
+ pgwin32_register_deadchild_callback(pi.hProcess, pi.dwProcessId);
- /* Don't close pi.hProcess here - waitpid() needs access to it */
+ /* Don't close pi.hProcess, it's owned by the deadchild callback now */
CloseHandle(pi.hThread);
@@ -6526,6 +6511,32 @@ pgwin32_deadchild_callback(PVOID lpParameter, BOOLEAN TimerOrWaitFired)
/* Queue SIGCHLD signal. */
pg_queue_signal(SIGCHLD);
}
+
+/*
+ * Queue a waiter to signal when this child dies. The wait will be handled
+ * automatically by an operating system thread pool. The memory and the
+ * process handle will be freed by a later call to waitpid().
+ */
+static void
+pgwin32_register_deadchild_callback(HANDLE procHandle, DWORD procId)
+{
+ win32_deadchild_waitinfo *childinfo;
+
+ childinfo = palloc(sizeof(win32_deadchild_waitinfo));
+ childinfo->procHandle = procHandle;
+ childinfo->procId = procId;
+
+ if (!RegisterWaitForSingleObject(&childinfo->waitHandle,
+ procHandle,
+ pgwin32_deadchild_callback,
+ childinfo,
+ INFINITE,
+ WT_EXECUTEONLYONCE | WT_EXECUTEINWAITTHREAD))
+ ereport(FATAL,
+ (errmsg_internal("could not register process for wait: error code %lu",
+ GetLastError())));
+}
+
#endif /* WIN32 */
/*
--
2.39.2
v13-0003-Move-some-functions-from-postmaster.c-to-a-new-s.patchtext/x-patch; charset=UTF-8; name=v13-0003-Move-some-functions-from-postmaster.c-to-a-new-s.patchDownload
From 30ae348c9b58c2bf58b1c58a523393514798e5c8 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Fri, 8 Dec 2023 10:31:57 +0200
Subject: [PATCH v13 3/6] Move some functions from postmaster.c to a new source
file
This just moves the functions, with no other changes, to make the next
commits smaller and easier to review. The moved functions are related
to forking and execing a new backend in EXEC_BACKEND mode.
---
src/backend/postmaster/Makefile | 1 +
src/backend/postmaster/launch_backend.c | 812 ++++++++++++++++++++++++
src/backend/postmaster/meson.build | 1 +
src/backend/postmaster/postmaster.c | 753 +---------------------
src/include/postmaster/postmaster.h | 5 +
5 files changed, 829 insertions(+), 743 deletions(-)
create mode 100644 src/backend/postmaster/launch_backend.c
diff --git a/src/backend/postmaster/Makefile b/src/backend/postmaster/Makefile
index 367a46c6177..db08543d195 100644
--- a/src/backend/postmaster/Makefile
+++ b/src/backend/postmaster/Makefile
@@ -20,6 +20,7 @@ OBJS = \
checkpointer.o \
fork_process.o \
interrupt.o \
+ launch_backend.o \
pgarch.o \
postmaster.o \
startup.o \
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
new file mode 100644
index 00000000000..cd81abd12a8
--- /dev/null
+++ b/src/backend/postmaster/launch_backend.c
@@ -0,0 +1,812 @@
+/*-------------------------------------------------------------------------
+ *
+ * launch_backend.c
+ * Functions for launching backends and other postmaster child
+ * processes.
+ *
+ * On Unix systems, a new child process is launched with fork(). It inherits
+ * all the global variables and data structures that had been initialized in
+ * the postmaster. After forking, the child process closes the file
+ * descriptors that are not needed in the child process, and sets up the
+ * mechanism to detect death of the parent postmaster process, etc. After
+ * that, it calls the right Main function depending on the kind of child
+ * process.
+ *
+ * In EXEC_BACKEND mode, which is used on Windows but can be enabled on other
+ * platforms for testing, the child process is launched by fork() + exec() (or
+ * CreateProcess() on Windows). It does not inherit the state from the
+ * postmaster, so it needs to re-attach to the shared memory, re-initialize
+ * global variables, re-load the config file etc.
+ *
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/postmaster/launch_backend.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include <unistd.h>
+
+#include "access/xlog.h"
+#include "common/file_utils.h"
+#include "libpq/libpq-be.h"
+#include "libpq/pqsignal.h"
+#include "miscadmin.h"
+#include "nodes/queryjumble.h"
+#include "port.h"
+#include "postmaster/autovacuum.h"
+#include "postmaster/auxprocess.h"
+#include "postmaster/bgworker_internals.h"
+#include "postmaster/bgwriter.h"
+#include "postmaster/fork_process.h"
+#include "postmaster/pgarch.h"
+#include "postmaster/postmaster.h"
+#include "postmaster/startup.h"
+#include "postmaster/syslogger.h"
+#include "postmaster/walwriter.h"
+#include "replication/walreceiver.h"
+#include "storage/fd.h"
+#include "storage/ipc.h"
+#include "storage/pg_shmem.h"
+#include "storage/pmsignal.h"
+#include "storage/proc.h"
+#include "tcop/tcopprot.h"
+#include "utils/builtins.h"
+#include "utils/datetime.h"
+#include "utils/guc.h"
+#include "utils/memutils.h"
+
+#include "utils/timestamp.h"
+
+#ifdef EXEC_BACKEND
+#include "nodes/queryjumble.h"
+#include "storage/pg_shmem.h"
+#include "storage/spin.h"
+#endif
+
+
+#ifdef EXEC_BACKEND
+
+/* Type for a socket that can be inherited to a client process */
+#ifdef WIN32
+typedef struct
+{
+ SOCKET origsocket; /* Original socket value, or PGINVALID_SOCKET
+ * if not a socket */
+ WSAPROTOCOL_INFO wsainfo;
+} InheritableSocket;
+#else
+typedef int InheritableSocket;
+#endif
+
+/*
+ * Structure contains all variables passed to exec:ed backends
+ */
+typedef struct
+{
+ bool has_client_sock;
+ ClientSocket client_sock;
+ InheritableSocket inh_sock;
+
+ bool has_bgworker;
+ BackgroundWorker bgworker;
+
+ char DataDir[MAXPGPATH];
+ int32 MyCancelKey;
+ int MyPMChildSlot;
+#ifndef WIN32
+ unsigned long UsedShmemSegID;
+#else
+ void *ShmemProtectiveRegion;
+ HANDLE UsedShmemSegID;
+#endif
+ void *UsedShmemSegAddr;
+ slock_t *ShmemLock;
+ struct bkend *ShmemBackendArray;
+#ifndef HAVE_SPINLOCKS
+ PGSemaphore *SpinlockSemaArray;
+#endif
+ int NamedLWLockTrancheRequests;
+ NamedLWLockTranche *NamedLWLockTrancheArray;
+ LWLockPadded *MainLWLockArray;
+ slock_t *ProcStructLock;
+ PROC_HDR *ProcGlobal;
+ PGPROC *AuxiliaryProcs;
+ PGPROC *PreparedXactProcs;
+ PMSignalData *PMSignalState;
+ pid_t PostmasterPid;
+ TimestampTz PgStartTime;
+ TimestampTz PgReloadTime;
+ pg_time_t first_syslogger_file_time;
+ bool redirection_done;
+ bool IsBinaryUpgrade;
+ bool query_id_enabled;
+ int max_safe_fds;
+ int MaxBackends;
+#ifdef WIN32
+ HANDLE PostmasterHandle;
+ HANDLE initial_signal_pipe;
+ HANDLE syslogPipe[2];
+#else
+ int postmaster_alive_fds[2];
+ int syslogPipe[2];
+#endif
+ char my_exec_path[MAXPGPATH];
+ char pkglib_path[MAXPGPATH];
+} BackendParameters;
+
+#define SizeOfBackendParameters(startup_data_len) (offsetof(BackendParameters, startup_data) + startup_data_len)
+
+void read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **worker);
+static void restore_backend_variables(BackendParameters *param, ClientSocket **client_sock, BackgroundWorker **worker);
+
+#ifndef WIN32
+static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker);
+#else
+static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker,
+ HANDLE childProcess, pid_t childPid);
+#endif
+
+pid_t internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker);
+
+#ifndef WIN32
+
+/*
+ * internal_forkexec non-win32 implementation
+ *
+ * - writes out backend variables to the parameter file
+ * - fork():s, and then exec():s the child process
+ */
+pid_t
+internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker)
+{
+ static unsigned long tmpBackendFileNum = 0;
+ pid_t pid;
+ char tmpfilename[MAXPGPATH];
+ BackendParameters param;
+ FILE *fp;
+
+ /*
+ * Make sure padding bytes are initialized, to prevent Valgrind from
+ * complaining about writing uninitialized bytes to the file. This isn't
+ * performance critical, and the win32 implementation initializes the
+ * padding bytes to zeros, so do it even when not using Valgrind.
+ */
+ memset(¶m, 0, sizeof(BackendParameters));
+
+ if (!save_backend_variables(¶m, client_sock, worker))
+ return -1; /* log made by save_backend_variables */
+
+ /* Calculate name for temp file */
+ snprintf(tmpfilename, MAXPGPATH, "%s/%s.backend_var.%d.%lu",
+ PG_TEMP_FILES_DIR, PG_TEMP_FILE_PREFIX,
+ MyProcPid, ++tmpBackendFileNum);
+
+ /* Open file */
+ fp = AllocateFile(tmpfilename, PG_BINARY_W);
+ if (!fp)
+ {
+ /*
+ * As in OpenTemporaryFileInTablespace, try to make the temp-file
+ * directory, ignoring errors.
+ */
+ (void) MakePGDirectory(PG_TEMP_FILES_DIR);
+
+ fp = AllocateFile(tmpfilename, PG_BINARY_W);
+ if (!fp)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not create file \"%s\": %m",
+ tmpfilename)));
+ return -1;
+ }
+ }
+
+ if (fwrite(¶m, sizeof(param), 1, fp) != 1)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not write to file \"%s\": %m", tmpfilename)));
+ FreeFile(fp);
+ return -1;
+ }
+
+ /* Release file */
+ if (FreeFile(fp))
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not write to file \"%s\": %m", tmpfilename)));
+ return -1;
+ }
+
+ /* Make sure caller set up argv properly */
+ Assert(argc >= 3);
+ Assert(argv[argc] == NULL);
+ Assert(strncmp(argv[1], "--fork", 6) == 0);
+ Assert(argv[2] == NULL);
+
+ /* Insert temp file name after --fork argument */
+ argv[2] = tmpfilename;
+
+ /* Fire off execv in child */
+ if ((pid = fork_process()) == 0)
+ {
+ if (execv(postgres_exec_path, argv) < 0)
+ {
+ ereport(LOG,
+ (errmsg("could not execute server process \"%s\": %m",
+ postgres_exec_path)));
+ /* We're already in the child process here, can't return */
+ exit(1);
+ }
+ }
+
+ return pid; /* Parent returns pid, or -1 on fork failure */
+}
+#else /* WIN32 */
+
+/*
+ * internal_forkexec win32 implementation
+ *
+ * - starts backend using CreateProcess(), in suspended state
+ * - writes out backend variables to the parameter file
+ * - during this, duplicates handles and sockets required for
+ * inheritance into the new process
+ * - resumes execution of the new process once the backend parameter
+ * file is complete.
+ */
+pid_t
+internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker)
+{
+ int retry_count = 0;
+ STARTUPINFO si;
+ PROCESS_INFORMATION pi;
+ int i;
+ int j;
+ char cmdLine[MAXPGPATH * 2];
+ HANDLE paramHandle;
+ BackendParameters *param;
+ SECURITY_ATTRIBUTES sa;
+ char paramHandleStr[32];
+ win32_deadchild_waitinfo *childinfo;
+
+ /* Make sure caller set up argv properly */
+ Assert(argc >= 3);
+ Assert(argv[argc] == NULL);
+ Assert(strncmp(argv[1], "--fork", 6) == 0);
+ Assert(argv[2] == NULL);
+
+ /* Resume here if we need to retry */
+retry:
+
+ /* Set up shared memory for parameter passing */
+ ZeroMemory(&sa, sizeof(sa));
+ sa.nLength = sizeof(sa);
+ sa.bInheritHandle = TRUE;
+ paramHandle = CreateFileMapping(INVALID_HANDLE_VALUE,
+ &sa,
+ PAGE_READWRITE,
+ 0,
+ sizeof(BackendParameters),
+ NULL);
+ if (paramHandle == INVALID_HANDLE_VALUE)
+ {
+ ereport(LOG,
+ (errmsg("could not create backend parameter file mapping: error code %lu",
+ GetLastError())));
+ return -1;
+ }
+
+ param = MapViewOfFile(paramHandle, FILE_MAP_WRITE, 0, 0, sizeof(BackendParameters));
+ if (!param)
+ {
+ ereport(LOG,
+ (errmsg("could not map backend parameter memory: error code %lu",
+ GetLastError())));
+ CloseHandle(paramHandle);
+ return -1;
+ }
+
+ /* Insert temp file name after --fork argument */
+#ifdef _WIN64
+ sprintf(paramHandleStr, "%llu", (LONG_PTR) paramHandle);
+#else
+ sprintf(paramHandleStr, "%lu", (DWORD) paramHandle);
+#endif
+ argv[2] = paramHandleStr;
+
+ /* Format the cmd line */
+ cmdLine[sizeof(cmdLine) - 1] = '\0';
+ cmdLine[sizeof(cmdLine) - 2] = '\0';
+ snprintf(cmdLine, sizeof(cmdLine) - 1, "\"%s\"", postgres_exec_path);
+ i = 0;
+ while (argv[++i] != NULL)
+ {
+ j = strlen(cmdLine);
+ snprintf(cmdLine + j, sizeof(cmdLine) - 1 - j, " \"%s\"", argv[i]);
+ }
+ if (cmdLine[sizeof(cmdLine) - 2] != '\0')
+ {
+ ereport(LOG,
+ (errmsg("subprocess command line too long")));
+ UnmapViewOfFile(param);
+ CloseHandle(paramHandle);
+ return -1;
+ }
+
+ memset(&pi, 0, sizeof(pi));
+ memset(&si, 0, sizeof(si));
+ si.cb = sizeof(si);
+
+ /*
+ * Create the subprocess in a suspended state. This will be resumed later,
+ * once we have written out the parameter file.
+ */
+ if (!CreateProcess(NULL, cmdLine, NULL, NULL, TRUE, CREATE_SUSPENDED,
+ NULL, NULL, &si, &pi))
+ {
+ ereport(LOG,
+ (errmsg("CreateProcess() call failed: %m (error code %lu)",
+ GetLastError())));
+ UnmapViewOfFile(param);
+ CloseHandle(paramHandle);
+ return -1;
+ }
+
+ if (!save_backend_variables(param, client_sock, worker, pi.hProcess, pi.dwProcessId))
+ {
+ /*
+ * log made by save_backend_variables, but we have to clean up the
+ * mess with the half-started process
+ */
+ if (!TerminateProcess(pi.hProcess, 255))
+ ereport(LOG,
+ (errmsg_internal("could not terminate unstarted process: error code %lu",
+ GetLastError())));
+ CloseHandle(pi.hProcess);
+ CloseHandle(pi.hThread);
+ UnmapViewOfFile(param);
+ CloseHandle(paramHandle);
+ return -1; /* log made by save_backend_variables */
+ }
+
+ /* Drop the parameter shared memory that is now inherited to the backend */
+ if (!UnmapViewOfFile(param))
+ ereport(LOG,
+ (errmsg("could not unmap view of backend parameter file: error code %lu",
+ GetLastError())));
+ if (!CloseHandle(paramHandle))
+ ereport(LOG,
+ (errmsg("could not close handle to backend parameter file: error code %lu",
+ GetLastError())));
+
+ /*
+ * Reserve the memory region used by our main shared memory segment before
+ * we resume the child process. Normally this should succeed, but if ASLR
+ * is active then it might sometimes fail due to the stack or heap having
+ * gotten mapped into that range. In that case, just terminate the
+ * process and retry.
+ */
+ if (!pgwin32_ReserveSharedMemoryRegion(pi.hProcess))
+ {
+ /* pgwin32_ReserveSharedMemoryRegion already made a log entry */
+ if (!TerminateProcess(pi.hProcess, 255))
+ ereport(LOG,
+ (errmsg_internal("could not terminate process that failed to reserve memory: error code %lu",
+ GetLastError())));
+ CloseHandle(pi.hProcess);
+ CloseHandle(pi.hThread);
+ if (++retry_count < 100)
+ goto retry;
+ ereport(LOG,
+ (errmsg("giving up after too many tries to reserve shared memory"),
+ errhint("This might be caused by ASLR or antivirus software.")));
+ return -1;
+ }
+
+ /*
+ * Now that the backend variables are written out, we start the child
+ * thread so it can start initializing while we set up the rest of the
+ * parent state.
+ */
+ if (ResumeThread(pi.hThread) == -1)
+ {
+ if (!TerminateProcess(pi.hProcess, 255))
+ {
+ ereport(LOG,
+ (errmsg_internal("could not terminate unstartable process: error code %lu",
+ GetLastError())));
+ CloseHandle(pi.hProcess);
+ CloseHandle(pi.hThread);
+ return -1;
+ }
+ CloseHandle(pi.hProcess);
+ CloseHandle(pi.hThread);
+ ereport(LOG,
+ (errmsg_internal("could not resume thread of unstarted process: error code %lu",
+ GetLastError())));
+ return -1;
+ }
+
+ /* Set up notification when the child process dies */
+ pgwin32_register_deadchild_callback(pi.hProcess, pi.dwProcessId);
+
+ /* Don't close pi.hProcess, it's owned by the deadchild callback now */
+
+ CloseHandle(pi.hThread);
+
+ return pi.dwProcessId;
+}
+#endif /* WIN32 */
+
+/*
+ * The following need to be available to the save/restore_backend_variables
+ * functions. They are marked NON_EXEC_STATIC in their home modules.
+ */
+extern slock_t *ShmemLock;
+extern slock_t *ProcStructLock;
+extern PGPROC *AuxiliaryProcs;
+extern PMSignalData *PMSignalState;
+extern pg_time_t first_syslogger_file_time;
+extern struct bkend *ShmemBackendArray;
+extern bool redirection_done;
+
+/* Save critical backend variables into the BackendParameters struct */
+#ifndef WIN32
+#define write_inheritable_socket(dest, src, childpid) ((*(dest) = (src)), true)
+#define read_inheritable_socket(dest, src) (*(dest) = *(src))
+#else
+static bool write_duplicated_handle(HANDLE *dest, HANDLE src, HANDLE child);
+static bool write_inheritable_socket(InheritableSocket *dest, SOCKET src,
+ pid_t childPid);
+static void read_inheritable_socket(SOCKET *dest, InheritableSocket *src);
+#endif
+
+#ifndef WIN32
+static bool
+save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker)
+#else
+static bool
+save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker,
+ HANDLE childProcess, pid_t childPid)
+#endif
+{
+ if (client_sock)
+ {
+ memcpy(¶m->client_sock, client_sock, sizeof(ClientSocket));
+ if (!write_inheritable_socket(¶m->inh_sock, client_sock->sock, childPid))
+ return false;
+ param->has_client_sock = true;
+ }
+ else
+ {
+ memset(¶m->client_sock, 0, sizeof(ClientSocket));
+ param->has_client_sock = false;
+ }
+
+ if (worker)
+ {
+ memcpy(¶m->bgworker, worker, sizeof(BackgroundWorker));
+ param->has_bgworker = true;
+ }
+ else
+ {
+ memset(¶m->bgworker, 0, sizeof(BackgroundWorker));
+ param->has_bgworker = false;
+ }
+
+ strlcpy(param->DataDir, DataDir, MAXPGPATH);
+
+ param->MyCancelKey = MyCancelKey;
+ param->MyPMChildSlot = MyPMChildSlot;
+
+#ifdef WIN32
+ param->ShmemProtectiveRegion = ShmemProtectiveRegion;
+#endif
+ param->UsedShmemSegID = UsedShmemSegID;
+ param->UsedShmemSegAddr = UsedShmemSegAddr;
+
+ param->ShmemLock = ShmemLock;
+ param->ShmemBackendArray = ShmemBackendArray;
+
+#ifndef HAVE_SPINLOCKS
+ param->SpinlockSemaArray = SpinlockSemaArray;
+#endif
+ param->NamedLWLockTrancheRequests = NamedLWLockTrancheRequests;
+ param->NamedLWLockTrancheArray = NamedLWLockTrancheArray;
+ param->MainLWLockArray = MainLWLockArray;
+ param->ProcStructLock = ProcStructLock;
+ param->ProcGlobal = ProcGlobal;
+ param->AuxiliaryProcs = AuxiliaryProcs;
+ param->PreparedXactProcs = PreparedXactProcs;
+ param->PMSignalState = PMSignalState;
+
+ param->PostmasterPid = PostmasterPid;
+ param->PgStartTime = PgStartTime;
+ param->PgReloadTime = PgReloadTime;
+ param->first_syslogger_file_time = first_syslogger_file_time;
+
+ param->redirection_done = redirection_done;
+ param->IsBinaryUpgrade = IsBinaryUpgrade;
+ param->query_id_enabled = query_id_enabled;
+ param->max_safe_fds = max_safe_fds;
+
+ param->MaxBackends = MaxBackends;
+
+#ifdef WIN32
+ param->PostmasterHandle = PostmasterHandle;
+ if (!write_duplicated_handle(¶m->initial_signal_pipe,
+ pgwin32_create_signal_listener(childPid),
+ childProcess))
+ return false;
+#else
+ memcpy(¶m->postmaster_alive_fds, &postmaster_alive_fds,
+ sizeof(postmaster_alive_fds));
+#endif
+
+ memcpy(¶m->syslogPipe, &syslogPipe, sizeof(syslogPipe));
+
+ strlcpy(param->my_exec_path, my_exec_path, MAXPGPATH);
+
+ strlcpy(param->pkglib_path, pkglib_path, MAXPGPATH);
+
+ return true;
+}
+
+#ifdef WIN32
+/*
+ * Duplicate a handle for usage in a child process, and write the child
+ * process instance of the handle to the parameter file.
+ */
+static bool
+write_duplicated_handle(HANDLE *dest, HANDLE src, HANDLE childProcess)
+{
+ HANDLE hChild = INVALID_HANDLE_VALUE;
+
+ if (!DuplicateHandle(GetCurrentProcess(),
+ src,
+ childProcess,
+ &hChild,
+ 0,
+ TRUE,
+ DUPLICATE_CLOSE_SOURCE | DUPLICATE_SAME_ACCESS))
+ {
+ ereport(LOG,
+ (errmsg_internal("could not duplicate handle to be written to backend parameter file: error code %lu",
+ GetLastError())));
+ return false;
+ }
+
+ *dest = hChild;
+ return true;
+}
+
+/*
+ * Duplicate a socket for usage in a child process, and write the resulting
+ * structure to the parameter file.
+ * This is required because a number of LSPs (Layered Service Providers) very
+ * common on Windows (antivirus, firewalls, download managers etc) break
+ * straight socket inheritance.
+ */
+static bool
+write_inheritable_socket(InheritableSocket *dest, SOCKET src, pid_t childpid)
+{
+ dest->origsocket = src;
+ if (src != 0 && src != PGINVALID_SOCKET)
+ {
+ /* Actual socket */
+ if (WSADuplicateSocket(src, childpid, &dest->wsainfo) != 0)
+ {
+ ereport(LOG,
+ (errmsg("could not duplicate socket %d for use in backend: error code %d",
+ (int) src, WSAGetLastError())));
+ return false;
+ }
+ }
+ return true;
+}
+
+/*
+ * Read a duplicate socket structure back, and get the socket descriptor.
+ */
+static void
+read_inheritable_socket(SOCKET *dest, InheritableSocket *src)
+{
+ SOCKET s;
+
+ if (src->origsocket == PGINVALID_SOCKET || src->origsocket == 0)
+ {
+ /* Not a real socket! */
+ *dest = src->origsocket;
+ }
+ else
+ {
+ /* Actual socket, so create from structure */
+ s = WSASocket(FROM_PROTOCOL_INFO,
+ FROM_PROTOCOL_INFO,
+ FROM_PROTOCOL_INFO,
+ &src->wsainfo,
+ 0,
+ 0);
+ if (s == INVALID_SOCKET)
+ {
+ write_stderr("could not create inherited socket: error code %d\n",
+ WSAGetLastError());
+ exit(1);
+ }
+ *dest = s;
+
+ /*
+ * To make sure we don't get two references to the same socket, close
+ * the original one. (This would happen when inheritance actually
+ * works..
+ */
+ closesocket(src->origsocket);
+ }
+}
+#endif
+
+void
+read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **worker)
+{
+ BackendParameters param;
+
+#ifndef WIN32
+ /* Non-win32 implementation reads from file */
+ FILE *fp;
+
+ /* Open file */
+ fp = AllocateFile(id, PG_BINARY_R);
+ if (!fp)
+ {
+ write_stderr("could not open backend variables file \"%s\": %m\n", id);
+ exit(1);
+ }
+
+ if (fread(¶m, sizeof(param), 1, fp) != 1)
+ {
+ write_stderr("could not read from backend variables file \"%s\": %m\n", id);
+ exit(1);
+ }
+
+ /* Release file */
+ FreeFile(fp);
+ if (unlink(id) != 0)
+ {
+ write_stderr("could not remove file \"%s\": %m\n", id);
+ exit(1);
+ }
+#else
+ /* Win32 version uses mapped file */
+ HANDLE paramHandle;
+ BackendParameters *paramp;
+
+#ifdef _WIN64
+ paramHandle = (HANDLE) _atoi64(id);
+#else
+ paramHandle = (HANDLE) atol(id);
+#endif
+ paramp = MapViewOfFile(paramHandle, FILE_MAP_READ, 0, 0, 0);
+ if (!paramp)
+ {
+ write_stderr("could not map view of backend variables: error code %lu\n",
+ GetLastError());
+ exit(1);
+ }
+
+ memcpy(¶m, paramp, sizeof(BackendParameters));
+
+ if (!UnmapViewOfFile(paramp))
+ {
+ write_stderr("could not unmap view of backend variables: error code %lu\n",
+ GetLastError());
+ exit(1);
+ }
+
+ if (!CloseHandle(paramHandle))
+ {
+ write_stderr("could not close handle to backend parameter variables: error code %lu\n",
+ GetLastError());
+ exit(1);
+ }
+#endif
+
+ restore_backend_variables(¶m, client_sock, worker);
+}
+
+/* Restore critical backend variables from the BackendParameters struct */
+static void
+restore_backend_variables(BackendParameters *param, ClientSocket **client_sock, BackgroundWorker **worker)
+{
+ if (param->has_client_sock)
+ {
+ *client_sock = (ClientSocket *) MemoryContextAlloc(TopMemoryContext, sizeof(ClientSocket));
+ memcpy(*client_sock, ¶m->client_sock, sizeof(ClientSocket));
+ read_inheritable_socket(&(*client_sock)->sock, ¶m->inh_sock);
+ }
+ else
+ *client_sock = NULL;
+
+ if (param->has_bgworker)
+ {
+ *worker = (BackgroundWorker *)
+ MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
+ memcpy(*worker, ¶m->bgworker, sizeof(BackgroundWorker));
+ }
+ else
+ *worker = NULL;
+
+ SetDataDir(param->DataDir);
+
+ MyCancelKey = param->MyCancelKey;
+ MyPMChildSlot = param->MyPMChildSlot;
+
+#ifdef WIN32
+ ShmemProtectiveRegion = param->ShmemProtectiveRegion;
+#endif
+ UsedShmemSegID = param->UsedShmemSegID;
+ UsedShmemSegAddr = param->UsedShmemSegAddr;
+
+ ShmemLock = param->ShmemLock;
+ ShmemBackendArray = param->ShmemBackendArray;
+
+#ifndef HAVE_SPINLOCKS
+ SpinlockSemaArray = param->SpinlockSemaArray;
+#endif
+ NamedLWLockTrancheRequests = param->NamedLWLockTrancheRequests;
+ NamedLWLockTrancheArray = param->NamedLWLockTrancheArray;
+ MainLWLockArray = param->MainLWLockArray;
+ ProcStructLock = param->ProcStructLock;
+ ProcGlobal = param->ProcGlobal;
+ AuxiliaryProcs = param->AuxiliaryProcs;
+ PreparedXactProcs = param->PreparedXactProcs;
+ PMSignalState = param->PMSignalState;
+
+ PostmasterPid = param->PostmasterPid;
+ PgStartTime = param->PgStartTime;
+ PgReloadTime = param->PgReloadTime;
+ first_syslogger_file_time = param->first_syslogger_file_time;
+
+ redirection_done = param->redirection_done;
+ IsBinaryUpgrade = param->IsBinaryUpgrade;
+ query_id_enabled = param->query_id_enabled;
+ max_safe_fds = param->max_safe_fds;
+
+ MaxBackends = param->MaxBackends;
+
+#ifdef WIN32
+ PostmasterHandle = param->PostmasterHandle;
+ pgwin32_initial_signal_pipe = param->initial_signal_pipe;
+#else
+ memcpy(&postmaster_alive_fds, ¶m->postmaster_alive_fds,
+ sizeof(postmaster_alive_fds));
+#endif
+
+ memcpy(&syslogPipe, ¶m->syslogPipe, sizeof(syslogPipe));
+
+ strlcpy(my_exec_path, param->my_exec_path, MAXPGPATH);
+
+ strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
+
+ /*
+ * We need to restore fd.c's counts of externally-opened FDs; to avoid
+ * confusion, be sure to do this after restoring max_safe_fds. (Note:
+ * BackendInitialize will handle this for (*client_sock)->sock.)
+ */
+#ifndef WIN32
+ if (postmaster_alive_fds[0] >= 0)
+ ReserveExternalFD();
+ if (postmaster_alive_fds[1] >= 0)
+ ReserveExternalFD();
+#endif
+}
+
+#endif /* EXEC_BACKEND */
diff --git a/src/backend/postmaster/meson.build b/src/backend/postmaster/meson.build
index 63a611b68be..0ea4bbe084e 100644
--- a/src/backend/postmaster/meson.build
+++ b/src/backend/postmaster/meson.build
@@ -8,6 +8,7 @@ backend_sources += files(
'checkpointer.c',
'fork_process.c',
'interrupt.c',
+ 'launch_backend.c',
'pgarch.c',
'postmaster.c',
'startup.c',
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 9cdca024887..6610bd5f866 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -129,9 +129,7 @@
#include "utils/varlena.h"
#ifdef EXEC_BACKEND
-#include "nodes/queryjumble.h"
#include "storage/pg_shmem.h"
-#include "storage/spin.h"
#endif
@@ -186,7 +184,7 @@ typedef struct bkend
static dlist_head BackendList = DLIST_STATIC_INIT(BackendList);
#ifdef EXEC_BACKEND
-static Backend *ShmemBackendArray;
+Backend *ShmemBackendArray;
#endif
BackgroundWorker *MyBgworkerEntry = NULL;
@@ -476,7 +474,6 @@ static void MaybeStartSlotSyncWorker(void);
static pid_t waitpid(pid_t pid, int *exitstatus, int options);
static void WINAPI pgwin32_deadchild_callback(PVOID lpParameter, BOOLEAN TimerOrWaitFired);
-static void pgwin32_register_deadchild_callback(HANDLE procHandle, DWORD procId);
static HANDLE win32ChildQueue;
@@ -489,85 +486,11 @@ typedef struct
#endif /* WIN32 */
static pid_t backend_forkexec(ClientSocket *client_sock, CAC_state cac);
-static pid_t internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker);
-/* Type for a socket that can be inherited to a client process */
-#ifdef WIN32
-typedef struct
-{
- SOCKET origsocket; /* Original socket value, or PGINVALID_SOCKET
- * if not a socket */
- WSAPROTOCOL_INFO wsainfo;
-} InheritableSocket;
-#else
-typedef int InheritableSocket;
-#endif
-
-/*
- * Structure contains all variables passed to exec:ed backends
- */
-typedef struct
-{
- bool has_client_sock;
- ClientSocket client_sock;
- InheritableSocket inh_sock;
-
- bool has_bgworker;
- BackgroundWorker bgworker;
-
- char DataDir[MAXPGPATH];
- int32 MyCancelKey;
- int MyPMChildSlot;
-#ifndef WIN32
- unsigned long UsedShmemSegID;
-#else
- void *ShmemProtectiveRegion;
- HANDLE UsedShmemSegID;
-#endif
- void *UsedShmemSegAddr;
- slock_t *ShmemLock;
- Backend *ShmemBackendArray;
-#ifndef HAVE_SPINLOCKS
- PGSemaphore *SpinlockSemaArray;
-#endif
- int NamedLWLockTrancheRequests;
- NamedLWLockTranche *NamedLWLockTrancheArray;
- LWLockPadded *MainLWLockArray;
- slock_t *ProcStructLock;
- PROC_HDR *ProcGlobal;
- PGPROC *AuxiliaryProcs;
- PGPROC *PreparedXactProcs;
- PMSignalData *PMSignalState;
- pid_t PostmasterPid;
- TimestampTz PgStartTime;
- TimestampTz PgReloadTime;
- pg_time_t first_syslogger_file_time;
- bool redirection_done;
- bool IsBinaryUpgrade;
- bool query_id_enabled;
- int max_safe_fds;
- int MaxBackends;
-#ifdef WIN32
- HANDLE PostmasterHandle;
- HANDLE initial_signal_pipe;
- HANDLE syslogPipe[2];
-#else
- int postmaster_alive_fds[2];
- int syslogPipe[2];
-#endif
- char my_exec_path[MAXPGPATH];
- char pkglib_path[MAXPGPATH];
-} BackendParameters;
-static void read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **worker);
-static void restore_backend_variables(BackendParameters *param, ClientSocket **client_sock, BackgroundWorker **worker);
-
-#ifndef WIN32
-static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker);
-#else
-static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker,
- HANDLE childProcess, pid_t childPid);
-#endif
+/* in launch_backend.c */
+extern pid_t internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker);
+extern void read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **worker);
static void ShmemBackendArrayAdd(Backend *bn);
static void ShmemBackendArrayRemove(Backend *bn);
@@ -1117,11 +1040,11 @@ PostmasterMain(int argc, char *argv[])
/*
* Clean out the temp directory used to transmit parameters to child
- * processes (see internal_forkexec, below). We must do this before
- * launching any child processes, else we have a race condition: we could
- * remove a parameter file before the child can read it. It should be
- * safe to do so now, because we verified earlier that there are no
- * conflicting Postgres processes in this data directory.
+ * processes (see internal_forkexec). We must do this before launching
+ * any child processes, else we have a race condition: we could remove a
+ * parameter file before the child can read it. It should be safe to do
+ * so now, because we verified earlier that there are no conflicting
+ * Postgres processes in this data directory.
*/
RemovePgTempFilesInDir(PG_TEMP_FILES_DIR, true, false);
#endif
@@ -4503,299 +4426,6 @@ backend_forkexec(ClientSocket *client_sock, CAC_state cac)
return internal_forkexec(ac, av, client_sock, NULL);
}
-#ifndef WIN32
-
-/*
- * internal_forkexec non-win32 implementation
- *
- * - writes out backend variables to the parameter file
- * - fork():s, and then exec():s the child process
- */
-static pid_t
-internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker)
-{
- static unsigned long tmpBackendFileNum = 0;
- pid_t pid;
- char tmpfilename[MAXPGPATH];
- BackendParameters param;
- FILE *fp;
-
- /*
- * Make sure padding bytes are initialized, to prevent Valgrind from
- * complaining about writing uninitialized bytes to the file. This isn't
- * performance critical, and the win32 implementation initializes the
- * padding bytes to zeros, so do it even when not using Valgrind.
- */
- memset(¶m, 0, sizeof(BackendParameters));
-
- if (!save_backend_variables(¶m, client_sock, worker))
- return -1; /* log made by save_backend_variables */
-
- /* Calculate name for temp file */
- snprintf(tmpfilename, MAXPGPATH, "%s/%s.backend_var.%d.%lu",
- PG_TEMP_FILES_DIR, PG_TEMP_FILE_PREFIX,
- MyProcPid, ++tmpBackendFileNum);
-
- /* Open file */
- fp = AllocateFile(tmpfilename, PG_BINARY_W);
- if (!fp)
- {
- /*
- * As in OpenTemporaryFileInTablespace, try to make the temp-file
- * directory, ignoring errors.
- */
- (void) MakePGDirectory(PG_TEMP_FILES_DIR);
-
- fp = AllocateFile(tmpfilename, PG_BINARY_W);
- if (!fp)
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not create file \"%s\": %m",
- tmpfilename)));
- return -1;
- }
- }
-
- if (fwrite(¶m, sizeof(param), 1, fp) != 1)
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not write to file \"%s\": %m", tmpfilename)));
- FreeFile(fp);
- return -1;
- }
-
- /* Release file */
- if (FreeFile(fp))
- {
- ereport(LOG,
- (errcode_for_file_access(),
- errmsg("could not write to file \"%s\": %m", tmpfilename)));
- return -1;
- }
-
- /* Make sure caller set up argv properly */
- Assert(argc >= 3);
- Assert(argv[argc] == NULL);
- Assert(strncmp(argv[1], "--fork", 6) == 0);
- Assert(argv[2] == NULL);
-
- /* Insert temp file name after --fork argument */
- argv[2] = tmpfilename;
-
- /* Fire off execv in child */
- if ((pid = fork_process()) == 0)
- {
- if (execv(postgres_exec_path, argv) < 0)
- {
- ereport(LOG,
- (errmsg("could not execute server process \"%s\": %m",
- postgres_exec_path)));
- /* We're already in the child process here, can't return */
- exit(1);
- }
- }
-
- return pid; /* Parent returns pid, or -1 on fork failure */
-}
-#else /* WIN32 */
-
-/*
- * internal_forkexec win32 implementation
- *
- * - starts backend using CreateProcess(), in suspended state
- * - writes out backend variables to the parameter file
- * - during this, duplicates handles and sockets required for
- * inheritance into the new process
- * - resumes execution of the new process once the backend parameter
- * file is complete.
- */
-static pid_t
-internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker)
-{
- int retry_count = 0;
- STARTUPINFO si;
- PROCESS_INFORMATION pi;
- int i;
- int j;
- char cmdLine[MAXPGPATH * 2];
- HANDLE paramHandle;
- BackendParameters *param;
- SECURITY_ATTRIBUTES sa;
- char paramHandleStr[32];
- win32_deadchild_waitinfo *childinfo;
-
- /* Make sure caller set up argv properly */
- Assert(argc >= 3);
- Assert(argv[argc] == NULL);
- Assert(strncmp(argv[1], "--fork", 6) == 0);
- Assert(argv[2] == NULL);
-
- /* Resume here if we need to retry */
-retry:
-
- /* Set up shared memory for parameter passing */
- ZeroMemory(&sa, sizeof(sa));
- sa.nLength = sizeof(sa);
- sa.bInheritHandle = TRUE;
- paramHandle = CreateFileMapping(INVALID_HANDLE_VALUE,
- &sa,
- PAGE_READWRITE,
- 0,
- sizeof(BackendParameters),
- NULL);
- if (paramHandle == INVALID_HANDLE_VALUE)
- {
- ereport(LOG,
- (errmsg("could not create backend parameter file mapping: error code %lu",
- GetLastError())));
- return -1;
- }
-
- param = MapViewOfFile(paramHandle, FILE_MAP_WRITE, 0, 0, sizeof(BackendParameters));
- if (!param)
- {
- ereport(LOG,
- (errmsg("could not map backend parameter memory: error code %lu",
- GetLastError())));
- CloseHandle(paramHandle);
- return -1;
- }
-
- /* Insert temp file name after --fork argument */
-#ifdef _WIN64
- sprintf(paramHandleStr, "%llu", (LONG_PTR) paramHandle);
-#else
- sprintf(paramHandleStr, "%lu", (DWORD) paramHandle);
-#endif
- argv[2] = paramHandleStr;
-
- /* Format the cmd line */
- cmdLine[sizeof(cmdLine) - 1] = '\0';
- cmdLine[sizeof(cmdLine) - 2] = '\0';
- snprintf(cmdLine, sizeof(cmdLine) - 1, "\"%s\"", postgres_exec_path);
- i = 0;
- while (argv[++i] != NULL)
- {
- j = strlen(cmdLine);
- snprintf(cmdLine + j, sizeof(cmdLine) - 1 - j, " \"%s\"", argv[i]);
- }
- if (cmdLine[sizeof(cmdLine) - 2] != '\0')
- {
- ereport(LOG,
- (errmsg("subprocess command line too long")));
- UnmapViewOfFile(param);
- CloseHandle(paramHandle);
- return -1;
- }
-
- memset(&pi, 0, sizeof(pi));
- memset(&si, 0, sizeof(si));
- si.cb = sizeof(si);
-
- /*
- * Create the subprocess in a suspended state. This will be resumed later,
- * once we have written out the parameter file.
- */
- if (!CreateProcess(NULL, cmdLine, NULL, NULL, TRUE, CREATE_SUSPENDED,
- NULL, NULL, &si, &pi))
- {
- ereport(LOG,
- (errmsg("CreateProcess() call failed: %m (error code %lu)",
- GetLastError())));
- UnmapViewOfFile(param);
- CloseHandle(paramHandle);
- return -1;
- }
-
- if (!save_backend_variables(param, client_sock, worker, pi.hProcess, pi.dwProcessId))
- {
- /*
- * log made by save_backend_variables, but we have to clean up the
- * mess with the half-started process
- */
- if (!TerminateProcess(pi.hProcess, 255))
- ereport(LOG,
- (errmsg_internal("could not terminate unstarted process: error code %lu",
- GetLastError())));
- CloseHandle(pi.hProcess);
- CloseHandle(pi.hThread);
- UnmapViewOfFile(param);
- CloseHandle(paramHandle);
- return -1; /* log made by save_backend_variables */
- }
-
- /* Drop the parameter shared memory that is now inherited to the backend */
- if (!UnmapViewOfFile(param))
- ereport(LOG,
- (errmsg("could not unmap view of backend parameter file: error code %lu",
- GetLastError())));
- if (!CloseHandle(paramHandle))
- ereport(LOG,
- (errmsg("could not close handle to backend parameter file: error code %lu",
- GetLastError())));
-
- /*
- * Reserve the memory region used by our main shared memory segment before
- * we resume the child process. Normally this should succeed, but if ASLR
- * is active then it might sometimes fail due to the stack or heap having
- * gotten mapped into that range. In that case, just terminate the
- * process and retry.
- */
- if (!pgwin32_ReserveSharedMemoryRegion(pi.hProcess))
- {
- /* pgwin32_ReserveSharedMemoryRegion already made a log entry */
- if (!TerminateProcess(pi.hProcess, 255))
- ereport(LOG,
- (errmsg_internal("could not terminate process that failed to reserve memory: error code %lu",
- GetLastError())));
- CloseHandle(pi.hProcess);
- CloseHandle(pi.hThread);
- if (++retry_count < 100)
- goto retry;
- ereport(LOG,
- (errmsg("giving up after too many tries to reserve shared memory"),
- errhint("This might be caused by ASLR or antivirus software.")));
- return -1;
- }
-
- /*
- * Now that the backend variables are written out, we start the child
- * thread so it can start initializing while we set up the rest of the
- * parent state.
- */
- if (ResumeThread(pi.hThread) == -1)
- {
- if (!TerminateProcess(pi.hProcess, 255))
- {
- ereport(LOG,
- (errmsg_internal("could not terminate unstartable process: error code %lu",
- GetLastError())));
- CloseHandle(pi.hProcess);
- CloseHandle(pi.hThread);
- return -1;
- }
- CloseHandle(pi.hProcess);
- CloseHandle(pi.hThread);
- ereport(LOG,
- (errmsg_internal("could not resume thread of unstarted process: error code %lu",
- GetLastError())));
- return -1;
- }
-
- /* Set up notification when the child process dies */
- pgwin32_register_deadchild_callback(pi.hProcess, pi.dwProcessId);
-
- /* Don't close pi.hProcess, it's owned by the deadchild callback now */
-
- CloseHandle(pi.hThread);
-
- return pi.dwProcessId;
-}
-#endif /* WIN32 */
-
-
/*
* SubPostmasterMain -- Get the fork/exec'd process into a state equivalent
* to what it would be if we'd simply forked on Unix, and then
@@ -6027,369 +5657,6 @@ PostmasterMarkPIDForWorkerNotify(int pid)
#ifdef EXEC_BACKEND
-/*
- * The following need to be available to the save/restore_backend_variables
- * functions. They are marked NON_EXEC_STATIC in their home modules.
- */
-extern slock_t *ShmemLock;
-extern slock_t *ProcStructLock;
-extern PGPROC *AuxiliaryProcs;
-extern PMSignalData *PMSignalState;
-extern pg_time_t first_syslogger_file_time;
-
-#ifndef WIN32
-#define write_inheritable_socket(dest, src, childpid) ((*(dest) = (src)), true)
-#define read_inheritable_socket(dest, src) (*(dest) = *(src))
-#else
-static bool write_duplicated_handle(HANDLE *dest, HANDLE src, HANDLE child);
-static bool write_inheritable_socket(InheritableSocket *dest, SOCKET src,
- pid_t childPid);
-static void read_inheritable_socket(SOCKET *dest, InheritableSocket *src);
-#endif
-
-
-/* Save critical backend variables into the BackendParameters struct */
-#ifndef WIN32
-static bool
-save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker)
-#else
-static bool
-save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker,
- HANDLE childProcess, pid_t childPid)
-#endif
-{
- if (client_sock)
- {
- memcpy(¶m->client_sock, client_sock, sizeof(ClientSocket));
- if (!write_inheritable_socket(¶m->inh_sock, client_sock->sock, childPid))
- return false;
- param->has_client_sock = true;
- }
- else
- {
- memset(¶m->client_sock, 0, sizeof(ClientSocket));
- param->has_client_sock = false;
- }
-
- if (worker)
- {
- memcpy(¶m->bgworker, worker, sizeof(BackgroundWorker));
- param->has_bgworker = true;
- }
- else
- {
- memset(¶m->bgworker, 0, sizeof(BackgroundWorker));
- param->has_bgworker = false;
- }
-
- strlcpy(param->DataDir, DataDir, MAXPGPATH);
-
- param->MyCancelKey = MyCancelKey;
- param->MyPMChildSlot = MyPMChildSlot;
-
-#ifdef WIN32
- param->ShmemProtectiveRegion = ShmemProtectiveRegion;
-#endif
- param->UsedShmemSegID = UsedShmemSegID;
- param->UsedShmemSegAddr = UsedShmemSegAddr;
-
- param->ShmemLock = ShmemLock;
- param->ShmemBackendArray = ShmemBackendArray;
-
-#ifndef HAVE_SPINLOCKS
- param->SpinlockSemaArray = SpinlockSemaArray;
-#endif
- param->NamedLWLockTrancheRequests = NamedLWLockTrancheRequests;
- param->NamedLWLockTrancheArray = NamedLWLockTrancheArray;
- param->MainLWLockArray = MainLWLockArray;
- param->ProcStructLock = ProcStructLock;
- param->ProcGlobal = ProcGlobal;
- param->AuxiliaryProcs = AuxiliaryProcs;
- param->PreparedXactProcs = PreparedXactProcs;
- param->PMSignalState = PMSignalState;
-
- param->PostmasterPid = PostmasterPid;
- param->PgStartTime = PgStartTime;
- param->PgReloadTime = PgReloadTime;
- param->first_syslogger_file_time = first_syslogger_file_time;
-
- param->redirection_done = redirection_done;
- param->IsBinaryUpgrade = IsBinaryUpgrade;
- param->query_id_enabled = query_id_enabled;
- param->max_safe_fds = max_safe_fds;
-
- param->MaxBackends = MaxBackends;
-
-#ifdef WIN32
- param->PostmasterHandle = PostmasterHandle;
- if (!write_duplicated_handle(¶m->initial_signal_pipe,
- pgwin32_create_signal_listener(childPid),
- childProcess))
- return false;
-#else
- memcpy(¶m->postmaster_alive_fds, &postmaster_alive_fds,
- sizeof(postmaster_alive_fds));
-#endif
-
- memcpy(¶m->syslogPipe, &syslogPipe, sizeof(syslogPipe));
-
- strlcpy(param->my_exec_path, my_exec_path, MAXPGPATH);
-
- strlcpy(param->pkglib_path, pkglib_path, MAXPGPATH);
-
- return true;
-}
-
-
-#ifdef WIN32
-/*
- * Duplicate a handle for usage in a child process, and write the child
- * process instance of the handle to the parameter file.
- */
-static bool
-write_duplicated_handle(HANDLE *dest, HANDLE src, HANDLE childProcess)
-{
- HANDLE hChild = INVALID_HANDLE_VALUE;
-
- if (!DuplicateHandle(GetCurrentProcess(),
- src,
- childProcess,
- &hChild,
- 0,
- TRUE,
- DUPLICATE_CLOSE_SOURCE | DUPLICATE_SAME_ACCESS))
- {
- ereport(LOG,
- (errmsg_internal("could not duplicate handle to be written to backend parameter file: error code %lu",
- GetLastError())));
- return false;
- }
-
- *dest = hChild;
- return true;
-}
-
-/*
- * Duplicate a socket for usage in a child process, and write the resulting
- * structure to the parameter file.
- * This is required because a number of LSPs (Layered Service Providers) very
- * common on Windows (antivirus, firewalls, download managers etc) break
- * straight socket inheritance.
- */
-static bool
-write_inheritable_socket(InheritableSocket *dest, SOCKET src, pid_t childpid)
-{
- dest->origsocket = src;
- if (src != 0 && src != PGINVALID_SOCKET)
- {
- /* Actual socket */
- if (WSADuplicateSocket(src, childpid, &dest->wsainfo) != 0)
- {
- ereport(LOG,
- (errmsg("could not duplicate socket %d for use in backend: error code %d",
- (int) src, WSAGetLastError())));
- return false;
- }
- }
- return true;
-}
-
-/*
- * Read a duplicate socket structure back, and get the socket descriptor.
- */
-static void
-read_inheritable_socket(SOCKET *dest, InheritableSocket *src)
-{
- SOCKET s;
-
- if (src->origsocket == PGINVALID_SOCKET || src->origsocket == 0)
- {
- /* Not a real socket! */
- *dest = src->origsocket;
- }
- else
- {
- /* Actual socket, so create from structure */
- s = WSASocket(FROM_PROTOCOL_INFO,
- FROM_PROTOCOL_INFO,
- FROM_PROTOCOL_INFO,
- &src->wsainfo,
- 0,
- 0);
- if (s == INVALID_SOCKET)
- {
- write_stderr("could not create inherited socket: error code %d\n",
- WSAGetLastError());
- exit(1);
- }
- *dest = s;
-
- /*
- * To make sure we don't get two references to the same socket, close
- * the original one. (This would happen when inheritance actually
- * works..
- */
- closesocket(src->origsocket);
- }
-}
-#endif
-
-static void
-read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **worker)
-{
- BackendParameters param;
-
-#ifndef WIN32
- /* Non-win32 implementation reads from file */
- FILE *fp;
-
- /* Open file */
- fp = AllocateFile(id, PG_BINARY_R);
- if (!fp)
- {
- write_stderr("could not open backend variables file \"%s\": %m\n", id);
- exit(1);
- }
-
- if (fread(¶m, sizeof(param), 1, fp) != 1)
- {
- write_stderr("could not read from backend variables file \"%s\": %m\n", id);
- exit(1);
- }
-
- /* Release file */
- FreeFile(fp);
- if (unlink(id) != 0)
- {
- write_stderr("could not remove file \"%s\": %m\n", id);
- exit(1);
- }
-#else
- /* Win32 version uses mapped file */
- HANDLE paramHandle;
- BackendParameters *paramp;
-
-#ifdef _WIN64
- paramHandle = (HANDLE) _atoi64(id);
-#else
- paramHandle = (HANDLE) atol(id);
-#endif
- paramp = MapViewOfFile(paramHandle, FILE_MAP_READ, 0, 0, 0);
- if (!paramp)
- {
- write_stderr("could not map view of backend variables: error code %lu\n",
- GetLastError());
- exit(1);
- }
-
- memcpy(¶m, paramp, sizeof(BackendParameters));
-
- if (!UnmapViewOfFile(paramp))
- {
- write_stderr("could not unmap view of backend variables: error code %lu\n",
- GetLastError());
- exit(1);
- }
-
- if (!CloseHandle(paramHandle))
- {
- write_stderr("could not close handle to backend parameter variables: error code %lu\n",
- GetLastError());
- exit(1);
- }
-#endif
-
- restore_backend_variables(¶m, client_sock, worker);
-}
-
-/* Restore critical backend variables from the BackendParameters struct */
-static void
-restore_backend_variables(BackendParameters *param, ClientSocket **client_sock, BackgroundWorker **worker)
-{
- if (param->has_client_sock)
- {
- *client_sock = (ClientSocket *) MemoryContextAlloc(TopMemoryContext, sizeof(ClientSocket));
- memcpy(*client_sock, ¶m->client_sock, sizeof(ClientSocket));
- read_inheritable_socket(&(*client_sock)->sock, ¶m->inh_sock);
- }
- else
- *client_sock = NULL;
-
- if (param->has_bgworker)
- {
- *worker = (BackgroundWorker *)
- MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
- memcpy(*worker, ¶m->bgworker, sizeof(BackgroundWorker));
- }
- else
- *worker = NULL;
-
- SetDataDir(param->DataDir);
-
- MyCancelKey = param->MyCancelKey;
- MyPMChildSlot = param->MyPMChildSlot;
-
-#ifdef WIN32
- ShmemProtectiveRegion = param->ShmemProtectiveRegion;
-#endif
- UsedShmemSegID = param->UsedShmemSegID;
- UsedShmemSegAddr = param->UsedShmemSegAddr;
-
- ShmemLock = param->ShmemLock;
- ShmemBackendArray = param->ShmemBackendArray;
-
-#ifndef HAVE_SPINLOCKS
- SpinlockSemaArray = param->SpinlockSemaArray;
-#endif
- NamedLWLockTrancheRequests = param->NamedLWLockTrancheRequests;
- NamedLWLockTrancheArray = param->NamedLWLockTrancheArray;
- MainLWLockArray = param->MainLWLockArray;
- ProcStructLock = param->ProcStructLock;
- ProcGlobal = param->ProcGlobal;
- AuxiliaryProcs = param->AuxiliaryProcs;
- PreparedXactProcs = param->PreparedXactProcs;
- PMSignalState = param->PMSignalState;
-
- PostmasterPid = param->PostmasterPid;
- PgStartTime = param->PgStartTime;
- PgReloadTime = param->PgReloadTime;
- first_syslogger_file_time = param->first_syslogger_file_time;
-
- redirection_done = param->redirection_done;
- IsBinaryUpgrade = param->IsBinaryUpgrade;
- query_id_enabled = param->query_id_enabled;
- max_safe_fds = param->max_safe_fds;
-
- MaxBackends = param->MaxBackends;
-
-#ifdef WIN32
- PostmasterHandle = param->PostmasterHandle;
- pgwin32_initial_signal_pipe = param->initial_signal_pipe;
-#else
- memcpy(&postmaster_alive_fds, ¶m->postmaster_alive_fds,
- sizeof(postmaster_alive_fds));
-#endif
-
- memcpy(&syslogPipe, ¶m->syslogPipe, sizeof(syslogPipe));
-
- strlcpy(my_exec_path, param->my_exec_path, MAXPGPATH);
-
- strlcpy(pkglib_path, param->pkglib_path, MAXPGPATH);
-
- /*
- * We need to restore fd.c's counts of externally-opened FDs; to avoid
- * confusion, be sure to do this after restoring max_safe_fds. (Note:
- * BackendInitialize will handle this for (*client_sock)->sock.)
- */
-#ifndef WIN32
- if (postmaster_alive_fds[0] >= 0)
- ReserveExternalFD();
- if (postmaster_alive_fds[1] >= 0)
- ReserveExternalFD();
-#endif
-}
-
-
Size
ShmemBackendArraySize(void)
{
@@ -6517,7 +5784,7 @@ pgwin32_deadchild_callback(PVOID lpParameter, BOOLEAN TimerOrWaitFired)
* automatically by an operating system thread pool. The memory and the
* process handle will be freed by a later call to waitpid().
*/
-static void
+void
pgwin32_register_deadchild_callback(HANDLE procHandle, DWORD procId)
{
win32_deadchild_waitinfo *childinfo;
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index 46f95f7c8a6..53f82bb5d43 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -59,11 +59,16 @@ extern int MaxLivePostmasterChildren(void);
extern bool PostmasterMarkPIDForWorkerNotify(int);
#ifdef EXEC_BACKEND
+
extern pid_t postmaster_forkexec(int argc, char *argv[]);
extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
extern Size ShmemBackendArraySize(void);
extern void ShmemBackendArrayAllocation(void);
+
+#ifdef WIN32
+extern void pgwin32_register_deadchild_callback(HANDLE procHandle, DWORD procId);
+#endif
#endif
/*
--
2.39.2
v13-0004-Refactor-AuxProcess-startup.patchtext/x-patch; charset=UTF-8; name=v13-0004-Refactor-AuxProcess-startup.patchDownload
From 08e0c8f15b83fdd7544bae2fd7cf83195e4f8da2 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Mon, 4 Dec 2023 13:24:01 +0200
Subject: [PATCH v13 4/6] Refactor AuxProcess startup
Old:
AuxProcessMain()
Initialize a bunch of stuff
<aux process specific Main function>()
New:
AuxProcessMain()
<aux process specific Main function>()
AuxiliaryProcessMainCommon()
Initialize a bunch of stuff
This isn't too useful as is, but the next commits will remove the
AuxProcessMain() function and dispatch directly to the aux-process
specific Main function, like this:
<aux process specific Main function>()
AuxiliaryProcessMainCommon()
Initialize a bunch of stuff
This commit makes that next commit smaller.
---
src/backend/postmaster/auxprocess.c | 90 +++++++++++++++-----------
src/backend/postmaster/bgwriter.c | 3 +
src/backend/postmaster/checkpointer.c | 3 +
src/backend/postmaster/pgarch.c | 3 +
src/backend/postmaster/startup.c | 3 +
src/backend/postmaster/walsummarizer.c | 3 +
src/backend/postmaster/walwriter.c | 3 +
src/backend/replication/walreceiver.c | 3 +
src/include/postmaster/auxprocess.h | 3 +-
9 files changed, 76 insertions(+), 38 deletions(-)
diff --git a/src/backend/postmaster/auxprocess.c b/src/backend/postmaster/auxprocess.c
index 2c86abdb719..b5a35f6c0ea 100644
--- a/src/backend/postmaster/auxprocess.c
+++ b/src/backend/postmaster/auxprocess.c
@@ -27,6 +27,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
+#include "utils/memutils.h"
#include "utils/ps_status.h"
@@ -47,6 +48,59 @@ AuxiliaryProcessMain(BackendType auxtype)
Assert(IsUnderPostmaster);
MyBackendType = auxtype;
+ switch (MyBackendType)
+ {
+ case B_STARTUP:
+ StartupProcessMain();
+ proc_exit(1);
+
+ case B_ARCHIVER:
+ PgArchiverMain();
+ proc_exit(1);
+
+ case B_BG_WRITER:
+ BackgroundWriterMain();
+ proc_exit(1);
+
+ case B_CHECKPOINTER:
+ CheckpointerMain();
+ proc_exit(1);
+
+ case B_WAL_WRITER:
+ WalWriterMain();
+ proc_exit(1);
+
+ case B_WAL_RECEIVER:
+ WalReceiverMain();
+ proc_exit(1);
+
+ case B_WAL_SUMMARIZER:
+ WalSummarizerMain();
+ proc_exit(1);
+
+ default:
+ elog(PANIC, "unrecognized process type: %d", (int) MyBackendType);
+ proc_exit(1);
+ }
+}
+
+/*
+ * AuxiliaryProcessMainCommon
+ *
+ * Common initialization code for auxiliary processes, such as the bgwriter,
+ * walwriter, walreceiver, bootstrapper and the shared memory checker code.
+ */
+void
+AuxiliaryProcessMainCommon(void)
+{
+ /* Release postmaster's working memory context */
+ if (PostmasterContext)
+ {
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
+ }
+
+ Assert(IsUnderPostmaster);
init_ps_display(NULL);
@@ -75,7 +129,6 @@ AuxiliaryProcessMain(BackendType auxtype)
*/
CreateAuxProcessResourceOwner();
-
/* Initialize backend status information */
pgstat_beinit();
pgstat_bestart();
@@ -84,41 +137,6 @@ AuxiliaryProcessMain(BackendType auxtype)
before_shmem_exit(ShutdownAuxiliaryProcess, 0);
SetProcessingMode(NormalProcessing);
-
- switch (MyBackendType)
- {
- case B_STARTUP:
- StartupProcessMain();
- proc_exit(1);
-
- case B_ARCHIVER:
- PgArchiverMain();
- proc_exit(1);
-
- case B_BG_WRITER:
- BackgroundWriterMain();
- proc_exit(1);
-
- case B_CHECKPOINTER:
- CheckpointerMain();
- proc_exit(1);
-
- case B_WAL_WRITER:
- WalWriterMain();
- proc_exit(1);
-
- case B_WAL_RECEIVER:
- WalReceiverMain();
- proc_exit(1);
-
- case B_WAL_SUMMARIZER:
- WalSummarizerMain();
- proc_exit(1);
-
- default:
- elog(PANIC, "unrecognized process type: %d", (int) MyBackendType);
- proc_exit(1);
- }
}
/*
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index da2d95b9261..ad500b7fa2a 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -35,6 +35,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/bgwriter.h"
#include "postmaster/interrupt.h"
#include "storage/buf_internals.h"
@@ -90,6 +91,8 @@ BackgroundWriterMain(void)
bool prev_hibernate;
WritebackContext wb_context;
+ AuxiliaryProcessMainCommon();
+
/*
* Properly accept or ignore signals that might be sent to us.
*/
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 46197d56f86..b8d943a550e 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -42,6 +42,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/bgwriter.h"
#include "postmaster/interrupt.h"
#include "replication/syncrep.h"
@@ -174,6 +175,8 @@ CheckpointerMain(void)
sigjmp_buf local_sigjmp_buf;
MemoryContext checkpointer_context;
+ AuxiliaryProcessMainCommon();
+
CheckpointerShmem->checkpointer_pid = MyProcPid;
/*
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index f97035ca03c..14e45bb7071 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -36,6 +36,7 @@
#include "lib/binaryheap.h"
#include "libpq/pqsignal.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/interrupt.h"
#include "postmaster/pgarch.h"
#include "storage/fd.h"
@@ -211,6 +212,8 @@ PgArchCanRestart(void)
void
PgArchiverMain(void)
{
+ AuxiliaryProcessMainCommon();
+
/*
* Ignore all signals usually bound to some action in the postmaster,
* except for SIGHUP, SIGTERM, SIGUSR1, SIGUSR2, and SIGQUIT.
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index 8b51e45bad0..ea7db2040f0 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -24,6 +24,7 @@
#include "access/xlogutils.h"
#include "libpq/pqsignal.h"
#include "miscadmin.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/startup.h"
#include "storage/ipc.h"
#include "storage/pmsignal.h"
@@ -214,6 +215,8 @@ StartupProcExit(int code, Datum arg)
void
StartupProcessMain(void)
{
+ AuxiliaryProcessMainCommon();
+
/* Arrange to clean up at startup process exit */
on_shmem_exit(StartupProcExit, 0);
diff --git a/src/backend/postmaster/walsummarizer.c b/src/backend/postmaster/walsummarizer.c
index 8bf8a03471a..8d9ee1f0f9b 100644
--- a/src/backend/postmaster/walsummarizer.c
+++ b/src/backend/postmaster/walsummarizer.c
@@ -33,6 +33,7 @@
#include "common/blkreftable.h"
#include "libpq/pqsignal.h"
#include "miscadmin.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/interrupt.h"
#include "postmaster/walsummarizer.h"
#include "replication/walreceiver.h"
@@ -228,6 +229,8 @@ WalSummarizerMain(void)
XLogRecPtr switch_lsn = InvalidXLogRecPtr;
TimeLineID switch_tli = 0;
+ AuxiliaryProcessMainCommon();
+
ereport(DEBUG1,
(errmsg_internal("WAL summarizer started")));
diff --git a/src/backend/postmaster/walwriter.c b/src/backend/postmaster/walwriter.c
index e079dc65c88..dc052854db8 100644
--- a/src/backend/postmaster/walwriter.c
+++ b/src/backend/postmaster/walwriter.c
@@ -48,6 +48,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/interrupt.h"
#include "postmaster/walwriter.h"
#include "storage/bufmgr.h"
@@ -92,6 +93,8 @@ WalWriterMain(void)
int left_till_hibernate;
bool hibernating;
+ AuxiliaryProcessMainCommon();
+
/*
* Properly accept or ignore signals the postmaster might send us
*
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index 5a0652c9426..bfe7e4a8894 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -63,6 +63,7 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "postmaster/auxprocess.h"
#include "postmaster/interrupt.h"
#include "replication/walreceiver.h"
#include "replication/walsender.h"
@@ -195,6 +196,8 @@ WalReceiverMain(void)
char *sender_host = NULL;
int sender_port = 0;
+ AuxiliaryProcessMainCommon();
+
/*
* WalRcv should be set up already (if we are a backend, we inherit this
* by fork() or EXEC_BACKEND mechanism from the postmaster).
diff --git a/src/include/postmaster/auxprocess.h b/src/include/postmaster/auxprocess.h
index 3e443edde70..2cbd29534c4 100644
--- a/src/include/postmaster/auxprocess.h
+++ b/src/include/postmaster/auxprocess.h
@@ -13,8 +13,7 @@
#ifndef AUXPROCESS_H
#define AUXPROCESS_H
-#include "miscadmin.h"
-
extern void AuxiliaryProcessMain(BackendType auxtype) pg_attribute_noreturn();
+extern void AuxiliaryProcessMainCommon(void);
#endif /* AUXPROCESS_H */
--
2.39.2
v13-0005-Refactor-postmaster-child-process-launching.patchtext/x-patch; charset=UTF-8; name=v13-0005-Refactor-postmaster-child-process-launching.patchDownload
From cdcd304acf85d8b41b344ba1c289b8f9f8365160 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Tue, 12 Mar 2024 14:02:32 +0200
Subject: [PATCH v13 5/6] Refactor postmaster child process launching
- Introduce new postmaster_child_launch() function that deals with the
differences between EXEC_BACKEND and fork mode.
- Refactor the mechanism of passing information from the parent to
child process. Instead of using different command-line arguments when
launching the child process in EXEC_BACKEND mode, pass a
variable-length blob of startup data along with all the global
variables. The contents of that blob depend on the kind of child
process being launched. In !EXEC_BACKEND mode, we use the same blob,
but it's simply inherited from the parent to child process.
Reviewed-by: Tristan Partin, Andres Freund
Discussion: https://www.postgresql.org/message-id/7a59b073-5b5b-151e-7ed3-8b01ff7ce9ef@iki.fi
---
src/backend/postmaster/autovacuum.c | 153 +-----
src/backend/postmaster/auxprocess.c | 50 --
src/backend/postmaster/bgworker.c | 14 +-
src/backend/postmaster/bgwriter.c | 5 +-
src/backend/postmaster/checkpointer.c | 5 +-
src/backend/postmaster/launch_backend.c | 419 ++++++++++++----
src/backend/postmaster/pgarch.c | 5 +-
src/backend/postmaster/postmaster.c | 515 +++-----------------
src/backend/postmaster/startup.c | 5 +-
src/backend/postmaster/syslogger.c | 275 +++++------
src/backend/postmaster/walsummarizer.c | 5 +-
src/backend/postmaster/walwriter.c | 5 +-
src/backend/replication/walreceiver.c | 5 +-
src/backend/utils/init/globals.c | 1 +
src/include/postmaster/autovacuum.h | 10 +-
src/include/postmaster/auxprocess.h | 1 -
src/include/postmaster/bgworker_internals.h | 2 +-
src/include/postmaster/bgwriter.h | 4 +-
src/include/postmaster/pgarch.h | 2 +-
src/include/postmaster/postmaster.h | 25 +-
src/include/postmaster/startup.h | 2 +-
src/include/postmaster/syslogger.h | 4 +-
src/include/postmaster/walsummarizer.h | 2 +-
src/include/postmaster/walwriter.h | 2 +-
src/include/replication/walreceiver.h | 2 +-
src/tools/pgindent/typedefs.list | 3 +
26 files changed, 595 insertions(+), 926 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 7d7fff6f5d5..b055fe1f740 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -85,7 +85,6 @@
#include "nodes/makefuncs.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "storage/bufmgr.h"
@@ -311,13 +310,6 @@ static WorkerInfo MyWorkerInfo = NULL;
/* PID of launcher, valid only in worker while shutting down */
int AutovacuumLauncherPid = 0;
-#ifdef EXEC_BACKEND
-static pid_t avlauncher_forkexec(void);
-static pid_t avworker_forkexec(void);
-#endif
-NON_EXEC_STATIC void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
-NON_EXEC_STATIC void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
-
static Oid do_start_worker(void);
static void HandleAutoVacLauncherInterrupts(void);
static void AutoVacLauncherShutdown(void) pg_attribute_noreturn();
@@ -361,76 +353,21 @@ static void avl_sigusr2_handler(SIGNAL_ARGS);
* AUTOVACUUM LAUNCHER CODE
********************************************************************/
-#ifdef EXEC_BACKEND
/*
- * forkexec routine for the autovacuum launcher process.
- *
- * Format up the arglist, then fork and exec.
- */
-static pid_t
-avlauncher_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkavlauncher";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif
-
-/*
- * Main entry point for autovacuum launcher process, to be called from the
- * postmaster.
+ * Main loop for the autovacuum launcher process.
*/
-int
-StartAutoVacLauncher(void)
+void
+AutoVacLauncherMain(char *startup_data, size_t startup_data_len)
{
- pid_t AutoVacPID;
+ sigjmp_buf local_sigjmp_buf;
-#ifdef EXEC_BACKEND
- switch ((AutoVacPID = avlauncher_forkexec()))
-#else
- switch ((AutoVacPID = fork_process()))
-#endif
+ /* Release postmaster's working memory context */
+ if (PostmasterContext)
{
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum launcher process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacLauncherMain(0, NULL);
- break;
-#endif
- default:
- return (int) AutoVacPID;
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
}
- /* shouldn't get here */
- return 0;
-}
-
-/*
- * Main loop for the autovacuum launcher process.
- */
-NON_EXEC_STATIC void
-AutoVacLauncherMain(int argc, char *argv[])
-{
- sigjmp_buf local_sigjmp_buf;
-
MyBackendType = B_AUTOVAC_LAUNCHER;
init_ps_display(NULL);
@@ -1412,78 +1349,22 @@ avl_sigusr2_handler(SIGNAL_ARGS)
* AUTOVACUUM WORKER CODE
********************************************************************/
-#ifdef EXEC_BACKEND
-/*
- * forkexec routines for the autovacuum worker.
- *
- * Format up the arglist, then fork and exec.
- */
-static pid_t
-avworker_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkavworker";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif
-
-/*
- * Main entry point for autovacuum worker process.
- *
- * This code is heavily based on pgarch.c, q.v.
- */
-int
-StartAutoVacWorker(void)
-{
- pid_t worker_pid;
-
-#ifdef EXEC_BACKEND
- switch ((worker_pid = avworker_forkexec()))
-#else
- switch ((worker_pid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum worker process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacWorkerMain(0, NULL);
- break;
-#endif
- default:
- return (int) worker_pid;
- }
-
- /* shouldn't get here */
- return 0;
-}
-
/*
* AutoVacWorkerMain
*/
-NON_EXEC_STATIC void
-AutoVacWorkerMain(int argc, char *argv[])
+void
+AutoVacWorkerMain(char *startup_data, size_t startup_data_len)
{
sigjmp_buf local_sigjmp_buf;
Oid dbid;
+ /* Release postmaster's working memory context */
+ if (PostmasterContext)
+ {
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
+ }
+
MyBackendType = B_AUTOVAC_WORKER;
init_ps_display(NULL);
diff --git a/src/backend/postmaster/auxprocess.c b/src/backend/postmaster/auxprocess.c
index b5a35f6c0ea..76adfbff147 100644
--- a/src/backend/postmaster/auxprocess.c
+++ b/src/backend/postmaster/auxprocess.c
@@ -34,56 +34,6 @@
static void ShutdownAuxiliaryProcess(int code, Datum arg);
-/*
- * AuxiliaryProcessMain
- *
- * The main entry point for auxiliary processes, such as the bgwriter,
- * walwriter, walreceiver, bootstrapper and the shared memory checker code.
- *
- * This code is here just because of historical reasons.
- */
-void
-AuxiliaryProcessMain(BackendType auxtype)
-{
- Assert(IsUnderPostmaster);
-
- MyBackendType = auxtype;
- switch (MyBackendType)
- {
- case B_STARTUP:
- StartupProcessMain();
- proc_exit(1);
-
- case B_ARCHIVER:
- PgArchiverMain();
- proc_exit(1);
-
- case B_BG_WRITER:
- BackgroundWriterMain();
- proc_exit(1);
-
- case B_CHECKPOINTER:
- CheckpointerMain();
- proc_exit(1);
-
- case B_WAL_WRITER:
- WalWriterMain();
- proc_exit(1);
-
- case B_WAL_RECEIVER:
- WalReceiverMain();
- proc_exit(1);
-
- case B_WAL_SUMMARIZER:
- WalSummarizerMain();
- proc_exit(1);
-
- default:
- elog(PANIC, "unrecognized process type: %d", (int) MyBackendType);
- proc_exit(1);
- }
-}
-
/*
* AuxiliaryProcessMainCommon
*
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index b73e91f0c86..a2a7e533119 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -720,15 +720,25 @@ bgworker_die(SIGNAL_ARGS)
* Main entry point for background worker processes.
*/
void
-BackgroundWorkerMain(void)
+BackgroundWorkerMain(char *startup_data, size_t startup_data_len)
{
sigjmp_buf local_sigjmp_buf;
- BackgroundWorker *worker = MyBgworkerEntry;
+ BackgroundWorker *worker;
bgworker_main_type entrypt;
+ /* Release postmaster's working memory context */
+ if (PostmasterContext)
+ {
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
+ }
+
+ Assert(startup_data_len == sizeof(BackgroundWorker));
+ worker = (BackgroundWorker *) startup_data;
if (worker == NULL)
elog(FATAL, "unable to find bgworker entry");
+ MyBgworkerEntry = worker;
MyBackendType = B_BG_WORKER;
init_ps_display(worker->bgw_name);
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index ad500b7fa2a..0f75548759a 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -84,13 +84,16 @@ static XLogRecPtr last_snapshot_lsn = InvalidXLogRecPtr;
* basic execution environment, but not enabled signals yet.
*/
void
-BackgroundWriterMain(void)
+BackgroundWriterMain(char *startup_data, size_t startup_data_len)
{
sigjmp_buf local_sigjmp_buf;
MemoryContext bgwriter_context;
bool prev_hibernate;
WritebackContext wb_context;
+ Assert(startup_data_len == 0);
+
+ MyBackendType = B_BG_WRITER;
AuxiliaryProcessMainCommon();
/*
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index b8d943a550e..8ef600ae72a 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -170,11 +170,14 @@ static void ReqCheckpointHandler(SIGNAL_ARGS);
* basic execution environment, but not enabled signals yet.
*/
void
-CheckpointerMain(void)
+CheckpointerMain(char *startup_data, size_t startup_data_len)
{
sigjmp_buf local_sigjmp_buf;
MemoryContext checkpointer_context;
+ Assert(startup_data_len == 0);
+
+ MyBackendType = B_CHECKPOINTER;
AuxiliaryProcessMainCommon();
CheckpointerShmem->checkpointer_pid = MyProcPid;
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index cd81abd12a8..5290d49957c 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -49,6 +49,7 @@
#include "postmaster/postmaster.h"
#include "postmaster/startup.h"
#include "postmaster/syslogger.h"
+#include "postmaster/walsummarizer.h"
#include "postmaster/walwriter.h"
#include "replication/walreceiver.h"
#include "storage/fd.h"
@@ -86,17 +87,10 @@ typedef int InheritableSocket;
#endif
/*
- * Structure contains all variables passed to exec:ed backends
+ * Structure contains all global variables passed to exec:ed backends
*/
typedef struct
{
- bool has_client_sock;
- ClientSocket client_sock;
- InheritableSocket inh_sock;
-
- bool has_bgworker;
- BackgroundWorker bgworker;
-
char DataDir[MAXPGPATH];
int32 MyCancelKey;
int MyPMChildSlot;
@@ -139,22 +133,136 @@ typedef struct
#endif
char my_exec_path[MAXPGPATH];
char pkglib_path[MAXPGPATH];
+
+ /*
+ * These are only used by backend processes, but it's here because passing
+ * a socket needs some special handling on Windows. 'client_sock' is an
+ * explicit argument to postmaster_child_launch, but is stored in
+ * MyClientSocket in the child process.
+ */
+ ClientSocket client_sock;
+ InheritableSocket inh_sock;
+
+ size_t startup_data_len;
+ /* startup data follows */
+ char startup_data[FLEXIBLE_ARRAY_MEMBER];
} BackendParameters;
#define SizeOfBackendParameters(startup_data_len) (offsetof(BackendParameters, startup_data) + startup_data_len)
-void read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **worker);
-static void restore_backend_variables(BackendParameters *param, ClientSocket **client_sock, BackgroundWorker **worker);
+static void read_backend_variables(char *id, char **startup_data, size_t *startup_data_len);
+static void restore_backend_variables(BackendParameters *param);
-#ifndef WIN32
-static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker);
-#else
-static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker,
- HANDLE childProcess, pid_t childPid);
+static bool save_backend_variables(BackendParameters *param, ClientSocket *client_sock,
+#ifdef WIN32
+ HANDLE childProcess, pid_t childPid,
#endif
+ char *startup_data, size_t startup_data_len);
+
+static pid_t internal_forkexec(const char *child_kind, char *startup_data, size_t startup_data_len, ClientSocket *client_sock);
+
+#endif /* EXEC_BACKEND */
+
+/*
+ * Information needed to launch different kinds of child processes.
+ */
+typedef struct
+{
+ const char *name;
+ void (*main_fn) (char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
+ bool shmem_attach;
+} child_process_kind;
+
+child_process_kind child_process_kinds[] = {
+ [B_BACKEND] = {"backend", BackendMain, true},
+ [B_AUTOVAC_LAUNCHER] = {"autovacuum launcher", AutoVacLauncherMain, true},
+ [B_AUTOVAC_WORKER] = {"autovacuum worker", AutoVacWorkerMain, true},
+ [B_BG_WORKER] = {"bgworker", BackgroundWorkerMain, true},
+ [B_WAL_SENDER] = {"wal sender", NULL, false},
+ [B_SLOTSYNC_WORKER] = {"slot sync worker", NULL, true},
+
+ [B_STANDALONE_BACKEND] = {"standalone backend", NULL, false},
+
+ [B_ARCHIVER] = {"archiver", PgArchiverMain, true},
+ [B_BG_WRITER] = {"bgwriter", BackgroundWriterMain, true},
+ [B_CHECKPOINTER] = {"checkpointer", CheckpointerMain, true},
+ [B_STARTUP] = {"startup", StartupProcessMain, true},
+ [B_WAL_RECEIVER] = {"wal_receiver", WalReceiverMain, true},
+ [B_WAL_SUMMARIZER] = {"wal_summarizer", WalSummarizerMain, true},
+ [B_WAL_WRITER] = {"wal_writer", WalWriterMain, true},
+
+ [B_LOGGER] = {"syslogger", SysLoggerMain, false},
+};
+
+const char *
+PostmasterChildName(BackendType child_type)
+{
+ Assert(child_type >= 0 && child_type < lengthof(child_process_kinds));
+ return child_process_kinds[child_type].name;
+}
+
+/*
+ * Start a new postmaster child process.
+ *
+ * The child process will be restored to roughly the same state, whether
+ * EXEC_BACKEND is used or not: it will be attached to shared memory, and fds
+ * and other resources that we've inherited from postmaster that are not
+ * needed in a child process have been closed.
+ *
+ * 'startup_data' is an optional contiguous chunk of data that is passed to
+ * the child process.
+ */
+pid_t
+postmaster_child_launch(BackendType child_type,
+ char *startup_data, size_t startup_data_len,
+ ClientSocket *client_sock)
+{
+ pid_t pid;
+
+ Assert(child_type >= 0 && child_type < lengthof(child_process_kinds));
+ Assert(IsPostmasterEnvironment && !IsUnderPostmaster);
+
+#ifdef EXEC_BACKEND
+ pid = internal_forkexec(child_process_kinds[child_type].name,
+ startup_data, startup_data_len, client_sock);
+ /* the child process will arrive in SubPostmasterMain */
+#else /* !EXEC_BACKEND */
+ pid = fork_process();
+ if (pid == 0) /* child */
+ {
+ /* Close the postmaster's sockets */
+ ClosePostmasterPorts(child_type == B_LOGGER);
-pid_t internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker);
+ /* Detangle from postmaster */
+ InitPostmasterChild();
+ /*
+ * Before blowing away PostmasterContext (in the Main function), save
+ * the startup data.
+ */
+ MemoryContextSwitchTo(TopMemoryContext);
+ if (startup_data != NULL)
+ {
+ char *cp = palloc(startup_data_len);
+
+ memcpy(cp, startup_data, startup_data_len);
+ startup_data = cp;
+ }
+
+ if (client_sock)
+ {
+ MyClientSocket = palloc(sizeof(ClientSocket));
+ memcpy(MyClientSocket, client_sock, sizeof(ClientSocket));
+ }
+
+ child_process_kinds[child_type].main_fn(startup_data, startup_data_len);
+ pg_unreachable(); /* main_fn never returns */
+ }
+#endif /* EXEC_BACKEND */
+ return pid;
+}
+
+#ifdef EXEC_BACKEND
#ifndef WIN32
/*
@@ -163,25 +271,25 @@ pid_t internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, Back
* - writes out backend variables to the parameter file
* - fork():s, and then exec():s the child process
*/
-pid_t
-internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker)
+static pid_t
+internal_forkexec(const char *child_kind, char *startup_data, size_t startup_data_len, ClientSocket *client_sock)
{
static unsigned long tmpBackendFileNum = 0;
pid_t pid;
char tmpfilename[MAXPGPATH];
- BackendParameters param;
+ size_t paramsz;
+ BackendParameters *param;
FILE *fp;
+ char *argv[4];
+ char forkav[MAXPGPATH];
- /*
- * Make sure padding bytes are initialized, to prevent Valgrind from
- * complaining about writing uninitialized bytes to the file. This isn't
- * performance critical, and the win32 implementation initializes the
- * padding bytes to zeros, so do it even when not using Valgrind.
- */
- memset(¶m, 0, sizeof(BackendParameters));
-
- if (!save_backend_variables(¶m, client_sock, worker))
+ paramsz = SizeOfBackendParameters(startup_data_len);
+ param = palloc(paramsz);
+ if (!save_backend_variables(param, client_sock, startup_data, startup_data_len))
+ {
+ pfree(param);
return -1; /* log made by save_backend_variables */
+ }
/* Calculate name for temp file */
snprintf(tmpfilename, MAXPGPATH, "%s/%s.backend_var.%d.%lu",
@@ -209,7 +317,7 @@ internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundW
}
}
- if (fwrite(¶m, sizeof(param), 1, fp) != 1)
+ if (fwrite(param, paramsz, 1, fp) != 1)
{
ereport(LOG,
(errcode_for_file_access(),
@@ -227,14 +335,13 @@ internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundW
return -1;
}
- /* Make sure caller set up argv properly */
- Assert(argc >= 3);
- Assert(argv[argc] == NULL);
- Assert(strncmp(argv[1], "--fork", 6) == 0);
- Assert(argv[2] == NULL);
-
- /* Insert temp file name after --fork argument */
+ /* set up argv properly */
+ argv[0] = "postgres";
+ snprintf(forkav, MAXPGPATH, "--forkchild=%s", child_kind);
+ argv[1] = forkav;
+ /* Insert temp file name after --forkchild argument */
argv[2] = tmpfilename;
+ argv[3] = NULL;
/* Fire off execv in child */
if ((pid = fork_process()) == 0)
@@ -263,26 +370,21 @@ internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundW
* - resumes execution of the new process once the backend parameter
* file is complete.
*/
-pid_t
-internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker)
+static pid_t
+internal_forkexec(const char *child_kind, char *startup_data, size_t startup_data_len, ClientSocket *client_sock)
{
int retry_count = 0;
STARTUPINFO si;
PROCESS_INFORMATION pi;
- int i;
- int j;
char cmdLine[MAXPGPATH * 2];
HANDLE paramHandle;
BackendParameters *param;
SECURITY_ATTRIBUTES sa;
+ size_t paramsz;
char paramHandleStr[32];
- win32_deadchild_waitinfo *childinfo;
+ int l;
- /* Make sure caller set up argv properly */
- Assert(argc >= 3);
- Assert(argv[argc] == NULL);
- Assert(strncmp(argv[1], "--fork", 6) == 0);
- Assert(argv[2] == NULL);
+ paramsz = SizeOfBackendParameters(startup_data_len);
/* Resume here if we need to retry */
retry:
@@ -295,7 +397,7 @@ retry:
&sa,
PAGE_READWRITE,
0,
- sizeof(BackendParameters),
+ paramsz,
NULL);
if (paramHandle == INVALID_HANDLE_VALUE)
{
@@ -304,8 +406,7 @@ retry:
GetLastError())));
return -1;
}
-
- param = MapViewOfFile(paramHandle, FILE_MAP_WRITE, 0, 0, sizeof(BackendParameters));
+ param = MapViewOfFile(paramHandle, FILE_MAP_WRITE, 0, 0, paramsz);
if (!param)
{
ereport(LOG,
@@ -315,25 +416,15 @@ retry:
return -1;
}
- /* Insert temp file name after --fork argument */
+ /* Format the cmd line */
#ifdef _WIN64
sprintf(paramHandleStr, "%llu", (LONG_PTR) paramHandle);
#else
sprintf(paramHandleStr, "%lu", (DWORD) paramHandle);
#endif
- argv[2] = paramHandleStr;
-
- /* Format the cmd line */
- cmdLine[sizeof(cmdLine) - 1] = '\0';
- cmdLine[sizeof(cmdLine) - 2] = '\0';
- snprintf(cmdLine, sizeof(cmdLine) - 1, "\"%s\"", postgres_exec_path);
- i = 0;
- while (argv[++i] != NULL)
- {
- j = strlen(cmdLine);
- snprintf(cmdLine + j, sizeof(cmdLine) - 1 - j, " \"%s\"", argv[i]);
- }
- if (cmdLine[sizeof(cmdLine) - 2] != '\0')
+ l = snprintf(cmdLine, sizeof(cmdLine) - 1, "\"%s\" --forkchild=\"%s\" %s",
+ postgres_exec_path, child_kind, paramHandleStr);
+ if (l >= sizeof(cmdLine))
{
ereport(LOG,
(errmsg("subprocess command line too long")));
@@ -361,7 +452,7 @@ retry:
return -1;
}
- if (!save_backend_variables(param, client_sock, worker, pi.hProcess, pi.dwProcessId))
+ if (!save_backend_variables(param, client_sock, pi.hProcess, pi.dwProcessId, startup_data, startup_data_len))
{
/*
* log made by save_backend_variables, but we have to clean up the
@@ -447,6 +538,117 @@ retry:
}
#endif /* WIN32 */
+/*
+ * SubPostmasterMain -- Get the fork/exec'd process into a state equivalent
+ * to what it would be if we'd simply forked on Unix, and then
+ * dispatch to the appropriate place.
+ *
+ * The first two command line arguments are expected to be "--forkchild=<name>",
+ * where <name> indicates which postmaster child we are to become, and
+ * the name of a variables file that we can read to load data that would
+ * have been inherited by fork() on Unix.
+ */
+void
+SubPostmasterMain(int argc, char *argv[])
+{
+ char *startup_data;
+ size_t startup_data_len;
+ char *child_kind;
+ PostmasterChildType child_type;
+ bool found = false;
+
+ /* In EXEC_BACKEND case we will not have inherited these settings */
+ IsPostmasterEnvironment = true;
+ whereToSendOutput = DestNone;
+
+ /* Setup essential subsystems (to ensure elog() behaves sanely) */
+ InitializeGUCOptions();
+
+ /* Check we got appropriate args */
+ if (argc != 3)
+ elog(FATAL, "invalid subpostmaster invocation");
+
+ /* Find the entry in child_process_kinds */
+ if (strncmp(argv[1], "--forkchild=", 12) != 0)
+ elog(FATAL, "invalid subpostmaster invocation (--forkchild argument missing)");
+ child_kind = argv[1] + 12;
+ found = false;
+ for (int idx = 0; idx < lengthof(child_process_kinds); idx++)
+ {
+ if (strcmp(child_process_kinds[idx].name, child_kind) == 0)
+ {
+ child_type = idx;
+ found = true;
+ break;
+ }
+ }
+ if (!found)
+ elog(ERROR, "unknown child kind %s", child_kind);
+
+ /* Read in the variables file */
+ read_backend_variables(argv[2], &startup_data, &startup_data_len);
+
+ /* Close the postmaster's sockets (as soon as we know them) */
+ ClosePostmasterPorts(child_type == PMC_SYSLOGGER);
+
+ /* Setup as postmaster child */
+ InitPostmasterChild();
+
+ /*
+ * If appropriate, physically re-attach to shared memory segment. We want
+ * to do this before going any further to ensure that we can attach at the
+ * same address the postmaster used. On the other hand, if we choose not
+ * to re-attach, we may have other cleanup to do.
+ *
+ * If testing EXEC_BACKEND on Linux, you should run this as root before
+ * starting the postmaster:
+ *
+ * sysctl -w kernel.randomize_va_space=0
+ *
+ * This prevents using randomized stack and code addresses that cause the
+ * child process's memory map to be different from the parent's, making it
+ * sometimes impossible to attach to shared memory at the desired address.
+ * Return the setting to its old value (usually '1' or '2') when finished.
+ */
+ if (child_process_kinds[child_type].shmem_attach)
+ PGSharedMemoryReAttach();
+ else
+ PGSharedMemoryNoReAttach();
+
+ /* Read in remaining GUC variables */
+ read_nondefault_variables();
+
+ /*
+ * Check that the data directory looks valid, which will also check the
+ * privileges on the data directory and update our umask and file/group
+ * variables for creating files later. Note: this should really be done
+ * before we create any files or directories.
+ */
+ checkDataDir();
+
+ /*
+ * (re-)read control file, as it contains config. The postmaster will
+ * already have read this, but this process doesn't know about that.
+ */
+ LocalProcessControlFile(false);
+
+ /*
+ * Reload any libraries that were preloaded by the postmaster. Since we
+ * exec'd this process, those libraries didn't come along with us; but we
+ * should load them into all child processes to be consistent with the
+ * non-EXEC_BACKEND behavior.
+ */
+ process_shared_preload_libraries();
+
+ /* Restore basic shared memory pointers */
+ if (UsedShmemSegAddr != NULL)
+ InitShmemAccess(UsedShmemSegAddr);
+
+ /* Run backend or appropriate child */
+ child_process_kinds[child_type].main_fn(startup_data, startup_data_len);
+ pg_unreachable(); /* main_fn never returns */
+}
+
/*
* The following need to be available to the save/restore_backend_variables
* functions. They are marked NON_EXEC_STATIC in their home modules.
@@ -470,38 +672,22 @@ static bool write_inheritable_socket(InheritableSocket *dest, SOCKET src,
static void read_inheritable_socket(SOCKET *dest, InheritableSocket *src);
#endif
-#ifndef WIN32
-static bool
-save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker)
-#else
+
static bool
-save_backend_variables(BackendParameters *param, ClientSocket *client_sock, BackgroundWorker *worker,
- HANDLE childProcess, pid_t childPid)
+save_backend_variables(BackendParameters *param, ClientSocket *client_sock,
+#ifdef WIN32
+ HANDLE childProcess, pid_t childPid,
#endif
+ char *startup_data, size_t startup_data_len)
{
if (client_sock)
- {
memcpy(¶m->client_sock, client_sock, sizeof(ClientSocket));
- if (!write_inheritable_socket(¶m->inh_sock, client_sock->sock, childPid))
- return false;
- param->has_client_sock = true;
- }
else
- {
memset(¶m->client_sock, 0, sizeof(ClientSocket));
- param->has_client_sock = false;
- }
-
- if (worker)
- {
- memcpy(¶m->bgworker, worker, sizeof(BackgroundWorker));
- param->has_bgworker = true;
- }
- else
- {
- memset(¶m->bgworker, 0, sizeof(BackgroundWorker));
- param->has_bgworker = false;
- }
+ if (!write_inheritable_socket(¶m->inh_sock,
+ client_sock ? client_sock->sock : PGINVALID_SOCKET,
+ childPid))
+ return false;
strlcpy(param->DataDir, DataDir, MAXPGPATH);
@@ -558,6 +744,9 @@ save_backend_variables(BackendParameters *param, ClientSocket *client_sock, Back
strlcpy(param->pkglib_path, pkglib_path, MAXPGPATH);
+ param->startup_data_len = startup_data_len;
+ memcpy(param->startup_data, startup_data, startup_data_len);
+
return true;
}
@@ -654,8 +843,8 @@ read_inheritable_socket(SOCKET *dest, InheritableSocket *src)
}
#endif
-void
-read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **worker)
+static void
+read_backend_variables(char *id, char **startup_data, size_t *startup_data_len)
{
BackendParameters param;
@@ -677,6 +866,21 @@ read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **
exit(1);
}
+ /* read startup data */
+ *startup_data_len = param.startup_data_len;
+ if (param.startup_data_len > 0)
+ {
+ *startup_data = palloc(*startup_data_len);
+ if (fread(*startup_data, *startup_data_len, 1, fp) != 1)
+ {
+ write_stderr("could not read startup data from backend variables file \"%s\": %s\n",
+ id, strerror(errno));
+ exit(1);
+ }
+ }
+ else
+ *startup_data = NULL;
+
/* Release file */
FreeFile(fp);
if (unlink(id) != 0)
@@ -704,6 +908,16 @@ read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **
memcpy(¶m, paramp, sizeof(BackendParameters));
+ /* read startup data */
+ *startup_data_len = param.startup_data_len;
+ if (param.startup_data_len > 0)
+ {
+ *startup_data = palloc(paramp->startup_data_len);
+ memcpy(*startup_data, paramp->startup_data, param.startup_data_len);
+ }
+ else
+ *startup_data = NULL;
+
if (!UnmapViewOfFile(paramp))
{
write_stderr("could not unmap view of backend variables: error code %lu\n",
@@ -719,30 +933,19 @@ read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **
}
#endif
- restore_backend_variables(¶m, client_sock, worker);
+ restore_backend_variables(¶m);
}
/* Restore critical backend variables from the BackendParameters struct */
static void
-restore_backend_variables(BackendParameters *param, ClientSocket **client_sock, BackgroundWorker **worker)
+restore_backend_variables(BackendParameters *param)
{
- if (param->has_client_sock)
- {
- *client_sock = (ClientSocket *) MemoryContextAlloc(TopMemoryContext, sizeof(ClientSocket));
- memcpy(*client_sock, ¶m->client_sock, sizeof(ClientSocket));
- read_inheritable_socket(&(*client_sock)->sock, ¶m->inh_sock);
- }
- else
- *client_sock = NULL;
-
- if (param->has_bgworker)
+ if (param->client_sock.sock != PGINVALID_SOCKET)
{
- *worker = (BackgroundWorker *)
- MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
- memcpy(*worker, ¶m->bgworker, sizeof(BackgroundWorker));
+ MyClientSocket = MemoryContextAlloc(TopMemoryContext, sizeof(ClientSocket));
+ memcpy(MyClientSocket, ¶m->client_sock, sizeof(ClientSocket));
+ read_inheritable_socket(&MyClientSocket->sock, ¶m->inh_sock);
}
- else
- *worker = NULL;
SetDataDir(param->DataDir);
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 14e45bb7071..c266904b579 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -210,8 +210,11 @@ PgArchCanRestart(void)
/* Main entry point for archiver process */
void
-PgArchiverMain(void)
+PgArchiverMain(char *startup_data, size_t startup_data_len)
{
+ Assert(startup_data_len == 0);
+
+ MyBackendType = B_ARCHIVER;
AuxiliaryProcessMainCommon();
/*
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 6610bd5f866..d3720717f25 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -2,9 +2,9 @@
*
* postmaster.c
* This program acts as a clearing house for requests to the
- * POSTGRES system. Frontend programs send a startup message
- * to the Postmaster and the postmaster uses the info in the
- * message to setup a backend process.
+ * POSTGRES system. Frontend programs connect to the Postmaster,
+ * and postmaster forks a new backend process to handle the
+ * connection.
*
* The postmaster also manages system-wide operations such as
* startup and shutdown. The postmaster itself doesn't do those
@@ -106,7 +106,6 @@
#include "postmaster/autovacuum.h"
#include "postmaster/auxprocess.h"
#include "postmaster/bgworker_internals.h"
-#include "postmaster/fork_process.h"
#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
#include "postmaster/syslogger.h"
@@ -427,7 +426,6 @@ typedef enum CAC_state
} CAC_state;
static void BackendInitialize(ClientSocket *client_sock, CAC_state cac);
-static void BackendRun(void) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
static int BackendStartup(ClientSocket *client_sock);
@@ -485,13 +483,6 @@ typedef struct
} win32_deadchild_waitinfo;
#endif /* WIN32 */
-static pid_t backend_forkexec(ClientSocket *client_sock, CAC_state cac);
-
-
-/* in launch_backend.c */
-extern pid_t internal_forkexec(int argc, char *argv[], ClientSocket *client_sock, BackgroundWorker *worker);
-extern void read_backend_variables(char *id, ClientSocket **client_sock, BackgroundWorker **worker);
-
static void ShmemBackendArrayAdd(Backend *bn);
static void ShmemBackendArrayRemove(Backend *bn);
#endif /* EXEC_BACKEND */
@@ -1748,7 +1739,7 @@ ServerLoop(void)
(AutoVacuumingActive() || start_autovac_launcher) &&
pmState == PM_RUN)
{
- AutoVacPID = StartAutoVacLauncher();
+ AutoVacPID = StartChildProcess(B_AUTOVAC_LAUNCHER);
if (AutoVacPID != 0)
start_autovac_launcher = false; /* signal processed */
}
@@ -2902,7 +2893,7 @@ process_pm_child_exit(void)
* situation, some of them may be alive already.
*/
if (!IsBinaryUpgrade && AutoVacuumingActive() && AutoVacPID == 0)
- AutoVacPID = StartAutoVacLauncher();
+ AutoVacPID = StartChildProcess(B_AUTOVAC_LAUNCHER);
if (PgArchStartupAllowed() && PgArchPID == 0)
PgArchPID = StartChildProcess(B_ARCHIVER);
MaybeStartSlotSyncWorker();
@@ -3964,6 +3955,12 @@ TerminateChildren(int signal)
signal_child(SlotSyncWorkerPID, signal);
}
+/* Information passed from postmaster to backend process */
+typedef struct BackendStartupData
+{
+ CAC_state canAcceptConnections;
+} BackendStartupData;
+
/*
* BackendStartup -- start backend process
*
@@ -3976,7 +3973,7 @@ BackendStartup(ClientSocket *client_sock)
{
Backend *bn; /* for backend cleanup */
pid_t pid;
- CAC_state cac;
+ BackendStartupData startup_data;
/*
* Create backend data structure. Better before the fork() so we can
@@ -4005,11 +4002,10 @@ BackendStartup(ClientSocket *client_sock)
return STATUS_ERROR;
}
- bn->cancel_key = MyCancelKey;
-
/* Pass down canAcceptConnections state */
- cac = canAcceptConnections(BACKEND_TYPE_NORMAL);
- bn->dead_end = (cac != CAC_OK);
+ startup_data.canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
+ bn->dead_end = (startup_data.canAcceptConnections != CAC_OK);
+ bn->cancel_key = MyCancelKey;
/*
* Unless it's a dead_end child, assign it a child slot number
@@ -4022,26 +4018,9 @@ BackendStartup(ClientSocket *client_sock)
/* Hasn't asked to be notified about any bgworkers yet */
bn->bgworker_notify = false;
-#ifdef EXEC_BACKEND
- pid = backend_forkexec(client_sock, cac);
-#else /* !EXEC_BACKEND */
- pid = fork_process();
- if (pid == 0) /* child */
- {
- /* Detangle from postmaster */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Perform additional initialization and collect startup packet */
- BackendInitialize(client_sock, cac);
-
- /* And run the backend */
- BackendRun();
- }
-#endif /* EXEC_BACKEND */
-
+ pid = postmaster_child_launch(B_BACKEND,
+ (char *) &startup_data, sizeof(startup_data),
+ client_sock);
if (pid < 0)
{
/* in parent, fork failed */
@@ -4351,269 +4330,57 @@ BackendInitialize(ClientSocket *client_sock, CAC_state cac)
set_ps_display("initializing");
}
-
-/*
- * BackendRun -- set up the backend's argument list and invoke PostgresMain()
- *
- * returns:
- * Doesn't return at all.
- */
-static void
-BackendRun(void)
-{
- /*
- * Create a per-backend PGPROC struct in shared memory. We must do this
- * before we can use LWLocks or access any shared memory.
- */
- InitProcess();
-
- /*
- * Make sure we aren't in PostmasterContext anymore. (We can't delete it
- * just yet, though, because InitPostgres will need the HBA data.)
- */
- MemoryContextSwitchTo(TopMemoryContext);
-
- PostgresMain(MyProcPort->database_name, MyProcPort->user_name);
-}
-
-
-#ifdef EXEC_BACKEND
-
-/*
- * postmaster_forkexec -- fork and exec a postmaster subprocess
- *
- * The caller must have set up the argv array already, except for argv[2]
- * which will be filled with the name of the temp variable file.
- *
- * Returns the child process PID, or -1 on fork failure (a suitable error
- * message has been logged on failure).
- *
- * All uses of this routine will dispatch to SubPostmasterMain in the
- * child process.
- */
-pid_t
-postmaster_forkexec(int argc, char *argv[])
-{
- return internal_forkexec(argc, argv, NULL, NULL);
-}
-
-/*
- * backend_forkexec -- fork/exec off a backend process
- *
- * Some operating systems (WIN32) don't have fork() so we have to simulate
- * it by storing parameters that need to be passed to the child and
- * then create a new child process.
- *
- * returns the pid of the fork/exec'd process, or -1 on failure
- */
-static pid_t
-backend_forkexec(ClientSocket *client_sock, CAC_state cac)
-{
- char *av[5];
- int ac = 0;
- char cacbuf[10];
-
- av[ac++] = "postgres";
- av[ac++] = "--forkbackend";
- av[ac++] = NULL; /* filled in by internal_forkexec */
-
- snprintf(cacbuf, sizeof(cacbuf), "%d", (int) cac);
- av[ac++] = cacbuf;
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return internal_forkexec(ac, av, client_sock, NULL);
-}
-
-/*
- * SubPostmasterMain -- Get the fork/exec'd process into a state equivalent
- * to what it would be if we'd simply forked on Unix, and then
- * dispatch to the appropriate place.
- *
- * The first two command line arguments are expected to be "--forkFOO"
- * (where FOO indicates which postmaster child we are to become), and
- * the name of a variables file that we can read to load data that would
- * have been inherited by fork() on Unix. Remaining arguments go to the
- * subprocess FooMain() routine.
- */
void
-SubPostmasterMain(int argc, char *argv[])
+BackendMain(char *startup_data, size_t startup_data_len)
{
- ClientSocket *client_sock;
- BackgroundWorker *worker;
-
- /* In EXEC_BACKEND case we will not have inherited these settings */
- IsPostmasterEnvironment = true;
- whereToSendOutput = DestNone;
-
- /* Setup essential subsystems (to ensure elog() behaves sanely) */
- InitializeGUCOptions();
-
- /* Check we got appropriate args */
- if (argc < 3)
- elog(FATAL, "invalid subpostmaster invocation");
-
- /* Read in the variables file */
- read_backend_variables(argv[2], &client_sock, &worker);
+ BackendStartupData *bsdata = (BackendStartupData *) startup_data;
- /* Close the postmaster's sockets (as soon as we know them) */
- ClosePostmasterPorts(strcmp(argv[1], "--forklog") == 0);
+ Assert(startup_data_len == sizeof(BackendStartupData));
+ Assert(MyClientSocket != NULL);
- /* Setup as postmaster child */
- InitPostmasterChild();
+#ifdef EXEC_BACKEND
/*
- * If appropriate, physically re-attach to shared memory segment. We want
- * to do this before going any further to ensure that we can attach at the
- * same address the postmaster used. On the other hand, if we choose not
- * to re-attach, we may have other cleanup to do.
- *
- * If testing EXEC_BACKEND on Linux, you should run this as root before
- * starting the postmaster:
+ * Need to reinitialize the SSL library in the backend, since the context
+ * structures contain function pointers and cannot be passed through the
+ * parameter file.
*
- * sysctl -w kernel.randomize_va_space=0
+ * If for some reason reload fails (maybe the user installed broken key
+ * files), soldier on without SSL; that's better than all connections
+ * becoming impossible.
*
- * This prevents using randomized stack and code addresses that cause the
- * child process's memory map to be different from the parent's, making it
- * sometimes impossible to attach to shared memory at the desired address.
- * Return the setting to its old value (usually '1' or '2') when finished.
+ * XXX should we do this in all child processes? For the moment it's
+ * enough to do it in backend children.
*/
- if (strcmp(argv[1], "--forkbackend") == 0 ||
- strcmp(argv[1], "--forkavlauncher") == 0 ||
- strcmp(argv[1], "--forkssworker") == 0 ||
- strcmp(argv[1], "--forkavworker") == 0 ||
- strcmp(argv[1], "--forkaux") == 0 ||
- strcmp(argv[1], "--forkbgworker") == 0)
- PGSharedMemoryReAttach();
- else
- PGSharedMemoryNoReAttach();
+#ifdef USE_SSL
+ if (EnableSSL)
+ {
+ if (secure_initialize(false) == 0)
+ LoadedSSL = true;
+ else
+ ereport(LOG,
+ (errmsg("SSL configuration could not be loaded in child process")));
+ }
+#endif
+#endif
- /* Read in remaining GUC variables */
- read_nondefault_variables();
+ /* Perform additional initialization and collect startup packet */
+ BackendInitialize(MyClientSocket, bsdata->canAcceptConnections);
/*
- * Check that the data directory looks valid, which will also check the
- * privileges on the data directory and update our umask and file/group
- * variables for creating files later. Note: this should really be done
- * before we create any files or directories.
- */
- checkDataDir();
-
- /*
- * (re-)read control file, as it contains config. The postmaster will
- * already have read this, but this process doesn't know about that.
+ * Create a per-backend PGPROC struct in shared memory. We must do this
+ * before we can use LWLocks or access any shared memory.
*/
- LocalProcessControlFile(false);
+ InitProcess();
/*
- * Reload any libraries that were preloaded by the postmaster. Since we
- * exec'd this process, those libraries didn't come along with us; but we
- * should load them into all child processes to be consistent with the
- * non-EXEC_BACKEND behavior.
+ * Make sure we aren't in PostmasterContext anymore. (We can't delete it
+ * just yet, though, because InitPostgres will need the HBA data.)
*/
- process_shared_preload_libraries();
-
- /* Run backend or appropriate child */
- if (strcmp(argv[1], "--forkbackend") == 0)
- {
- CAC_state cac;
-
- Assert(argc == 4);
- cac = (CAC_state) atoi(argv[3]);
-
- /*
- * Need to reinitialize the SSL library in the backend, since the
- * context structures contain function pointers and cannot be passed
- * through the parameter file.
- *
- * If for some reason reload fails (maybe the user installed broken
- * key files), soldier on without SSL; that's better than all
- * connections becoming impossible.
- *
- * XXX should we do this in all child processes? For the moment it's
- * enough to do it in backend children.
- */
-#ifdef USE_SSL
- if (EnableSSL)
- {
- if (secure_initialize(false) == 0)
- LoadedSSL = true;
- else
- ereport(LOG,
- (errmsg("SSL configuration could not be loaded in child process")));
- }
-#endif
-
- /*
- * Perform additional initialization and collect startup packet.
- *
- * We want to do this before InitProcess() for a couple of reasons: 1.
- * so that we aren't eating up a PGPROC slot while waiting on the
- * client. 2. so that if InitProcess() fails due to being out of
- * PGPROC slots, we have already initialized libpq and are able to
- * report the error to the client.
- */
- BackendInitialize(client_sock, cac);
-
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* And run the backend */
- BackendRun(); /* does not return */
-
- }
- if (strcmp(argv[1], "--forkaux") == 0)
- {
- BackendType auxtype;
-
- Assert(argc == 4);
-
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- auxtype = atoi(argv[3]);
- AuxiliaryProcessMain(auxtype); /* does not return */
- }
- if (strcmp(argv[1], "--forkavlauncher") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- AutoVacLauncherMain(argc - 2, argv + 2); /* does not return */
- }
- if (strcmp(argv[1], "--forkavworker") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
- }
- if (strcmp(argv[1], "--forkssworker") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- ReplSlotSyncWorkerMain(argc - 2, argv + 2); /* does not return */
- }
- if (strcmp(argv[1], "--forkbgworker") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- MyBgworkerEntry = worker;
- BackgroundWorkerMain();
- }
- if (strcmp(argv[1], "--forklog") == 0)
- {
- /* Do not want to attach to shared memory */
-
- SysLoggerMain(argc, argv); /* does not return */
- }
+ MemoryContextSwitchTo(TopMemoryContext);
- abort(); /* shouldn't get here */
+ PostgresMain(MyProcPort->database_name, MyProcPort->user_name);
}
-#endif /* EXEC_BACKEND */
/*
@@ -4912,87 +4679,12 @@ StartChildProcess(BackendType type)
{
pid_t pid;
-#ifdef EXEC_BACKEND
- {
- char *av[10];
- int ac = 0;
- char typebuf[32];
-
- /*
- * Set up command-line arguments for subprocess
- */
- av[ac++] = "postgres";
- av[ac++] = "--forkaux";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- snprintf(typebuf, sizeof(typebuf), "%d", type);
- av[ac++] = typebuf;
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- pid = postmaster_forkexec(ac, av);
- }
-#else /* !EXEC_BACKEND */
- pid = fork_process();
-
- if (pid == 0) /* child */
- {
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Release postmaster's working memory context */
- MemoryContextSwitchTo(TopMemoryContext);
- MemoryContextDelete(PostmasterContext);
- PostmasterContext = NULL;
-
- AuxiliaryProcessMain(type); /* does not return */
- }
-#endif /* EXEC_BACKEND */
-
+ pid = postmaster_child_launch(type, NULL, 0, NULL);
if (pid < 0)
{
/* in parent, fork failed */
- int save_errno = errno;
-
- errno = save_errno;
- switch (type)
- {
- case B_STARTUP:
- ereport(LOG,
- (errmsg("could not fork startup process: %m")));
- break;
- case B_ARCHIVER:
- ereport(LOG,
- (errmsg("could not fork archiver process: %m")));
- break;
- case B_BG_WRITER:
- ereport(LOG,
- (errmsg("could not fork background writer process: %m")));
- break;
- case B_CHECKPOINTER:
- ereport(LOG,
- (errmsg("could not fork checkpointer process: %m")));
- break;
- case B_WAL_WRITER:
- ereport(LOG,
- (errmsg("could not fork WAL writer process: %m")));
- break;
- case B_WAL_RECEIVER:
- ereport(LOG,
- (errmsg("could not fork WAL receiver process: %m")));
- break;
- case B_WAL_SUMMARIZER:
- ereport(LOG,
- (errmsg("could not fork WAL summarizer process: %m")));
- break;
- default:
- ereport(LOG,
- (errmsg("could not fork process: %m")));
- break;
- }
+ ereport(LOG,
+ (errmsg("could not fork \"%s\" process: %m", PostmasterChildName(type))));
/*
* fork failure is fatal during startup, but there's no need to choke
@@ -5056,7 +4748,7 @@ StartAutovacuumWorker(void)
bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
bn->bgworker_notify = false;
- bn->pid = StartAutoVacWorker();
+ bn->pid = StartChildProcess(B_AUTOVAC_WORKER);
if (bn->pid > 0)
{
bn->bkend_type = BACKEND_TYPE_AUTOVAC;
@@ -5070,7 +4762,7 @@ StartAutovacuumWorker(void)
/*
* fork failed, fall through to report -- actual error message was
- * logged by StartAutoVacWorker
+ * logged by StartChildProcess
*/
(void) ReleasePostmasterChildSlot(bn->child_slot);
pfree(bn);
@@ -5293,24 +4985,6 @@ BackgroundWorkerUnblockSignals(void)
sigprocmask(SIG_SETMASK, &UnBlockSig, NULL);
}
-#ifdef EXEC_BACKEND
-static pid_t
-bgworker_forkexec(BackgroundWorker *worker)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkbgworker";
- av[ac++] = NULL; /* filled in by internal_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return internal_forkexec(ac, av, NULL, worker);
-}
-#endif
-
/*
* Start a new bgworker.
* Starting time conditions must have been checked already.
@@ -5347,65 +5021,32 @@ do_start_bgworker(RegisteredBgWorker *rw)
(errmsg_internal("starting background worker process \"%s\"",
rw->rw_worker.bgw_name)));
-#ifdef EXEC_BACKEND
- switch ((worker_pid = bgworker_forkexec(&rw->rw_worker)))
-#else
- switch ((worker_pid = fork_process()))
-#endif
+ worker_pid = postmaster_child_launch(B_BG_WORKER, (char *) &rw->rw_worker, sizeof(BackgroundWorker), NULL);
+ if (worker_pid == -1)
{
- case -1:
- /* in postmaster, fork failed ... */
- ereport(LOG,
- (errmsg("could not fork background worker process: %m")));
- /* undo what assign_backendlist_entry did */
- ReleasePostmasterChildSlot(rw->rw_child_slot);
- rw->rw_child_slot = 0;
- pfree(rw->rw_backend);
- rw->rw_backend = NULL;
- /* mark entry as crashed, so we'll try again later */
- rw->rw_crashed_at = GetCurrentTimestamp();
- break;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /*
- * Before blowing away PostmasterContext, save this bgworker's
- * data where it can find it.
- */
- MyBgworkerEntry = (BackgroundWorker *)
- MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
- memcpy(MyBgworkerEntry, &rw->rw_worker, sizeof(BackgroundWorker));
-
- /* Release postmaster's working memory context */
- MemoryContextSwitchTo(TopMemoryContext);
- MemoryContextDelete(PostmasterContext);
- PostmasterContext = NULL;
-
- BackgroundWorkerMain();
+ /* in postmaster, fork failed ... */
+ ereport(LOG,
+ (errmsg("could not fork background worker process: %m")));
+ /* undo what assign_backendlist_entry did */
+ ReleasePostmasterChildSlot(rw->rw_child_slot);
+ rw->rw_child_slot = 0;
+ pfree(rw->rw_backend);
+ rw->rw_backend = NULL;
+ /* mark entry as crashed, so we'll try again later */
+ rw->rw_crashed_at = GetCurrentTimestamp();
+ return false;
+ }
- exit(1); /* should not get here */
- break;
-#endif
- default:
- /* in postmaster, fork successful ... */
- rw->rw_pid = worker_pid;
- rw->rw_backend->pid = rw->rw_pid;
- ReportBackgroundWorkerPID(rw);
- /* add new worker to lists of backends */
- dlist_push_head(&BackendList, &rw->rw_backend->elem);
+ /* in postmaster, fork successful ... */
+ rw->rw_pid = worker_pid;
+ rw->rw_backend->pid = rw->rw_pid;
+ ReportBackgroundWorkerPID(rw);
+ /* add new worker to lists of backends */
+ dlist_push_head(&BackendList, &rw->rw_backend->elem);
#ifdef EXEC_BACKEND
- ShmemBackendArrayAdd(rw->rw_backend);
+ ShmemBackendArrayAdd(rw->rw_backend);
#endif
- return true;
- }
-
- return false;
+ return true;
}
/*
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index ea7db2040f0..ef6f98ebcd7 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -213,8 +213,11 @@ StartupProcExit(int code, Datum arg)
* ----------------------------------
*/
void
-StartupProcessMain(void)
+StartupProcessMain(char *startup_data, size_t startup_data_len)
{
+ Assert(startup_data_len == 0);
+
+ MyBackendType = B_STARTUP;
AuxiliaryProcessMainCommon();
/* Arrange to clean up at startup process exit */
diff --git a/src/backend/postmaster/syslogger.c b/src/backend/postmaster/syslogger.c
index d9d042f5628..d2a778c244f 100644
--- a/src/backend/postmaster/syslogger.c
+++ b/src/backend/postmaster/syslogger.c
@@ -39,7 +39,6 @@
#include "pgstat.h"
#include "pgtime.h"
#include "port/pg_bitutils.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "postmaster/syslogger.h"
@@ -50,6 +49,7 @@
#include "storage/pg_shmem.h"
#include "tcop/tcopprot.h"
#include "utils/guc.h"
+#include "utils/memutils.h"
#include "utils/ps_status.h"
/*
@@ -133,10 +133,7 @@ static volatile sig_atomic_t rotation_requested = false;
#ifdef EXEC_BACKEND
static int syslogger_fdget(FILE *file);
static FILE *syslogger_fdopen(int fd);
-static pid_t syslogger_forkexec(void);
-static void syslogger_parseArgs(int argc, char *argv[]);
#endif
-NON_EXEC_STATIC void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
static void process_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static void flush_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static FILE *logfile_open(const char *filename, const char *mode,
@@ -155,13 +152,19 @@ static void set_next_rotation_time(void);
static void sigUsr1Handler(SIGNAL_ARGS);
static void update_metainfo_datafile(void);
+typedef struct
+{
+ int syslogFile;
+ int csvlogFile;
+ int jsonlogFile;
+} SysloggerStartupData;
/*
* Main entry point for syslogger process
* argc/argv parameters are valid only in EXEC_BACKEND case.
*/
-NON_EXEC_STATIC void
-SysLoggerMain(int argc, char *argv[])
+void
+SysLoggerMain(char *startup_data, size_t startup_data_len)
{
#ifndef WIN32
char logbuffer[READ_BUF_SIZE];
@@ -173,11 +176,34 @@ SysLoggerMain(int argc, char *argv[])
pg_time_t now;
WaitEventSet *wes;
- now = MyStartTime;
+ /* Release postmaster's working memory context */
+ if (PostmasterContext)
+ {
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
+ }
+ /*
+ * Re-open the error output files that were opened by SysLogger_Start().
+ *
+ * We expect this will always succeed, which is too optimistic, but if it
+ * fails there's not a lot we can do to report the problem anyway. As
+ * coded, we'll just crash on a null pointer dereference after failure...
+ */
#ifdef EXEC_BACKEND
- syslogger_parseArgs(argc, argv);
-#endif /* EXEC_BACKEND */
+ {
+ SysLoggerStartupData *slsdata = (SysLoggerStartupData *) startup_data;
+
+ Assert(startup_data_len == sizeof(*slsdata));
+ syslogFile = syslogger_fdopen(slsdata->syslogFile);
+ csvlogFile = syslogger_fdopen(slsdata->csvlogFile);
+ jsonlogFile = syslogger_fdopen(slsdata->jsonlogFile);
+ }
+#else
+ Assert(startup_data_len == 0);
+#endif
+
+ now = MyStartTime;
MyBackendType = B_LOGGER;
init_ps_display(NULL);
@@ -567,6 +593,9 @@ SysLogger_Start(void)
{
pid_t sysloggerPid;
char *filename;
+#ifdef EXEC_BACKEND
+ SysloggerStartupData startup_data;
+#endif /* EXEC_BACKEND */
if (!Logging_collector)
return 0;
@@ -666,112 +695,95 @@ SysLogger_Start(void)
}
#ifdef EXEC_BACKEND
- switch ((sysloggerPid = syslogger_forkexec()))
+ startup_data.syslogFile = syslogger_fdget(syslogFile);
+ startup_data.csvlogFile = syslogger_fdget(csvlogFile);
+ startup_data.jsonlogFile = syslogger_fdget(jsonlogFile);
+ sysloggerPid = postmaster_child_launch(B_LOGGER, (char *) &startup_data, sizeof(startup_data), NULL);
#else
- switch ((sysloggerPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork system logger: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(true);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
+ sysloggerPid = postmaster_child_launch(B_LOGGER, NULL, 0, NULL);
+#endif /* EXEC_BACKEND */
- /* do the work */
- SysLoggerMain(0, NULL);
- break;
-#endif
+ if (sysloggerPid == -1)
+ {
+ ereport(LOG,
+ (errmsg("could not fork system logger: %m")));
+ return 0;
+ }
- default:
- /* success, in postmaster */
+ /* success, in postmaster */
- /* now we redirect stderr, if not done already */
- if (!redirection_done)
- {
+ /* now we redirect stderr, if not done already */
+ if (!redirection_done)
+ {
#ifdef WIN32
- int fd;
+ int fd;
#endif
- /*
- * Leave a breadcrumb trail when redirecting, in case the user
- * forgets that redirection is active and looks only at the
- * original stderr target file.
- */
- ereport(LOG,
- (errmsg("redirecting log output to logging collector process"),
- errhint("Future log output will appear in directory \"%s\".",
- Log_directory)));
+ /*
+ * Leave a breadcrumb trail when redirecting, in case the user forgets
+ * that redirection is active and looks only at the original stderr
+ * target file.
+ */
+ ereport(LOG,
+ (errmsg("redirecting log output to logging collector process"),
+ errhint("Future log output will appear in directory \"%s\".",
+ Log_directory)));
#ifndef WIN32
- fflush(stdout);
- if (dup2(syslogPipe[1], STDOUT_FILENO) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stdout: %m")));
- fflush(stderr);
- if (dup2(syslogPipe[1], STDERR_FILENO) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stderr: %m")));
- /* Now we are done with the write end of the pipe. */
- close(syslogPipe[1]);
- syslogPipe[1] = -1;
+ fflush(stdout);
+ if (dup2(syslogPipe[1], STDOUT_FILENO) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stdout: %m")));
+ fflush(stderr);
+ if (dup2(syslogPipe[1], STDERR_FILENO) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stderr: %m")));
+ /* Now we are done with the write end of the pipe. */
+ close(syslogPipe[1]);
+ syslogPipe[1] = -1;
#else
- /*
- * open the pipe in binary mode and make sure stderr is binary
- * after it's been dup'ed into, to avoid disturbing the pipe
- * chunking protocol.
- */
- fflush(stderr);
- fd = _open_osfhandle((intptr_t) syslogPipe[1],
- _O_APPEND | _O_BINARY);
- if (dup2(fd, STDERR_FILENO) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stderr: %m")));
- close(fd);
- _setmode(STDERR_FILENO, _O_BINARY);
+ /*
+ * open the pipe in binary mode and make sure stderr is binary after
+ * it's been dup'ed into, to avoid disturbing the pipe chunking
+ * protocol.
+ */
+ fflush(stderr);
+ fd = _open_osfhandle((intptr_t) syslogPipe[1],
+ _O_APPEND | _O_BINARY);
+ if (dup2(fd, STDERR_FILENO) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stderr: %m")));
+ close(fd);
+ _setmode(STDERR_FILENO, _O_BINARY);
- /*
- * Now we are done with the write end of the pipe.
- * CloseHandle() must not be called because the preceding
- * close() closes the underlying handle.
- */
- syslogPipe[1] = 0;
+ /*
+ * Now we are done with the write end of the pipe. CloseHandle() must
+ * not be called because the preceding close() closes the underlying
+ * handle.
+ */
+ syslogPipe[1] = 0;
#endif
- redirection_done = true;
- }
-
- /* postmaster will never write the file(s); close 'em */
- fclose(syslogFile);
- syslogFile = NULL;
- if (csvlogFile != NULL)
- {
- fclose(csvlogFile);
- csvlogFile = NULL;
- }
- if (jsonlogFile != NULL)
- {
- fclose(jsonlogFile);
- jsonlogFile = NULL;
- }
- return (int) sysloggerPid;
+ redirection_done = true;
}
- /* we should never reach here */
- return 0;
+ /* postmaster will never write the file(s); close 'em */
+ fclose(syslogFile);
+ syslogFile = NULL;
+ if (csvlogFile != NULL)
+ {
+ fclose(csvlogFile);
+ csvlogFile = NULL;
+ }
+ if (jsonlogFile != NULL)
+ {
+ fclose(jsonlogFile);
+ jsonlogFile = NULL;
+ }
+ return (int) sysloggerPid;
}
@@ -830,69 +842,6 @@ syslogger_fdopen(int fd)
return file;
}
-
-/*
- * syslogger_forkexec() -
- *
- * Format up the arglist for, then fork and exec, a syslogger process
- */
-static pid_t
-syslogger_forkexec(void)
-{
- char *av[10];
- int ac = 0;
- char filenobuf[32];
- char csvfilenobuf[32];
- char jsonfilenobuf[32];
-
- av[ac++] = "postgres";
- av[ac++] = "--forklog";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- /* static variables (those not passed by write_backend_variables) */
- snprintf(filenobuf, sizeof(filenobuf), "%d",
- syslogger_fdget(syslogFile));
- av[ac++] = filenobuf;
- snprintf(csvfilenobuf, sizeof(csvfilenobuf), "%d",
- syslogger_fdget(csvlogFile));
- av[ac++] = csvfilenobuf;
- snprintf(jsonfilenobuf, sizeof(jsonfilenobuf), "%d",
- syslogger_fdget(jsonlogFile));
- av[ac++] = jsonfilenobuf;
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-
-/*
- * syslogger_parseArgs() -
- *
- * Extract data from the arglist for exec'ed syslogger process
- */
-static void
-syslogger_parseArgs(int argc, char *argv[])
-{
- int fd;
-
- Assert(argc == 6);
- argv += 3;
-
- /*
- * Re-open the error output files that were opened by SysLogger_Start().
- *
- * We expect this will always succeed, which is too optimistic, but if it
- * fails there's not a lot we can do to report the problem anyway. As
- * coded, we'll just crash on a null pointer dereference after failure...
- */
- fd = atoi(*argv++);
- syslogFile = syslogger_fdopen(fd);
- fd = atoi(*argv++);
- csvlogFile = syslogger_fdopen(fd);
- fd = atoi(*argv++);
- jsonlogFile = syslogger_fdopen(fd);
-}
#endif /* EXEC_BACKEND */
diff --git a/src/backend/postmaster/walsummarizer.c b/src/backend/postmaster/walsummarizer.c
index 8d9ee1f0f9b..a31bacbc55a 100644
--- a/src/backend/postmaster/walsummarizer.c
+++ b/src/backend/postmaster/walsummarizer.c
@@ -207,7 +207,7 @@ WalSummarizerShmemInit(void)
* Entry point for walsummarizer process.
*/
void
-WalSummarizerMain(void)
+WalSummarizerMain(char *startup_data, size_t startup_data_len)
{
sigjmp_buf local_sigjmp_buf;
MemoryContext context;
@@ -229,6 +229,9 @@ WalSummarizerMain(void)
XLogRecPtr switch_lsn = InvalidXLogRecPtr;
TimeLineID switch_tli = 0;
+ Assert(startup_data_len == 0);
+
+ MyBackendType = B_WAL_SUMMARIZER;
AuxiliaryProcessMainCommon();
ereport(DEBUG1,
diff --git a/src/backend/postmaster/walwriter.c b/src/backend/postmaster/walwriter.c
index dc052854db8..6e7918a78d4 100644
--- a/src/backend/postmaster/walwriter.c
+++ b/src/backend/postmaster/walwriter.c
@@ -86,13 +86,16 @@ int WalWriterFlushAfter = DEFAULT_WAL_WRITER_FLUSH_AFTER;
* basic execution environment, but not enabled signals yet.
*/
void
-WalWriterMain(void)
+WalWriterMain(char *startup_data, size_t startup_data_len)
{
sigjmp_buf local_sigjmp_buf;
MemoryContext walwriter_context;
int left_till_hibernate;
bool hibernating;
+ Assert(startup_data_len == 0);
+
+ MyBackendType = B_WAL_WRITER;
AuxiliaryProcessMainCommon();
/*
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index bfe7e4a8894..acda5f68d9a 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -180,7 +180,7 @@ ProcessWalRcvInterrupts(void)
/* Main entry point for walreceiver process */
void
-WalReceiverMain(void)
+WalReceiverMain(char *startup_data, size_t startup_data_len)
{
char conninfo[MAXCONNINFO];
char *tmp_conninfo;
@@ -196,6 +196,9 @@ WalReceiverMain(void)
char *sender_host = NULL;
int sender_port = 0;
+ Assert(startup_data_len == 0);
+
+ MyBackendType = B_WAL_RECEIVER;
AuxiliaryProcessMainCommon();
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 5b536ac50d1..3e38bb1311d 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -45,6 +45,7 @@ volatile uint32 CritSectionCount = 0;
int MyProcPid;
pg_time_t MyStartTime;
TimestampTz MyStartTimestamp;
+struct ClientSocket *MyClientSocket;
struct Port *MyProcPort;
int32 MyCancelKey;
int MyPMChildSlot;
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 80cf4cdd969..cae1e8b3294 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -50,18 +50,14 @@ extern PGDLLIMPORT int Log_autovacuum_min_duration;
/* Status inquiry functions */
extern bool AutoVacuumingActive(void);
-/* Functions to start autovacuum process, called from postmaster */
+/* called from postmaster at server startup */
extern void autovac_init(void);
-extern int StartAutoVacLauncher(void);
-extern int StartAutoVacWorker(void);
/* called from postmaster when a worker could not be forked */
extern void AutoVacWorkerFailed(void);
-#ifdef EXEC_BACKEND
-extern void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
-extern void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
+extern void AutoVacLauncherMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
+extern void AutoVacWorkerMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern bool AutoVacuumRequestWork(AutoVacuumWorkItemType type,
Oid relationId, BlockNumber blkno);
diff --git a/src/include/postmaster/auxprocess.h b/src/include/postmaster/auxprocess.h
index 2cbd29534c4..4e80b1cfec7 100644
--- a/src/include/postmaster/auxprocess.h
+++ b/src/include/postmaster/auxprocess.h
@@ -13,7 +13,6 @@
#ifndef AUXPROCESS_H
#define AUXPROCESS_H
-extern void AuxiliaryProcessMain(BackendType auxtype) pg_attribute_noreturn();
extern void AuxiliaryProcessMainCommon(void);
#endif /* AUXPROCESS_H */
diff --git a/src/include/postmaster/bgworker_internals.h b/src/include/postmaster/bgworker_internals.h
index 5e30525364b..9106a0ef3f0 100644
--- a/src/include/postmaster/bgworker_internals.h
+++ b/src/include/postmaster/bgworker_internals.h
@@ -55,6 +55,6 @@ extern void ForgetUnstartedBackgroundWorkers(void);
extern void ResetBackgroundWorkerCrashTimes(void);
/* Entry point for background worker processes */
-extern void BackgroundWorkerMain(void) pg_attribute_noreturn();
+extern void BackgroundWorkerMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
#endif /* BGWORKER_INTERNALS_H */
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index b52dc19ef0b..407f26e5302 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -27,8 +27,8 @@ extern PGDLLIMPORT int CheckPointTimeout;
extern PGDLLIMPORT int CheckPointWarning;
extern PGDLLIMPORT double CheckPointCompletionTarget;
-extern void BackgroundWriterMain(void) pg_attribute_noreturn();
-extern void CheckpointerMain(void) pg_attribute_noreturn();
+extern void BackgroundWriterMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
+extern void CheckpointerMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern void RequestCheckpoint(int flags);
extern void CheckpointWriteDelay(int flags, double progress);
diff --git a/src/include/postmaster/pgarch.h b/src/include/postmaster/pgarch.h
index 6135a9718d6..a7a417226b0 100644
--- a/src/include/postmaster/pgarch.h
+++ b/src/include/postmaster/pgarch.h
@@ -29,7 +29,7 @@
extern Size PgArchShmemSize(void);
extern void PgArchShmemInit(void);
extern bool PgArchCanRestart(void);
-extern void PgArchiverMain(void) pg_attribute_noreturn();
+extern void PgArchiverMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern void PgArchWakeup(void);
extern void PgArchForceDirScan(void);
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index 53f82bb5d43..76280cdf453 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -13,6 +13,8 @@
#ifndef _POSTMASTER_H
#define _POSTMASTER_H
+#include "miscadmin.h"
+
/* GUC options */
extern PGDLLIMPORT bool EnableSSL;
extern PGDLLIMPORT int SuperuserReservedConnections;
@@ -58,11 +60,9 @@ extern int MaxLivePostmasterChildren(void);
extern bool PostmasterMarkPIDForWorkerNotify(int);
-#ifdef EXEC_BACKEND
-
-extern pid_t postmaster_forkexec(int argc, char *argv[]);
-extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
+extern void BackendMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
+#ifdef EXEC_BACKEND
extern Size ShmemBackendArraySize(void);
extern void ShmemBackendArrayAllocation(void);
@@ -71,6 +71,23 @@ extern void pgwin32_register_deadchild_callback(HANDLE procHandle, DWORD procId)
#endif
#endif
+/* defined in globals.c */
+extern struct ClientSocket *MyClientSocket;
+
+/* in launch_backend.c */
+
+extern pid_t postmaster_child_launch(BackendType child_type, char *startup_data, size_t startup_data_len, struct ClientSocket *sock);
+
+#ifdef EXEC_BACKEND
+extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
+#endif
+
+#ifdef WIN32
+extern void pgwin32_register_deadchild_callback(HANDLE procHandle, DWORD procId);
+#endif
+
+const char *PostmasterChildName(BackendType child_type);
+
/*
* Note: MAX_BACKENDS is limited to 2^18-1 because that's the width reserved
* for buffer references in buf_internals.h. This limitation could be lifted
diff --git a/src/include/postmaster/startup.h b/src/include/postmaster/startup.h
index cf7a43e38cc..dde7ebde881 100644
--- a/src/include/postmaster/startup.h
+++ b/src/include/postmaster/startup.h
@@ -26,7 +26,7 @@
extern PGDLLIMPORT int log_startup_progress_interval;
extern void HandleStartupProcInterrupts(void);
-extern void StartupProcessMain(void) pg_attribute_noreturn();
+extern void StartupProcessMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern void PreRestoreCommand(void);
extern void PostRestoreCommand(void);
extern bool IsPromoteSignaled(void);
diff --git a/src/include/postmaster/syslogger.h b/src/include/postmaster/syslogger.h
index b26a2f29473..0f28ebcba55 100644
--- a/src/include/postmaster/syslogger.h
+++ b/src/include/postmaster/syslogger.h
@@ -86,9 +86,7 @@ extern int SysLogger_Start(void);
extern void write_syslogger_file(const char *buffer, int count, int destination);
-#ifdef EXEC_BACKEND
-extern void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
+extern void SysLoggerMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern bool CheckLogrotateSignal(void);
extern void RemoveLogrotateSignalFiles(void);
diff --git a/src/include/postmaster/walsummarizer.h b/src/include/postmaster/walsummarizer.h
index ecb4ea37fba..ad346d0c119 100644
--- a/src/include/postmaster/walsummarizer.h
+++ b/src/include/postmaster/walsummarizer.h
@@ -21,7 +21,7 @@ extern PGDLLIMPORT int wal_summary_keep_time;
extern Size WalSummarizerShmemSize(void);
extern void WalSummarizerShmemInit(void);
-extern void WalSummarizerMain(void) pg_attribute_noreturn();
+extern void WalSummarizerMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern void GetWalSummarizerState(TimeLineID *summarized_tli,
XLogRecPtr *summarized_lsn,
diff --git a/src/include/postmaster/walwriter.h b/src/include/postmaster/walwriter.h
index 60f23618cbd..5884d69fed1 100644
--- a/src/include/postmaster/walwriter.h
+++ b/src/include/postmaster/walwriter.h
@@ -18,6 +18,6 @@
extern PGDLLIMPORT int WalWriterDelay;
extern PGDLLIMPORT int WalWriterFlushAfter;
-extern void WalWriterMain(void) pg_attribute_noreturn();
+extern void WalWriterMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
#endif /* _WALWRITER_H */
diff --git a/src/include/replication/walreceiver.h b/src/include/replication/walreceiver.h
index b906bb5ce83..12f71fa99b0 100644
--- a/src/include/replication/walreceiver.h
+++ b/src/include/replication/walreceiver.h
@@ -483,7 +483,7 @@ walrcv_clear_result(WalRcvExecResult *walres)
}
/* prototypes for functions in walreceiver.c */
-extern void WalReceiverMain(void) pg_attribute_noreturn();
+extern void WalReceiverMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
extern void ProcessWalRcvInterrupts(void);
extern void WalRcvForceReply(void);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index a3052a181d1..1b30bcfc526 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -231,6 +231,7 @@ BY_HANDLE_FILE_INFORMATION
Backend
BackendId
BackendParameters
+BackendStartupData
BackendState
BackendType
BackgroundWorker
@@ -2133,6 +2134,7 @@ PortalStrategy
PostParseColumnRefHook
PostgresPollingStatusType
PostingItem
+PostmasterChildType
PreParseColumnRefHook
PredClass
PredIterInfo
@@ -4039,6 +4041,7 @@ BlockRefTableReader
BlockRefTableSerializedEntry
BlockRefTableWriter
SummarizerReadLocalXLogPrivate
+SysloggerStartupData
WalSummarizerData
WalSummaryFile
WalSummaryIO
--
2.39.2
v13-0006-Move-code-for-backend-startup-to-separate-file.patchtext/x-patch; charset=UTF-8; name=v13-0006-Move-code-for-backend-startup-to-separate-file.patchDownload
From 2b6e50907c2e766a346455a07246228606e683d3 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Tue, 12 Mar 2024 14:14:36 +0200
Subject: [PATCH v13 6/6] Move code for backend startup to separate file
This is code that runs in the backend process after forking.
---
src/backend/postmaster/postmaster.c | 764 +--------------------------
src/backend/tcop/Makefile | 1 +
src/backend/tcop/backend_startup.c | 774 ++++++++++++++++++++++++++++
src/backend/tcop/meson.build | 1 +
src/include/postmaster/postmaster.h | 3 +
src/include/tcop/backend_startup.h | 41 ++
6 files changed, 825 insertions(+), 759 deletions(-)
create mode 100644 src/backend/tcop/backend_startup.c
create mode 100644 src/include/tcop/backend_startup.h
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index d3720717f25..a217cbcb485 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -95,10 +95,8 @@
#include "common/file_utils.h"
#include "common/ip.h"
#include "common/pg_prng.h"
-#include "common/string.h"
#include "lib/ilist.h"
#include "libpq/libpq.h"
-#include "libpq/pqformat.h"
#include "libpq/pqsignal.h"
#include "pg_getopt.h"
#include "pgstat.h"
@@ -117,13 +115,11 @@
#include "storage/ipc.h"
#include "storage/pmsignal.h"
#include "storage/proc.h"
+#include "tcop/backend_startup.h"
#include "tcop/tcopprot.h"
-#include "utils/builtins.h"
#include "utils/datetime.h"
#include "utils/memutils.h"
#include "utils/pidfile.h"
-#include "utils/ps_status.h"
-#include "utils/timeout.h"
#include "utils/timestamp.h"
#include "utils/varlena.h"
@@ -382,7 +378,7 @@ static WaitEventSet *pm_wait_set;
#ifdef USE_SSL
/* Set when and if SSL has been initialized properly */
-static bool LoadedSSL = false;
+bool LoadedSSL = false;
#endif
#ifdef USE_BONJOUR
@@ -404,9 +400,7 @@ static void process_pm_pmsignal(void);
static void process_pm_child_exit(void);
static void process_pm_reload_request(void);
static void process_pm_shutdown_request(void);
-static void process_startup_packet_die(SIGNAL_ARGS);
static void dummy_handler(SIGNAL_ARGS);
-static void StartupPacketTimeoutHandler(void);
static void CleanupBackend(int pid, int exitstatus);
static bool CleanupBackgroundWorker(int pid, int exitstatus);
static void HandleChildCrash(int pid, int exitstatus, const char *procname);
@@ -414,24 +408,9 @@ static void LogChildExit(int lev, const char *procname,
int pid, int exitstatus);
static void PostmasterStateMachine(void);
-/* Return value of canAcceptConnections() */
-typedef enum CAC_state
-{
- CAC_OK,
- CAC_STARTUP,
- CAC_SHUTDOWN,
- CAC_RECOVERY,
- CAC_NOTCONSISTENT,
- CAC_TOOMANY,
-} CAC_state;
-
-static void BackendInitialize(ClientSocket *client_sock, CAC_state cac);
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
-static int BackendStartup(ClientSocket *client_sock);
-static int ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done);
-static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
-static void processCancelRequest(Port *port, void *pkt);
+static int BackendStartup(ClientSocket *port);
static void report_fork_failure_to_client(ClientSocket *client_sock, int errnum);
static CAC_state canAcceptConnections(int backend_type);
static bool RandomCancelKey(int32 *cancel_key);
@@ -1847,412 +1826,14 @@ ServerLoop(void)
}
}
-/*
- * Read a client's startup packet and do something according to it.
- *
- * Returns STATUS_OK or STATUS_ERROR, or might call ereport(FATAL) and
- * not return at all.
- *
- * (Note that ereport(FATAL) stuff is sent to the client, so only use it
- * if that's what you want. Return STATUS_ERROR if you don't want to
- * send anything to the client, which would typically be appropriate
- * if we detect a communications failure.)
- *
- * Set ssl_done and/or gss_done when negotiation of an encrypted layer
- * (currently, TLS or GSSAPI) is completed. A successful negotiation of either
- * encryption layer sets both flags, but a rejected negotiation sets only the
- * flag for that layer, since the client may wish to try the other one. We
- * should make no assumption here about the order in which the client may make
- * requests.
- */
-static int
-ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
-{
- int32 len;
- char *buf;
- ProtocolVersion proto;
- MemoryContext oldcontext;
-
- pq_startmsgread();
-
- /*
- * Grab the first byte of the length word separately, so that we can tell
- * whether we have no data at all or an incomplete packet. (This might
- * sound inefficient, but it's not really, because of buffering in
- * pqcomm.c.)
- */
- if (pq_getbytes((char *) &len, 1) == EOF)
- {
- /*
- * If we get no data at all, don't clutter the log with a complaint;
- * such cases often occur for legitimate reasons. An example is that
- * we might be here after responding to NEGOTIATE_SSL_CODE, and if the
- * client didn't like our response, it'll probably just drop the
- * connection. Service-monitoring software also often just opens and
- * closes a connection without sending anything. (So do port
- * scanners, which may be less benign, but it's not really our job to
- * notice those.)
- */
- return STATUS_ERROR;
- }
-
- if (pq_getbytes(((char *) &len) + 1, 3) == EOF)
- {
- /* Got a partial length word, so bleat about that */
- if (!ssl_done && !gss_done)
- ereport(COMMERROR,
- (errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("incomplete startup packet")));
- return STATUS_ERROR;
- }
-
- len = pg_ntoh32(len);
- len -= 4;
-
- if (len < (int32) sizeof(ProtocolVersion) ||
- len > MAX_STARTUP_PACKET_LENGTH)
- {
- ereport(COMMERROR,
- (errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("invalid length of startup packet")));
- return STATUS_ERROR;
- }
-
- /*
- * Allocate space to hold the startup packet, plus one extra byte that's
- * initialized to be zero. This ensures we will have null termination of
- * all strings inside the packet.
- */
- buf = palloc(len + 1);
- buf[len] = '\0';
-
- if (pq_getbytes(buf, len) == EOF)
- {
- ereport(COMMERROR,
- (errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("incomplete startup packet")));
- return STATUS_ERROR;
- }
- pq_endmsgread();
-
- /*
- * The first field is either a protocol version number or a special
- * request code.
- */
- port->proto = proto = pg_ntoh32(*((ProtocolVersion *) buf));
-
- if (proto == CANCEL_REQUEST_CODE)
- {
- if (len != sizeof(CancelRequestPacket))
- {
- ereport(COMMERROR,
- (errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("invalid length of startup packet")));
- return STATUS_ERROR;
- }
- processCancelRequest(port, buf);
- /* Not really an error, but we don't want to proceed further */
- return STATUS_ERROR;
- }
-
- if (proto == NEGOTIATE_SSL_CODE && !ssl_done)
- {
- char SSLok;
-
-#ifdef USE_SSL
- /* No SSL when disabled or on Unix sockets */
- if (!LoadedSSL || port->laddr.addr.ss_family == AF_UNIX)
- SSLok = 'N';
- else
- SSLok = 'S'; /* Support for SSL */
-#else
- SSLok = 'N'; /* No support for SSL */
-#endif
-
-retry1:
- if (send(port->sock, &SSLok, 1, 0) != 1)
- {
- if (errno == EINTR)
- goto retry1; /* if interrupted, just retry */
- ereport(COMMERROR,
- (errcode_for_socket_access(),
- errmsg("failed to send SSL negotiation response: %m")));
- return STATUS_ERROR; /* close the connection */
- }
-
-#ifdef USE_SSL
- if (SSLok == 'S' && secure_open_server(port) == -1)
- return STATUS_ERROR;
-#endif
-
- /*
- * At this point we should have no data already buffered. If we do,
- * it was received before we performed the SSL handshake, so it wasn't
- * encrypted and indeed may have been injected by a man-in-the-middle.
- * We report this case to the client.
- */
- if (pq_buffer_has_data())
- ereport(FATAL,
- (errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("received unencrypted data after SSL request"),
- errdetail("This could be either a client-software bug or evidence of an attempted man-in-the-middle attack.")));
-
- /*
- * regular startup packet, cancel, etc packet should follow, but not
- * another SSL negotiation request, and a GSS request should only
- * follow if SSL was rejected (client may negotiate in either order)
- */
- return ProcessStartupPacket(port, true, SSLok == 'S');
- }
- else if (proto == NEGOTIATE_GSS_CODE && !gss_done)
- {
- char GSSok = 'N';
-
-#ifdef ENABLE_GSS
- /* No GSSAPI encryption when on Unix socket */
- if (port->laddr.addr.ss_family != AF_UNIX)
- GSSok = 'G';
-#endif
-
- while (send(port->sock, &GSSok, 1, 0) != 1)
- {
- if (errno == EINTR)
- continue;
- ereport(COMMERROR,
- (errcode_for_socket_access(),
- errmsg("failed to send GSSAPI negotiation response: %m")));
- return STATUS_ERROR; /* close the connection */
- }
-
-#ifdef ENABLE_GSS
- if (GSSok == 'G' && secure_open_gssapi(port) == -1)
- return STATUS_ERROR;
-#endif
-
- /*
- * At this point we should have no data already buffered. If we do,
- * it was received before we performed the GSS handshake, so it wasn't
- * encrypted and indeed may have been injected by a man-in-the-middle.
- * We report this case to the client.
- */
- if (pq_buffer_has_data())
- ereport(FATAL,
- (errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("received unencrypted data after GSSAPI encryption request"),
- errdetail("This could be either a client-software bug or evidence of an attempted man-in-the-middle attack.")));
-
- /*
- * regular startup packet, cancel, etc packet should follow, but not
- * another GSS negotiation request, and an SSL request should only
- * follow if GSS was rejected (client may negotiate in either order)
- */
- return ProcessStartupPacket(port, GSSok == 'G', true);
- }
-
- /* Could add additional special packet types here */
-
- /*
- * Set FrontendProtocol now so that ereport() knows what format to send if
- * we fail during startup.
- */
- FrontendProtocol = proto;
-
- /* Check that the major protocol version is in range. */
- if (PG_PROTOCOL_MAJOR(proto) < PG_PROTOCOL_MAJOR(PG_PROTOCOL_EARLIEST) ||
- PG_PROTOCOL_MAJOR(proto) > PG_PROTOCOL_MAJOR(PG_PROTOCOL_LATEST))
- ereport(FATAL,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("unsupported frontend protocol %u.%u: server supports %u.0 to %u.%u",
- PG_PROTOCOL_MAJOR(proto), PG_PROTOCOL_MINOR(proto),
- PG_PROTOCOL_MAJOR(PG_PROTOCOL_EARLIEST),
- PG_PROTOCOL_MAJOR(PG_PROTOCOL_LATEST),
- PG_PROTOCOL_MINOR(PG_PROTOCOL_LATEST))));
-
- /*
- * Now fetch parameters out of startup packet and save them into the Port
- * structure.
- */
- oldcontext = MemoryContextSwitchTo(TopMemoryContext);
-
- /* Handle protocol version 3 startup packet */
- {
- int32 offset = sizeof(ProtocolVersion);
- List *unrecognized_protocol_options = NIL;
-
- /*
- * Scan packet body for name/option pairs. We can assume any string
- * beginning within the packet body is null-terminated, thanks to
- * zeroing extra byte above.
- */
- port->guc_options = NIL;
-
- while (offset < len)
- {
- char *nameptr = buf + offset;
- int32 valoffset;
- char *valptr;
-
- if (*nameptr == '\0')
- break; /* found packet terminator */
- valoffset = offset + strlen(nameptr) + 1;
- if (valoffset >= len)
- break; /* missing value, will complain below */
- valptr = buf + valoffset;
-
- if (strcmp(nameptr, "database") == 0)
- port->database_name = pstrdup(valptr);
- else if (strcmp(nameptr, "user") == 0)
- port->user_name = pstrdup(valptr);
- else if (strcmp(nameptr, "options") == 0)
- port->cmdline_options = pstrdup(valptr);
- else if (strcmp(nameptr, "replication") == 0)
- {
- /*
- * Due to backward compatibility concerns the replication
- * parameter is a hybrid beast which allows the value to be
- * either boolean or the string 'database'. The latter
- * connects to a specific database which is e.g. required for
- * logical decoding while.
- */
- if (strcmp(valptr, "database") == 0)
- {
- am_walsender = true;
- am_db_walsender = true;
- }
- else if (!parse_bool(valptr, &am_walsender))
- ereport(FATAL,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("invalid value for parameter \"%s\": \"%s\"",
- "replication",
- valptr),
- errhint("Valid values are: \"false\", 0, \"true\", 1, \"database\".")));
- }
- else if (strncmp(nameptr, "_pq_.", 5) == 0)
- {
- /*
- * Any option beginning with _pq_. is reserved for use as a
- * protocol-level option, but at present no such options are
- * defined.
- */
- unrecognized_protocol_options =
- lappend(unrecognized_protocol_options, pstrdup(nameptr));
- }
- else
- {
- /* Assume it's a generic GUC option */
- port->guc_options = lappend(port->guc_options,
- pstrdup(nameptr));
- port->guc_options = lappend(port->guc_options,
- pstrdup(valptr));
-
- /*
- * Copy application_name to port if we come across it. This
- * is done so we can log the application_name in the
- * connection authorization message. Note that the GUC would
- * be used but we haven't gone through GUC setup yet.
- */
- if (strcmp(nameptr, "application_name") == 0)
- {
- port->application_name = pg_clean_ascii(valptr, 0);
- }
- }
- offset = valoffset + strlen(valptr) + 1;
- }
-
- /*
- * If we didn't find a packet terminator exactly at the end of the
- * given packet length, complain.
- */
- if (offset != len - 1)
- ereport(FATAL,
- (errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("invalid startup packet layout: expected terminator as last byte")));
-
- /*
- * If the client requested a newer protocol version or if the client
- * requested any protocol options we didn't recognize, let them know
- * the newest minor protocol version we do support and the names of
- * any unrecognized options.
- */
- if (PG_PROTOCOL_MINOR(proto) > PG_PROTOCOL_MINOR(PG_PROTOCOL_LATEST) ||
- unrecognized_protocol_options != NIL)
- SendNegotiateProtocolVersion(unrecognized_protocol_options);
- }
-
- /* Check a user name was given. */
- if (port->user_name == NULL || port->user_name[0] == '\0')
- ereport(FATAL,
- (errcode(ERRCODE_INVALID_AUTHORIZATION_SPECIFICATION),
- errmsg("no PostgreSQL user name specified in startup packet")));
-
- /* The database defaults to the user name. */
- if (port->database_name == NULL || port->database_name[0] == '\0')
- port->database_name = pstrdup(port->user_name);
-
- if (am_walsender)
- MyBackendType = B_WAL_SENDER;
- else
- MyBackendType = B_BACKEND;
-
- /*
- * Normal walsender backends, e.g. for streaming replication, are not
- * connected to a particular database. But walsenders used for logical
- * replication need to connect to a specific database. We allow streaming
- * replication commands to be issued even if connected to a database as it
- * can make sense to first make a basebackup and then stream changes
- * starting from that.
- */
- if (am_walsender && !am_db_walsender)
- port->database_name[0] = '\0';
-
- /*
- * Done filling the Port structure
- */
- MemoryContextSwitchTo(oldcontext);
-
- return STATUS_OK;
-}
-
-/*
- * Send a NegotiateProtocolVersion to the client. This lets the client know
- * that they have requested a newer minor protocol version than we are able
- * to speak. We'll speak the highest version we know about; the client can,
- * of course, abandon the connection if that's a problem.
- *
- * We also include in the response a list of protocol options we didn't
- * understand. This allows clients to include optional parameters that might
- * be present either in newer protocol versions or third-party protocol
- * extensions without fear of having to reconnect if those options are not
- * understood, while at the same time making certain that the client is aware
- * of which options were actually accepted.
- */
-static void
-SendNegotiateProtocolVersion(List *unrecognized_protocol_options)
-{
- StringInfoData buf;
- ListCell *lc;
-
- pq_beginmessage(&buf, PqMsg_NegotiateProtocolVersion);
- pq_sendint32(&buf, PG_PROTOCOL_LATEST);
- pq_sendint32(&buf, list_length(unrecognized_protocol_options));
- foreach(lc, unrecognized_protocol_options)
- pq_sendstring(&buf, lfirst(lc));
- pq_endmessage(&buf);
-
- /* no need to flush, some other message will follow */
-}
-
/*
* The client has sent a cancel request packet, not a normal
* start-a-new-connection packet. Perform the necessary processing.
* Nothing is sent back to the client.
*/
-static void
-processCancelRequest(Port *port, void *pkt)
+void
+processCancelRequest(int backendPID, int32 cancelAuthCode)
{
- CancelRequestPacket *canc = (CancelRequestPacket *) pkt;
- int backendPID;
- int32 cancelAuthCode;
Backend *bp;
#ifndef EXEC_BACKEND
@@ -2261,9 +1842,6 @@ processCancelRequest(Port *port, void *pkt)
int i;
#endif
- backendPID = (int) pg_ntoh32(canc->backendPID);
- cancelAuthCode = (int32) pg_ntoh32(canc->cancelAuthCode);
-
/*
* See if we have a matching backend. In the EXEC_BACKEND case, we can no
* longer access the postmaster's own backend list, and must rely on the
@@ -3955,12 +3533,6 @@ TerminateChildren(int signal)
signal_child(SlotSyncWorkerPID, signal);
}
-/* Information passed from postmaster to backend process */
-typedef struct BackendStartupData
-{
- CAC_state canAcceptConnections;
-} BackendStartupData;
-
/*
* BackendStartup -- start backend process
*
@@ -4087,302 +3659,6 @@ report_fork_failure_to_client(ClientSocket *client_sock, int errnum)
} while (rc < 0 && errno == EINTR);
}
-
-/*
- * BackendInitialize -- initialize an interactive (postmaster-child)
- * backend process, and collect the client's startup packet.
- *
- * returns: nothing. Will not return at all if there's any failure.
- *
- * Note: this code does not depend on having any access to shared memory.
- * Indeed, our approach to SIGTERM/timeout handling *requires* that
- * shared memory not have been touched yet; see comments within.
- * In the EXEC_BACKEND case, we are physically attached to shared memory
- * but have not yet set up most of our local pointers to shmem structures.
- */
-static void
-BackendInitialize(ClientSocket *client_sock, CAC_state cac)
-{
- int status;
- int ret;
- Port *port;
- char remote_host[NI_MAXHOST];
- char remote_port[NI_MAXSERV];
- StringInfoData ps_data;
- MemoryContext oldcontext;
-
- /* Tell fd.c about the long-lived FD associated with the client_sock */
- ReserveExternalFD();
-
- /*
- * PreAuthDelay is a debugging aid for investigating problems in the
- * authentication cycle: it can be set in postgresql.conf to allow time to
- * attach to the newly-forked backend with a debugger. (See also
- * PostAuthDelay, which we allow clients to pass through PGOPTIONS, but it
- * is not honored until after authentication.)
- */
- if (PreAuthDelay > 0)
- pg_usleep(PreAuthDelay * 1000000L);
-
- /* This flag will remain set until InitPostgres finishes authentication */
- ClientAuthInProgress = true; /* limit visibility of log messages */
-
- /*
- * Initialize libpq and enable reporting of ereport errors to the client.
- * Must do this now because authentication uses libpq to send messages.
- *
- * The Port structure and all data structures attached to it are allocated
- * in TopMemoryContext, so that they survive into PostgresMain execution.
- * We need not worry about leaking this storage on failure, since we
- * aren't in the postmaster process anymore.
- */
- oldcontext = MemoryContextSwitchTo(TopMemoryContext);
- port = MyProcPort = pq_init(client_sock);
- MemoryContextSwitchTo(oldcontext);
-
- whereToSendOutput = DestRemote; /* now safe to ereport to client */
-
- /* set these to empty in case they are needed before we set them up */
- port->remote_host = "";
- port->remote_port = "";
-
- /*
- * We arrange to do _exit(1) if we receive SIGTERM or timeout while trying
- * to collect the startup packet; while SIGQUIT results in _exit(2).
- * Otherwise the postmaster cannot shutdown the database FAST or IMMED
- * cleanly if a buggy client fails to send the packet promptly.
- *
- * Exiting with _exit(1) is only possible because we have not yet touched
- * shared memory; therefore no outside-the-process state needs to get
- * cleaned up.
- */
- pqsignal(SIGTERM, process_startup_packet_die);
- /* SIGQUIT handler was already set up by InitPostmasterChild */
- InitializeTimeouts(); /* establishes SIGALRM handler */
- sigprocmask(SIG_SETMASK, &StartupBlockSig, NULL);
-
- /*
- * Get the remote host name and port for logging and status display.
- */
- remote_host[0] = '\0';
- remote_port[0] = '\0';
- if ((ret = pg_getnameinfo_all(&port->raddr.addr, port->raddr.salen,
- remote_host, sizeof(remote_host),
- remote_port, sizeof(remote_port),
- (log_hostname ? 0 : NI_NUMERICHOST) | NI_NUMERICSERV)) != 0)
- ereport(WARNING,
- (errmsg_internal("pg_getnameinfo_all() failed: %s",
- gai_strerror(ret))));
-
- /*
- * Save remote_host and remote_port in port structure (after this, they
- * will appear in log_line_prefix data for log messages).
- */
- oldcontext = MemoryContextSwitchTo(TopMemoryContext);
- port->remote_host = pstrdup(remote_host);
- port->remote_port = pstrdup(remote_port);
-
- /* And now we can issue the Log_connections message, if wanted */
- if (Log_connections)
- {
- if (remote_port[0])
- ereport(LOG,
- (errmsg("connection received: host=%s port=%s",
- remote_host,
- remote_port)));
- else
- ereport(LOG,
- (errmsg("connection received: host=%s",
- remote_host)));
- }
-
- /*
- * If we did a reverse lookup to name, we might as well save the results
- * rather than possibly repeating the lookup during authentication.
- *
- * Note that we don't want to specify NI_NAMEREQD above, because then we'd
- * get nothing useful for a client without an rDNS entry. Therefore, we
- * must check whether we got a numeric IPv4 or IPv6 address, and not save
- * it into remote_hostname if so. (This test is conservative and might
- * sometimes classify a hostname as numeric, but an error in that
- * direction is safe; it only results in a possible extra lookup.)
- */
- if (log_hostname &&
- ret == 0 &&
- strspn(remote_host, "0123456789.") < strlen(remote_host) &&
- strspn(remote_host, "0123456789ABCDEFabcdef:") < strlen(remote_host))
- {
- port->remote_hostname = pstrdup(remote_host);
- }
- MemoryContextSwitchTo(oldcontext);
-
- /*
- * Ready to begin client interaction. We will give up and _exit(1) after
- * a time delay, so that a broken client can't hog a connection
- * indefinitely. PreAuthDelay and any DNS interactions above don't count
- * against the time limit.
- *
- * Note: AuthenticationTimeout is applied here while waiting for the
- * startup packet, and then again in InitPostgres for the duration of any
- * authentication operations. So a hostile client could tie up the
- * process for nearly twice AuthenticationTimeout before we kick him off.
- *
- * Note: because PostgresMain will call InitializeTimeouts again, the
- * registration of STARTUP_PACKET_TIMEOUT will be lost. This is okay
- * since we never use it again after this function.
- */
- RegisterTimeout(STARTUP_PACKET_TIMEOUT, StartupPacketTimeoutHandler);
- enable_timeout_after(STARTUP_PACKET_TIMEOUT, AuthenticationTimeout * 1000);
-
- /*
- * Receive the startup packet (which might turn out to be a cancel request
- * packet).
- */
- status = ProcessStartupPacket(port, false, false);
-
- /*
- * If we're going to reject the connection due to database state, say so
- * now instead of wasting cycles on an authentication exchange. (This also
- * allows a pg_ping utility to be written.)
- */
- if (status == STATUS_OK)
- {
- switch (cac)
- {
- case CAC_STARTUP:
- ereport(FATAL,
- (errcode(ERRCODE_CANNOT_CONNECT_NOW),
- errmsg("the database system is starting up")));
- break;
- case CAC_NOTCONSISTENT:
- if (EnableHotStandby)
- ereport(FATAL,
- (errcode(ERRCODE_CANNOT_CONNECT_NOW),
- errmsg("the database system is not yet accepting connections"),
- errdetail("Consistent recovery state has not been yet reached.")));
- else
- ereport(FATAL,
- (errcode(ERRCODE_CANNOT_CONNECT_NOW),
- errmsg("the database system is not accepting connections"),
- errdetail("Hot standby mode is disabled.")));
- break;
- case CAC_SHUTDOWN:
- ereport(FATAL,
- (errcode(ERRCODE_CANNOT_CONNECT_NOW),
- errmsg("the database system is shutting down")));
- break;
- case CAC_RECOVERY:
- ereport(FATAL,
- (errcode(ERRCODE_CANNOT_CONNECT_NOW),
- errmsg("the database system is in recovery mode")));
- break;
- case CAC_TOOMANY:
- ereport(FATAL,
- (errcode(ERRCODE_TOO_MANY_CONNECTIONS),
- errmsg("sorry, too many clients already")));
- break;
- case CAC_OK:
- break;
- }
- }
-
- /*
- * Disable the timeout, and prevent SIGTERM again.
- */
- disable_timeout(STARTUP_PACKET_TIMEOUT, false);
- sigprocmask(SIG_SETMASK, &BlockSig, NULL);
-
- /*
- * As a safety check that nothing in startup has yet performed
- * shared-memory modifications that would need to be undone if we had
- * exited through SIGTERM or timeout above, check that no on_shmem_exit
- * handlers have been registered yet. (This isn't terribly bulletproof,
- * since someone might misuse an on_proc_exit handler for shmem cleanup,
- * but it's a cheap and helpful check. We cannot disallow on_proc_exit
- * handlers unfortunately, since pq_init() already registered one.)
- */
- check_on_shmem_exit_lists_are_empty();
-
- /*
- * Stop here if it was bad or a cancel packet. ProcessStartupPacket
- * already did any appropriate error reporting.
- */
- if (status != STATUS_OK)
- proc_exit(0);
-
- /*
- * Now that we have the user and database name, we can set the process
- * title for ps. It's good to do this as early as possible in startup.
- */
- initStringInfo(&ps_data);
- if (am_walsender)
- appendStringInfo(&ps_data, "%s ", GetBackendTypeDesc(B_WAL_SENDER));
- appendStringInfo(&ps_data, "%s ", port->user_name);
- if (port->database_name[0] != '\0')
- appendStringInfo(&ps_data, "%s ", port->database_name);
- appendStringInfoString(&ps_data, port->remote_host);
- if (port->remote_port[0] != '\0')
- appendStringInfo(&ps_data, "(%s)", port->remote_port);
-
- init_ps_display(ps_data.data);
- pfree(ps_data.data);
-
- set_ps_display("initializing");
-}
-
-void
-BackendMain(char *startup_data, size_t startup_data_len)
-{
- BackendStartupData *bsdata = (BackendStartupData *) startup_data;
-
- Assert(startup_data_len == sizeof(BackendStartupData));
- Assert(MyClientSocket != NULL);
-
-#ifdef EXEC_BACKEND
-
- /*
- * Need to reinitialize the SSL library in the backend, since the context
- * structures contain function pointers and cannot be passed through the
- * parameter file.
- *
- * If for some reason reload fails (maybe the user installed broken key
- * files), soldier on without SSL; that's better than all connections
- * becoming impossible.
- *
- * XXX should we do this in all child processes? For the moment it's
- * enough to do it in backend children.
- */
-#ifdef USE_SSL
- if (EnableSSL)
- {
- if (secure_initialize(false) == 0)
- LoadedSSL = true;
- else
- ereport(LOG,
- (errmsg("SSL configuration could not be loaded in child process")));
- }
-#endif
-#endif
-
- /* Perform additional initialization and collect startup packet */
- BackendInitialize(MyClientSocket, bsdata->canAcceptConnections);
-
- /*
- * Create a per-backend PGPROC struct in shared memory. We must do this
- * before we can use LWLocks or access any shared memory.
- */
- InitProcess();
-
- /*
- * Make sure we aren't in PostmasterContext anymore. (We can't delete it
- * just yet, though, because InitPostgres will need the HBA data.)
- */
- MemoryContextSwitchTo(TopMemoryContext);
-
- PostgresMain(MyProcPort->database_name, MyProcPort->user_name);
-}
-
-
/*
* ExitPostmaster -- cleanup
*
@@ -4571,25 +3847,6 @@ process_pm_pmsignal(void)
}
}
-/*
- * SIGTERM while processing startup packet.
- *
- * Running proc_exit() from a signal handler would be quite unsafe.
- * However, since we have not yet touched shared memory, we can just
- * pull the plug and exit without running any atexit handlers.
- *
- * One might be tempted to try to send a message, or log one, indicating
- * why we are disconnecting. However, that would be quite unsafe in itself.
- * Also, it seems undesirable to provide clues about the database's state
- * to a client that has not yet completed authentication, or even sent us
- * a startup packet.
- */
-static void
-process_startup_packet_die(SIGNAL_ARGS)
-{
- _exit(1);
-}
-
/*
* Dummy signal handler
*
@@ -4604,17 +3861,6 @@ dummy_handler(SIGNAL_ARGS)
{
}
-/*
- * Timeout while processing startup packet.
- * As for process_startup_packet_die(), we exit via _exit(1).
- */
-static void
-StartupPacketTimeoutHandler(void)
-{
- _exit(1);
-}
-
-
/*
* Generate a random cancel key.
*/
diff --git a/src/backend/tcop/Makefile b/src/backend/tcop/Makefile
index f662a7dd1cf..9119667345a 100644
--- a/src/backend/tcop/Makefile
+++ b/src/backend/tcop/Makefile
@@ -13,6 +13,7 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
OBJS = \
+ backend_startup.o \
cmdtag.o \
dest.o \
fastpath.o \
diff --git a/src/backend/tcop/backend_startup.c b/src/backend/tcop/backend_startup.c
new file mode 100644
index 00000000000..abd6b4ef317
--- /dev/null
+++ b/src/backend/tcop/backend_startup.c
@@ -0,0 +1,774 @@
+/*-------------------------------------------------------------------------
+ *
+ * backend_startup.c
+ * Backend startup code
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/tcop/backend_startup.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include <unistd.h>
+
+#include "access/xlog.h"
+#include "common/ip.h"
+#include "common/string.h"
+#include "libpq/libpq.h"
+#include "libpq/libpq-be.h"
+#include "libpq/pqformat.h"
+#include "libpq/pqsignal.h"
+#include "miscadmin.h"
+#include "postmaster/postmaster.h"
+#include "replication/walsender.h"
+#include "storage/fd.h"
+#include "storage/ipc.h"
+#include "storage/proc.h"
+#include "tcop/backend_startup.h"
+#include "tcop/tcopprot.h"
+#include "utils/builtins.h"
+#include "utils/memutils.h"
+#include "utils/ps_status.h"
+#include "utils/timeout.h"
+
+static void BackendInitialize(ClientSocket *client_sock, CAC_state cac);
+static int ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done);
+static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
+static void process_startup_packet_die(SIGNAL_ARGS);
+static void StartupPacketTimeoutHandler(void);
+
+/*
+ * Entry point for a new backend process.
+ *
+ * Initialize the connection, read the startup packet.
+ */
+void
+BackendMain(char *startup_data, size_t startup_data_len)
+{
+ BackendStartupData *bsdata = (BackendStartupData *) startup_data;
+
+ Assert(startup_data_len == sizeof(BackendStartupData));
+ Assert(MyClientSocket != NULL);
+
+#ifdef EXEC_BACKEND
+
+ /*
+ * Need to reinitialize the SSL library in the backend, since the context
+ * structures contain function pointers and cannot be passed through the
+ * parameter file.
+ *
+ * If for some reason reload fails (maybe the user installed broken key
+ * files), soldier on without SSL; that's better than all connections
+ * becoming impossible.
+ *
+ * XXX should we do this in all child processes? For the moment it's
+ * enough to do it in backend children.
+ */
+#ifdef USE_SSL
+ if (EnableSSL)
+ {
+ if (secure_initialize(false) == 0)
+ LoadedSSL = true;
+ else
+ ereport(LOG,
+ (errmsg("SSL configuration could not be loaded in child process")));
+ }
+#endif
+#endif
+
+ /* Perform additional initialization and collect startup packet */
+ BackendInitialize(MyClientSocket, bsdata->canAcceptConnections);
+
+ /*
+ * Create a per-backend PGPROC struct in shared memory. We must do this
+ * before we can use LWLocks or access any shared memory.
+ */
+ InitProcess();
+
+ /*
+ * Make sure we aren't in PostmasterContext anymore. (We can't delete it
+ * just yet, though, because InitPostgres will need the HBA data.)
+ */
+ MemoryContextSwitchTo(TopMemoryContext);
+
+ PostgresMain(MyProcPort->database_name, MyProcPort->user_name);
+}
+
+
+/*
+ * BackendInitialize -- initialize an interactive (postmaster-child)
+ * backend process, and collect the client's startup packet.
+ *
+ * returns: nothing. Will not return at all if there's any failure.
+ *
+ * Note: this code does not depend on having any access to shared memory.
+ * Indeed, our approach to SIGTERM/timeout handling *requires* that
+ * shared memory not have been touched yet; see comments within.
+ * In the EXEC_BACKEND case, we are physically attached to shared memory
+ * but have not yet set up most of our local pointers to shmem structures.
+ */
+static void
+BackendInitialize(ClientSocket *client_sock, CAC_state cac)
+{
+ int status;
+ int ret;
+ Port *port;
+ char remote_host[NI_MAXHOST];
+ char remote_port[NI_MAXSERV];
+ StringInfoData ps_data;
+ MemoryContext oldcontext;
+
+ /* Tell fd.c about the long-lived FD associated with the client_sock */
+ ReserveExternalFD();
+
+ /*
+ * PreAuthDelay is a debugging aid for investigating problems in the
+ * authentication cycle: it can be set in postgresql.conf to allow time to
+ * attach to the newly-forked backend with a debugger. (See also
+ * PostAuthDelay, which we allow clients to pass through PGOPTIONS, but it
+ * is not honored until after authentication.)
+ */
+ if (PreAuthDelay > 0)
+ pg_usleep(PreAuthDelay * 1000000L);
+
+ /*
+ * Initialize libpq and enable reporting of ereport errors to the client.
+ * Must do this now because authentication uses libpq to send messages.
+ *
+ * The Port structure and all data structures attached to it are allocated
+ * in TopMemoryContext, so that they survive into PostgresMain execution.
+ * We need not worry about leaking this storage on failure, since we
+ * aren't in the postmaster process anymore.
+ */
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
+ port = MyProcPort = pq_init(client_sock);
+ MemoryContextSwitchTo(oldcontext);
+
+ whereToSendOutput = DestRemote; /* now safe to ereport to client */
+
+ /* set these to empty in case they are needed before we set them up */
+ port->remote_host = "";
+ port->remote_port = "";
+
+ /*
+ * We arrange to do _exit(1) if we receive SIGTERM or timeout while trying
+ * to collect the startup packet; while SIGQUIT results in _exit(2).
+ * Otherwise the postmaster cannot shutdown the database FAST or IMMED
+ * cleanly if a buggy client fails to send the packet promptly.
+ *
+ * Exiting with _exit(1) is only possible because we have not yet touched
+ * shared memory; therefore no outside-the-process state needs to get
+ * cleaned up.
+ */
+ pqsignal(SIGTERM, process_startup_packet_die);
+ /* SIGQUIT handler was already set up by InitPostmasterChild */
+ InitializeTimeouts(); /* establishes SIGALRM handler */
+ sigprocmask(SIG_SETMASK, &StartupBlockSig, NULL);
+
+ /*
+ * Get the remote host name and port for logging and status display.
+ */
+ remote_host[0] = '\0';
+ remote_port[0] = '\0';
+ if ((ret = pg_getnameinfo_all(&port->raddr.addr, port->raddr.salen,
+ remote_host, sizeof(remote_host),
+ remote_port, sizeof(remote_port),
+ (log_hostname ? 0 : NI_NUMERICHOST) | NI_NUMERICSERV)) != 0)
+ ereport(WARNING,
+ (errmsg_internal("pg_getnameinfo_all() failed: %s",
+ gai_strerror(ret))));
+
+ /*
+ * Save remote_host and remote_port in port structure (after this, they
+ * will appear in log_line_prefix data for log messages).
+ */
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
+ port->remote_host = pstrdup(remote_host);
+ port->remote_port = pstrdup(remote_port);
+
+ /* And now we can issue the Log_connections message, if wanted */
+ if (Log_connections)
+ {
+ if (remote_port[0])
+ ereport(LOG,
+ (errmsg("connection received: host=%s port=%s",
+ remote_host,
+ remote_port)));
+ else
+ ereport(LOG,
+ (errmsg("connection received: host=%s",
+ remote_host)));
+ }
+
+ /*
+ * If we did a reverse lookup to name, we might as well save the results
+ * rather than possibly repeating the lookup during authentication.
+ *
+ * Note that we don't want to specify NI_NAMEREQD above, because then we'd
+ * get nothing useful for a client without an rDNS entry. Therefore, we
+ * must check whether we got a numeric IPv4 or IPv6 address, and not save
+ * it into remote_hostname if so. (This test is conservative and might
+ * sometimes classify a hostname as numeric, but an error in that
+ * direction is safe; it only results in a possible extra lookup.)
+ */
+ if (log_hostname &&
+ ret == 0 &&
+ strspn(remote_host, "0123456789.") < strlen(remote_host) &&
+ strspn(remote_host, "0123456789ABCDEFabcdef:") < strlen(remote_host))
+ {
+ port->remote_hostname = pstrdup(remote_host);
+ }
+ MemoryContextSwitchTo(oldcontext);
+
+ /*
+ * Ready to begin client interaction. We will give up and _exit(1) after
+ * a time delay, so that a broken client can't hog a connection
+ * indefinitely. PreAuthDelay and any DNS interactions above don't count
+ * against the time limit.
+ *
+ * Note: AuthenticationTimeout is applied here while waiting for the
+ * startup packet, and then again in InitPostgres for the duration of any
+ * authentication operations. So a hostile client could tie up the
+ * process for nearly twice AuthenticationTimeout before we kick him off.
+ *
+ * Note: because PostgresMain will call InitializeTimeouts again, the
+ * registration of STARTUP_PACKET_TIMEOUT will be lost. This is okay
+ * since we never use it again after this function.
+ */
+ RegisterTimeout(STARTUP_PACKET_TIMEOUT, StartupPacketTimeoutHandler);
+ enable_timeout_after(STARTUP_PACKET_TIMEOUT, AuthenticationTimeout * 1000);
+
+ /*
+ * Receive the startup packet (which might turn out to be a cancel request
+ * packet).
+ */
+ status = ProcessStartupPacket(port, false, false);
+
+ /*
+ * If we're going to reject the connection due to database state, say so
+ * now instead of wasting cycles on an authentication exchange. (This also
+ * allows a pg_ping utility to be written.)
+ */
+ if (status == STATUS_OK)
+ {
+ switch (cac)
+ {
+ case CAC_STARTUP:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is starting up")));
+ break;
+ case CAC_NOTCONSISTENT:
+ if (EnableHotStandby)
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is not yet accepting connections"),
+ errdetail("Consistent recovery state has not been yet reached.")));
+ else
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is not accepting connections"),
+ errdetail("Hot standby mode is disabled.")));
+ break;
+ case CAC_SHUTDOWN:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is shutting down")));
+ break;
+ case CAC_RECOVERY:
+ ereport(FATAL,
+ (errcode(ERRCODE_CANNOT_CONNECT_NOW),
+ errmsg("the database system is in recovery mode")));
+ break;
+ case CAC_TOOMANY:
+ ereport(FATAL,
+ (errcode(ERRCODE_TOO_MANY_CONNECTIONS),
+ errmsg("sorry, too many clients already")));
+ break;
+ case CAC_OK:
+ break;
+ }
+ }
+
+ /*
+ * Disable the timeout, and prevent SIGTERM again.
+ */
+ disable_timeout(STARTUP_PACKET_TIMEOUT, false);
+ sigprocmask(SIG_SETMASK, &BlockSig, NULL);
+
+ /*
+ * As a safety check that nothing in startup has yet performed
+ * shared-memory modifications that would need to be undone if we had
+ * exited through SIGTERM or timeout above, check that no on_shmem_exit
+ * handlers have been registered yet. (This isn't terribly bulletproof,
+ * since someone might misuse an on_proc_exit handler for shmem cleanup,
+ * but it's a cheap and helpful check. We cannot disallow on_proc_exit
+ * handlers unfortunately, since pq_init() already registered one.)
+ */
+ check_on_shmem_exit_lists_are_empty();
+
+ /*
+ * Stop here if it was bad or a cancel packet. ProcessStartupPacket
+ * already did any appropriate error reporting.
+ */
+ if (status != STATUS_OK)
+ proc_exit(0);
+
+ /*
+ * Now that we have the user and database name, we can set the process
+ * title for ps. It's good to do this as early as possible in startup.
+ */
+ initStringInfo(&ps_data);
+ if (am_walsender)
+ appendStringInfo(&ps_data, "%s ", GetBackendTypeDesc(B_WAL_SENDER));
+ appendStringInfo(&ps_data, "%s ", port->user_name);
+ if (port->database_name[0] != '\0')
+ appendStringInfo(&ps_data, "%s ", port->database_name);
+ appendStringInfoString(&ps_data, port->remote_host);
+ if (port->remote_port[0] != '\0')
+ appendStringInfo(&ps_data, "(%s)", port->remote_port);
+
+ init_ps_display(ps_data.data);
+ pfree(ps_data.data);
+
+ set_ps_display("initializing");
+}
+
+/*
+ * Read a client's startup packet and do something according to it.
+ *
+ * Returns STATUS_OK or STATUS_ERROR, or might call ereport(FATAL) and
+ * not return at all.
+ *
+ * (Note that ereport(FATAL) stuff is sent to the client, so only use it
+ * if that's what you want. Return STATUS_ERROR if you don't want to
+ * send anything to the client, which would typically be appropriate
+ * if we detect a communications failure.)
+ *
+ * Set ssl_done and/or gss_done when negotiation of an encrypted layer
+ * (currently, TLS or GSSAPI) is completed. A successful negotiation of either
+ * encryption layer sets both flags, but a rejected negotiation sets only the
+ * flag for that layer, since the client may wish to try the other one. We
+ * should make no assumption here about the order in which the client may make
+ * requests.
+ */
+static int
+ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
+{
+ int32 len;
+ char *buf;
+ ProtocolVersion proto;
+ MemoryContext oldcontext;
+
+ pq_startmsgread();
+
+ /*
+ * Grab the first byte of the length word separately, so that we can tell
+ * whether we have no data at all or an incomplete packet. (This might
+ * sound inefficient, but it's not really, because of buffering in
+ * pqcomm.c.)
+ */
+ if (pq_getbytes((char *) &len, 1) == EOF)
+ {
+ /*
+ * If we get no data at all, don't clutter the log with a complaint;
+ * such cases often occur for legitimate reasons. An example is that
+ * we might be here after responding to NEGOTIATE_SSL_CODE, and if the
+ * client didn't like our response, it'll probably just drop the
+ * connection. Service-monitoring software also often just opens and
+ * closes a connection without sending anything. (So do port
+ * scanners, which may be less benign, but it's not really our job to
+ * notice those.)
+ */
+ return STATUS_ERROR;
+ }
+
+ if (pq_getbytes(((char *) &len) + 1, 3) == EOF)
+ {
+ /* Got a partial length word, so bleat about that */
+ if (!ssl_done && !gss_done)
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("incomplete startup packet")));
+ return STATUS_ERROR;
+ }
+
+ len = pg_ntoh32(len);
+ len -= 4;
+
+ if (len < (int32) sizeof(ProtocolVersion) ||
+ len > MAX_STARTUP_PACKET_LENGTH)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("invalid length of startup packet")));
+ return STATUS_ERROR;
+ }
+
+ /*
+ * Allocate space to hold the startup packet, plus one extra byte that's
+ * initialized to be zero. This ensures we will have null termination of
+ * all strings inside the packet.
+ */
+ buf = palloc(len + 1);
+ buf[len] = '\0';
+
+ if (pq_getbytes(buf, len) == EOF)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("incomplete startup packet")));
+ return STATUS_ERROR;
+ }
+ pq_endmsgread();
+
+ /*
+ * The first field is either a protocol version number or a special
+ * request code.
+ */
+ port->proto = proto = pg_ntoh32(*((ProtocolVersion *) buf));
+
+ if (proto == CANCEL_REQUEST_CODE)
+ {
+ CancelRequestPacket *canc;
+ int backendPID;
+ int32 cancelAuthCode;
+
+ if (len != sizeof(CancelRequestPacket))
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("invalid length of startup packet")));
+ return STATUS_ERROR;
+ }
+ canc = (CancelRequestPacket *) buf;
+ backendPID = (int) pg_ntoh32(canc->backendPID);
+ cancelAuthCode = (int32) pg_ntoh32(canc->cancelAuthCode);
+
+ processCancelRequest(backendPID, cancelAuthCode);
+ /* Not really an error, but we don't want to proceed further */
+ return STATUS_ERROR;
+ }
+
+ if (proto == NEGOTIATE_SSL_CODE && !ssl_done)
+ {
+ char SSLok;
+
+#ifdef USE_SSL
+ /* No SSL when disabled or on Unix sockets */
+ if (!LoadedSSL || port->laddr.addr.ss_family == AF_UNIX)
+ SSLok = 'N';
+ else
+ SSLok = 'S'; /* Support for SSL */
+#else
+ SSLok = 'N'; /* No support for SSL */
+#endif
+
+retry1:
+ if (send(port->sock, &SSLok, 1, 0) != 1)
+ {
+ if (errno == EINTR)
+ goto retry1; /* if interrupted, just retry */
+ ereport(COMMERROR,
+ (errcode_for_socket_access(),
+ errmsg("failed to send SSL negotiation response: %m")));
+ return STATUS_ERROR; /* close the connection */
+ }
+
+#ifdef USE_SSL
+ if (SSLok == 'S' && secure_open_server(port) == -1)
+ return STATUS_ERROR;
+#endif
+
+ /*
+ * At this point we should have no data already buffered. If we do,
+ * it was received before we performed the SSL handshake, so it wasn't
+ * encrypted and indeed may have been injected by a man-in-the-middle.
+ * We report this case to the client.
+ */
+ if (pq_buffer_has_data())
+ ereport(FATAL,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("received unencrypted data after SSL request"),
+ errdetail("This could be either a client-software bug or evidence of an attempted man-in-the-middle attack.")));
+
+ /*
+ * regular startup packet, cancel, etc packet should follow, but not
+ * another SSL negotiation request, and a GSS request should only
+ * follow if SSL was rejected (client may negotiate in either order)
+ */
+ return ProcessStartupPacket(port, true, SSLok == 'S');
+ }
+ else if (proto == NEGOTIATE_GSS_CODE && !gss_done)
+ {
+ char GSSok = 'N';
+
+#ifdef ENABLE_GSS
+ /* No GSSAPI encryption when on Unix socket */
+ if (port->laddr.addr.ss_family != AF_UNIX)
+ GSSok = 'G';
+#endif
+
+ while (send(port->sock, &GSSok, 1, 0) != 1)
+ {
+ if (errno == EINTR)
+ continue;
+ ereport(COMMERROR,
+ (errcode_for_socket_access(),
+ errmsg("failed to send GSSAPI negotiation response: %m")));
+ return STATUS_ERROR; /* close the connection */
+ }
+
+#ifdef ENABLE_GSS
+ if (GSSok == 'G' && secure_open_gssapi(port) == -1)
+ return STATUS_ERROR;
+#endif
+
+ /*
+ * At this point we should have no data already buffered. If we do,
+ * it was received before we performed the GSS handshake, so it wasn't
+ * encrypted and indeed may have been injected by a man-in-the-middle.
+ * We report this case to the client.
+ */
+ if (pq_buffer_has_data())
+ ereport(FATAL,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("received unencrypted data after GSSAPI encryption request"),
+ errdetail("This could be either a client-software bug or evidence of an attempted man-in-the-middle attack.")));
+
+ /*
+ * regular startup packet, cancel, etc packet should follow, but not
+ * another GSS negotiation request, and an SSL request should only
+ * follow if GSS was rejected (client may negotiate in either order)
+ */
+ return ProcessStartupPacket(port, GSSok == 'G', true);
+ }
+
+ /* Could add additional special packet types here */
+
+ /*
+ * Set FrontendProtocol now so that ereport() knows what format to send if
+ * we fail during startup.
+ */
+ FrontendProtocol = proto;
+
+ /* Check that the major protocol version is in range. */
+ if (PG_PROTOCOL_MAJOR(proto) < PG_PROTOCOL_MAJOR(PG_PROTOCOL_EARLIEST) ||
+ PG_PROTOCOL_MAJOR(proto) > PG_PROTOCOL_MAJOR(PG_PROTOCOL_LATEST))
+ ereport(FATAL,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("unsupported frontend protocol %u.%u: server supports %u.0 to %u.%u",
+ PG_PROTOCOL_MAJOR(proto), PG_PROTOCOL_MINOR(proto),
+ PG_PROTOCOL_MAJOR(PG_PROTOCOL_EARLIEST),
+ PG_PROTOCOL_MAJOR(PG_PROTOCOL_LATEST),
+ PG_PROTOCOL_MINOR(PG_PROTOCOL_LATEST))));
+
+ /*
+ * Now fetch parameters out of startup packet and save them into the Port
+ * structure.
+ */
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
+
+ /* Handle protocol version 3 startup packet */
+ {
+ int32 offset = sizeof(ProtocolVersion);
+ List *unrecognized_protocol_options = NIL;
+
+ /*
+ * Scan packet body for name/option pairs. We can assume any string
+ * beginning within the packet body is null-terminated, thanks to
+ * zeroing extra byte above.
+ */
+ port->guc_options = NIL;
+
+ while (offset < len)
+ {
+ char *nameptr = buf + offset;
+ int32 valoffset;
+ char *valptr;
+
+ if (*nameptr == '\0')
+ break; /* found packet terminator */
+ valoffset = offset + strlen(nameptr) + 1;
+ if (valoffset >= len)
+ break; /* missing value, will complain below */
+ valptr = buf + valoffset;
+
+ if (strcmp(nameptr, "database") == 0)
+ port->database_name = pstrdup(valptr);
+ else if (strcmp(nameptr, "user") == 0)
+ port->user_name = pstrdup(valptr);
+ else if (strcmp(nameptr, "options") == 0)
+ port->cmdline_options = pstrdup(valptr);
+ else if (strcmp(nameptr, "replication") == 0)
+ {
+ /*
+ * Due to backward compatibility concerns the replication
+ * parameter is a hybrid beast which allows the value to be
+ * either boolean or the string 'database'. The latter
+ * connects to a specific database which is e.g. required for
+ * logical decoding while.
+ */
+ if (strcmp(valptr, "database") == 0)
+ {
+ am_walsender = true;
+ am_db_walsender = true;
+ }
+ else if (!parse_bool(valptr, &am_walsender))
+ ereport(FATAL,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid value for parameter \"%s\": \"%s\"",
+ "replication",
+ valptr),
+ errhint("Valid values are: \"false\", 0, \"true\", 1, \"database\".")));
+ }
+ else if (strncmp(nameptr, "_pq_.", 5) == 0)
+ {
+ /*
+ * Any option beginning with _pq_. is reserved for use as a
+ * protocol-level option, but at present no such options are
+ * defined.
+ */
+ unrecognized_protocol_options =
+ lappend(unrecognized_protocol_options, pstrdup(nameptr));
+ }
+ else
+ {
+ /* Assume it's a generic GUC option */
+ port->guc_options = lappend(port->guc_options,
+ pstrdup(nameptr));
+ port->guc_options = lappend(port->guc_options,
+ pstrdup(valptr));
+
+ /*
+ * Copy application_name to port if we come across it. This
+ * is done so we can log the application_name in the
+ * connection authorization message. Note that the GUC would
+ * be used but we haven't gone through GUC setup yet.
+ */
+ if (strcmp(nameptr, "application_name") == 0)
+ {
+ port->application_name = pg_clean_ascii(valptr, 0);
+ }
+ }
+ offset = valoffset + strlen(valptr) + 1;
+ }
+
+ /*
+ * If we didn't find a packet terminator exactly at the end of the
+ * given packet length, complain.
+ */
+ if (offset != len - 1)
+ ereport(FATAL,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("invalid startup packet layout: expected terminator as last byte")));
+
+ /*
+ * If the client requested a newer protocol version or if the client
+ * requested any protocol options we didn't recognize, let them know
+ * the newest minor protocol version we do support and the names of
+ * any unrecognized options.
+ */
+ if (PG_PROTOCOL_MINOR(proto) > PG_PROTOCOL_MINOR(PG_PROTOCOL_LATEST) ||
+ unrecognized_protocol_options != NIL)
+ SendNegotiateProtocolVersion(unrecognized_protocol_options);
+ }
+
+ /* Check a user name was given. */
+ if (port->user_name == NULL || port->user_name[0] == '\0')
+ ereport(FATAL,
+ (errcode(ERRCODE_INVALID_AUTHORIZATION_SPECIFICATION),
+ errmsg("no PostgreSQL user name specified in startup packet")));
+
+ /* The database defaults to the user name. */
+ if (port->database_name == NULL || port->database_name[0] == '\0')
+ port->database_name = pstrdup(port->user_name);
+
+ if (am_walsender)
+ MyBackendType = B_WAL_SENDER;
+ else
+ MyBackendType = B_BACKEND;
+
+ /*
+ * Normal walsender backends, e.g. for streaming replication, are not
+ * connected to a particular database. But walsenders used for logical
+ * replication need to connect to a specific database. We allow streaming
+ * replication commands to be issued even if connected to a database as it
+ * can make sense to first make a basebackup and then stream changes
+ * starting from that.
+ */
+ if (am_walsender && !am_db_walsender)
+ port->database_name[0] = '\0';
+
+ /*
+ * Done filling the Port structure
+ */
+ MemoryContextSwitchTo(oldcontext);
+
+ return STATUS_OK;
+}
+
+/*
+ * Send a NegotiateProtocolVersion to the client. This lets the client know
+ * that they have requested a newer minor protocol version than we are able
+ * to speak. We'll speak the highest version we know about; the client can,
+ * of course, abandon the connection if that's a problem.
+ *
+ * We also include in the response a list of protocol options we didn't
+ * understand. This allows clients to include optional parameters that might
+ * be present either in newer protocol versions or third-party protocol
+ * extensions without fear of having to reconnect if those options are not
+ * understood, while at the same time making certain that the client is aware
+ * of which options were actually accepted.
+ */
+static void
+SendNegotiateProtocolVersion(List *unrecognized_protocol_options)
+{
+ StringInfoData buf;
+ ListCell *lc;
+
+ pq_beginmessage(&buf, PqMsg_NegotiateProtocolVersion);
+ pq_sendint32(&buf, PG_PROTOCOL_LATEST);
+ pq_sendint32(&buf, list_length(unrecognized_protocol_options));
+ foreach(lc, unrecognized_protocol_options)
+ pq_sendstring(&buf, lfirst(lc));
+ pq_endmessage(&buf);
+
+ /* no need to flush, some other message will follow */
+}
+
+
+/*
+ * SIGTERM while processing startup packet.
+ *
+ * Running proc_exit() from a signal handler would be quite unsafe.
+ * However, since we have not yet touched shared memory, we can just
+ * pull the plug and exit without running any atexit handlers.
+ *
+ * One might be tempted to try to send a message, or log one, indicating
+ * why we are disconnecting. However, that would be quite unsafe in itself.
+ * Also, it seems undesirable to provide clues about the database's state
+ * to a client that has not yet completed authentication, or even sent us
+ * a startup packet.
+ */
+static void
+process_startup_packet_die(SIGNAL_ARGS)
+{
+ _exit(1);
+}
+
+/*
+ * Timeout while processing startup packet.
+ * As for process_startup_packet_die(), we exit via _exit(1).
+ */
+static void
+StartupPacketTimeoutHandler(void)
+{
+ _exit(1);
+}
diff --git a/src/backend/tcop/meson.build b/src/backend/tcop/meson.build
index 6104f746f9e..19a97bbf55e 100644
--- a/src/backend/tcop/meson.build
+++ b/src/backend/tcop/meson.build
@@ -1,6 +1,7 @@
# Copyright (c) 2022-2024, PostgreSQL Global Development Group
backend_sources += files(
+ 'backend_startup.c',
'cmdtag.c',
'dest.c',
'fastpath.c',
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index 76280cdf453..8cc2ea9a5ed 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -52,6 +52,8 @@ extern PGDLLIMPORT int postmaster_alive_fds[2];
extern PGDLLIMPORT const char *progname;
+extern bool LoadedSSL;
+
extern void PostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
extern void ClosePostmasterPorts(bool am_syslogger);
extern void InitProcessGlobals(void);
@@ -61,6 +63,7 @@ extern int MaxLivePostmasterChildren(void);
extern bool PostmasterMarkPIDForWorkerNotify(int);
extern void BackendMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
+extern void processCancelRequest(int backendPID, int32 cancelAuthCode);
#ifdef EXEC_BACKEND
extern Size ShmemBackendArraySize(void);
diff --git a/src/include/tcop/backend_startup.h b/src/include/tcop/backend_startup.h
new file mode 100644
index 00000000000..109cc1c4609
--- /dev/null
+++ b/src/include/tcop/backend_startup.h
@@ -0,0 +1,41 @@
+/*-------------------------------------------------------------------------
+ *
+ * backend_startup.h
+ * prototypes for backend_startup.c.
+ *
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/tcop/backend_startup.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef BACKEND_STARTUP_H
+#define BACKEND_STARTUP_H
+
+/*
+ * CAC_state is passed from postmaster to the backend process, to indicate
+ * whether the connection should be accepted, or if the process should just
+ * send an error to the client and close the cnnection. Note that the
+ * connection can fail for various reasons even if postmaster passed CAC_OK.
+ */
+typedef enum CAC_state
+{
+ CAC_OK,
+ CAC_STARTUP,
+ CAC_SHUTDOWN,
+ CAC_RECOVERY,
+ CAC_NOTCONSISTENT,
+ CAC_TOOMANY,
+} CAC_state;
+
+/* Information passed from postmaster to backend process in 'startup_data' */
+typedef struct BackendStartupData
+{
+ CAC_state canAcceptConnections;
+} BackendStartupData;
+
+extern void BackendMain(char *startup_data, size_t startup_data_len) pg_attribute_noreturn();
+
+#endif /* BACKEND_STARTUP_H */
--
2.39.2
Re: Refactoring backend fork+exec code
On 13/03/2024 09:30, Heikki Linnakangas wrote:
Attached is a new version of the remaining patches.
Committed, with some final cosmetic cleanups. Thanks everyone!
--
Heikki Linnakangas
Neon (https://neon.tech)
Re: Refactoring backend fork+exec code
Heikki Linnakangas <hlinnaka@iki.fi> writes:
Committed, with some final cosmetic cleanups. Thanks everyone!
A couple of buildfarm animals don't like these tests:
Assert(child_type >= 0 && child_type < lengthof(child_process_kinds));
for example
ayu | 2024-03-19 13:08:05 | launch_backend.c:211:39: warning: comparison of constant 16 with expression of type 'BackendType' (aka 'enum BackendType') is always true [-Wtautological-constant-out-of-range-compare]
ayu | 2024-03-19 13:08:05 | launch_backend.c:233:39: warning: comparison of constant 16 with expression of type 'BackendType' (aka 'enum BackendType') is always true [-Wtautological-constant-out-of-range-compare]
I'm not real sure why it's moaning about the "<" check but not the
">= 0" check, which ought to be equally tautological given the
assumption that the input is a valid member of the enum. But
in any case, exactly how much value do these assertions carry?
If you're intent on keeping them, perhaps casting child_type to
int here would suppress the warnings. But personally I think
I'd lose the Asserts.
regards, tom lane
Re: Refactoring backend fork+exec code
On 20/03/2024 07:37, Tom Lane wrote:
A couple of buildfarm animals don't like these tests:
Assert(child_type >= 0 && child_type < lengthof(child_process_kinds));
for example
ayu | 2024-03-19 13:08:05 | launch_backend.c:211:39: warning: comparison of constant 16 with expression of type 'BackendType' (aka 'enum BackendType') is always true [-Wtautological-constant-out-of-range-compare]
ayu | 2024-03-19 13:08:05 | launch_backend.c:233:39: warning: comparison of constant 16 with expression of type 'BackendType' (aka 'enum BackendType') is always true [-Wtautological-constant-out-of-range-compare]I'm not real sure why it's moaning about the "<" check but not the
">= 0" check, which ought to be equally tautological given the
assumption that the input is a valid member of the enum. But
in any case, exactly how much value do these assertions carry?
If you're intent on keeping them, perhaps casting child_type to
int here would suppress the warnings. But personally I think
I'd lose the Asserts.
Yeah, it's not a very valuable assertion. Removed, thanks!
--
Heikki Linnakangas
Neon (https://neon.tech)
Re: Refactoring backend fork+exec code
On Wed, 20 Mar 2024 at 08:16, Heikki Linnakangas <hlinnaka@iki.fi> wrote:
Yeah, it's not a very valuable assertion. Removed, thanks!
How about we add it as a static assert instead of removing it, like we
have for many other similar arrays.
Attachments:
v1-0001-Add-child_process_kinds-static-assert.patchapplication/octet-stream; name=v1-0001-Add-child_process_kinds-static-assert.patchDownload
From b68b7d94eaf6b8e485af7038cd10c6ef032181ec Mon Sep 17 00:00:00 2001
From: Jelte Fennema-Nio <jelte.fennema@microsoft.com>
Date: Thu, 21 Mar 2024 11:24:48 +0100
Subject: [PATCH v1] Add child_process_kinds static assert
Some asserts were removed in d63d486d6c39 due to certain build farm
animals complaining. This adds their equivalent back as a static assert,
like we have for many similar arrays in the codebase.
---
src/backend/postmaster/launch_backend.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index cb0c3e2f8ab..8d3ef01d25a 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -205,6 +205,9 @@ child_process_kind child_process_kinds[] = {
[B_LOGGER] = {"syslogger", SysLoggerMain, false},
};
+StaticAssertDecl(lengthof(child_process_kinds) == BACKEND_NUM_TYPES,
+ "array length mismatch");
+
const char *
PostmasterChildName(BackendType child_type)
{
base-commit: 1db689715d44276407dc4d6fadbc11da8d391bd9
--
2.34.1
Re: Refactoring backend fork+exec code
Hello!
Maybe add PGDLLIMPORT to
extern bool LoadedSSL;
and
extern struct ClientSocket *MyClientSocket;
definitions in the src/include/postmaster/postmaster.h ?
With the best regards,
--
Anton A. Melnikov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Re: Refactoring backend fork+exec code
On 27/04/2024 11:27, Anton A. Melnikov wrote:
Hello!
Maybe add PGDLLIMPORT to
extern bool LoadedSSL;
and
extern struct ClientSocket *MyClientSocket;
definitions in the src/include/postmaster/postmaster.h ?
Peter E noticed and Michael fixed them in commit 768ceeeaa1 already.
--
Heikki Linnakangas
Neon (https://neon.tech)
Re: Refactoring backend fork+exec code
On 28.04.2024 22:36, Heikki Linnakangas wrote:
Peter E noticed and Michael fixed them in commit 768ceeeaa1 already.
Didn't check that is already fixed in the current master. Sorry!
Thanks for pointing this out!
With the best wishes,
--
Anton A. Melnikov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Re: Refactoring backend fork+exec code
On Mon, Mar 18, 2024 at 10:41 PM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
Committed, with some final cosmetic cleanups. Thanks everyone!
Nitpicking from UBSan with EXEC_BACKEND on Linux (line numbers may be
a bit off, from a branch of mine):
../src/backend/postmaster/launch_backend.c:772:2: runtime error: null
pointer passed as argument 2, which is declared to never be null
==13303==Using libbacktrace symbolizer.
#0 0x5555564b0202 in save_backend_variables
../src/backend/postmaster/launch_backend.c:772
#1 0x5555564b0242 in internal_forkexec
../src/backend/postmaster/launch_backend.c:311
#2 0x5555564b0bdd in postmaster_child_launch
../src/backend/postmaster/launch_backend.c:244
#3 0x5555564b3121 in StartChildProcess
../src/backend/postmaster/postmaster.c:3928
#4 0x5555564b933a in PostmasterMain
../src/backend/postmaster/postmaster.c:1357
#5 0x5555562de4ad in main ../src/backend/main/main.c:197
#6 0x7ffff667ad09 in __libc_start_main
(/lib/x86_64-linux-gnu/libc.so.6+0x23d09)
#7 0x555555e34279 in _start
(/tmp/cirrus-ci-build/build/tmp_install/usr/local/pgsql/bin/postgres+0x8e0279)
This silences it:
- memcpy(param->startup_data, startup_data, startup_data_len);
+ if (startup_data_len > 0)
+ memcpy(param->startup_data, startup_data, startup_data_len);(I found that out by testing EXEC_BACKEND on CI. I also learned that
the Mac and FreeBSD tasks fail with EXEC_BACKEND because of SysV shmem
bleating. We probably should go and crank up the relevant sysctls in
the .cirrus.tasks.yml...)
Re: Refactoring backend fork+exec code
While looking into [0]/messages/by-id/CAKAnmmJkZtZAiSryho=gYpbvC7H-HNjEDAh16F3SoC9LPu8rqQ@mail.gmail.com, I noticed that main() still only checks for the
--fork prefix, but IIUC commit aafc05d removed all --fork* options except
for --forkchild. I've attached a patch to strengthen the check in main().
This is definitely just a nitpick.
[0]: /messages/by-id/CAKAnmmJkZtZAiSryho=gYpbvC7H-HNjEDAh16F3SoC9LPu8rqQ@mail.gmail.com
--
nathan
Attachments:
forkchild_check.patchtext/plain; charset=us-asciiDownload
diff --git a/src/backend/main/main.c b/src/backend/main/main.c
index bfd0c5ed65..4672aab837 100644
--- a/src/backend/main/main.c
+++ b/src/backend/main/main.c
@@ -185,7 +185,7 @@ main(int argc, char *argv[])
else if (argc > 1 && strcmp(argv[1], "--boot") == 0)
BootstrapModeMain(argc, argv, false);
#ifdef EXEC_BACKEND
- else if (argc > 1 && strncmp(argv[1], "--fork", 6) == 0)
+ else if (argc > 1 && strncmp(argv[1], "--forkchild", 11) == 0)
SubPostmasterMain(argc, argv);
#endif
else if (argc > 1 && strcmp(argv[1], "--describe-config") == 0)
Re: Refactoring backend fork+exec code
On 18/05/2024 08:24, Thomas Munro wrote:
Nitpicking from UBSan with EXEC_BACKEND on Linux (line numbers may be
a bit off, from a branch of mine):../src/backend/postmaster/launch_backend.c:772:2: runtime error: null
pointer passed as argument 2, which is declared to never be null
==13303==Using libbacktrace symbolizer.
#0 0x5555564b0202 in save_backend_variables
../src/backend/postmaster/launch_backend.c:772
#1 0x5555564b0242 in internal_forkexec
../src/backend/postmaster/launch_backend.c:311
#2 0x5555564b0bdd in postmaster_child_launch
../src/backend/postmaster/launch_backend.c:244
#3 0x5555564b3121 in StartChildProcess
../src/backend/postmaster/postmaster.c:3928
#4 0x5555564b933a in PostmasterMain
../src/backend/postmaster/postmaster.c:1357
#5 0x5555562de4ad in main ../src/backend/main/main.c:197
#6 0x7ffff667ad09 in __libc_start_main
(/lib/x86_64-linux-gnu/libc.so.6+0x23d09)
#7 0x555555e34279 in _start
(/tmp/cirrus-ci-build/build/tmp_install/usr/local/pgsql/bin/postgres+0x8e0279)This silences it:
- memcpy(param->startup_data, startup_data, startup_data_len); + if (startup_data_len > 0) + memcpy(param->startup_data, startup_data, startup_data_len);
Fixed, thanks!
On 17/06/2024 21:36, Nathan Bossart wrote:
While looking into [0], I noticed that main() still only checks for the
--fork prefix, but IIUC commit aafc05d removed all --fork* options except
for --forkchild. I've attached a patch to strengthen the check in main().
This is definitely just a nitpick.
Fixed this too, thanks!
--
Heikki Linnakangas
Neon (https://neon.tech)