Weaker shmem interlock w/o postmaster.pid
If a starting postmaster's CreateLockFile() finds an existing postmaster.pid,
it subjects the shared memory segment named therein to the careful scrutiny of
PGSharedMemoryIsInUse(). If that segment matches the current data directory
and has any attached processes, we bail with the "pre-existing shared memory
block ... is still in use" error. When the postmaster.pid file is missing,
there's inherently less we can do to reliably detect this situation; in
particular, an old postmaster could have chosen an unusual key due to the
usual 1+(port*1000) key being in use. That being said, PGSharedMemoryCreate()
typically will stumble upon the old segment, and it (its sysv variant, anyway)
applies checks much weaker than those of PGSharedMemoryIsInUse(). If the
segment has a PGShmemHeader and the postmaster PID named in that header is not
alive, PGSharedMemoryCreate() will delete the segment and proceed. Shouldn't
it instead check the same things as PGSharedMemoryIsInUse()?
The concrete situation in which I encountered this involved PostgreSQL 9.2 and
an immediate shutdown with a backend that had blocked SIGQUIT. The backend
survived the immediate shutdown as one would expect. The postmaster
nonetheless removed postmaster.pid before exiting, and I could immediately
restart PostgreSQL despite the survival of the SIGQUIT-blocked backend. If I
instead SIGKILL the postmaster, postmaster.pid remains, and I must kill stray
backends before restarting. The postmaster should not remove postmaster.pid
unless it has verified that its children have exited. Concretely, that means
not removing postmaster.pid on immediate shutdown in 9.3 and earlier. That's
consistent with the rough nature of an immediate shutdown, anyway.
I'm thinking to preserve postmaster.pid at immediate shutdown in all released
versions, but I'm less sure about back-patching a change to make
PGSharedMemoryCreate() pickier. On the one hand, allowing startup to proceed
with backends still active in the same data directory is a corruption hazard.
On the other hand, it could break weird shutdown/restart patterns that permit
trivial lifespan overlap between backends of different postmasters. Opinions?
Thanks,
nm
--
Noah Misch
EnterpriseDB http://www.enterprisedb.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
* Noah Misch (noah@leadboat.com) wrote:
Shouldn't it instead check the same things as PGSharedMemoryIsInUse()?
Offhand, I tend to agree that we should really be doing a very careful
job of looking at if an existing segment is still in use.
The concrete situation in which I encountered this involved PostgreSQL 9.2 and
an immediate shutdown with a backend that had blocked SIGQUIT. The backend
survived the immediate shutdown as one would expect.
Well.. We expect this now because of the analysis you did in the
adjacent thread showing how it can happen.
The postmaster
nonetheless removed postmaster.pid before exiting, and I could immediately
restart PostgreSQL despite the survival of the SIGQUIT-blocked backend. If I
instead SIGKILL the postmaster, postmaster.pid remains, and I must kill stray
backends before restarting. The postmaster should not remove postmaster.pid
unless it has verified that its children have exited.
This makes sense, however..
Concretely, that means
not removing postmaster.pid on immediate shutdown in 9.3 and earlier. That's
consistent with the rough nature of an immediate shutdown, anyway.
I don't like leaving the postmaster.pid file around, even on an
immediate shutdown. I don't have any great suggestions regarding what
to do, given what we try to do wrt 'immediate', so perhaps it's
acceptable for future releases.
I'm thinking to preserve postmaster.pid at immediate shutdown in all released
versions, but I'm less sure about back-patching a change to make
PGSharedMemoryCreate() pickier. On the one hand, allowing startup to proceed
with backends still active in the same data directory is a corruption hazard.
The corruption risk, imv anyway, is sufficient to backpatch the change
and overrides the concerns around very fast shutdown/restarts.
Thanks,
Stephen
On Tue, Sep 10, 2013 at 11:33 PM, Noah Misch <noah@leadboat.com> wrote:
I'm thinking to preserve postmaster.pid at immediate shutdown in all released
versions, but I'm less sure about back-patching a change to make
PGSharedMemoryCreate() pickier. On the one hand, allowing startup to proceed
with backends still active in the same data directory is a corruption hazard.
On the other hand, it could break weird shutdown/restart patterns that permit
trivial lifespan overlap between backends of different postmasters. Opinions?
I'm more sanguine about the second change than the first. Leaving
postmaster.pid around seems like a clear user-visible behavior change
that could break user scripts or have other consequences that we don't
foresee; thus, I would vote against back-patching it. Indeed, I'm not
sure it's a good idea to do that even in master. On the other hand,
tightening the checks in PGSharedMemoryCreate() seems very much worth
doing, and I think it might also be safe enough to back-patch.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Wed, Sep 11, 2013 at 11:32:01AM -0400, Stephen Frost wrote:
* Noah Misch (noah@leadboat.com) wrote:
The concrete situation in which I encountered this involved PostgreSQL 9.2 and
an immediate shutdown with a backend that had blocked SIGQUIT. The backend
survived the immediate shutdown as one would expect.Well.. We expect this now because of the analysis you did in the
adjacent thread showing how it can happen.
That was a surprising way for it to happen, but there have long been known
vectors like a SIGSTOP'd backend or a backend that has blocked SIGQUIT.
Concretely, that means
not removing postmaster.pid on immediate shutdown in 9.3 and earlier. That's
consistent with the rough nature of an immediate shutdown, anyway.I don't like leaving the postmaster.pid file around, even on an
immediate shutdown. I don't have any great suggestions regarding what
to do, given what we try to do wrt 'immediate', so perhaps it's
acceptable for future releases.
Fair enough.
I'm thinking to preserve postmaster.pid at immediate shutdown in all released
versions, but I'm less sure about back-patching a change to make
PGSharedMemoryCreate() pickier. On the one hand, allowing startup to proceed
with backends still active in the same data directory is a corruption hazard.The corruption risk, imv anyway, is sufficient to backpatch the change
and overrides the concerns around very fast shutdown/restarts.
Making PGSharedMemoryCreate() pickier in all branches will greatly diminish
the marginal value of preserving postmaster.pid, so I'm fine with dropping the
postmaster.pid side of the proposal.
Thanks,
nm
--
Noah Misch
EnterpriseDB http://www.enterprisedb.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Noah Misch-2 wrote
I'm thinking to preserve postmaster.pid at immediate shutdown in all
released
versions, but I'm less sure about back-patching a change to make
PGSharedMemoryCreate() pickier. On the one hand, allowing startup toproceed
with backends still active in the same data directory is a corruption
hazard.
The corruption risk, imv anyway, is sufficient to backpatch the change
and overrides the concerns around very fast shutdown/restarts.Making PGSharedMemoryCreate() pickier in all branches will greatly
diminish
the marginal value of preserving postmaster.pid, so I'm fine with dropping
the
postmaster.pid side of the proposal.
Its probably still worth a fresh look at the immediate shutdown process to
see whether the current location where postmaster.pid is removed is
acceptable. It may not be necessary to leave it in place always but:
1) if there is a section of shared memory that can only be reached/found if
one knows the pid, and
2) postmaster.pid is removed before that area is "secured from future
clobbering"
then there may be a risk that can still be mitigated by moving its removal
without having to go to the extreme.
David J.
--
View this message in context: http://postgresql.1045698.n5.nabble.com/Weaker-shmem-interlock-w-o-postmaster-pid-tp5770399p5770559.html
Sent from the PostgreSQL - hackers mailing list archive at Nabble.com.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Wed, Sep 11, 2013 at 07:57:22PM -0700, David Johnston wrote:
Noah Misch-2 wrote
Making PGSharedMemoryCreate() pickier in all branches will greatly
diminish
the marginal value of preserving postmaster.pid, so I'm fine with dropping
the
postmaster.pid side of the proposal.Its probably still worth a fresh look at the immediate shutdown process to
see whether the current location where postmaster.pid is removed is
acceptable. It may not be necessary to leave it in place always but:1) if there is a section of shared memory that can only be reached/found if
one knows the pid, and
Similar: one needs a sysv shared memory key to find the segment, and
postmaster.pid records that key. The chosen key is almost always the same
from run to run, so a new postmaster typically does find the old segment even
without postmaster.pid.
2) postmaster.pid is removed before that area is "secured from future
clobbering"
Clobbering shared memory is not the actual threat here. We use the shared
memory segment as a witness to the fact that PostgreSQL processes are active
in a data directory. If we let children of different postmasters operate in
the same directory simultaneously, they can corrupt data.
then there may be a risk that can still be mitigated by moving its removal
without having to go to the extreme.
--
Noah Misch
EnterpriseDB http://www.enterprisedb.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Wed, Sep 11, 2013 at 02:10:45PM -0400, Robert Haas wrote:
On Tue, Sep 10, 2013 at 11:33 PM, Noah Misch <noah@leadboat.com> wrote:
I'm thinking to preserve postmaster.pid at immediate shutdown in all released
versions, but I'm less sure about back-patching a change to make
PGSharedMemoryCreate() pickier. On the one hand, allowing startup to proceed
with backends still active in the same data directory is a corruption hazard.
On the other hand, it could break weird shutdown/restart patterns that permit
trivial lifespan overlap between backends of different postmasters. Opinions?I'm more sanguine about the second change than the first. Leaving
postmaster.pid around seems like a clear user-visible behavior change
that could break user scripts or have other consequences that we don't
foresee; thus, I would vote against back-patching it. Indeed, I'm not
sure it's a good idea to do that even in master. On the other hand,
tightening the checks in PGSharedMemoryCreate() seems very much worth
doing, and I think it might also be safe enough to back-patch.
Were these changes every applied? I don't see them.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Wed, Feb 12, 2014 at 06:06:40PM -0500, Bruce Momjian wrote:
On Wed, Sep 11, 2013 at 02:10:45PM -0400, Robert Haas wrote:
On Tue, Sep 10, 2013 at 11:33 PM, Noah Misch <noah@leadboat.com> wrote:
I'm thinking to preserve postmaster.pid at immediate shutdown in all released
versions, but I'm less sure about back-patching a change to make
PGSharedMemoryCreate() pickier. On the one hand, allowing startup to proceed
with backends still active in the same data directory is a corruption hazard.
On the other hand, it could break weird shutdown/restart patterns that permit
trivial lifespan overlap between backends of different postmasters. Opinions?I'm more sanguine about the second change than the first. Leaving
postmaster.pid around seems like a clear user-visible behavior change
that could break user scripts or have other consequences that we don't
foresee; thus, I would vote against back-patching it. Indeed, I'm not
sure it's a good idea to do that even in master. On the other hand,
tightening the checks in PGSharedMemoryCreate() seems very much worth
doing, and I think it might also be safe enough to back-patch.Were these changes every applied? I don't see them.
No, I haven't gotten around to writing them.
--
Noah Misch
EnterpriseDB http://www.enterprisedb.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Wed, Sep 11, 2013 at 10:28:00PM -0400, Noah Misch wrote:
On Wed, Sep 11, 2013 at 11:32:01AM -0400, Stephen Frost wrote:
* Noah Misch (noah@leadboat.com) wrote:
Concretely, that means
not removing postmaster.pid on immediate shutdown in 9.3 and earlier. That's
consistent with the rough nature of an immediate shutdown, anyway.
With 9.3 having a few months left, that's less interesting, but ...
I don't like leaving the postmaster.pid file around, even on an
immediate shutdown. I don't have any great suggestions regarding what
to do, given what we try to do wrt 'immediate', so perhaps it's
acceptable for future releases.Fair enough.
... we could still worry about folks doing "kill -9 `head -n1 postmaster.pid`
&& rm postmaster.pid". A possible defense is to record a copy of the shm
identifiers in a second file ("sysv_shmem_key"?) within the data directory.
Since users would have no expectations about a new file's lifecycle, we could
remove it when and only when we verify a segment doesn't exist. However, a
DBA determined to remove postmaster.pid would likely remove both files.
I'm thinking to preserve postmaster.pid at immediate shutdown in all released
versions, but I'm less sure about back-patching a change to make
PGSharedMemoryCreate() pickier. On the one hand, allowing startup to proceed
with backends still active in the same data directory is a corruption hazard.The corruption risk, imv anyway, is sufficient to backpatch the change
and overrides the concerns around very fast shutdown/restarts.Making PGSharedMemoryCreate() pickier in all branches will greatly diminish
the marginal value of preserving postmaster.pid, so I'm fine with dropping the
postmaster.pid side of the proposal.
Here's an implementation. The first, small patch replaces an obsolete
errhint() about manually removing shared memory blocks. In its place,
recommend killing processes. Today's text, added in commit 4d14fe0, is too
rarely pertinent to justify a HINT. (You might use ipcrm if a PostgreSQL
process is stuck in D state and has SIGKILL pending, so it will never become
runnable again.) Removal of the ipcclean script ten years ago (39627b1) and
its non-replacement corroborate that manual shm removal is now a niche goal.
In the main patch, one of the tests covers an unsafe case that sysv_shmem.c
still doesn't detect. I could pursue a fix via the aforementioned
sysv_shmem_key file, modulo the possibility of a DBA removing it. I could
also, when postmaster.pid is missing, make sysv_shmem.c check the first N
(N=100?) keys applicable to the selected port. My gut feeling is that neither
thing is worthwhile, but I'm interested to hear other opinions.
This removes the creatorPID test, which commit c715fde added before commit
4d14fe0 added the shm_nattch test. The pair of tests is not materially
stronger than the shm_nattch test by itself. For reasons explained in the
comment at "enum IpcMemoryState", this patch has us cease recycling segments
pertaining to data directories other than our own. This isn't ideal for the
buildfarm, where a postmaster crash bug would permanently leak a shm segment.
I think the benefits for production use cases eclipse this drawback. Did I
miss a reason to like the old behavior?
Single-user mode has been protected even less than normal execution, because
it selected a shm key as though using port zero. Thus, starting single-user
mode after an incomplete shutdown would never detect the pre-shutdown shm.
The patch makes single-user mode work like normal execution. Until commit
de98a7e, single-user mode used malloc(), not a shm segment. That commit also
restricted single-user mode to not recycle old segments. I didn't find a
rationale for that restriction, but a contributing factor may have been the
fact that every single-user process uses the same port-0 shm key space. Also,
initdb starts various single-user backends, and reaping stale segments would
be an odd side effect for initdb. With single-user mode using a real port
number and PostgreSQL no longer recycling segments of other data directories,
I removed that restriction. By the way, as of this writing, my regular
development system has 41 stale segments, all in the single-user key space
(0x00000001 to 0x0000002d with gaps). It's clear how they persisted once
leaked, but I don't know how I leaked them in the first place.
I ran 9327 iterations of the test file, and it did not leak a shm segment in
that time. I tested on Red Hat and on Windows Server 2016; I won't be shocked
if the test (not the code under test) breaks on other Windows configurations.
The patch adds PostgresNode features to enable that test coverage.
The comment referring to a CreateDataDirLockFile() bug refers to this one:
/messages/by-id/flat/20120803145635.GE9683@tornado.leadboat.com
Thanks,
nm
Attachments:
ipcrm-hint-v1.patchtext/plain; charset=us-asciiDownload
--- a/src/backend/utils/init/miscinit.c
+++ b/src/backend/utils/init/miscinit.c
@@ -1070,14 +1070,10 @@ CreateLockFile(const char *filename, bool amPostmaster,
if (PGSharedMemoryIsInUse(id1, id2))
ereport(FATAL,
(errcode(ERRCODE_LOCK_FILE_EXISTS),
- errmsg("pre-existing shared memory block "
- "(key %lu, ID %lu) is still in use",
+ errmsg("pre-existing shared memory block (key %lu, ID %lu) is still in use",
id1, id2),
- errhint("If you're sure there are no old "
- "server processes still running, remove "
- "the shared memory block "
- "or just delete the file \"%s\".",
- filename)));
+ errhint("Terminate any old server processes associated with data directory \"%s\".",
+ refName)));
}
}
PGSharedMemoryCreate-safety-v1.patchtext/plain; charset=us-asciiDownload
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 741c455..7a9a68e 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -71,6 +71,26 @@
typedef key_t IpcMemoryKey; /* shared memory key passed to shmget(2) */
typedef int IpcMemoryId; /* shared memory ID returned by shmget(2) */
+/*
+ * How does a given IpcMemoryId relate to this PostgreSQL process?
+ *
+ * One could recycle unattached segments of different data directories if we
+ * distinguished that case from other SHMSTATE_FOREIGN cases. Doing so would
+ * cause us to visit less of the key space, making us less likely to detect a
+ * SHMSTATE_ATTACHED key. It would also complicate the concurrency analysis,
+ * in that postmasters of different data directories could simultaneously
+ * attempt to recycle a given key. We'll waste keys longer in some cases, but
+ * avoiding the problems of the alternative justifies that loss.
+ */
+enum IpcMemoryState
+{
+ SHMSTATE_ANALYSIS_FAILURE, /* unexpected failure to analyze the ID */
+ SHMSTATE_ATTACHED, /* pertinent to DataDir, has attached PIDs */
+ SHMSTATE_EEXISTS, /* no segment of that ID */
+ SHMSTATE_FOREIGN, /* exists, but not pertinent to DataDir */
+ SHMSTATE_UNATTACHED /* pertinent to DataDir, no attached PIDs */
+};
+
unsigned long UsedShmemSegID = 0;
void *UsedShmemSegAddr = NULL;
@@ -83,6 +103,7 @@ static void *AnonymousShmem = NULL;
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
+static enum IpcMemoryState IpcMemoryAnalyze(IpcMemoryId shmId);
static PGShmemHeader *PGSharedMemoryAttach(IpcMemoryKey key,
IpcMemoryId *shmid);
@@ -288,7 +309,23 @@ IpcMemoryDelete(int status, Datum shmId)
bool
PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2)
{
- IpcMemoryId shmId = (IpcMemoryId) id2;
+ switch (IpcMemoryAnalyze((IpcMemoryId) id2))
+ {
+ case SHMSTATE_EEXISTS:
+ case SHMSTATE_FOREIGN:
+ case SHMSTATE_UNATTACHED:
+ return false;
+ case SHMSTATE_ANALYSIS_FAILURE:
+ case SHMSTATE_ATTACHED:
+ return true;
+ }
+ return true;
+}
+
+/* See comment at enum IpcMemoryState. */
+static enum IpcMemoryState
+IpcMemoryAnalyze(IpcMemoryId shmId)
+{
struct shmid_ds shmStat;
struct stat statbuf;
PGShmemHeader *hdr;
@@ -305,7 +342,7 @@ PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2)
* exists.
*/
if (errno == EINVAL)
- return false;
+ return SHMSTATE_EEXISTS;
/*
* EACCES implies that the segment belongs to some other userid, which
@@ -313,7 +350,7 @@ PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2)
* is relevant to our data directory).
*/
if (errno == EACCES)
- return false;
+ return SHMSTATE_FOREIGN;
/*
* Some Linux kernel versions (in fact, all of them as of July 2007)
@@ -324,7 +361,7 @@ PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2)
*/
#ifdef HAVE_LINUX_EIDRM_BUG
if (errno == EIDRM)
- return false;
+ return SHMSTATE_EEXISTS;
#endif
/*
@@ -332,25 +369,26 @@ PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2)
* only likely case is EIDRM, which implies that the segment has been
* IPC_RMID'd but there are still processes attached to it.
*/
- return true;
+ return SHMSTATE_ANALYSIS_FAILURE;
}
- /* If it has no attached processes, it's not in use */
- if (shmStat.shm_nattch == 0)
- return false;
-
/*
* Try to attach to the segment and see if it matches our data directory.
* This avoids shmid-conflict problems on machines that are running
* several postmasters under the same userid.
*/
if (stat(DataDir, &statbuf) < 0)
- return true; /* if can't stat, be conservative */
+ return SHMSTATE_ANALYSIS_FAILURE; /* can't stat; be conservative */
+ /*
+ * If we can't attach, be conservative. This may fail if postmaster.pid
+ * furnished the shmId and another user created a world-readable segment
+ * of the same shmId. It shouldn't fail if PGSharedMemoryAttach()
+ * furnished the shmId, because that function tests shmat().
+ */
hdr = (PGShmemHeader *) shmat(shmId, NULL, PG_SHMAT_FLAGS);
-
if (hdr == (PGShmemHeader *) -1)
- return true; /* if can't attach, be conservative */
+ return SHMSTATE_ANALYSIS_FAILURE;
if (hdr->magic != PGShmemMagic ||
hdr->device != statbuf.st_dev ||
@@ -358,16 +396,15 @@ PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2)
{
/*
* It's either not a Postgres segment, or not one for my data
- * directory. In either case it poses no threat.
+ * directory.
*/
shmdt((void *) hdr);
- return false;
+ return SHMSTATE_FOREIGN;
}
- /* Trouble --- looks a lot like there's still live backends */
shmdt((void *) hdr);
- return true;
+ return shmStat.shm_nattch == 0 ? SHMSTATE_UNATTACHED : SHMSTATE_ATTACHED;
}
#ifdef USE_ANONYMOUS_SHMEM
@@ -543,19 +580,16 @@ AnonymousShmemDetach(int status, Datum arg)
* standard header. Also, register an on_shmem_exit callback to release
* the storage.
*
- * Dead Postgres segments are recycled if found, but we do not fail upon
- * collision with non-Postgres shmem segments. The idea here is to detect and
- * re-use keys that may have been assigned by a crashed postmaster or backend.
- *
- * makePrivate means to always create a new segment, rather than attach to
- * or recycle any existing segment.
+ * Dead Postgres segments pertinent to this DataDir are recycled if found, but
+ * we do not fail upon collision with foreign shmem segments. The idea here
+ * is to detect and re-use keys that may have been assigned by a crashed
+ * postmaster or backend.
*
* The port number is passed for possible use as a key (for SysV, we use
- * it to generate the starting shmem key). In a standalone backend,
- * zero will be passed.
+ * it to generate the starting shmem key).
*/
PGShmemHeader *
-PGSharedMemoryCreate(Size size, bool makePrivate, int port,
+PGSharedMemoryCreate(Size size, int port,
PGShmemHeader **shim)
{
IpcMemoryKey NextShmemSegID;
@@ -592,11 +626,18 @@ PGSharedMemoryCreate(Size size, bool makePrivate, int port,
/* Make sure PGSharedMemoryAttach doesn't fail without need */
UsedShmemSegAddr = NULL;
- /* Loop till we find a free IPC key */
- NextShmemSegID = port * 1000;
+ /*
+ * Loop till we find a free IPC key. Trust CreateDataDirLockFile() to
+ * ensure no more than one postmaster per data directory can enter this
+ * loop simultaneously. (CreateDataDirLockFile() does not ensure that,
+ * but prefer fixing it over coping here.)
+ */
+ NextShmemSegID = 1 + port * 1000;
- for (NextShmemSegID++;; NextShmemSegID++)
+ for (;;)
{
+ dsm_handle hdr_dsm;
+
/* Try to create new segment */
memAddress = InternalIpcMemoryCreate(NextShmemSegID, sysvsize);
if (memAddress)
@@ -604,58 +645,62 @@ PGSharedMemoryCreate(Size size, bool makePrivate, int port,
/* Check shared memory and possibly remove and recreate */
- if (makePrivate) /* a standalone backend shouldn't do this */
- continue;
-
if ((memAddress = PGSharedMemoryAttach(NextShmemSegID, &shmid)) == NULL)
- continue; /* can't attach, not one of mine */
-
- /*
- * If I am not the creator and it belongs to an extant process,
- * continue.
- */
- hdr = (PGShmemHeader *) memAddress;
- if (hdr->creatorPID != getpid())
{
- if (kill(hdr->creatorPID, 0) == 0 || errno != ESRCH)
- {
- shmdt(memAddress);
- continue; /* segment belongs to a live process */
- }
+ NextShmemSegID++;
+ continue;
}
-
- /*
- * The segment appears to be from a dead Postgres process, or from a
- * previous cycle of life in this same process. Zap it, if possible,
- * and any associated dynamic shared memory segments, as well. This
- * probably shouldn't fail, but if it does, assume the segment belongs
- * to someone else after all, and continue quietly.
- */
- if (hdr->dsm_control != 0)
- dsm_cleanup_using_control_segment(hdr->dsm_control);
+ hdr_dsm = ((PGShmemHeader *) memAddress)->dsm_control;
shmdt(memAddress);
- if (shmctl(shmid, IPC_RMID, NULL) < 0)
- continue;
+ memAddress = NULL;
- /*
- * Now try again to create the segment.
- */
- memAddress = InternalIpcMemoryCreate(NextShmemSegID, sysvsize);
- if (memAddress)
- break; /* successful create and attach */
+ switch (IpcMemoryAnalyze(shmid))
+ {
+ case SHMSTATE_ANALYSIS_FAILURE:
+ case SHMSTATE_ATTACHED:
+ ereport(FATAL,
+ (errcode(ERRCODE_LOCK_FILE_EXISTS),
+ errmsg("pre-existing shared memory block (key %lu, ID %lu) is still in use",
+ (unsigned long) NextShmemSegID,
+ (unsigned long) shmid),
+ errhint("Terminate any old server processes associated with data directory \"%s\".",
+ DataDir)));
+ case SHMSTATE_EEXISTS:
- /*
- * Can only get here if some other process managed to create the same
- * shmem key before we did. Let him have that one, loop around to try
- * next key.
- */
+ /*
+ * To our surprise, some other process deleted the segment
+ * between InternalIpcMemoryCreate() and IpcMemoryAnalyze().
+ * Moments earlier, we would have seen SHMSTATE_FOREIGN. Try
+ * that same ID again.
+ */
+ elog(LOG,
+ "shared memory block (key %lu, ID %lu) deleted during startup",
+ (unsigned long) NextShmemSegID,
+ (unsigned long) shmid);
+ break;
+ case SHMSTATE_FOREIGN:
+ NextShmemSegID++;
+ break;
+ case SHMSTATE_UNATTACHED:
+
+ /*
+ * The segment pertains to DataDir, and every process that had
+ * used it has died or detached. Zap it, if possible, and any
+ * associated dynamic shared memory segments, as well. This
+ * shouldn't fail, but if it does, assume the segment belongs
+ * to someone else after all, and try the next candidate.
+ * Otherwise, try again to create the segment. That may fail
+ * if some other process creates the same shmem key before we
+ * do, in which case we'll try the next key.
+ */
+ if (hdr_dsm != 0)
+ dsm_cleanup_using_control_segment(hdr_dsm);
+ if (shmctl(shmid, IPC_RMID, NULL) < 0)
+ NextShmemSegID++;
+ }
}
- /*
- * OK, we created a new segment. Mark it as created by this process. The
- * order of assignments here is critical so that another Postgres process
- * can't see the header as valid but belonging to an invalid PID!
- */
+ /* Initialize new segment. */
hdr = (PGShmemHeader *) memAddress;
hdr->creatorPID = getpid();
hdr->magic = PGShmemMagic;
@@ -816,7 +861,10 @@ PGSharedMemoryDetach(void)
/*
* Attach to shared memory and make sure it has a Postgres header
*
- * Returns attach address if OK, else NULL
+ * Returns attach address if OK, else NULL. Treat a NULL return value like
+ * SHMSTATE_FOREIGN. (EACCES is common. ENOENT, a narrow possibility,
+ * implies SHMSTATE_EEXISTS, but one can safely treat SHMSTATE_EEXISTS like
+ * SHMSTATE_FOREIGN.)
*/
static PGShmemHeader *
PGSharedMemoryAttach(IpcMemoryKey key, IpcMemoryId *shmid)
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index f8ca52e..3888c26 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -170,14 +170,9 @@ EnableLockPagesPrivilege(int elevel)
*
* Create a shared memory segment of the given size and initialize its
* standard header.
- *
- * makePrivate means to always create a new segment, rather than attach to
- * or recycle any existing segment. On win32, we always create a new segment,
- * since there is no need for recycling (segments go away automatically
- * when the last backend exits)
*/
PGShmemHeader *
-PGSharedMemoryCreate(Size size, bool makePrivate, int port,
+PGSharedMemoryCreate(Size size, int port,
PGShmemHeader **shim)
{
void *memAddress;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index a4b53b3..cf66c76 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -2509,7 +2509,7 @@ reset_shared(int port)
* determine IPC keys. This helps ensure that we will clean up dead IPC
* objects if the postmaster crashes and is restarted.
*/
- CreateSharedMemoryAndSemaphores(false, port);
+ CreateSharedMemoryAndSemaphores(port);
}
@@ -4877,7 +4877,7 @@ SubPostmasterMain(int argc, char *argv[])
InitProcess();
/* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores(false, 0);
+ CreateSharedMemoryAndSemaphores(0);
/* And run the backend */
BackendRun(&port); /* does not return */
@@ -4891,7 +4891,7 @@ SubPostmasterMain(int argc, char *argv[])
InitAuxiliaryProcess();
/* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores(false, 0);
+ CreateSharedMemoryAndSemaphores(0);
AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
}
@@ -4904,7 +4904,7 @@ SubPostmasterMain(int argc, char *argv[])
InitProcess();
/* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores(false, 0);
+ CreateSharedMemoryAndSemaphores(0);
AutoVacLauncherMain(argc - 2, argv + 2); /* does not return */
}
@@ -4917,7 +4917,7 @@ SubPostmasterMain(int argc, char *argv[])
InitProcess();
/* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores(false, 0);
+ CreateSharedMemoryAndSemaphores(0);
AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
}
@@ -4935,7 +4935,7 @@ SubPostmasterMain(int argc, char *argv[])
InitProcess();
/* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores(false, 0);
+ CreateSharedMemoryAndSemaphores(0);
/* Fetch MyBgworkerEntry from shared memory */
shmem_slot = atoi(argv[1] + 15);
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 0c86a58..091244d 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -88,12 +88,9 @@ RequestAddinShmemSpace(Size size)
* through the same code as before. (Note that the called routines mostly
* check IsUnderPostmaster, rather than EXEC_BACKEND, to detect this case.
* This is a bit code-wasteful and could be cleaned up.)
- *
- * If "makePrivate" is true then we only need private memory, not shared
- * memory. This is true for a standalone backend, false for a postmaster.
*/
void
-CreateSharedMemoryAndSemaphores(bool makePrivate, int port)
+CreateSharedMemoryAndSemaphores(int port)
{
PGShmemHeader *shim = NULL;
@@ -166,7 +163,7 @@ CreateSharedMemoryAndSemaphores(bool makePrivate, int port)
/*
* Create the shmem segment
*/
- seghdr = PGSharedMemoryCreate(size, makePrivate, port, &shim);
+ seghdr = PGSharedMemoryCreate(size, port, &shim);
InitShmemAccess(seghdr);
@@ -187,12 +184,9 @@ CreateSharedMemoryAndSemaphores(bool makePrivate, int port)
{
/*
* We are reattaching to an existing shared memory segment. This
- * should only be reached in the EXEC_BACKEND case, and even then only
- * with makePrivate == false.
+ * should only be reached in the EXEC_BACKEND case.
*/
-#ifdef EXEC_BACKEND
- Assert(!makePrivate);
-#else
+#ifndef EXEC_BACKEND
elog(PANIC, "should be attached to shared memory already");
#endif
}
diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c
index 865119d..dee9279 100644
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index 5ef6315..3f3c3c8 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -422,9 +422,11 @@ InitCommunication(void)
{
/*
* We're running a postgres bootstrap process or a standalone backend.
- * Create private "shmem" and semaphores.
+ * Though we won't listen on PostPortNumber, use it to select a shmem
+ * key. This increases the chance of detecting a leftover live
+ * backend of this DataDir.
*/
- CreateSharedMemoryAndSemaphores(true, 0);
+ CreateSharedMemoryAndSemaphores(PostPortNumber);
}
}
diff --git a/src/include/storage/ipc.h b/src/include/storage/ipc.h
index 6a05a89..02f4b1b 100644
--- a/src/include/storage/ipc.h
+++ b/src/include/storage/ipc.h
@@ -76,6 +76,6 @@ extern void on_exit_reset(void);
/* ipci.c */
extern PGDLLIMPORT shmem_startup_hook_type shmem_startup_hook;
-extern void CreateSharedMemoryAndSemaphores(bool makePrivate, int port);
+extern void CreateSharedMemoryAndSemaphores(int port);
#endif /* IPC_H */
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 6b1e040..0a65196 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -30,7 +30,7 @@ typedef struct PGShmemHeader /* standard header for all Postgres shmem */
{
int32 magic; /* magic # to identify Postgres segments */
#define PGShmemMagic 679834894
- pid_t creatorPID; /* PID of creating process */
+ pid_t creatorPID; /* PID of creating process (set but unread) */
Size totalsize; /* total size of segment */
Size freeoffset; /* offset to first free space */
dsm_handle dsm_control; /* ID of dynamic shared memory control seg */
@@ -64,8 +64,8 @@ extern void PGSharedMemoryReAttach(void);
extern void PGSharedMemoryNoReAttach(void);
#endif
-extern PGShmemHeader *PGSharedMemoryCreate(Size size, bool makePrivate,
- int port, PGShmemHeader **shim);
+extern PGShmemHeader *PGSharedMemoryCreate(Size size, int port,
+ PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 79fb457..e1e9931 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -104,7 +104,8 @@ our @EXPORT = qw(
get_new_node
);
-our ($test_localhost, $test_pghost, $last_port_assigned, @all_nodes, $died);
+our ($use_tcp, $test_localhost, $test_pghost, $last_host_assigned,
+ $last_port_assigned, @all_nodes, $died);
# Windows path to virtual file system root
@@ -118,13 +119,14 @@ if ($Config{osname} eq 'msys')
INIT
{
- # PGHOST is set once and for all through a single series of tests when
- # this module is loaded.
- $test_localhost = "127.0.0.1";
- $test_pghost =
- $TestLib::windows_os ? $test_localhost : TestLib::tempdir_short;
- $ENV{PGHOST} = $test_pghost;
- $ENV{PGDATABASE} = 'postgres';
+ # Set PGHOST for backward compatibility. This doesn't work for own_host
+ # nodes, so prefer to not rely on this when writing new tests.
+ $use_tcp = $TestLib::windows_os;
+ $test_localhost = "127.0.0.1";
+ $last_host_assigned = 1;
+ $test_pghost = $use_tcp ? $test_localhost : TestLib::tempdir_short;
+ $ENV{PGHOST} = $test_pghost;
+ $ENV{PGDATABASE} = 'postgres';
# Tracking of last port value assigned to accelerate free port lookup.
$last_port_assigned = int(rand() * 16384) + 49152;
@@ -155,7 +157,9 @@ sub new
_host => $pghost,
_basedir => "$TestLib::tmp_check/t_${testname}_${name}_data",
_name => $name,
- _logfile => "$TestLib::log_path/${testname}_${name}.log"
+ _logfile_generation => 0,
+ _logfile_base => "$TestLib::log_path/${testname}_${name}",
+ _logfile => "$TestLib::log_path/${testname}_${name}.log"
};
bless $self, $class;
@@ -473,8 +477,9 @@ sub init
print $conf "max_wal_senders = 0\n";
}
- if ($TestLib::windows_os)
+ if ($use_tcp)
{
+ print $conf "unix_socket_directories = ''\n";
print $conf "listen_addresses = '$host'\n";
}
else
@@ -536,12 +541,11 @@ sub backup
{
my ($self, $backup_name) = @_;
my $backup_path = $self->backup_dir . '/' . $backup_name;
- my $port = $self->port;
my $name = $self->name;
print "# Taking pg_basebackup $backup_name from node \"$name\"\n";
- TestLib::system_or_bail('pg_basebackup', '-D', $backup_path, '-p', $port,
- '--no-sync');
+ TestLib::system_or_bail('pg_basebackup', '-D', $backup_path, '-h',
+ $self->host, '-p', $self->port, '--no-sync');
print "# Backup finished\n";
return;
}
@@ -653,6 +657,7 @@ sub init_from_backup
{
my ($self, $root_node, $backup_name, %params) = @_;
my $backup_path = $root_node->backup_dir . '/' . $backup_name;
+ my $host = $self->host;
my $port = $self->port;
my $node_name = $self->name;
my $root_name = $root_node->name;
@@ -679,6 +684,15 @@ sub init_from_backup
qq(
port = $port
));
+ if ($use_tcp)
+ {
+ $self->append_conf('postgresql.conf', "listen_addresses = '$host'");
+ }
+ else
+ {
+ $self->append_conf('postgresql.conf',
+ "unix_socket_directories = '$host'");
+ }
$self->enable_streaming($root_node) if $params{has_streaming};
$self->enable_restoring($root_node) if $params{has_restoring};
return;
@@ -686,17 +700,45 @@ port = $port
=pod
-=item $node->start()
+=item $node->rotate_logfile()
+
+Switch to a new PostgreSQL log file. This does not alter any running
+PostgreSQL process. Subsequent method calls, including pg_ctl invocations,
+will use the new name. Return the new name.
+
+=cut
+
+sub rotate_logfile
+{
+ my ($self) = @_;
+ $self->{_logfile} = sprintf('%s_%d.log',
+ $self->{_logfile_base},
+ ++$self->{_logfile_generation});
+ return $self->{_logfile};
+}
+
+=pod
+
+=item $node->start(%params) => success_or_failure
Wrapper for pg_ctl start
Start the node and wait until it is ready to accept connections.
+=over
+
+=item fail_ok => 1
+
+By default, failure terminates the entire F<prove> invocation. If given,
+instead return a true or false value to indicate success or failure.
+
+=back
+
=cut
sub start
{
- my ($self) = @_;
+ my ($self, %params) = @_;
my $port = $self->port;
my $pgdata = $self->data_dir;
my $name = $self->name;
@@ -709,10 +751,34 @@ sub start
{
print "# pg_ctl start failed; logfile:\n";
print TestLib::slurp_file($self->logfile);
- BAIL_OUT("pg_ctl start failed");
+ BAIL_OUT("pg_ctl start failed") unless $params{fail_ok};
+ return 0;
}
$self->_update_pid(1);
+ return 1;
+}
+
+=pod
+
+=item $node->kill9()
+
+Send SIGKILL (signal 9) to the postmaster.
+
+Note: if the node is already known stopped, this does nothing.
+However, if we think it's running and it's not, it's important for
+this to fail. Otherwise, tests might fail to detect server crashes.
+
+=cut
+
+sub kill9
+{
+ my ($self) = @_;
+ my $name = $self->name;
+ return unless defined $self->{_pid};
+ print "### Killing node \"$name\" using signal 9\n";
+ kill(9, $self->{_pid}) or BAIL_OUT("kill(9, $self->{_pid}) failed");
+ $self->{_pid} = undef;
return;
}
@@ -908,7 +974,7 @@ sub _update_pid
=pod
-=item PostgresNode->get_new_node(node_name)
+=item PostgresNode->get_new_node(node_name, %params)
Build a new object of class C<PostgresNode> (or of a subclass, if you have
one), assigning a free port number. Remembers the node, to prevent its port
@@ -917,6 +983,22 @@ shut down when the test script exits.
You should generally use this instead of C<PostgresNode::new(...)>.
+=over
+
+=item port => [1,65535]
+
+By default, this function assigns a port number to each node. Specify this to
+force a particular port number. The caller is responsible for evaluating
+potential conflicts and privilege requirements.
+
+=item own_host => 1
+
+By default, all nodes use the same PGHOST value. If specified, generate a
+PGHOST specific to this node. This allows multiple nodes to use the same
+port.
+
+=back
+
For backwards compatibility, it is also exported as a standalone function,
which can only create objects of class C<PostgresNode>.
@@ -925,10 +1007,11 @@ which can only create objects of class C<PostgresNode>.
sub get_new_node
{
my $class = 'PostgresNode';
- $class = shift if 1 < scalar @_;
- my $name = shift;
- my $found = 0;
- my $port = $last_port_assigned;
+ $class = shift if scalar(@_) % 2 != 1;
+ my ($name, %params) = @_;
+ my $port_is_forced = defined $params{port};
+ my $found = $port_is_forced;
+ my $port = $port_is_forced ? $params{port} : $last_port_assigned;
while ($found == 0)
{
@@ -945,13 +1028,15 @@ sub get_new_node
$found = 0 if ($node->port == $port);
}
- # Check to see if anything else is listening on this TCP port.
- # This is *necessary* on Windows, and seems like a good idea
- # on Unixen as well, even though we don't ask the postmaster
- # to open a TCP port on Unix.
+ # Check to see if anything else is listening on this TCP port. Accept
+ # only ports available for all possible listen_addresses values, so
+ # the caller can harness this port for the widest range of purposes.
+ # This is *necessary* on Windows, and seems like a good idea on Unixen
+ # as well, even though we don't ask the postmaster to open a TCP port
+ # on Unix.
if ($found == 1)
{
- my $iaddr = inet_aton($test_localhost);
+ my $iaddr = inet_aton('0.0.0.0');
my $paddr = sockaddr_in($port, $iaddr);
my $proto = getprotobyname("tcp");
@@ -967,16 +1052,35 @@ sub get_new_node
}
}
- print "# Found free port $port\n";
+ print "# Found port $port\n";
+
+ # Select a host.
+ my $host = $test_pghost;
+ if ($params{own_host})
+ {
+ if ($use_tcp)
+ {
+ # This assumes $use_tcp platforms treat every address in
+ # 127.0.0.1/24, not just 127.0.0.1, as a usable loopback.
+ $last_host_assigned++;
+ $last_host_assigned > 254 and BAIL_OUT("too many own_host nodes");
+ $host = '127.0.0.' . $last_host_assigned;
+ }
+ else
+ {
+ $host = "$test_pghost/$name"; # Assume $name =~ /^[-_a-zA-Z0-9]+$/
+ mkdir $host;
+ }
+ }
# Lock port number found by creating a new node
- my $node = $class->new($name, $test_pghost, $port);
+ my $node = $class->new($name, $host, $port);
# Add node to list of nodes
push(@all_nodes, $node);
# And update port for next time
- $last_port_assigned = $port;
+ $port_is_forced or $last_port_assigned = $port;
return $node;
}
@@ -1358,9 +1462,8 @@ sub poll_query_until
=item $node->command_ok(...)
-Runs a shell command like TestLib::command_ok, but with PGPORT
-set so that the command will default to connecting to this
-PostgresNode.
+Runs a shell command like TestLib::command_ok, but with PGHOST and PGPORT set
+so that the command will default to connecting to this PostgresNode.
=cut
@@ -1370,6 +1473,7 @@ sub command_ok
my $self = shift;
+ local $ENV{PGHOST} = $self->host;
local $ENV{PGPORT} = $self->port;
TestLib::command_ok(@_);
@@ -1380,7 +1484,7 @@ sub command_ok
=item $node->command_fails(...)
-TestLib::command_fails with our PGPORT. See command_ok(...)
+TestLib::command_fails with our connection parameters. See command_ok(...)
=cut
@@ -1390,6 +1494,7 @@ sub command_fails
my $self = shift;
+ local $ENV{PGHOST} = $self->host;
local $ENV{PGPORT} = $self->port;
TestLib::command_fails(@_);
@@ -1400,7 +1505,7 @@ sub command_fails
=item $node->command_like(...)
-TestLib::command_like with our PGPORT. See command_ok(...)
+TestLib::command_like with our connection parameters. See command_ok(...)
=cut
@@ -1410,6 +1515,7 @@ sub command_like
my $self = shift;
+ local $ENV{PGHOST} = $self->host;
local $ENV{PGPORT} = $self->port;
TestLib::command_like(@_);
@@ -1420,7 +1526,8 @@ sub command_like
=item $node->command_checks_all(...)
-TestLib::command_checks_all with our PGPORT. See command_ok(...)
+TestLib::command_checks_all with our connection parameters. See
+command_ok(...)
=cut
@@ -1430,6 +1537,7 @@ sub command_checks_all
my $self = shift;
+ local $ENV{PGHOST} = $self->host;
local $ENV{PGPORT} = $self->port;
TestLib::command_checks_all(@_);
@@ -1454,6 +1562,7 @@ sub issues_sql_like
my ($self, $cmd, $expected_sql, $test_name) = @_;
+ local $ENV{PGHOST} = $self->host;
local $ENV{PGPORT} = $self->port;
truncate $self->logfile, 0;
@@ -1468,8 +1577,8 @@ sub issues_sql_like
=item $node->run_log(...)
-Runs a shell command like TestLib::run_log, but with PGPORT set so
-that the command will default to connecting to this PostgresNode.
+Runs a shell command like TestLib::run_log, but with connection parameters set
+so that the command will default to connecting to this PostgresNode.
=cut
@@ -1477,6 +1586,7 @@ sub run_log
{
my $self = shift;
+ local $ENV{PGHOST} = $self->host;
local $ENV{PGPORT} = $self->port;
TestLib::run_log(@_);
diff --git a/src/test/recovery/t/016_shm.pl b/src/test/recovery/t/016_shm.pl
new file mode 100644
index 0000000..58e56f4
--- /dev/null
+++ b/src/test/recovery/t/016_shm.pl
@@ -0,0 +1,146 @@
+#
+# Tests of pg_shmem.h functions
+#
+use strict;
+use warnings;
+use IPC::Run 'run';
+use PostgresNode;
+use Test::More;
+use TestLib;
+
+plan tests => 6;
+
+my $tempdir = TestLib::tempdir;
+my $port;
+
+# When using Unix sockets, we can dictate the port number. In the absence of
+# collisions from other shmget() activity, gnat starts with key 0x7d001
+# (512001), and flea starts with key 0x7d002 (512002).
+$port = 512 unless $PostgresNode::use_tcp;
+
+# Log "ipcs" diffs on a best-effort basis, swallowing any error.
+my $ipcs_before = "$tempdir/ipcs_before";
+eval { run_log [ 'ipcs', '-am' ], '>', $ipcs_before; };
+
+sub log_ipcs
+{
+ eval { run_log [ 'ipcs', '-am' ], '|', [ 'diff', $ipcs_before, '-' ] };
+ return;
+}
+
+# Node setup.
+sub init_start
+{
+ my $name = shift;
+ my $ret = PostgresNode->get_new_node($name, port => $port, own_host => 1);
+ defined($port) or $port = $ret->port; # same port for all nodes
+ $ret->init;
+ $ret->start;
+ log_ipcs();
+ return $ret;
+}
+my $gnat = init_start 'gnat';
+my $flea = init_start 'flea';
+
+# Upon postmaster death, postmaster children exit automatically.
+$gnat->kill9;
+log_ipcs();
+$flea->restart; # flea ignores the shm key gnat abandoned.
+log_ipcs();
+poll_start($gnat); # gnat recycles its former shm key.
+log_ipcs();
+
+# After clean shutdown, the nodes swap shm keys.
+$gnat->stop;
+$flea->restart;
+log_ipcs();
+$gnat->start;
+log_ipcs();
+
+# Scenarios involving no postmaster.pid, dead postmaster, and a live backend.
+my $sleep_query = 'SELECT pg_sleep(3600)';
+my ($stdout, $stderr);
+my $asleep = IPC::Run::start(
+ [
+ 'psql', '-X', '-qAt', '-d', $gnat->connstr('postgres'),
+ '-c', $sleep_query
+ ],
+ '<',
+ \undef,
+ '>',
+ \$stdout,
+ '2>',
+ \$stderr,
+ IPC::Run::timeout(900)); # five times the poll_query_until timeout
+ok( $gnat->poll_query_until(
+ 'postgres',
+ "SELECT 1 FROM pg_stat_activity WHERE query = '$sleep_query'", '1'),
+ 'pg_sleep() started');
+my $asleep_backend_pid = $gnat->safe_psql('postgres',
+ "SELECT pid FROM pg_stat_activity WHERE query = '$sleep_query'");
+$gnat->kill9;
+unlink($gnat->data_dir . '/postmaster.pid');
+$gnat->rotate_logfile; # on Windows, can't open old log for writing
+log_ipcs();
+# Reject ordinary startup.
+ok(!$gnat->start(fail_ok => 1), 'live pg_sleep() blocks restart');
+like(
+ slurp_file($gnat->logfile),
+ qr/pre-existing shared memory block/,
+ 'detected live backend via shared memory');
+# Reject single-user startup.
+my $single_stderr;
+ok( !run_log(
+ [ 'postgres', '--single', '-D', $gnat->data_dir, 'template1' ],
+ '<', \('SELECT 1 + 1'), '2>', \$single_stderr),
+ 'live pg_sleep() blocks --single');
+print STDERR $single_stderr;
+like(
+ $single_stderr,
+ qr/pre-existing shared memory block/,
+ 'single-user mode detected live backend via shared memory');
+log_ipcs();
+# Fail to reject startup if shm key N has become available and we crash while
+# using key N+1. This is unwanted, but expected. Windows is immune, because
+# its GetSharedMemName() use DataDir strings, not numeric keys.
+$flea->stop; # release first key
+is( $gnat->start(fail_ok => 1),
+ $TestLib::windows_os ? 0 : 1,
+ 'key turnover fools only sysv_shmem.c');
+$gnat->stop; # release first key (no-op on $TestLib::windows_os)
+$flea->start; # grab first key
+# cleanup
+TestLib::system_log('pg_ctl', 'kill', 'QUIT', $asleep_backend_pid);
+$asleep->finish; # client has detected backend termination
+log_ipcs();
+poll_start($gnat); # recycle second key
+
+$gnat->stop;
+$flea->stop;
+log_ipcs();
+
+
+# When postmaster children are slow to exit after postmaster death, we may
+# need retries to start a new postmaster.
+sub poll_start
+{
+ my ($node) = @_;
+
+ my $max_attempts = 180 * 10;
+ my $attempts = 0;
+
+ while ($attempts < $max_attempts)
+ {
+ $node->start(fail_ok => 1) && return 1;
+
+ # Wait 0.1 second before retrying.
+ usleep(100_000);
+
+ $attempts++;
+ }
+
+ # No success within 180 seconds. Try one last time without fail_ok, which
+ # will BAIL_OUT unless it succeeds.
+ $node->start && return 1;
+ return 0;
+}
On Sun, Aug 12, 2018 at 8:48 AM Noah Misch <noah@leadboat.com> wrote:
On Wed, Sep 11, 2013 at 10:28:00PM -0400, Noah Misch wrote:
On Wed, Sep 11, 2013 at 11:32:01AM -0400, Stephen Frost wrote:
* Noah Misch (noah@leadboat.com) wrote:
Concretely, that means
not removing postmaster.pid on immediate shutdown in 9.3 and earlier. That's
consistent with the rough nature of an immediate shutdown, anyway.With 9.3 having a few months left, that's less interesting, but ...
Thanks for the patch!
I'm a bit out of context, but taking into account that 9.3 is already beyond
EOL, is it still interesting? I'm just thinking about whether it's worth to
move it to the next CF, or mark as rejected, since no one reviewed the patch so
far?
As a side note, with this patch recovery tests are failing now on 016_shm.pl
# Failed test 'detected live backend via shared memory'
# at t/016_shm.pl line 87.
# '2018-11-28 13:08:08.504 UTC [21924] LOG:
listening on Unix socket "/tmp/yV2oDNcG8e/gnat/.s.PGSQL.512"
# 2018-11-28 13:08:08.512 UTC [21925] LOG: database system was
interrupted; last known up at 2018-11-28 13:08:08 UTC
# 2018-11-28 13:08:08.512 UTC [21925] LOG: database system was not
properly shut down; automatic recovery in progress
# 2018-11-28 13:08:08.512 UTC [21925] LOG: invalid record length at
0/165FEF8: wanted 24, got 0
# 2018-11-28 13:08:08.512 UTC [21925] LOG: redo is not required
# 2018-11-28 13:08:08.516 UTC [21924] LOG: database system is ready
to accept connections
# '
# doesn't match '(?^:pre-existing shared memory block)'
On Thu, Nov 29, 2018 at 01:10:57PM +0100, Dmitry Dolgov wrote:
On Sun, Aug 12, 2018 at 8:48 AM Noah Misch <noah@leadboat.com> wrote:
With 9.3 having a few months left, that's less interesting, but ...
I'm a bit out of context, but taking into account that 9.3 is already beyond
EOL, is it still interesting?
Yes.
As a side note, with this patch recovery tests are failing now on 016_shm.pl
# Failed test 'detected live backend via shared memory'
# at t/016_shm.pl line 87.
# '2018-11-28 13:08:08.504 UTC [21924] LOG:
listening on Unix socket "/tmp/yV2oDNcG8e/gnat/.s.PGSQL.512"
# 2018-11-28 13:08:08.512 UTC [21925] LOG: database system was
interrupted; last known up at 2018-11-28 13:08:08 UTC
# 2018-11-28 13:08:08.512 UTC [21925] LOG: database system was not
properly shut down; automatic recovery in progress
# 2018-11-28 13:08:08.512 UTC [21925] LOG: invalid record length at
0/165FEF8: wanted 24, got 0
# 2018-11-28 13:08:08.512 UTC [21925] LOG: redo is not required
# 2018-11-28 13:08:08.516 UTC [21924] LOG: database system is ready
to accept connections
# '
# doesn't match '(?^:pre-existing shared memory block)'
Thanks for the report. Since commit cfdf4dc made pg_sleep() react to
postmaster death, the test will need a different way to stall a backend. This
doesn't affect non-test code, and the test still passes against cfdf4dc^ and
against REL_11_STABLE. I've queued a task to update the test code, but review
can proceed in parallel.
On Thu, Nov 29, 2018 at 10:51:40PM -0800, Noah Misch wrote:
On Thu, Nov 29, 2018 at 01:10:57PM +0100, Dmitry Dolgov wrote:
As a side note, with this patch recovery tests are failing now on 016_shm.pl
# Failed test 'detected live backend via shared memory'
# at t/016_shm.pl line 87.
# '2018-11-28 13:08:08.504 UTC [21924] LOG:
listening on Unix socket "/tmp/yV2oDNcG8e/gnat/.s.PGSQL.512"
# 2018-11-28 13:08:08.512 UTC [21925] LOG: database system was
interrupted; last known up at 2018-11-28 13:08:08 UTC
# 2018-11-28 13:08:08.512 UTC [21925] LOG: database system was not
properly shut down; automatic recovery in progress
# 2018-11-28 13:08:08.512 UTC [21925] LOG: invalid record length at
0/165FEF8: wanted 24, got 0
# 2018-11-28 13:08:08.512 UTC [21925] LOG: redo is not required
# 2018-11-28 13:08:08.516 UTC [21924] LOG: database system is ready
to accept connections
# '
# doesn't match '(?^:pre-existing shared memory block)'Thanks for the report. Since commit cfdf4dc made pg_sleep() react to
postmaster death, the test will need a different way to stall a backend. This
doesn't affect non-test code, and the test still passes against cfdf4dc^ and
against REL_11_STABLE. I've queued a task to update the test code, but review
can proceed in parallel.
Here's a version fixing that test for post-cfdf4dc backends.
Attachments:
ipcrm-hint-v1.patchtext/plain; charset=us-asciiDownload
--- a/src/backend/utils/init/miscinit.c
+++ b/src/backend/utils/init/miscinit.c
@@ -1070,14 +1070,10 @@ CreateLockFile(const char *filename, bool amPostmaster,
if (PGSharedMemoryIsInUse(id1, id2))
ereport(FATAL,
(errcode(ERRCODE_LOCK_FILE_EXISTS),
- errmsg("pre-existing shared memory block "
- "(key %lu, ID %lu) is still in use",
+ errmsg("pre-existing shared memory block (key %lu, ID %lu) is still in use",
id1, id2),
- errhint("If you're sure there are no old "
- "server processes still running, remove "
- "the shared memory block "
- "or just delete the file \"%s\".",
- filename)));
+ errhint("Terminate any old server processes associated with data directory \"%s\".",
+ refName)));
}
}
PGSharedMemoryCreate-safety-v2.patchtext/plain; charset=us-asciiDownload
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 956fd27..a67455b 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -421,13 +421,13 @@ ifeq ($(enable_tap_tests),yes)
define prove_installcheck
rm -rf '$(CURDIR)'/tmp_check
$(MKDIR_P) '$(CURDIR)'/tmp_check
-cd $(srcdir) && TESTDIR='$(CURDIR)' PATH="$(bindir):$$PATH" PGPORT='6$(DEF_PGPORT)' top_builddir='$(CURDIR)/$(top_builddir)' PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
+cd $(srcdir) && TESTDIR='$(CURDIR)' PATH="$(bindir):$$PATH" PGPORT='6$(DEF_PGPORT)' top_builddir='$(CURDIR)/$(top_builddir)' PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' REGRESS_SHLIB='$(abs_top_builddir)/src/test/regress/regress$(DLSUFFIX)' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
endef
define prove_check
rm -rf '$(CURDIR)'/tmp_check
$(MKDIR_P) '$(CURDIR)'/tmp_check
-cd $(srcdir) && TESTDIR='$(CURDIR)' $(with_temp_install) PGPORT='6$(DEF_PGPORT)' PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
+cd $(srcdir) && TESTDIR='$(CURDIR)' $(with_temp_install) PGPORT='6$(DEF_PGPORT)' PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' REGRESS_SHLIB='$(abs_top_builddir)/src/test/regress/regress$(DLSUFFIX)' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
endef
else
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 741c455..7a9a68e 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -71,6 +71,26 @@
typedef key_t IpcMemoryKey; /* shared memory key passed to shmget(2) */
typedef int IpcMemoryId; /* shared memory ID returned by shmget(2) */
+/*
+ * How does a given IpcMemoryId relate to this PostgreSQL process?
+ *
+ * One could recycle unattached segments of different data directories if we
+ * distinguished that case from other SHMSTATE_FOREIGN cases. Doing so would
+ * cause us to visit less of the key space, making us less likely to detect a
+ * SHMSTATE_ATTACHED key. It would also complicate the concurrency analysis,
+ * in that postmasters of different data directories could simultaneously
+ * attempt to recycle a given key. We'll waste keys longer in some cases, but
+ * avoiding the problems of the alternative justifies that loss.
+ */
+enum IpcMemoryState
+{
+ SHMSTATE_ANALYSIS_FAILURE, /* unexpected failure to analyze the ID */
+ SHMSTATE_ATTACHED, /* pertinent to DataDir, has attached PIDs */
+ SHMSTATE_EEXISTS, /* no segment of that ID */
+ SHMSTATE_FOREIGN, /* exists, but not pertinent to DataDir */
+ SHMSTATE_UNATTACHED /* pertinent to DataDir, no attached PIDs */
+};
+
unsigned long UsedShmemSegID = 0;
void *UsedShmemSegAddr = NULL;
@@ -83,6 +103,7 @@ static void *AnonymousShmem = NULL;
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
+static enum IpcMemoryState IpcMemoryAnalyze(IpcMemoryId shmId);
static PGShmemHeader *PGSharedMemoryAttach(IpcMemoryKey key,
IpcMemoryId *shmid);
@@ -288,7 +309,23 @@ IpcMemoryDelete(int status, Datum shmId)
bool
PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2)
{
- IpcMemoryId shmId = (IpcMemoryId) id2;
+ switch (IpcMemoryAnalyze((IpcMemoryId) id2))
+ {
+ case SHMSTATE_EEXISTS:
+ case SHMSTATE_FOREIGN:
+ case SHMSTATE_UNATTACHED:
+ return false;
+ case SHMSTATE_ANALYSIS_FAILURE:
+ case SHMSTATE_ATTACHED:
+ return true;
+ }
+ return true;
+}
+
+/* See comment at enum IpcMemoryState. */
+static enum IpcMemoryState
+IpcMemoryAnalyze(IpcMemoryId shmId)
+{
struct shmid_ds shmStat;
struct stat statbuf;
PGShmemHeader *hdr;
@@ -305,7 +342,7 @@ PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2)
* exists.
*/
if (errno == EINVAL)
- return false;
+ return SHMSTATE_EEXISTS;
/*
* EACCES implies that the segment belongs to some other userid, which
@@ -313,7 +350,7 @@ PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2)
* is relevant to our data directory).
*/
if (errno == EACCES)
- return false;
+ return SHMSTATE_FOREIGN;
/*
* Some Linux kernel versions (in fact, all of them as of July 2007)
@@ -324,7 +361,7 @@ PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2)
*/
#ifdef HAVE_LINUX_EIDRM_BUG
if (errno == EIDRM)
- return false;
+ return SHMSTATE_EEXISTS;
#endif
/*
@@ -332,25 +369,26 @@ PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2)
* only likely case is EIDRM, which implies that the segment has been
* IPC_RMID'd but there are still processes attached to it.
*/
- return true;
+ return SHMSTATE_ANALYSIS_FAILURE;
}
- /* If it has no attached processes, it's not in use */
- if (shmStat.shm_nattch == 0)
- return false;
-
/*
* Try to attach to the segment and see if it matches our data directory.
* This avoids shmid-conflict problems on machines that are running
* several postmasters under the same userid.
*/
if (stat(DataDir, &statbuf) < 0)
- return true; /* if can't stat, be conservative */
+ return SHMSTATE_ANALYSIS_FAILURE; /* can't stat; be conservative */
+ /*
+ * If we can't attach, be conservative. This may fail if postmaster.pid
+ * furnished the shmId and another user created a world-readable segment
+ * of the same shmId. It shouldn't fail if PGSharedMemoryAttach()
+ * furnished the shmId, because that function tests shmat().
+ */
hdr = (PGShmemHeader *) shmat(shmId, NULL, PG_SHMAT_FLAGS);
-
if (hdr == (PGShmemHeader *) -1)
- return true; /* if can't attach, be conservative */
+ return SHMSTATE_ANALYSIS_FAILURE;
if (hdr->magic != PGShmemMagic ||
hdr->device != statbuf.st_dev ||
@@ -358,16 +396,15 @@ PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2)
{
/*
* It's either not a Postgres segment, or not one for my data
- * directory. In either case it poses no threat.
+ * directory.
*/
shmdt((void *) hdr);
- return false;
+ return SHMSTATE_FOREIGN;
}
- /* Trouble --- looks a lot like there's still live backends */
shmdt((void *) hdr);
- return true;
+ return shmStat.shm_nattch == 0 ? SHMSTATE_UNATTACHED : SHMSTATE_ATTACHED;
}
#ifdef USE_ANONYMOUS_SHMEM
@@ -543,19 +580,16 @@ AnonymousShmemDetach(int status, Datum arg)
* standard header. Also, register an on_shmem_exit callback to release
* the storage.
*
- * Dead Postgres segments are recycled if found, but we do not fail upon
- * collision with non-Postgres shmem segments. The idea here is to detect and
- * re-use keys that may have been assigned by a crashed postmaster or backend.
- *
- * makePrivate means to always create a new segment, rather than attach to
- * or recycle any existing segment.
+ * Dead Postgres segments pertinent to this DataDir are recycled if found, but
+ * we do not fail upon collision with foreign shmem segments. The idea here
+ * is to detect and re-use keys that may have been assigned by a crashed
+ * postmaster or backend.
*
* The port number is passed for possible use as a key (for SysV, we use
- * it to generate the starting shmem key). In a standalone backend,
- * zero will be passed.
+ * it to generate the starting shmem key).
*/
PGShmemHeader *
-PGSharedMemoryCreate(Size size, bool makePrivate, int port,
+PGSharedMemoryCreate(Size size, int port,
PGShmemHeader **shim)
{
IpcMemoryKey NextShmemSegID;
@@ -592,11 +626,18 @@ PGSharedMemoryCreate(Size size, bool makePrivate, int port,
/* Make sure PGSharedMemoryAttach doesn't fail without need */
UsedShmemSegAddr = NULL;
- /* Loop till we find a free IPC key */
- NextShmemSegID = port * 1000;
+ /*
+ * Loop till we find a free IPC key. Trust CreateDataDirLockFile() to
+ * ensure no more than one postmaster per data directory can enter this
+ * loop simultaneously. (CreateDataDirLockFile() does not ensure that,
+ * but prefer fixing it over coping here.)
+ */
+ NextShmemSegID = 1 + port * 1000;
- for (NextShmemSegID++;; NextShmemSegID++)
+ for (;;)
{
+ dsm_handle hdr_dsm;
+
/* Try to create new segment */
memAddress = InternalIpcMemoryCreate(NextShmemSegID, sysvsize);
if (memAddress)
@@ -604,58 +645,62 @@ PGSharedMemoryCreate(Size size, bool makePrivate, int port,
/* Check shared memory and possibly remove and recreate */
- if (makePrivate) /* a standalone backend shouldn't do this */
- continue;
-
if ((memAddress = PGSharedMemoryAttach(NextShmemSegID, &shmid)) == NULL)
- continue; /* can't attach, not one of mine */
-
- /*
- * If I am not the creator and it belongs to an extant process,
- * continue.
- */
- hdr = (PGShmemHeader *) memAddress;
- if (hdr->creatorPID != getpid())
{
- if (kill(hdr->creatorPID, 0) == 0 || errno != ESRCH)
- {
- shmdt(memAddress);
- continue; /* segment belongs to a live process */
- }
+ NextShmemSegID++;
+ continue;
}
-
- /*
- * The segment appears to be from a dead Postgres process, or from a
- * previous cycle of life in this same process. Zap it, if possible,
- * and any associated dynamic shared memory segments, as well. This
- * probably shouldn't fail, but if it does, assume the segment belongs
- * to someone else after all, and continue quietly.
- */
- if (hdr->dsm_control != 0)
- dsm_cleanup_using_control_segment(hdr->dsm_control);
+ hdr_dsm = ((PGShmemHeader *) memAddress)->dsm_control;
shmdt(memAddress);
- if (shmctl(shmid, IPC_RMID, NULL) < 0)
- continue;
+ memAddress = NULL;
- /*
- * Now try again to create the segment.
- */
- memAddress = InternalIpcMemoryCreate(NextShmemSegID, sysvsize);
- if (memAddress)
- break; /* successful create and attach */
+ switch (IpcMemoryAnalyze(shmid))
+ {
+ case SHMSTATE_ANALYSIS_FAILURE:
+ case SHMSTATE_ATTACHED:
+ ereport(FATAL,
+ (errcode(ERRCODE_LOCK_FILE_EXISTS),
+ errmsg("pre-existing shared memory block (key %lu, ID %lu) is still in use",
+ (unsigned long) NextShmemSegID,
+ (unsigned long) shmid),
+ errhint("Terminate any old server processes associated with data directory \"%s\".",
+ DataDir)));
+ case SHMSTATE_EEXISTS:
- /*
- * Can only get here if some other process managed to create the same
- * shmem key before we did. Let him have that one, loop around to try
- * next key.
- */
+ /*
+ * To our surprise, some other process deleted the segment
+ * between InternalIpcMemoryCreate() and IpcMemoryAnalyze().
+ * Moments earlier, we would have seen SHMSTATE_FOREIGN. Try
+ * that same ID again.
+ */
+ elog(LOG,
+ "shared memory block (key %lu, ID %lu) deleted during startup",
+ (unsigned long) NextShmemSegID,
+ (unsigned long) shmid);
+ break;
+ case SHMSTATE_FOREIGN:
+ NextShmemSegID++;
+ break;
+ case SHMSTATE_UNATTACHED:
+
+ /*
+ * The segment pertains to DataDir, and every process that had
+ * used it has died or detached. Zap it, if possible, and any
+ * associated dynamic shared memory segments, as well. This
+ * shouldn't fail, but if it does, assume the segment belongs
+ * to someone else after all, and try the next candidate.
+ * Otherwise, try again to create the segment. That may fail
+ * if some other process creates the same shmem key before we
+ * do, in which case we'll try the next key.
+ */
+ if (hdr_dsm != 0)
+ dsm_cleanup_using_control_segment(hdr_dsm);
+ if (shmctl(shmid, IPC_RMID, NULL) < 0)
+ NextShmemSegID++;
+ }
}
- /*
- * OK, we created a new segment. Mark it as created by this process. The
- * order of assignments here is critical so that another Postgres process
- * can't see the header as valid but belonging to an invalid PID!
- */
+ /* Initialize new segment. */
hdr = (PGShmemHeader *) memAddress;
hdr->creatorPID = getpid();
hdr->magic = PGShmemMagic;
@@ -816,7 +861,10 @@ PGSharedMemoryDetach(void)
/*
* Attach to shared memory and make sure it has a Postgres header
*
- * Returns attach address if OK, else NULL
+ * Returns attach address if OK, else NULL. Treat a NULL return value like
+ * SHMSTATE_FOREIGN. (EACCES is common. ENOENT, a narrow possibility,
+ * implies SHMSTATE_EEXISTS, but one can safely treat SHMSTATE_EEXISTS like
+ * SHMSTATE_FOREIGN.)
*/
static PGShmemHeader *
PGSharedMemoryAttach(IpcMemoryKey key, IpcMemoryId *shmid)
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index f8ca52e..3888c26 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -170,14 +170,9 @@ EnableLockPagesPrivilege(int elevel)
*
* Create a shared memory segment of the given size and initialize its
* standard header.
- *
- * makePrivate means to always create a new segment, rather than attach to
- * or recycle any existing segment. On win32, we always create a new segment,
- * since there is no need for recycling (segments go away automatically
- * when the last backend exits)
*/
PGShmemHeader *
-PGSharedMemoryCreate(Size size, bool makePrivate, int port,
+PGSharedMemoryCreate(Size size, int port,
PGShmemHeader **shim)
{
void *memAddress;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index a33a131..3db9f34 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -2565,7 +2565,7 @@ reset_shared(int port)
* determine IPC keys. This helps ensure that we will clean up dead IPC
* objects if the postmaster crashes and is restarted.
*/
- CreateSharedMemoryAndSemaphores(false, port);
+ CreateSharedMemoryAndSemaphores(port);
}
@@ -4914,7 +4914,7 @@ SubPostmasterMain(int argc, char *argv[])
InitProcess();
/* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores(false, 0);
+ CreateSharedMemoryAndSemaphores(0);
/* And run the backend */
BackendRun(&port); /* does not return */
@@ -4928,7 +4928,7 @@ SubPostmasterMain(int argc, char *argv[])
InitAuxiliaryProcess();
/* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores(false, 0);
+ CreateSharedMemoryAndSemaphores(0);
AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
}
@@ -4941,7 +4941,7 @@ SubPostmasterMain(int argc, char *argv[])
InitProcess();
/* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores(false, 0);
+ CreateSharedMemoryAndSemaphores(0);
AutoVacLauncherMain(argc - 2, argv + 2); /* does not return */
}
@@ -4954,7 +4954,7 @@ SubPostmasterMain(int argc, char *argv[])
InitProcess();
/* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores(false, 0);
+ CreateSharedMemoryAndSemaphores(0);
AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
}
@@ -4972,7 +4972,7 @@ SubPostmasterMain(int argc, char *argv[])
InitProcess();
/* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores(false, 0);
+ CreateSharedMemoryAndSemaphores(0);
/* Fetch MyBgworkerEntry from shared memory */
shmem_slot = atoi(argv[1] + 15);
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 0c86a58..091244d 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -88,12 +88,9 @@ RequestAddinShmemSpace(Size size)
* through the same code as before. (Note that the called routines mostly
* check IsUnderPostmaster, rather than EXEC_BACKEND, to detect this case.
* This is a bit code-wasteful and could be cleaned up.)
- *
- * If "makePrivate" is true then we only need private memory, not shared
- * memory. This is true for a standalone backend, false for a postmaster.
*/
void
-CreateSharedMemoryAndSemaphores(bool makePrivate, int port)
+CreateSharedMemoryAndSemaphores(int port)
{
PGShmemHeader *shim = NULL;
@@ -166,7 +163,7 @@ CreateSharedMemoryAndSemaphores(bool makePrivate, int port)
/*
* Create the shmem segment
*/
- seghdr = PGSharedMemoryCreate(size, makePrivate, port, &shim);
+ seghdr = PGSharedMemoryCreate(size, port, &shim);
InitShmemAccess(seghdr);
@@ -187,12 +184,9 @@ CreateSharedMemoryAndSemaphores(bool makePrivate, int port)
{
/*
* We are reattaching to an existing shared memory segment. This
- * should only be reached in the EXEC_BACKEND case, and even then only
- * with makePrivate == false.
+ * should only be reached in the EXEC_BACKEND case.
*/
-#ifdef EXEC_BACKEND
- Assert(!makePrivate);
-#else
+#ifndef EXEC_BACKEND
elog(PANIC, "should be attached to shared memory already");
#endif
}
diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c
index 3d10aa5..9b2ba8c 100644
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index b636b1e..ab31ee3e 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -444,9 +444,11 @@ InitCommunication(void)
{
/*
* We're running a postgres bootstrap process or a standalone backend.
- * Create private "shmem" and semaphores.
+ * Though we won't listen on PostPortNumber, use it to select a shmem
+ * key. This increases the chance of detecting a leftover live
+ * backend of this DataDir.
*/
- CreateSharedMemoryAndSemaphores(true, 0);
+ CreateSharedMemoryAndSemaphores(PostPortNumber);
}
}
diff --git a/src/include/storage/ipc.h b/src/include/storage/ipc.h
index 6a05a89..02f4b1b 100644
--- a/src/include/storage/ipc.h
+++ b/src/include/storage/ipc.h
@@ -76,6 +76,6 @@ extern void on_exit_reset(void);
/* ipci.c */
extern PGDLLIMPORT shmem_startup_hook_type shmem_startup_hook;
-extern void CreateSharedMemoryAndSemaphores(bool makePrivate, int port);
+extern void CreateSharedMemoryAndSemaphores(int port);
#endif /* IPC_H */
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 6b1e040..0a65196 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -30,7 +30,7 @@ typedef struct PGShmemHeader /* standard header for all Postgres shmem */
{
int32 magic; /* magic # to identify Postgres segments */
#define PGShmemMagic 679834894
- pid_t creatorPID; /* PID of creating process */
+ pid_t creatorPID; /* PID of creating process (set but unread) */
Size totalsize; /* total size of segment */
Size freeoffset; /* offset to first free space */
dsm_handle dsm_control; /* ID of dynamic shared memory control seg */
@@ -64,8 +64,8 @@ extern void PGSharedMemoryReAttach(void);
extern void PGSharedMemoryNoReAttach(void);
#endif
-extern PGShmemHeader *PGSharedMemoryCreate(Size size, bool makePrivate,
- int port, PGShmemHeader **shim);
+extern PGShmemHeader *PGSharedMemoryCreate(Size size, int port,
+ PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8a2c6fc..5c5f90f 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -104,7 +104,8 @@ our @EXPORT = qw(
get_new_node
);
-our ($test_localhost, $test_pghost, $last_port_assigned, @all_nodes, $died);
+our ($use_tcp, $test_localhost, $test_pghost, $last_host_assigned,
+ $last_port_assigned, @all_nodes, $died);
# Windows path to virtual file system root
@@ -118,13 +119,14 @@ if ($Config{osname} eq 'msys')
INIT
{
- # PGHOST is set once and for all through a single series of tests when
- # this module is loaded.
- $test_localhost = "127.0.0.1";
- $test_pghost =
- $TestLib::windows_os ? $test_localhost : TestLib::tempdir_short;
- $ENV{PGHOST} = $test_pghost;
- $ENV{PGDATABASE} = 'postgres';
+ # Set PGHOST for backward compatibility. This doesn't work for own_host
+ # nodes, so prefer to not rely on this when writing new tests.
+ $use_tcp = $TestLib::windows_os;
+ $test_localhost = "127.0.0.1";
+ $last_host_assigned = 1;
+ $test_pghost = $use_tcp ? $test_localhost : TestLib::tempdir_short;
+ $ENV{PGHOST} = $test_pghost;
+ $ENV{PGDATABASE} = 'postgres';
# Tracking of last port value assigned to accelerate free port lookup.
$last_port_assigned = int(rand() * 16384) + 49152;
@@ -155,7 +157,9 @@ sub new
_host => $pghost,
_basedir => "$TestLib::tmp_check/t_${testname}_${name}_data",
_name => $name,
- _logfile => "$TestLib::log_path/${testname}_${name}.log"
+ _logfile_generation => 0,
+ _logfile_base => "$TestLib::log_path/${testname}_${name}",
+ _logfile => "$TestLib::log_path/${testname}_${name}.log"
};
bless $self, $class;
@@ -473,8 +477,9 @@ sub init
print $conf "max_wal_senders = 0\n";
}
- if ($TestLib::windows_os)
+ if ($use_tcp)
{
+ print $conf "unix_socket_directories = ''\n";
print $conf "listen_addresses = '$host'\n";
}
else
@@ -536,12 +541,11 @@ sub backup
{
my ($self, $backup_name) = @_;
my $backup_path = $self->backup_dir . '/' . $backup_name;
- my $port = $self->port;
my $name = $self->name;
print "# Taking pg_basebackup $backup_name from node \"$name\"\n";
- TestLib::system_or_bail('pg_basebackup', '-D', $backup_path, '-p', $port,
- '--no-sync');
+ TestLib::system_or_bail('pg_basebackup', '-D', $backup_path, '-h',
+ $self->host, '-p', $self->port, '--no-sync');
print "# Backup finished\n";
return;
}
@@ -651,6 +655,7 @@ sub init_from_backup
{
my ($self, $root_node, $backup_name, %params) = @_;
my $backup_path = $root_node->backup_dir . '/' . $backup_name;
+ my $host = $self->host;
my $port = $self->port;
my $node_name = $self->name;
my $root_name = $root_node->name;
@@ -677,6 +682,15 @@ sub init_from_backup
qq(
port = $port
));
+ if ($use_tcp)
+ {
+ $self->append_conf('postgresql.conf', "listen_addresses = '$host'");
+ }
+ else
+ {
+ $self->append_conf('postgresql.conf',
+ "unix_socket_directories = '$host'");
+ }
$self->enable_streaming($root_node) if $params{has_streaming};
$self->enable_restoring($root_node) if $params{has_restoring};
return;
@@ -684,17 +698,45 @@ port = $port
=pod
-=item $node->start()
+=item $node->rotate_logfile()
+
+Switch to a new PostgreSQL log file. This does not alter any running
+PostgreSQL process. Subsequent method calls, including pg_ctl invocations,
+will use the new name. Return the new name.
+
+=cut
+
+sub rotate_logfile
+{
+ my ($self) = @_;
+ $self->{_logfile} = sprintf('%s_%d.log',
+ $self->{_logfile_base},
+ ++$self->{_logfile_generation});
+ return $self->{_logfile};
+}
+
+=pod
+
+=item $node->start(%params) => success_or_failure
Wrapper for pg_ctl start
Start the node and wait until it is ready to accept connections.
+=over
+
+=item fail_ok => 1
+
+By default, failure terminates the entire F<prove> invocation. If given,
+instead return a true or false value to indicate success or failure.
+
+=back
+
=cut
sub start
{
- my ($self) = @_;
+ my ($self, %params) = @_;
my $port = $self->port;
my $pgdata = $self->data_dir;
my $name = $self->name;
@@ -707,10 +749,34 @@ sub start
{
print "# pg_ctl start failed; logfile:\n";
print TestLib::slurp_file($self->logfile);
- BAIL_OUT("pg_ctl start failed");
+ BAIL_OUT("pg_ctl start failed") unless $params{fail_ok};
+ return 0;
}
$self->_update_pid(1);
+ return 1;
+}
+
+=pod
+
+=item $node->kill9()
+
+Send SIGKILL (signal 9) to the postmaster.
+
+Note: if the node is already known stopped, this does nothing.
+However, if we think it's running and it's not, it's important for
+this to fail. Otherwise, tests might fail to detect server crashes.
+
+=cut
+
+sub kill9
+{
+ my ($self) = @_;
+ my $name = $self->name;
+ return unless defined $self->{_pid};
+ print "### Killing node \"$name\" using signal 9\n";
+ kill(9, $self->{_pid}) or BAIL_OUT("kill(9, $self->{_pid}) failed");
+ $self->{_pid} = undef;
return;
}
@@ -943,7 +1009,7 @@ sub _update_pid
=pod
-=item PostgresNode->get_new_node(node_name)
+=item PostgresNode->get_new_node(node_name, %params)
Build a new object of class C<PostgresNode> (or of a subclass, if you have
one), assigning a free port number. Remembers the node, to prevent its port
@@ -952,6 +1018,22 @@ shut down when the test script exits.
You should generally use this instead of C<PostgresNode::new(...)>.
+=over
+
+=item port => [1,65535]
+
+By default, this function assigns a port number to each node. Specify this to
+force a particular port number. The caller is responsible for evaluating
+potential conflicts and privilege requirements.
+
+=item own_host => 1
+
+By default, all nodes use the same PGHOST value. If specified, generate a
+PGHOST specific to this node. This allows multiple nodes to use the same
+port.
+
+=back
+
For backwards compatibility, it is also exported as a standalone function,
which can only create objects of class C<PostgresNode>.
@@ -960,10 +1042,11 @@ which can only create objects of class C<PostgresNode>.
sub get_new_node
{
my $class = 'PostgresNode';
- $class = shift if 1 < scalar @_;
- my $name = shift;
- my $found = 0;
- my $port = $last_port_assigned;
+ $class = shift if scalar(@_) % 2 != 1;
+ my ($name, %params) = @_;
+ my $port_is_forced = defined $params{port};
+ my $found = $port_is_forced;
+ my $port = $port_is_forced ? $params{port} : $last_port_assigned;
while ($found == 0)
{
@@ -980,13 +1063,15 @@ sub get_new_node
$found = 0 if ($node->port == $port);
}
- # Check to see if anything else is listening on this TCP port.
- # This is *necessary* on Windows, and seems like a good idea
- # on Unixen as well, even though we don't ask the postmaster
- # to open a TCP port on Unix.
+ # Check to see if anything else is listening on this TCP port. Accept
+ # only ports available for all possible listen_addresses values, so
+ # the caller can harness this port for the widest range of purposes.
+ # This is *necessary* on Windows, and seems like a good idea on Unixen
+ # as well, even though we don't ask the postmaster to open a TCP port
+ # on Unix.
if ($found == 1)
{
- my $iaddr = inet_aton($test_localhost);
+ my $iaddr = inet_aton('0.0.0.0');
my $paddr = sockaddr_in($port, $iaddr);
my $proto = getprotobyname("tcp");
@@ -1002,16 +1087,35 @@ sub get_new_node
}
}
- print "# Found free port $port\n";
+ print "# Found port $port\n";
+
+ # Select a host.
+ my $host = $test_pghost;
+ if ($params{own_host})
+ {
+ if ($use_tcp)
+ {
+ # This assumes $use_tcp platforms treat every address in
+ # 127.0.0.1/24, not just 127.0.0.1, as a usable loopback.
+ $last_host_assigned++;
+ $last_host_assigned > 254 and BAIL_OUT("too many own_host nodes");
+ $host = '127.0.0.' . $last_host_assigned;
+ }
+ else
+ {
+ $host = "$test_pghost/$name"; # Assume $name =~ /^[-_a-zA-Z0-9]+$/
+ mkdir $host;
+ }
+ }
# Lock port number found by creating a new node
- my $node = $class->new($name, $test_pghost, $port);
+ my $node = $class->new($name, $host, $port);
# Add node to list of nodes
push(@all_nodes, $node);
# And update port for next time
- $last_port_assigned = $port;
+ $port_is_forced or $last_port_assigned = $port;
return $node;
}
@@ -1402,9 +1506,8 @@ $stderr);
=item $node->command_ok(...)
-Runs a shell command like TestLib::command_ok, but with PGPORT
-set so that the command will default to connecting to this
-PostgresNode.
+Runs a shell command like TestLib::command_ok, but with PGHOST and PGPORT set
+so that the command will default to connecting to this PostgresNode.
=cut
@@ -1414,6 +1517,7 @@ sub command_ok
my $self = shift;
+ local $ENV{PGHOST} = $self->host;
local $ENV{PGPORT} = $self->port;
TestLib::command_ok(@_);
@@ -1424,7 +1528,7 @@ sub command_ok
=item $node->command_fails(...)
-TestLib::command_fails with our PGPORT. See command_ok(...)
+TestLib::command_fails with our connection parameters. See command_ok(...)
=cut
@@ -1434,6 +1538,7 @@ sub command_fails
my $self = shift;
+ local $ENV{PGHOST} = $self->host;
local $ENV{PGPORT} = $self->port;
TestLib::command_fails(@_);
@@ -1444,7 +1549,7 @@ sub command_fails
=item $node->command_like(...)
-TestLib::command_like with our PGPORT. See command_ok(...)
+TestLib::command_like with our connection parameters. See command_ok(...)
=cut
@@ -1454,6 +1559,7 @@ sub command_like
my $self = shift;
+ local $ENV{PGHOST} = $self->host;
local $ENV{PGPORT} = $self->port;
TestLib::command_like(@_);
@@ -1464,7 +1570,8 @@ sub command_like
=item $node->command_checks_all(...)
-TestLib::command_checks_all with our PGPORT. See command_ok(...)
+TestLib::command_checks_all with our connection parameters. See
+command_ok(...)
=cut
@@ -1474,6 +1581,7 @@ sub command_checks_all
my $self = shift;
+ local $ENV{PGHOST} = $self->host;
local $ENV{PGPORT} = $self->port;
TestLib::command_checks_all(@_);
@@ -1498,6 +1606,7 @@ sub issues_sql_like
my ($self, $cmd, $expected_sql, $test_name) = @_;
+ local $ENV{PGHOST} = $self->host;
local $ENV{PGPORT} = $self->port;
truncate $self->logfile, 0;
@@ -1512,8 +1621,8 @@ sub issues_sql_like
=item $node->run_log(...)
-Runs a shell command like TestLib::run_log, but with PGPORT set so
-that the command will default to connecting to this PostgresNode.
+Runs a shell command like TestLib::run_log, but with connection parameters set
+so that the command will default to connecting to this PostgresNode.
=cut
@@ -1521,6 +1630,7 @@ sub run_log
{
my $self = shift;
+ local $ENV{PGHOST} = $self->host;
local $ENV{PGPORT} = $self->port;
TestLib::run_log(@_);
diff --git a/src/test/recovery/t/016_shm.pl b/src/test/recovery/t/016_shm.pl
new file mode 100644
index 0000000..9021562
--- /dev/null
+++ b/src/test/recovery/t/016_shm.pl
@@ -0,0 +1,154 @@
+#
+# Tests of pg_shmem.h functions
+#
+use strict;
+use warnings;
+use IPC::Run 'run';
+use PostgresNode;
+use Test::More;
+use TestLib;
+
+plan tests => 6;
+
+my $tempdir = TestLib::tempdir;
+my $port;
+
+# When using Unix sockets, we can dictate the port number. In the absence of
+# collisions from other shmget() activity, gnat starts with key 0x7d001
+# (512001), and flea starts with key 0x7d002 (512002).
+$port = 512 unless $PostgresNode::use_tcp;
+
+# Log "ipcs" diffs on a best-effort basis, swallowing any error.
+my $ipcs_before = "$tempdir/ipcs_before";
+eval { run_log [ 'ipcs', '-am' ], '>', $ipcs_before; };
+
+sub log_ipcs
+{
+ eval { run_log [ 'ipcs', '-am' ], '|', [ 'diff', $ipcs_before, '-' ] };
+ return;
+}
+
+# Node setup.
+sub init_start
+{
+ my $name = shift;
+ my $ret = PostgresNode->get_new_node($name, port => $port, own_host => 1);
+ defined($port) or $port = $ret->port; # same port for all nodes
+ $ret->init;
+ $ret->start;
+ log_ipcs();
+ return $ret;
+}
+my $gnat = init_start 'gnat';
+my $flea = init_start 'flea';
+
+# Upon postmaster death, postmaster children exit automatically.
+$gnat->kill9;
+log_ipcs();
+$flea->restart; # flea ignores the shm key gnat abandoned.
+log_ipcs();
+poll_start($gnat); # gnat recycles its former shm key.
+log_ipcs();
+
+# After clean shutdown, the nodes swap shm keys.
+$gnat->stop;
+$flea->restart;
+log_ipcs();
+$gnat->start;
+log_ipcs();
+
+# Scenarios involving no postmaster.pid, dead postmaster, and a live backend.
+# Use a regress.c function to emulate the responsiveness of a backend working
+# through a CPU-intensive task.
+$gnat->safe_psql('postgres', <<EOSQL);
+CREATE FUNCTION wait_pid(int)
+ RETURNS void
+ AS '$ENV{REGRESS_SHLIB}'
+ LANGUAGE C STRICT;
+EOSQL
+my $slow_query = 'SELECT wait_pid(pg_backend_pid())';
+my ($stdout, $stderr);
+my $slow_client = IPC::Run::start(
+ [
+ 'psql', '-X', '-qAt', '-d', $gnat->connstr('postgres'),
+ '-c', $slow_query
+ ],
+ '<',
+ \undef,
+ '>',
+ \$stdout,
+ '2>',
+ \$stderr,
+ IPC::Run::timeout(900)); # five times the poll_query_until timeout
+ok( $gnat->poll_query_until(
+ 'postgres',
+ "SELECT 1 FROM pg_stat_activity WHERE query = '$slow_query'", '1'),
+ 'slow query started');
+my $slow_pid = $gnat->safe_psql('postgres',
+ "SELECT pid FROM pg_stat_activity WHERE query = '$slow_query'");
+$gnat->kill9;
+unlink($gnat->data_dir . '/postmaster.pid');
+$gnat->rotate_logfile; # on Windows, can't open old log for writing
+log_ipcs();
+# Reject ordinary startup.
+ok(!$gnat->start(fail_ok => 1), 'live query blocks restart');
+like(
+ slurp_file($gnat->logfile),
+ qr/pre-existing shared memory block/,
+ 'detected live backend via shared memory');
+# Reject single-user startup.
+my $single_stderr;
+ok( !run_log(
+ [ 'postgres', '--single', '-D', $gnat->data_dir, 'template1' ],
+ '<', \('SELECT 1 + 1'), '2>', \$single_stderr),
+ 'live query blocks --single');
+print STDERR $single_stderr;
+like(
+ $single_stderr,
+ qr/pre-existing shared memory block/,
+ 'single-user mode detected live backend via shared memory');
+log_ipcs();
+# Fail to reject startup if shm key N has become available and we crash while
+# using key N+1. This is unwanted, but expected. Windows is immune, because
+# its GetSharedMemName() use DataDir strings, not numeric keys.
+$flea->stop; # release first key
+is( $gnat->start(fail_ok => 1),
+ $TestLib::windows_os ? 0 : 1,
+ 'key turnover fools only sysv_shmem.c');
+$gnat->stop; # release first key (no-op on $TestLib::windows_os)
+$flea->start; # grab first key
+# cleanup
+TestLib::system_log('pg_ctl', 'kill', 'QUIT', $slow_pid);
+$slow_client->finish; # client has detected backend termination
+log_ipcs();
+poll_start($gnat); # recycle second key
+
+$gnat->stop;
+$flea->stop;
+log_ipcs();
+
+
+# When postmaster children are slow to exit after postmaster death, we may
+# need retries to start a new postmaster.
+sub poll_start
+{
+ my ($node) = @_;
+
+ my $max_attempts = 180 * 10;
+ my $attempts = 0;
+
+ while ($attempts < $max_attempts)
+ {
+ $node->start(fail_ok => 1) && return 1;
+
+ # Wait 0.1 second before retrying.
+ usleep(100_000);
+
+ $attempts++;
+ }
+
+ # No success within 180 seconds. Try one last time without fail_ok, which
+ # will BAIL_OUT unless it succeeds.
+ $node->start && return 1;
+ return 0;
+}
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index 4018313..4a1e6e3 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -205,6 +205,7 @@ sub tap_check
local %ENV = %ENV;
$ENV{PERL5LIB} = "$topdir/src/test/perl;$ENV{PERL5LIB}";
$ENV{PG_REGRESS} = "$topdir/$Config/pg_regress/pg_regress";
+ $ENV{REGRESS_SHLIB} = "$topdir/src/test/regress/regress.dll";
$ENV{TESTDIR} = "$dir";
On Sun, Dec 02, 2018 at 04:41:06PM -0800, Noah Misch wrote:
Here's a version fixing that test for post-cfdf4dc backends.
This patch set still applies and needs review, so moved to next CF.
--
Michael
Hello.
At Sun, 2 Dec 2018 16:41:06 -0800, Noah Misch <noah@leadboat.com> wrote in <20181203004106.GA2860387@rfd.leadboat.com>
On Thu, Nov 29, 2018 at 10:51:40PM -0800, Noah Misch wrote:
On Thu, Nov 29, 2018 at 01:10:57PM +0100, Dmitry Dolgov wrote:
As a side note, with this patch recovery tests are failing now on 016_shm.pl
# Failed test 'detected live backend via shared memory'
# at t/016_shm.pl line 87.
# '2018-11-28 13:08:08.504 UTC [21924] LOG:
listening on Unix socket "/tmp/yV2oDNcG8e/gnat/.s.PGSQL.512"
# 2018-11-28 13:08:08.512 UTC [21925] LOG: database system was
interrupted; last known up at 2018-11-28 13:08:08 UTC
# 2018-11-28 13:08:08.512 UTC [21925] LOG: database system was not
properly shut down; automatic recovery in progress
# 2018-11-28 13:08:08.512 UTC [21925] LOG: invalid record length at
0/165FEF8: wanted 24, got 0
# 2018-11-28 13:08:08.512 UTC [21925] LOG: redo is not required
# 2018-11-28 13:08:08.516 UTC [21924] LOG: database system is ready
to accept connections
# '
# doesn't match '(?^:pre-existing shared memory block)'Thanks for the report. Since commit cfdf4dc made pg_sleep() react to
postmaster death, the test will need a different way to stall a backend. This
doesn't affect non-test code, and the test still passes against cfdf4dc^ and
against REL_11_STABLE. I've queued a task to update the test code, but review
can proceed in parallel.
I found that I have 65(h) segments left alone on my environment:p
At Sat, 11 Aug 2018 23:48:15 -0700, Noah Misch <noah@leadboat.com> wrote in <20180812064815.GB2301738@rfd.leadboat.com>
still doesn't detect. I could pursue a fix via the aforementioned
sysv_shmem_key file, modulo the possibility of a DBA removing it. I could
also, when postmaster.pid is missing, make sysv_shmem.c check the first N
(N=100?) keys applicable to the selected port. My gut feeling is that neither
thing is worthwhile, but I'm interested to hear other opinions.
# Though I don't get the meaning of the "modulo" there..
I think the only thing we must avoid here is sharing the same
shmem segment with a living-dead server. If we can do that
without the pid file, it would be better than relying on it. We
could remove orphaned segments automatically, but I don't think
we should do that going so far as relying on a dedicated
file. Also, I don't think it's worth stopping shmem id scanning
at a certain number since I don't come up with an appropriate
number for it. But looking "port * 1000", it might be expected
that a usable segment id will found while scanning that number of
ids (1000).
Here's a version fixing that test for post-cfdf4dc backends.
This moves what were in PGSharedMmoeryIsInUse into a new function
IpcMemoryAnalyze for reusing then adds distinction among
EEXISTS/FOREIGN in not-in-use cases and ATTACHED/ANALYSIS_FAILURE
in an in-use case. UNATTACHED is changed to a not-in-use case by
this patch. As the result PGSharedMemoryIsInUse() is changed so
that it reutrns "not-in-use" for the UNATTACHED case. It looks
fine.
PGSharedMemoryCreate changed to choose a usable shmem id using
the IpcMemoryAnalyze(). But some of the statuses from
IpcMemoryAnalyze() is concealed by failure of
PGSharedMemoryAttach() and ignored silently opposed to what the
code is intending to do. (By the way SHMSTATE_EEXISTS seems
suggesting oppsite thing from EEXIST, which would be confusing.)
PGSharedMemoryCreate() repeats shmat/shmdt twice in every
iteration. It won't harm so much but it would be better if we
could get rid of that.
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
Import Notes
Reply to msg id not found: 20181203004106.GA2860387@rfd.leadboat.com20180812064815.GB2301738@rfd.leadboat.com
On Mon, Mar 04, 2019 at 06:04:20PM +0900, Kyotaro HORIGUCHI wrote:
I found that I have 65(h) segments left alone on my environment:p
Did patched PostgreSQL create those, or did unpatched PostgreSQL create them?
At Sat, 11 Aug 2018 23:48:15 -0700, Noah Misch <noah@leadboat.com> wrote in <20180812064815.GB2301738@rfd.leadboat.com>
still doesn't detect. I could pursue a fix via the aforementioned
sysv_shmem_key file, modulo the possibility of a DBA removing it. I could
also, when postmaster.pid is missing, make sysv_shmem.c check the first N
(N=100?) keys applicable to the selected port. My gut feeling is that neither
thing is worthwhile, but I'm interested to hear other opinions.# Though I don't get the meaning of the "modulo" there..
It wasn't clear. Adding a separate sysv_shmem_key file would not protect a
cluster if the DBA does "rm -f postmaster.pid sysv_shmem_key". It would,
however, fix this test case:
+# Fail to reject startup if shm key N has become available and we crash while
+# using key N+1. This is unwanted, but expected. Windows is immune, because
+# its GetSharedMemName() use DataDir strings, not numeric keys.
+$flea->stop; # release first key
+is( $gnat->start(fail_ok => 1),
+ $TestLib::windows_os ? 0 : 1,
+ 'key turnover fools only sysv_shmem.c');
I think the only thing we must avoid here is sharing the same
shmem segment with a living-dead server. If we can do that
without the pid file, it would be better than relying on it. We
could remove orphaned segments automatically, but I don't think
we should do that going so far as relying on a dedicated
file.
There are two things to avoid. First, during postmaster startup, avoid
sharing a segment with any other process. Second, avoid sharing a data
directory with a running PostgreSQL process that was a child of some other
postmaster. I think that's what you mean by "living-dead server". The first
case is already prevented thoroughly, because we only proceed with startup
after creating a new segment with IPC_CREAT | IPC_EXCL. The second case is
what this patch seeks to improve.
Also, I don't think it's worth stopping shmem id scanning
at a certain number since I don't come up with an appropriate
number for it. But looking "port * 1000", it might be expected
that a usable segment id will found while scanning that number of
ids (1000).
If we find thousands of unusable segments, we do keep going until we find a
usable segment. I was referring to a different situation. Today, if there's
no postmaster.pid file and we're using port 5432, we first try segment ID
5432000. If segment ID 5432000 is usable, we use it without examining any
other segment ID. If segment ID 5432010 were in use by a "living-dead
server", we won't discover that fact. We could increase the chance of
discovering that fact by checking every segment ID from 5432000 to 5432999.
(Once finished with that scan, we'd still create and use 5432000.) I didn't
implement that idea in the patch, but I shared it to see if anyone thought it
was worth adding.
PGSharedMemoryCreate changed to choose a usable shmem id using
the IpcMemoryAnalyze(). But some of the statuses from
IpcMemoryAnalyze() is concealed by failure of
PGSharedMemoryAttach() and ignored silently opposed to what the
code is intending to do.
SHMSTATE_FOREIGN is ignored silently. The patch modifies the
PGSharedMemoryAttach() header comment to direct callers to treat
PGSharedMemoryAttach() failure like SHMSTATE_FOREIGN. I don't think the code
had an unintentional outcome due to the situation you describe. Perhaps I
could have made the situation clearer.
(By the way SHMSTATE_EEXISTS seems
suggesting oppsite thing from EEXIST, which would be confusing.)
Good catch. Is SHMSTATE_ENOENT better?
PGSharedMemoryCreate() repeats shmat/shmdt twice in every
iteration. It won't harm so much but it would be better if we
could get rid of that.
I'll try to achieve that and see if the code turns out cleaner.
Thanks,
nm
On Wed, Mar 06, 2019 at 07:24:22PM -0800, Noah Misch wrote:
On Mon, Mar 04, 2019 at 06:04:20PM +0900, Kyotaro HORIGUCHI wrote:
PGSharedMemoryCreate changed to choose a usable shmem id using
the IpcMemoryAnalyze(). But some of the statuses from
IpcMemoryAnalyze() is concealed by failure of
PGSharedMemoryAttach() and ignored silently opposed to what the
code is intending to do.SHMSTATE_FOREIGN is ignored silently. The patch modifies the
PGSharedMemoryAttach() header comment to direct callers to treat
PGSharedMemoryAttach() failure like SHMSTATE_FOREIGN. I don't think the code
had an unintentional outcome due to the situation you describe. Perhaps I
could have made the situation clearer.
I think v3, attached, avoids this appearance of unintended behavior.
(By the way SHMSTATE_EEXISTS seems
suggesting oppsite thing from EEXIST, which would be confusing.)Good catch. Is SHMSTATE_ENOENT better?
I did s/SHMSTATE_EEXISTS/SHMSTATE_ENOENT/.
PGSharedMemoryCreate() repeats shmat/shmdt twice in every
iteration. It won't harm so much but it would be better if we
could get rid of that.I'll try to achieve that and see if the code turns out cleaner.
I renamed IpcMemoryAnalyze() to PGSharedMemoryAttach() and deleted the old
function of that name. Now, this function never calls shmdt(); the caller is
responsible for that. I do like things better this way. What do you think?
Attachments:
ipcrm-hint-v1.patchtext/plain; charset=us-asciiDownload
--- a/src/backend/utils/init/miscinit.c
+++ b/src/backend/utils/init/miscinit.c
@@ -1070,14 +1070,10 @@ CreateLockFile(const char *filename, bool amPostmaster,
if (PGSharedMemoryIsInUse(id1, id2))
ereport(FATAL,
(errcode(ERRCODE_LOCK_FILE_EXISTS),
- errmsg("pre-existing shared memory block "
- "(key %lu, ID %lu) is still in use",
+ errmsg("pre-existing shared memory block (key %lu, ID %lu) is still in use",
id1, id2),
- errhint("If you're sure there are no old "
- "server processes still running, remove "
- "the shared memory block "
- "or just delete the file \"%s\".",
- filename)));
+ errhint("Terminate any old server processes associated with data directory \"%s\".",
+ refName)));
}
}
PGSharedMemoryCreate-safety-v3.patchtext/plain; charset=us-asciiDownload
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index c118f64..e8f6050 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -430,13 +430,13 @@ ifeq ($(enable_tap_tests),yes)
define prove_installcheck
rm -rf '$(CURDIR)'/tmp_check
$(MKDIR_P) '$(CURDIR)'/tmp_check
-cd $(srcdir) && TESTDIR='$(CURDIR)' PATH="$(bindir):$$PATH" PGPORT='6$(DEF_PGPORT)' top_builddir='$(CURDIR)/$(top_builddir)' PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
+cd $(srcdir) && TESTDIR='$(CURDIR)' PATH="$(bindir):$$PATH" PGPORT='6$(DEF_PGPORT)' top_builddir='$(CURDIR)/$(top_builddir)' PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' REGRESS_SHLIB='$(abs_top_builddir)/src/test/regress/regress$(DLSUFFIX)' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
endef
define prove_check
rm -rf '$(CURDIR)'/tmp_check
$(MKDIR_P) '$(CURDIR)'/tmp_check
-cd $(srcdir) && TESTDIR='$(CURDIR)' $(with_temp_install) PGPORT='6$(DEF_PGPORT)' PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
+cd $(srcdir) && TESTDIR='$(CURDIR)' $(with_temp_install) PGPORT='6$(DEF_PGPORT)' PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' REGRESS_SHLIB='$(abs_top_builddir)/src/test/regress/regress$(DLSUFFIX)' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
endef
else
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 1511a61..c9f6e43 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -72,6 +72,26 @@
typedef key_t IpcMemoryKey; /* shared memory key passed to shmget(2) */
typedef int IpcMemoryId; /* shared memory ID returned by shmget(2) */
+/*
+ * How does a given IpcMemoryId relate to this PostgreSQL process?
+ *
+ * One could recycle unattached segments of different data directories if we
+ * distinguished that case from other SHMSTATE_FOREIGN cases. Doing so would
+ * cause us to visit less of the key space, making us less likely to detect a
+ * SHMSTATE_ATTACHED key. It would also complicate the concurrency analysis,
+ * in that postmasters of different data directories could simultaneously
+ * attempt to recycle a given key. We'll waste keys longer in some cases, but
+ * avoiding the problems of the alternative justifies that loss.
+ */
+typedef enum
+{
+ SHMSTATE_ANALYSIS_FAILURE, /* unexpected failure to analyze the ID */
+ SHMSTATE_ATTACHED, /* pertinent to DataDir, has attached PIDs */
+ SHMSTATE_ENOENT, /* no segment of that ID */
+ SHMSTATE_FOREIGN, /* exists, but not pertinent to DataDir */
+ SHMSTATE_UNATTACHED /* pertinent to DataDir, no attached PIDs */
+} IpcMemoryState;
+
unsigned long UsedShmemSegID = 0;
void *UsedShmemSegAddr = NULL;
@@ -82,8 +102,8 @@ static void *AnonymousShmem = NULL;
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
-static PGShmemHeader *PGSharedMemoryAttach(IpcMemoryKey key,
- IpcMemoryId *shmid);
+static IpcMemoryState PGSharedMemoryAttach(IpcMemoryId shmId,
+ PGShmemHeader **addr);
/*
@@ -287,11 +307,36 @@ IpcMemoryDelete(int status, Datum shmId)
bool
PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2)
{
- IpcMemoryId shmId = (IpcMemoryId) id2;
+ PGShmemHeader *memAddress;
+ IpcMemoryState state;
+
+ state = PGSharedMemoryAttach((IpcMemoryId) id2, &memAddress);
+ if (memAddress)
+ shmdt(memAddress);
+ switch (state)
+ {
+ case SHMSTATE_ENOENT:
+ case SHMSTATE_FOREIGN:
+ case SHMSTATE_UNATTACHED:
+ return false;
+ case SHMSTATE_ANALYSIS_FAILURE:
+ case SHMSTATE_ATTACHED:
+ return true;
+ }
+ return true;
+}
+
+/* See comment at IpcMemoryState. */
+static IpcMemoryState
+PGSharedMemoryAttach(IpcMemoryId shmId,
+ PGShmemHeader **addr)
+{
struct shmid_ds shmStat;
struct stat statbuf;
PGShmemHeader *hdr;
+ *addr = NULL;
+
/*
* We detect whether a shared memory segment is in use by seeing whether
* it (a) exists and (b) has any processes attached to it.
@@ -304,7 +349,7 @@ PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2)
* exists.
*/
if (errno == EINVAL)
- return false;
+ return SHMSTATE_ENOENT;
/*
* EACCES implies that the segment belongs to some other userid, which
@@ -312,7 +357,7 @@ PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2)
* is relevant to our data directory).
*/
if (errno == EACCES)
- return false;
+ return SHMSTATE_FOREIGN;
/*
* Some Linux kernel versions (in fact, all of them as of July 2007)
@@ -323,7 +368,7 @@ PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2)
*/
#ifdef HAVE_LINUX_EIDRM_BUG
if (errno == EIDRM)
- return false;
+ return SHMSTATE_ENOENT;
#endif
/*
@@ -331,25 +376,26 @@ PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2)
* only likely case is EIDRM, which implies that the segment has been
* IPC_RMID'd but there are still processes attached to it.
*/
- return true;
+ return SHMSTATE_ANALYSIS_FAILURE;
}
- /* If it has no attached processes, it's not in use */
- if (shmStat.shm_nattch == 0)
- return false;
-
/*
* Try to attach to the segment and see if it matches our data directory.
* This avoids shmid-conflict problems on machines that are running
* several postmasters under the same userid.
*/
if (stat(DataDir, &statbuf) < 0)
- return true; /* if can't stat, be conservative */
-
- hdr = (PGShmemHeader *) shmat(shmId, NULL, PG_SHMAT_FLAGS);
+ return SHMSTATE_ANALYSIS_FAILURE; /* can't stat; be conservative */
+ /*
+ * If we can't attach, be conservative. This may fail if postmaster.pid
+ * furnished the shmId and another user created a world-readable segment
+ * of the same shmId.
+ */
+ hdr = (PGShmemHeader *) shmat(shmId, UsedShmemSegAddr, PG_SHMAT_FLAGS);
if (hdr == (PGShmemHeader *) -1)
- return true; /* if can't attach, be conservative */
+ return SHMSTATE_ANALYSIS_FAILURE;
+ *addr = hdr;
if (hdr->magic != PGShmemMagic ||
hdr->device != statbuf.st_dev ||
@@ -357,16 +403,12 @@ PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2)
{
/*
* It's either not a Postgres segment, or not one for my data
- * directory. In either case it poses no threat.
+ * directory.
*/
- shmdt((void *) hdr);
- return false;
+ return SHMSTATE_FOREIGN;
}
- /* Trouble --- looks a lot like there's still live backends */
- shmdt((void *) hdr);
-
- return true;
+ return shmStat.shm_nattch == 0 ? SHMSTATE_UNATTACHED : SHMSTATE_ATTACHED;
}
#ifdef MAP_HUGETLB
@@ -538,25 +580,21 @@ AnonymousShmemDetach(int status, Datum arg)
* standard header. Also, register an on_shmem_exit callback to release
* the storage.
*
- * Dead Postgres segments are recycled if found, but we do not fail upon
- * collision with non-Postgres shmem segments. The idea here is to detect and
- * re-use keys that may have been assigned by a crashed postmaster or backend.
- *
- * makePrivate means to always create a new segment, rather than attach to
- * or recycle any existing segment.
+ * Dead Postgres segments pertinent to this DataDir are recycled if found, but
+ * we do not fail upon collision with foreign shmem segments. The idea here
+ * is to detect and re-use keys that may have been assigned by a crashed
+ * postmaster or backend.
*
* The port number is passed for possible use as a key (for SysV, we use
- * it to generate the starting shmem key). In a standalone backend,
- * zero will be passed.
+ * it to generate the starting shmem key).
*/
PGShmemHeader *
-PGSharedMemoryCreate(Size size, bool makePrivate, int port,
+PGSharedMemoryCreate(Size size, int port,
PGShmemHeader **shim)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
PGShmemHeader *hdr;
- IpcMemoryId shmid;
struct stat statbuf;
Size sysvsize;
@@ -588,11 +626,20 @@ PGSharedMemoryCreate(Size size, bool makePrivate, int port,
/* Make sure PGSharedMemoryAttach doesn't fail without need */
UsedShmemSegAddr = NULL;
- /* Loop till we find a free IPC key */
- NextShmemSegID = port * 1000;
+ /*
+ * Loop till we find a free IPC key. Trust CreateDataDirLockFile() to
+ * ensure no more than one postmaster per data directory can enter this
+ * loop simultaneously. (CreateDataDirLockFile() does not ensure that,
+ * but prefer fixing it over coping here.)
+ */
+ NextShmemSegID = 1 + port * 1000;
- for (NextShmemSegID++;; NextShmemSegID++)
+ for (;;)
{
+ IpcMemoryId shmid;
+ PGShmemHeader *oldhdr;
+ IpcMemoryState state;
+
/* Try to create new segment */
memAddress = InternalIpcMemoryCreate(NextShmemSegID, sysvsize);
if (memAddress)
@@ -600,58 +647,69 @@ PGSharedMemoryCreate(Size size, bool makePrivate, int port,
/* Check shared memory and possibly remove and recreate */
- if (makePrivate) /* a standalone backend shouldn't do this */
- continue;
-
- if ((memAddress = PGSharedMemoryAttach(NextShmemSegID, &shmid)) == NULL)
- continue; /* can't attach, not one of mine */
-
/*
- * If I am not the creator and it belongs to an extant process,
- * continue.
+ * shmget() failure is typically EACCES, hence SHMSTATE_FOREIGN.
+ * ENOENT, a narrow possibility, implies SHMSTATE_ENOENT, but one can
+ * safely treat SHMSTATE_ENOENT like SHMSTATE_FOREIGN.
*/
- hdr = (PGShmemHeader *) memAddress;
- if (hdr->creatorPID != getpid())
+ shmid = shmget(NextShmemSegID, sizeof(PGShmemHeader), 0);
+ if (shmid < 0)
{
- if (kill(hdr->creatorPID, 0) == 0 || errno != ESRCH)
- {
- shmdt(memAddress);
- continue; /* segment belongs to a live process */
- }
+ oldhdr = NULL;
+ state = SHMSTATE_FOREIGN;
}
+ else
+ state = PGSharedMemoryAttach(shmid, &oldhdr);
- /*
- * The segment appears to be from a dead Postgres process, or from a
- * previous cycle of life in this same process. Zap it, if possible,
- * and any associated dynamic shared memory segments, as well. This
- * probably shouldn't fail, but if it does, assume the segment belongs
- * to someone else after all, and continue quietly.
- */
- if (hdr->dsm_control != 0)
- dsm_cleanup_using_control_segment(hdr->dsm_control);
- shmdt(memAddress);
- if (shmctl(shmid, IPC_RMID, NULL) < 0)
- continue;
+ switch (state)
+ {
+ case SHMSTATE_ANALYSIS_FAILURE:
+ case SHMSTATE_ATTACHED:
+ ereport(FATAL,
+ (errcode(ERRCODE_LOCK_FILE_EXISTS),
+ errmsg("pre-existing shared memory block (key %lu, ID %lu) is still in use",
+ (unsigned long) NextShmemSegID,
+ (unsigned long) shmid),
+ errhint("Terminate any old server processes associated with data directory \"%s\".",
+ DataDir)));
+ case SHMSTATE_ENOENT:
- /*
- * Now try again to create the segment.
- */
- memAddress = InternalIpcMemoryCreate(NextShmemSegID, sysvsize);
- if (memAddress)
- break; /* successful create and attach */
+ /*
+ * To our surprise, some other process deleted since our last
+ * InternalIpcMemoryCreate(). Moments earlier, we would have
+ * seen SHMSTATE_FOREIGN. Try that same ID again.
+ */
+ elog(LOG,
+ "shared memory block (key %lu, ID %lu) deleted during startup",
+ (unsigned long) NextShmemSegID,
+ (unsigned long) shmid);
+ break;
+ case SHMSTATE_FOREIGN:
+ NextShmemSegID++;
+ break;
+ case SHMSTATE_UNATTACHED:
- /*
- * Can only get here if some other process managed to create the same
- * shmem key before we did. Let him have that one, loop around to try
- * next key.
- */
+ /*
+ * The segment pertains to DataDir, and every process that had
+ * used it has died or detached. Zap it, if possible, and any
+ * associated dynamic shared memory segments, as well. This
+ * shouldn't fail, but if it does, assume the segment belongs
+ * to someone else after all, and try the next candidate.
+ * Otherwise, try again to create the segment. That may fail
+ * if some other process creates the same shmem key before we
+ * do, in which case we'll try the next key.
+ */
+ if (oldhdr->dsm_control != 0)
+ dsm_cleanup_using_control_segment(oldhdr->dsm_control);
+ if (shmctl(shmid, IPC_RMID, NULL) < 0)
+ NextShmemSegID++;
+ }
+
+ if (oldhdr)
+ shmdt(oldhdr);
}
- /*
- * OK, we created a new segment. Mark it as created by this process. The
- * order of assignments here is critical so that another Postgres process
- * can't see the header as valid but belonging to an invalid PID!
- */
+ /* Initialize new segment. */
hdr = (PGShmemHeader *) memAddress;
hdr->creatorPID = getpid();
hdr->magic = PGShmemMagic;
@@ -707,7 +765,8 @@ void
PGSharedMemoryReAttach(void)
{
IpcMemoryId shmid;
- void *hdr;
+ PGShmemHeader *hdr;
+ IpcMemoryState state;
void *origUsedShmemSegAddr = UsedShmemSegAddr;
Assert(UsedShmemSegAddr != NULL);
@@ -720,14 +779,18 @@ PGSharedMemoryReAttach(void)
#endif
elog(DEBUG3, "attaching to %p", UsedShmemSegAddr);
- hdr = (void *) PGSharedMemoryAttach((IpcMemoryKey) UsedShmemSegID, &shmid);
- if (hdr == NULL)
+ shmid = shmget(UsedShmemSegID, sizeof(PGShmemHeader), 0);
+ if (shmid < 0)
+ state = SHMSTATE_FOREIGN;
+ else
+ state = PGSharedMemoryAttach(shmid, &hdr);
+ if (state != SHMSTATE_ATTACHED)
elog(FATAL, "could not reattach to shared memory (key=%d, addr=%p): %m",
(int) UsedShmemSegID, UsedShmemSegAddr);
if (hdr != origUsedShmemSegAddr)
elog(FATAL, "reattaching to shared memory returned unexpected address (got %p, expected %p)",
hdr, origUsedShmemSegAddr);
- dsm_set_control_handle(((PGShmemHeader *) hdr)->dsm_control);
+ dsm_set_control_handle(hdr->dsm_control);
UsedShmemSegAddr = hdr; /* probably redundant */
}
@@ -801,31 +864,3 @@ PGSharedMemoryDetach(void)
AnonymousShmem = NULL;
}
}
-
-
-/*
- * Attach to shared memory and make sure it has a Postgres header
- *
- * Returns attach address if OK, else NULL
- */
-static PGShmemHeader *
-PGSharedMemoryAttach(IpcMemoryKey key, IpcMemoryId *shmid)
-{
- PGShmemHeader *hdr;
-
- if ((*shmid = shmget(key, sizeof(PGShmemHeader), 0)) < 0)
- return NULL;
-
- hdr = (PGShmemHeader *) shmat(*shmid, UsedShmemSegAddr, PG_SHMAT_FLAGS);
-
- if (hdr == (PGShmemHeader *) -1)
- return NULL; /* failed: must be some other app's */
-
- if (hdr->magic != PGShmemMagic)
- {
- shmdt((void *) hdr);
- return NULL; /* segment belongs to a non-Postgres app */
- }
-
- return hdr;
-}
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index bc1e946..b583166 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -170,14 +170,9 @@ EnableLockPagesPrivilege(int elevel)
*
* Create a shared memory segment of the given size and initialize its
* standard header.
- *
- * makePrivate means to always create a new segment, rather than attach to
- * or recycle any existing segment. On win32, we always create a new segment,
- * since there is no need for recycling (segments go away automatically
- * when the last backend exits)
*/
PGShmemHeader *
-PGSharedMemoryCreate(Size size, bool makePrivate, int port,
+PGSharedMemoryCreate(Size size, int port,
PGShmemHeader **shim)
{
void *memAddress;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index fe59963..04e7be9 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -2574,7 +2574,7 @@ reset_shared(int port)
* determine IPC keys. This helps ensure that we will clean up dead IPC
* objects if the postmaster crashes and is restarted.
*/
- CreateSharedMemoryAndSemaphores(false, port);
+ CreateSharedMemoryAndSemaphores(port);
}
@@ -4915,7 +4915,7 @@ SubPostmasterMain(int argc, char *argv[])
InitProcess();
/* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores(false, 0);
+ CreateSharedMemoryAndSemaphores(0);
/* And run the backend */
BackendRun(&port); /* does not return */
@@ -4929,7 +4929,7 @@ SubPostmasterMain(int argc, char *argv[])
InitAuxiliaryProcess();
/* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores(false, 0);
+ CreateSharedMemoryAndSemaphores(0);
AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
}
@@ -4942,7 +4942,7 @@ SubPostmasterMain(int argc, char *argv[])
InitProcess();
/* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores(false, 0);
+ CreateSharedMemoryAndSemaphores(0);
AutoVacLauncherMain(argc - 2, argv + 2); /* does not return */
}
@@ -4955,7 +4955,7 @@ SubPostmasterMain(int argc, char *argv[])
InitProcess();
/* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores(false, 0);
+ CreateSharedMemoryAndSemaphores(0);
AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
}
@@ -4973,7 +4973,7 @@ SubPostmasterMain(int argc, char *argv[])
InitProcess();
/* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores(false, 0);
+ CreateSharedMemoryAndSemaphores(0);
/* Fetch MyBgworkerEntry from shared memory */
shmem_slot = atoi(argv[1] + 15);
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 5965d36..d7d7335 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -89,12 +89,9 @@ RequestAddinShmemSpace(Size size)
* through the same code as before. (Note that the called routines mostly
* check IsUnderPostmaster, rather than EXEC_BACKEND, to detect this case.
* This is a bit code-wasteful and could be cleaned up.)
- *
- * If "makePrivate" is true then we only need private memory, not shared
- * memory. This is true for a standalone backend, false for a postmaster.
*/
void
-CreateSharedMemoryAndSemaphores(bool makePrivate, int port)
+CreateSharedMemoryAndSemaphores(int port)
{
PGShmemHeader *shim = NULL;
@@ -166,7 +163,7 @@ CreateSharedMemoryAndSemaphores(bool makePrivate, int port)
/*
* Create the shmem segment
*/
- seghdr = PGSharedMemoryCreate(size, makePrivate, port, &shim);
+ seghdr = PGSharedMemoryCreate(size, port, &shim);
InitShmemAccess(seghdr);
@@ -187,12 +184,9 @@ CreateSharedMemoryAndSemaphores(bool makePrivate, int port)
{
/*
* We are reattaching to an existing shared memory segment. This
- * should only be reached in the EXEC_BACKEND case, and even then only
- * with makePrivate == false.
+ * should only be reached in the EXEC_BACKEND case.
*/
-#ifdef EXEC_BACKEND
- Assert(!makePrivate);
-#else
+#ifndef EXEC_BACKEND
elog(PANIC, "should be attached to shared memory already");
#endif
}
diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c
index bd2e4e8..c180a99 100644
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index a5ee209..bd28cdb 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -444,9 +444,11 @@ InitCommunication(void)
{
/*
* We're running a postgres bootstrap process or a standalone backend.
- * Create private "shmem" and semaphores.
+ * Though we won't listen on PostPortNumber, use it to select a shmem
+ * key. This increases the chance of detecting a leftover live
+ * backend of this DataDir.
*/
- CreateSharedMemoryAndSemaphores(true, 0);
+ CreateSharedMemoryAndSemaphores(PostPortNumber);
}
}
diff --git a/src/include/storage/ipc.h b/src/include/storage/ipc.h
index 0660252..e9b243f 100644
--- a/src/include/storage/ipc.h
+++ b/src/include/storage/ipc.h
@@ -76,6 +76,6 @@ extern void on_exit_reset(void);
/* ipci.c */
extern PGDLLIMPORT shmem_startup_hook_type shmem_startup_hook;
-extern void CreateSharedMemoryAndSemaphores(bool makePrivate, int port);
+extern void CreateSharedMemoryAndSemaphores(int port);
#endif /* IPC_H */
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 57d3107..50bf9f0 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -30,7 +30,7 @@ typedef struct PGShmemHeader /* standard header for all Postgres shmem */
{
int32 magic; /* magic # to identify Postgres segments */
#define PGShmemMagic 679834894
- pid_t creatorPID; /* PID of creating process */
+ pid_t creatorPID; /* PID of creating process (set but unread) */
Size totalsize; /* total size of segment */
Size freeoffset; /* offset to first free space */
dsm_handle dsm_control; /* ID of dynamic shared memory control seg */
@@ -81,8 +81,8 @@ extern void PGSharedMemoryReAttach(void);
extern void PGSharedMemoryNoReAttach(void);
#endif
-extern PGShmemHeader *PGSharedMemoryCreate(Size size, bool makePrivate,
- int port, PGShmemHeader **shim);
+extern PGShmemHeader *PGSharedMemoryCreate(Size size, int port,
+ PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 0634aef..8dfbab1 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -104,7 +104,8 @@ our @EXPORT = qw(
get_new_node
);
-our ($test_localhost, $test_pghost, $last_port_assigned, @all_nodes, $died);
+our ($use_tcp, $test_localhost, $test_pghost, $last_host_assigned,
+ $last_port_assigned, @all_nodes, $died);
# Windows path to virtual file system root
@@ -118,13 +119,14 @@ if ($Config{osname} eq 'msys')
INIT
{
- # PGHOST is set once and for all through a single series of tests when
- # this module is loaded.
- $test_localhost = "127.0.0.1";
- $test_pghost =
- $TestLib::windows_os ? $test_localhost : TestLib::tempdir_short;
- $ENV{PGHOST} = $test_pghost;
- $ENV{PGDATABASE} = 'postgres';
+ # Set PGHOST for backward compatibility. This doesn't work for own_host
+ # nodes, so prefer to not rely on this when writing new tests.
+ $use_tcp = $TestLib::windows_os;
+ $test_localhost = "127.0.0.1";
+ $last_host_assigned = 1;
+ $test_pghost = $use_tcp ? $test_localhost : TestLib::tempdir_short;
+ $ENV{PGHOST} = $test_pghost;
+ $ENV{PGDATABASE} = 'postgres';
# Tracking of last port value assigned to accelerate free port lookup.
$last_port_assigned = int(rand() * 16384) + 49152;
@@ -155,7 +157,9 @@ sub new
_host => $pghost,
_basedir => "$TestLib::tmp_check/t_${testname}_${name}_data",
_name => $name,
- _logfile => "$TestLib::log_path/${testname}_${name}.log"
+ _logfile_generation => 0,
+ _logfile_base => "$TestLib::log_path/${testname}_${name}",
+ _logfile => "$TestLib::log_path/${testname}_${name}.log"
};
bless $self, $class;
@@ -473,8 +477,9 @@ sub init
print $conf "max_wal_senders = 0\n";
}
- if ($TestLib::windows_os)
+ if ($use_tcp)
{
+ print $conf "unix_socket_directories = ''\n";
print $conf "listen_addresses = '$host'\n";
}
else
@@ -536,12 +541,11 @@ sub backup
{
my ($self, $backup_name) = @_;
my $backup_path = $self->backup_dir . '/' . $backup_name;
- my $port = $self->port;
my $name = $self->name;
print "# Taking pg_basebackup $backup_name from node \"$name\"\n";
- TestLib::system_or_bail('pg_basebackup', '-D', $backup_path, '-p', $port,
- '--no-sync');
+ TestLib::system_or_bail('pg_basebackup', '-D', $backup_path, '-h',
+ $self->host, '-p', $self->port, '--no-sync');
print "# Backup finished\n";
return;
}
@@ -651,6 +655,7 @@ sub init_from_backup
{
my ($self, $root_node, $backup_name, %params) = @_;
my $backup_path = $root_node->backup_dir . '/' . $backup_name;
+ my $host = $self->host;
my $port = $self->port;
my $node_name = $self->name;
my $root_name = $root_node->name;
@@ -677,6 +682,15 @@ sub init_from_backup
qq(
port = $port
));
+ if ($use_tcp)
+ {
+ $self->append_conf('postgresql.conf', "listen_addresses = '$host'");
+ }
+ else
+ {
+ $self->append_conf('postgresql.conf',
+ "unix_socket_directories = '$host'");
+ }
$self->enable_streaming($root_node) if $params{has_streaming};
$self->enable_restoring($root_node) if $params{has_restoring};
return;
@@ -684,17 +698,45 @@ port = $port
=pod
-=item $node->start()
+=item $node->rotate_logfile()
+
+Switch to a new PostgreSQL log file. This does not alter any running
+PostgreSQL process. Subsequent method calls, including pg_ctl invocations,
+will use the new name. Return the new name.
+
+=cut
+
+sub rotate_logfile
+{
+ my ($self) = @_;
+ $self->{_logfile} = sprintf('%s_%d.log',
+ $self->{_logfile_base},
+ ++$self->{_logfile_generation});
+ return $self->{_logfile};
+}
+
+=pod
+
+=item $node->start(%params) => success_or_failure
Wrapper for pg_ctl start
Start the node and wait until it is ready to accept connections.
+=over
+
+=item fail_ok => 1
+
+By default, failure terminates the entire F<prove> invocation. If given,
+instead return a true or false value to indicate success or failure.
+
+=back
+
=cut
sub start
{
- my ($self) = @_;
+ my ($self, %params) = @_;
my $port = $self->port;
my $pgdata = $self->data_dir;
my $name = $self->name;
@@ -709,10 +751,34 @@ sub start
{
print "# pg_ctl start failed; logfile:\n";
print TestLib::slurp_file($self->logfile);
- BAIL_OUT("pg_ctl start failed");
+ BAIL_OUT("pg_ctl start failed") unless $params{fail_ok};
+ return 0;
}
$self->_update_pid(1);
+ return 1;
+}
+
+=pod
+
+=item $node->kill9()
+
+Send SIGKILL (signal 9) to the postmaster.
+
+Note: if the node is already known stopped, this does nothing.
+However, if we think it's running and it's not, it's important for
+this to fail. Otherwise, tests might fail to detect server crashes.
+
+=cut
+
+sub kill9
+{
+ my ($self) = @_;
+ my $name = $self->name;
+ return unless defined $self->{_pid};
+ print "### Killing node \"$name\" using signal 9\n";
+ kill(9, $self->{_pid}) or BAIL_OUT("kill(9, $self->{_pid}) failed");
+ $self->{_pid} = undef;
return;
}
@@ -945,7 +1011,7 @@ sub _update_pid
=pod
-=item PostgresNode->get_new_node(node_name)
+=item PostgresNode->get_new_node(node_name, %params)
Build a new object of class C<PostgresNode> (or of a subclass, if you have
one), assigning a free port number. Remembers the node, to prevent its port
@@ -954,6 +1020,22 @@ shut down when the test script exits.
You should generally use this instead of C<PostgresNode::new(...)>.
+=over
+
+=item port => [1,65535]
+
+By default, this function assigns a port number to each node. Specify this to
+force a particular port number. The caller is responsible for evaluating
+potential conflicts and privilege requirements.
+
+=item own_host => 1
+
+By default, all nodes use the same PGHOST value. If specified, generate a
+PGHOST specific to this node. This allows multiple nodes to use the same
+port.
+
+=back
+
For backwards compatibility, it is also exported as a standalone function,
which can only create objects of class C<PostgresNode>.
@@ -962,10 +1044,11 @@ which can only create objects of class C<PostgresNode>.
sub get_new_node
{
my $class = 'PostgresNode';
- $class = shift if 1 < scalar @_;
- my $name = shift;
- my $found = 0;
- my $port = $last_port_assigned;
+ $class = shift if scalar(@_) % 2 != 1;
+ my ($name, %params) = @_;
+ my $port_is_forced = defined $params{port};
+ my $found = $port_is_forced;
+ my $port = $port_is_forced ? $params{port} : $last_port_assigned;
while ($found == 0)
{
@@ -982,13 +1065,15 @@ sub get_new_node
$found = 0 if ($node->port == $port);
}
- # Check to see if anything else is listening on this TCP port.
- # This is *necessary* on Windows, and seems like a good idea
- # on Unixen as well, even though we don't ask the postmaster
- # to open a TCP port on Unix.
+ # Check to see if anything else is listening on this TCP port. Accept
+ # only ports available for all possible listen_addresses values, so
+ # the caller can harness this port for the widest range of purposes.
+ # This is *necessary* on Windows, and seems like a good idea on Unixen
+ # as well, even though we don't ask the postmaster to open a TCP port
+ # on Unix.
if ($found == 1)
{
- my $iaddr = inet_aton($test_localhost);
+ my $iaddr = inet_aton('0.0.0.0');
my $paddr = sockaddr_in($port, $iaddr);
my $proto = getprotobyname("tcp");
@@ -1004,16 +1089,35 @@ sub get_new_node
}
}
- print "# Found free port $port\n";
+ print "# Found port $port\n";
+
+ # Select a host.
+ my $host = $test_pghost;
+ if ($params{own_host})
+ {
+ if ($use_tcp)
+ {
+ # This assumes $use_tcp platforms treat every address in
+ # 127.0.0.1/24, not just 127.0.0.1, as a usable loopback.
+ $last_host_assigned++;
+ $last_host_assigned > 254 and BAIL_OUT("too many own_host nodes");
+ $host = '127.0.0.' . $last_host_assigned;
+ }
+ else
+ {
+ $host = "$test_pghost/$name"; # Assume $name =~ /^[-_a-zA-Z0-9]+$/
+ mkdir $host;
+ }
+ }
# Lock port number found by creating a new node
- my $node = $class->new($name, $test_pghost, $port);
+ my $node = $class->new($name, $host, $port);
# Add node to list of nodes
push(@all_nodes, $node);
# And update port for next time
- $last_port_assigned = $port;
+ $port_is_forced or $last_port_assigned = $port;
return $node;
}
@@ -1404,9 +1508,8 @@ $stderr);
=item $node->command_ok(...)
-Runs a shell command like TestLib::command_ok, but with PGPORT
-set so that the command will default to connecting to this
-PostgresNode.
+Runs a shell command like TestLib::command_ok, but with PGHOST and PGPORT set
+so that the command will default to connecting to this PostgresNode.
=cut
@@ -1416,6 +1519,7 @@ sub command_ok
my $self = shift;
+ local $ENV{PGHOST} = $self->host;
local $ENV{PGPORT} = $self->port;
TestLib::command_ok(@_);
@@ -1426,7 +1530,7 @@ sub command_ok
=item $node->command_fails(...)
-TestLib::command_fails with our PGPORT. See command_ok(...)
+TestLib::command_fails with our connection parameters. See command_ok(...)
=cut
@@ -1436,6 +1540,7 @@ sub command_fails
my $self = shift;
+ local $ENV{PGHOST} = $self->host;
local $ENV{PGPORT} = $self->port;
TestLib::command_fails(@_);
@@ -1446,7 +1551,7 @@ sub command_fails
=item $node->command_like(...)
-TestLib::command_like with our PGPORT. See command_ok(...)
+TestLib::command_like with our connection parameters. See command_ok(...)
=cut
@@ -1456,6 +1561,7 @@ sub command_like
my $self = shift;
+ local $ENV{PGHOST} = $self->host;
local $ENV{PGPORT} = $self->port;
TestLib::command_like(@_);
@@ -1466,7 +1572,8 @@ sub command_like
=item $node->command_checks_all(...)
-TestLib::command_checks_all with our PGPORT. See command_ok(...)
+TestLib::command_checks_all with our connection parameters. See
+command_ok(...)
=cut
@@ -1476,6 +1583,7 @@ sub command_checks_all
my $self = shift;
+ local $ENV{PGHOST} = $self->host;
local $ENV{PGPORT} = $self->port;
TestLib::command_checks_all(@_);
@@ -1500,6 +1608,7 @@ sub issues_sql_like
my ($self, $cmd, $expected_sql, $test_name) = @_;
+ local $ENV{PGHOST} = $self->host;
local $ENV{PGPORT} = $self->port;
truncate $self->logfile, 0;
@@ -1514,8 +1623,8 @@ sub issues_sql_like
=item $node->run_log(...)
-Runs a shell command like TestLib::run_log, but with PGPORT set so
-that the command will default to connecting to this PostgresNode.
+Runs a shell command like TestLib::run_log, but with connection parameters set
+so that the command will default to connecting to this PostgresNode.
=cut
@@ -1523,6 +1632,7 @@ sub run_log
{
my $self = shift;
+ local $ENV{PGHOST} = $self->host;
local $ENV{PGPORT} = $self->port;
TestLib::run_log(@_);
diff --git a/src/test/recovery/t/016_shm.pl b/src/test/recovery/t/016_shm.pl
new file mode 100644
index 0000000..9021562
--- /dev/null
+++ b/src/test/recovery/t/016_shm.pl
@@ -0,0 +1,154 @@
+#
+# Tests of pg_shmem.h functions
+#
+use strict;
+use warnings;
+use IPC::Run 'run';
+use PostgresNode;
+use Test::More;
+use TestLib;
+
+plan tests => 6;
+
+my $tempdir = TestLib::tempdir;
+my $port;
+
+# When using Unix sockets, we can dictate the port number. In the absence of
+# collisions from other shmget() activity, gnat starts with key 0x7d001
+# (512001), and flea starts with key 0x7d002 (512002).
+$port = 512 unless $PostgresNode::use_tcp;
+
+# Log "ipcs" diffs on a best-effort basis, swallowing any error.
+my $ipcs_before = "$tempdir/ipcs_before";
+eval { run_log [ 'ipcs', '-am' ], '>', $ipcs_before; };
+
+sub log_ipcs
+{
+ eval { run_log [ 'ipcs', '-am' ], '|', [ 'diff', $ipcs_before, '-' ] };
+ return;
+}
+
+# Node setup.
+sub init_start
+{
+ my $name = shift;
+ my $ret = PostgresNode->get_new_node($name, port => $port, own_host => 1);
+ defined($port) or $port = $ret->port; # same port for all nodes
+ $ret->init;
+ $ret->start;
+ log_ipcs();
+ return $ret;
+}
+my $gnat = init_start 'gnat';
+my $flea = init_start 'flea';
+
+# Upon postmaster death, postmaster children exit automatically.
+$gnat->kill9;
+log_ipcs();
+$flea->restart; # flea ignores the shm key gnat abandoned.
+log_ipcs();
+poll_start($gnat); # gnat recycles its former shm key.
+log_ipcs();
+
+# After clean shutdown, the nodes swap shm keys.
+$gnat->stop;
+$flea->restart;
+log_ipcs();
+$gnat->start;
+log_ipcs();
+
+# Scenarios involving no postmaster.pid, dead postmaster, and a live backend.
+# Use a regress.c function to emulate the responsiveness of a backend working
+# through a CPU-intensive task.
+$gnat->safe_psql('postgres', <<EOSQL);
+CREATE FUNCTION wait_pid(int)
+ RETURNS void
+ AS '$ENV{REGRESS_SHLIB}'
+ LANGUAGE C STRICT;
+EOSQL
+my $slow_query = 'SELECT wait_pid(pg_backend_pid())';
+my ($stdout, $stderr);
+my $slow_client = IPC::Run::start(
+ [
+ 'psql', '-X', '-qAt', '-d', $gnat->connstr('postgres'),
+ '-c', $slow_query
+ ],
+ '<',
+ \undef,
+ '>',
+ \$stdout,
+ '2>',
+ \$stderr,
+ IPC::Run::timeout(900)); # five times the poll_query_until timeout
+ok( $gnat->poll_query_until(
+ 'postgres',
+ "SELECT 1 FROM pg_stat_activity WHERE query = '$slow_query'", '1'),
+ 'slow query started');
+my $slow_pid = $gnat->safe_psql('postgres',
+ "SELECT pid FROM pg_stat_activity WHERE query = '$slow_query'");
+$gnat->kill9;
+unlink($gnat->data_dir . '/postmaster.pid');
+$gnat->rotate_logfile; # on Windows, can't open old log for writing
+log_ipcs();
+# Reject ordinary startup.
+ok(!$gnat->start(fail_ok => 1), 'live query blocks restart');
+like(
+ slurp_file($gnat->logfile),
+ qr/pre-existing shared memory block/,
+ 'detected live backend via shared memory');
+# Reject single-user startup.
+my $single_stderr;
+ok( !run_log(
+ [ 'postgres', '--single', '-D', $gnat->data_dir, 'template1' ],
+ '<', \('SELECT 1 + 1'), '2>', \$single_stderr),
+ 'live query blocks --single');
+print STDERR $single_stderr;
+like(
+ $single_stderr,
+ qr/pre-existing shared memory block/,
+ 'single-user mode detected live backend via shared memory');
+log_ipcs();
+# Fail to reject startup if shm key N has become available and we crash while
+# using key N+1. This is unwanted, but expected. Windows is immune, because
+# its GetSharedMemName() use DataDir strings, not numeric keys.
+$flea->stop; # release first key
+is( $gnat->start(fail_ok => 1),
+ $TestLib::windows_os ? 0 : 1,
+ 'key turnover fools only sysv_shmem.c');
+$gnat->stop; # release first key (no-op on $TestLib::windows_os)
+$flea->start; # grab first key
+# cleanup
+TestLib::system_log('pg_ctl', 'kill', 'QUIT', $slow_pid);
+$slow_client->finish; # client has detected backend termination
+log_ipcs();
+poll_start($gnat); # recycle second key
+
+$gnat->stop;
+$flea->stop;
+log_ipcs();
+
+
+# When postmaster children are slow to exit after postmaster death, we may
+# need retries to start a new postmaster.
+sub poll_start
+{
+ my ($node) = @_;
+
+ my $max_attempts = 180 * 10;
+ my $attempts = 0;
+
+ while ($attempts < $max_attempts)
+ {
+ $node->start(fail_ok => 1) && return 1;
+
+ # Wait 0.1 second before retrying.
+ usleep(100_000);
+
+ $attempts++;
+ }
+
+ # No success within 180 seconds. Try one last time without fail_ok, which
+ # will BAIL_OUT unless it succeeds.
+ $node->start && return 1;
+ return 0;
+}
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index d0e8c68..2aa29ab 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -205,6 +205,7 @@ sub tap_check
local %ENV = %ENV;
$ENV{PERL5LIB} = "$topdir/src/test/perl;$ENV{PERL5LIB}";
$ENV{PG_REGRESS} = "$topdir/$Config/pg_regress/pg_regress";
+ $ENV{REGRESS_SHLIB} = "$topdir/src/test/regress/regress.dll";
$ENV{TESTDIR} = "$dir";
On Saturday, March 9, 2019 8:16 AM, Noah Misch <noah@leadboat.com> wrote:
This patch is not really in my wheelhouse, so I might very well be testing it
in the wrong way, but whats the fun in staying in shallow end. Below is my
attempt at reviewing this patchset.
Both patches applies with a little bit of fuzz, and passes check-world. No new
perlcritic warnings are introduced, but 016_shm.pl needs to be renamed to 017
since commit b0825d28ea83e44139bd319e6d1db2c499c claimed 016. They absolutely
seem to do what they on the tin, and to my understanding this seems like an
improvement we definitely want.
I renamed IpcMemoryAnalyze() to PGSharedMemoryAttach() and deleted the old
function of that name. Now, this function never calls shmdt(); the caller is
responsible for that. I do like things better this way. What do you think?
I think it makes for a good API for the caller to be responsible, but it does
warrant a comment on the function to explicitly state that.
A few other small comments:
+ state = PGSharedMemoryAttach((IpcMemoryId) id2, &memAddress);
+ if (memAddress)
+ shmdt(memAddress);
This seems like a case where it would be useful to log a shmdt() error or do
an Assert() around the success of the operation perhaps?
+ * Loop till we find a free IPC key. Trust CreateDataDirLockFile() to
+ * ensure no more than one postmaster per data directory can enter this
+ * loop simultaneously. (CreateDataDirLockFile() does not ensure that,
+ * but prefer fixing it over coping here.)
This comment make it seem like there is a fix to CreateLockFile() missing to
his patch, is that correct? If so, do you have an idea for that patch?
I tested on Red Hat and on Windows Server 2016; I won't be shocked
if the test (not the code under test) breaks on other Windows configurations.
IIRC there are Windows versions where Win32::Process::KillProcess is required
for this, but thats admittedly somewhat dated knowledge. If the buildfarm goes
red on older Windows animals it might be something to look at perhaps.
Switching this to Ready for Committer since I can't see anything but tiny things.
cheers ./daniel
On Fri, Mar 29, 2019 at 09:53:51AM +0000, Daniel Gustafsson wrote:
On Saturday, March 9, 2019 8:16 AM, Noah Misch <noah@leadboat.com> wrote:
I renamed IpcMemoryAnalyze() to PGSharedMemoryAttach() and deleted the old
function of that name. Now, this function never calls shmdt(); the caller is
responsible for that. I do like things better this way. What do you think?I think it makes for a good API for the caller to be responsible, but it does
warrant a comment on the function to explicitly state that.
The name "PGSharedMemoryAttach" makes that fact sufficiently obvious, I think.
A few other small comments:
+ state = PGSharedMemoryAttach((IpcMemoryId) id2, &memAddress); + if (memAddress) + shmdt(memAddress);This seems like a case where it would be useful to log a shmdt() error or do
an Assert() around the success of the operation perhaps?
I'll add the same elog(LOG) we have at other shmdt() sites. I can't think of
a site where we Assert() about the results of a system call. While shmdt()
might be a justified exception, elog(LOG) seems reasonable.
+ * Loop till we find a free IPC key. Trust CreateDataDirLockFile() to + * ensure no more than one postmaster per data directory can enter this + * loop simultaneously. (CreateDataDirLockFile() does not ensure that, + * but prefer fixing it over coping here.)This comment make it seem like there is a fix to CreateLockFile() missing to
his patch, is that correct? If so, do you have an idea for that patch?
That comment refers to
/messages/by-id/flat/20120803145635.GE9683@tornado.leadboat.com
Switching this to Ready for Committer since I can't see anything but tiny things.
Thanks.
On Monday, April 1, 2019 12:42 AM, Noah Misch <noah@leadboat.com> wrote:
On Fri, Mar 29, 2019 at 09:53:51AM +0000, Daniel Gustafsson wrote:
This seems like a case where it would be useful to log a shmdt() error or do
an Assert() around the success of the operation perhaps?I'll add the same elog(LOG) we have at other shmdt() sites. I can't think of
a site where we Assert() about the results of a system call. While shmdt()
might be a justified exception, elog(LOG) seems reasonable.
Agreed, seems reasonable.
- - Loop till we find a free IPC key. Trust CreateDataDirLockFile() to
- - ensure no more than one postmaster per data directory can enter this
- - loop simultaneously. (CreateDataDirLockFile() does not ensure that,
- - but prefer fixing it over coping here.)This comment make it seem like there is a fix to CreateLockFile() missing to
his patch, is that correct? If so, do you have an idea for that patch?That comment refers to
/messages/by-id/flat/20120803145635.GE9683@tornado.leadboat.com
Gotcha, thanks for the clarification.
cheers ./daniel
On Mon, Apr 01, 2019 at 08:19:56AM +0000, Daniel Gustafsson wrote:
On Monday, April 1, 2019 12:42 AM, Noah Misch <noah@leadboat.com> wrote:
On Fri, Mar 29, 2019 at 09:53:51AM +0000, Daniel Gustafsson wrote:
This seems like a case where it would be useful to log a shmdt() error or do
an Assert() around the success of the operation perhaps?I'll add the same elog(LOG) we have at other shmdt() sites. I can't think of
a site where we Assert() about the results of a system call. While shmdt()
might be a justified exception, elog(LOG) seems reasonable.Agreed, seems reasonable.
Pushed, but that broke two buildfarm members:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=idiacanthus&dt=2019-04-04%2000%3A33%3A14
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=komodoensis&dt=2019-04-04%2000%3A33%3A13
I think the problem arose because these animals run on the same machine, and
their test execution was synchronized to the second. Two copies of the new
test ran concurrently. It doesn't tolerate that, owing to expectations about
which shared memory keys are in use. My initial thought is to fix this by
having a third postmaster that runs throughout the test and represents
ownership of a given port. If that postmaster gets something other than the
first shm key pertaining to its port, switch ports and try again.
I'll also include fixes for the warnings Andres reported on the
pgsql-committers thread.
On Wed, Apr 03, 2019 at 07:05:43PM -0700, Noah Misch wrote:
On Mon, Apr 01, 2019 at 08:19:56AM +0000, Daniel Gustafsson wrote:
On Monday, April 1, 2019 12:42 AM, Noah Misch <noah@leadboat.com> wrote:
On Fri, Mar 29, 2019 at 09:53:51AM +0000, Daniel Gustafsson wrote:
This seems like a case where it would be useful to log a shmdt() error or do
an Assert() around the success of the operation perhaps?I'll add the same elog(LOG) we have at other shmdt() sites. I can't think of
a site where we Assert() about the results of a system call. While shmdt()
might be a justified exception, elog(LOG) seems reasonable.Agreed, seems reasonable.
Pushed, but that broke two buildfarm members:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=idiacanthus&dt=2019-04-04%2000%3A33%3A14
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=komodoensis&dt=2019-04-04%2000%3A33%3A13I think the problem arose because these animals run on the same machine, and
their test execution was synchronized to the second. Two copies of the new
test ran concurrently. It doesn't tolerate that, owing to expectations about
which shared memory keys are in use. My initial thought is to fix this by
having a third postmaster that runs throughout the test and represents
ownership of a given port. If that postmaster gets something other than the
first shm key pertaining to its port, switch ports and try again.I'll also include fixes for the warnings Andres reported on the
pgsql-committers thread.
This thread's 2019-04-03 patches still break buildfarm members in multiple
ways. I plan to revert them. I'll wait a day or two before doing that, in
case more failure types show up.
On Thu, Apr 04, 2019 at 07:53:19AM -0700, Noah Misch wrote:
On Wed, Apr 03, 2019 at 07:05:43PM -0700, Noah Misch wrote:
Pushed, but that broke two buildfarm members:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=idiacanthus&dt=2019-04-04%2000%3A33%3A14
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=komodoensis&dt=2019-04-04%2000%3A33%3A13I think the problem arose because these animals run on the same machine, and
their test execution was synchronized to the second. Two copies of the new
test ran concurrently. It doesn't tolerate that, owing to expectations about
which shared memory keys are in use. My initial thought is to fix this by
having a third postmaster that runs throughout the test and represents
ownership of a given port. If that postmaster gets something other than the
first shm key pertaining to its port, switch ports and try again.I'll also include fixes for the warnings Andres reported on the
pgsql-committers thread.This thread's 2019-04-03 patches still break buildfarm members in multiple
ways. I plan to revert them. I'll wait a day or two before doing that, in
case more failure types show up.
Notable classes of buildfarm failure:
- AIX animals failed two ways. First, I missed a "use" statement such that
poll_start() would fail if it needed more than one attempt. Second, I
assumed $pid would be gone as soon as kill(9, $pid) returned[1]POSIX says "sig or at least one pending unblocked signal shall be delivered to the sending thread before kill() returns." I doubt the postmaster had another signal pending often enough to explain the failures, so AIX probably doesn't follow POSIX in this respect..
- komodoensis and idiacanthus failed due to 16ee6ea not fully resolving the
problems with concurrent execution. I reproduced the various concurrency
bugs by setting up four vpath build trees and looping the one test in each:
for dir in 0 1 2 3; do (until [ -f /tmp/stopprove ]; do make -C $dir/src/test/recovery installcheck PROVE_TESTS=t/017_shm.pl; done) & done; wait; rm /tmp/stopprove
- elver failed due to semaphore exhaustion. I'm reducing max_connections.
- lorikeet's FailedAssertion("!(vmq->mq_sender == ((void *)0))" looked
suspicious, but this happened six other times in the past year[2]All examples in the last year: sysname │ stage │ branch │ snapshot │ url ──────────┼────────────────┼───────────────┼─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────── lorikeet │ InstallCheck-C │ REL_10_STABLE │ 2018-05-04 09:49:55 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=lorikeet&dt=2018-05-04%2009:49:55 lorikeet │ InstallCheck-C │ REL_10_STABLE │ 2018-05-05 13:15:24 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=lorikeet&dt=2018-05-05%2013:15:24 lorikeet │ InstallCheck-C │ REL_10_STABLE │ 2018-05-06 09:33:35 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=lorikeet&dt=2018-05-06%2009:33:35 lorikeet │ InstallCheck-C │ REL_10_STABLE │ 2018-05-15 20:52:36 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=lorikeet&dt=2018-05-15%2020:52:36 lorikeet │ Check │ REL_10_STABLE │ 2019-02-20 10:40:40 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=lorikeet&dt=2019-02-20%2010:40:40 lorikeet │ InstallCheck-C │ REL_10_STABLE │ 2019-03-06 09:31:24 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=lorikeet&dt=2019-03-06%2009:31:24 lorikeet │ Check │ REL_10_STABLE │ 2019-04-04 09:47:02 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=lorikeet&dt=2019-04-04%2009:47:02, always on
v10 lorikeet.
- Commit 0aa0ccf, a wrong back-patch, saw 100% failure of the new test.
While it didn't cause a buildfarm failure, I'm changing the non-test code to
treat shmat() EACCESS as SHMSTATE_FOREIGN, so we ignore that key and move to
another. In the previous version, I treated it as SHMSTATE_ANALYSIS_FAILURE
and blocked startup. In HEAD today, shmat() failure blocks startup if and
only if we got the shmid from postmaster.pid; there's no distinction between
EACCES and other causes.
Attached v4 fixes all the above. I've also attached the incremental diff
versus the code I reverted.
[1]: POSIX says "sig or at least one pending unblocked signal shall be delivered to the sending thread before kill() returns." I doubt the postmaster had another signal pending often enough to explain the failures, so AIX probably doesn't follow POSIX in this respect.
delivered to the sending thread before kill() returns." I doubt the
postmaster had another signal pending often enough to explain the failures, so
AIX probably doesn't follow POSIX in this respect.
[2]: All examples in the last year: sysname │ stage │ branch │ snapshot │ url ──────────┼────────────────┼───────────────┼─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────── lorikeet │ InstallCheck-C │ REL_10_STABLE │ 2018-05-04 09:49:55 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=lorikeet&dt=2018-05-04%2009:49:55 lorikeet │ InstallCheck-C │ REL_10_STABLE │ 2018-05-05 13:15:24 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=lorikeet&dt=2018-05-05%2013:15:24 lorikeet │ InstallCheck-C │ REL_10_STABLE │ 2018-05-06 09:33:35 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=lorikeet&dt=2018-05-06%2009:33:35 lorikeet │ InstallCheck-C │ REL_10_STABLE │ 2018-05-15 20:52:36 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=lorikeet&dt=2018-05-15%2020:52:36 lorikeet │ Check │ REL_10_STABLE │ 2019-02-20 10:40:40 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=lorikeet&dt=2019-02-20%2010:40:40 lorikeet │ InstallCheck-C │ REL_10_STABLE │ 2019-03-06 09:31:24 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=lorikeet&dt=2019-03-06%2009:31:24 lorikeet │ Check │ REL_10_STABLE │ 2019-04-04 09:47:02 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=lorikeet&dt=2019-04-04%2009:47:02
sysname │ stage │ branch │ snapshot │ url
──────────┼────────────────┼───────────────┼─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────
lorikeet │ InstallCheck-C │ REL_10_STABLE │ 2018-05-04 09:49:55 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=lorikeet&dt=2018-05-04%2009:49:55
lorikeet │ InstallCheck-C │ REL_10_STABLE │ 2018-05-05 13:15:24 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=lorikeet&dt=2018-05-05%2013:15:24
lorikeet │ InstallCheck-C │ REL_10_STABLE │ 2018-05-06 09:33:35 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=lorikeet&dt=2018-05-06%2009:33:35
lorikeet │ InstallCheck-C │ REL_10_STABLE │ 2018-05-15 20:52:36 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=lorikeet&dt=2018-05-15%2020:52:36
lorikeet │ Check │ REL_10_STABLE │ 2019-02-20 10:40:40 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=lorikeet&dt=2019-02-20%2010:40:40
lorikeet │ InstallCheck-C │ REL_10_STABLE │ 2019-03-06 09:31:24 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=lorikeet&dt=2019-03-06%2009:31:24
lorikeet │ Check │ REL_10_STABLE │ 2019-04-04 09:47:02 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=lorikeet&dt=2019-04-04%2009:47:02
Attachments:
PGSharedMemoryCreate-safety-v4.patchtext/plain; charset=us-asciiDownload
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 6b98d53..b9d86ac 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -430,13 +430,13 @@ ifeq ($(enable_tap_tests),yes)
define prove_installcheck
rm -rf '$(CURDIR)'/tmp_check
$(MKDIR_P) '$(CURDIR)'/tmp_check
-cd $(srcdir) && TESTDIR='$(CURDIR)' PATH="$(bindir):$$PATH" PGPORT='6$(DEF_PGPORT)' top_builddir='$(CURDIR)/$(top_builddir)' PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
+cd $(srcdir) && TESTDIR='$(CURDIR)' PATH="$(bindir):$$PATH" PGPORT='6$(DEF_PGPORT)' top_builddir='$(CURDIR)/$(top_builddir)' PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' REGRESS_SHLIB='$(abs_top_builddir)/src/test/regress/regress$(DLSUFFIX)' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
endef
define prove_check
rm -rf '$(CURDIR)'/tmp_check
$(MKDIR_P) '$(CURDIR)'/tmp_check
-cd $(srcdir) && TESTDIR='$(CURDIR)' $(with_temp_install) PGPORT='6$(DEF_PGPORT)' PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
+cd $(srcdir) && TESTDIR='$(CURDIR)' $(with_temp_install) PGPORT='6$(DEF_PGPORT)' PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' REGRESS_SHLIB='$(abs_top_builddir)/src/test/regress/regress$(DLSUFFIX)' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
endef
else
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index 1511a61..a9d7bf9 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -72,6 +72,26 @@
typedef key_t IpcMemoryKey; /* shared memory key passed to shmget(2) */
typedef int IpcMemoryId; /* shared memory ID returned by shmget(2) */
+/*
+ * How does a given IpcMemoryId relate to this PostgreSQL process?
+ *
+ * One could recycle unattached segments of different data directories if we
+ * distinguished that case from other SHMSTATE_FOREIGN cases. Doing so would
+ * cause us to visit less of the key space, making us less likely to detect a
+ * SHMSTATE_ATTACHED key. It would also complicate the concurrency analysis,
+ * in that postmasters of different data directories could simultaneously
+ * attempt to recycle a given key. We'll waste keys longer in some cases, but
+ * avoiding the problems of the alternative justifies that loss.
+ */
+typedef enum
+{
+ SHMSTATE_ANALYSIS_FAILURE, /* unexpected failure to analyze the ID */
+ SHMSTATE_ATTACHED, /* pertinent to DataDir, has attached PIDs */
+ SHMSTATE_ENOENT, /* no segment of that ID */
+ SHMSTATE_FOREIGN, /* exists, but not pertinent to DataDir */
+ SHMSTATE_UNATTACHED /* pertinent to DataDir, no attached PIDs */
+} IpcMemoryState;
+
unsigned long UsedShmemSegID = 0;
void *UsedShmemSegAddr = NULL;
@@ -82,8 +102,8 @@ static void *AnonymousShmem = NULL;
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
static void IpcMemoryDelete(int status, Datum shmId);
-static PGShmemHeader *PGSharedMemoryAttach(IpcMemoryKey key,
- IpcMemoryId *shmid);
+static IpcMemoryState PGSharedMemoryAttach(IpcMemoryId shmId,
+ PGShmemHeader **addr);
/*
@@ -287,11 +307,36 @@ IpcMemoryDelete(int status, Datum shmId)
bool
PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2)
{
- IpcMemoryId shmId = (IpcMemoryId) id2;
+ PGShmemHeader *memAddress;
+ IpcMemoryState state;
+
+ state = PGSharedMemoryAttach((IpcMemoryId) id2, &memAddress);
+ if (memAddress && shmdt(memAddress) < 0)
+ elog(LOG, "shmdt(%p) failed: %m", memAddress);
+ switch (state)
+ {
+ case SHMSTATE_ENOENT:
+ case SHMSTATE_FOREIGN:
+ case SHMSTATE_UNATTACHED:
+ return false;
+ case SHMSTATE_ANALYSIS_FAILURE:
+ case SHMSTATE_ATTACHED:
+ return true;
+ }
+ return true;
+}
+
+/* See comment at IpcMemoryState. */
+static IpcMemoryState
+PGSharedMemoryAttach(IpcMemoryId shmId,
+ PGShmemHeader **addr)
+{
struct shmid_ds shmStat;
struct stat statbuf;
PGShmemHeader *hdr;
+ *addr = NULL;
+
/*
* We detect whether a shared memory segment is in use by seeing whether
* it (a) exists and (b) has any processes attached to it.
@@ -304,15 +349,15 @@ PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2)
* exists.
*/
if (errno == EINVAL)
- return false;
+ return SHMSTATE_ENOENT;
/*
- * EACCES implies that the segment belongs to some other userid, which
- * means it is not a Postgres shmem segment (or at least, not one that
- * is relevant to our data directory).
+ * EACCES implies we have no read permission, which means it is not a
+ * Postgres shmem segment (or at least, not one that is relevant to
+ * our data directory).
*/
if (errno == EACCES)
- return false;
+ return SHMSTATE_FOREIGN;
/*
* Some Linux kernel versions (in fact, all of them as of July 2007)
@@ -323,7 +368,7 @@ PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2)
*/
#ifdef HAVE_LINUX_EIDRM_BUG
if (errno == EIDRM)
- return false;
+ return SHMSTATE_ENOENT;
#endif
/*
@@ -331,25 +376,32 @@ PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2)
* only likely case is EIDRM, which implies that the segment has been
* IPC_RMID'd but there are still processes attached to it.
*/
- return true;
+ return SHMSTATE_ANALYSIS_FAILURE;
}
- /* If it has no attached processes, it's not in use */
- if (shmStat.shm_nattch == 0)
- return false;
-
/*
* Try to attach to the segment and see if it matches our data directory.
* This avoids shmid-conflict problems on machines that are running
* several postmasters under the same userid.
*/
if (stat(DataDir, &statbuf) < 0)
- return true; /* if can't stat, be conservative */
-
- hdr = (PGShmemHeader *) shmat(shmId, NULL, PG_SHMAT_FLAGS);
+ return SHMSTATE_ANALYSIS_FAILURE; /* can't stat; be conservative */
+ /*
+ * Attachment fails if we have no write permission. Since that will never
+ * happen with Postgres IPCProtection, such a failure shows the segment is
+ * not a Postgres segment. If attachment fails for some other reason, be
+ * conservative.
+ */
+ hdr = (PGShmemHeader *) shmat(shmId, UsedShmemSegAddr, PG_SHMAT_FLAGS);
if (hdr == (PGShmemHeader *) -1)
- return true; /* if can't attach, be conservative */
+ {
+ if (errno == EACCES)
+ return SHMSTATE_FOREIGN;
+ else
+ return SHMSTATE_ANALYSIS_FAILURE;
+ }
+ *addr = hdr;
if (hdr->magic != PGShmemMagic ||
hdr->device != statbuf.st_dev ||
@@ -357,16 +409,12 @@ PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2)
{
/*
* It's either not a Postgres segment, or not one for my data
- * directory. In either case it poses no threat.
+ * directory.
*/
- shmdt((void *) hdr);
- return false;
+ return SHMSTATE_FOREIGN;
}
- /* Trouble --- looks a lot like there's still live backends */
- shmdt((void *) hdr);
-
- return true;
+ return shmStat.shm_nattch == 0 ? SHMSTATE_UNATTACHED : SHMSTATE_ATTACHED;
}
#ifdef MAP_HUGETLB
@@ -538,25 +586,21 @@ AnonymousShmemDetach(int status, Datum arg)
* standard header. Also, register an on_shmem_exit callback to release
* the storage.
*
- * Dead Postgres segments are recycled if found, but we do not fail upon
- * collision with non-Postgres shmem segments. The idea here is to detect and
- * re-use keys that may have been assigned by a crashed postmaster or backend.
- *
- * makePrivate means to always create a new segment, rather than attach to
- * or recycle any existing segment.
+ * Dead Postgres segments pertinent to this DataDir are recycled if found, but
+ * we do not fail upon collision with foreign shmem segments. The idea here
+ * is to detect and re-use keys that may have been assigned by a crashed
+ * postmaster or backend.
*
* The port number is passed for possible use as a key (for SysV, we use
- * it to generate the starting shmem key). In a standalone backend,
- * zero will be passed.
+ * it to generate the starting shmem key).
*/
PGShmemHeader *
-PGSharedMemoryCreate(Size size, bool makePrivate, int port,
+PGSharedMemoryCreate(Size size, int port,
PGShmemHeader **shim)
{
IpcMemoryKey NextShmemSegID;
void *memAddress;
PGShmemHeader *hdr;
- IpcMemoryId shmid;
struct stat statbuf;
Size sysvsize;
@@ -588,11 +632,20 @@ PGSharedMemoryCreate(Size size, bool makePrivate, int port,
/* Make sure PGSharedMemoryAttach doesn't fail without need */
UsedShmemSegAddr = NULL;
- /* Loop till we find a free IPC key */
- NextShmemSegID = port * 1000;
+ /*
+ * Loop till we find a free IPC key. Trust CreateDataDirLockFile() to
+ * ensure no more than one postmaster per data directory can enter this
+ * loop simultaneously. (CreateDataDirLockFile() does not ensure that,
+ * but prefer fixing it over coping here.)
+ */
+ NextShmemSegID = 1 + port * 1000;
- for (NextShmemSegID++;; NextShmemSegID++)
+ for (;;)
{
+ IpcMemoryId shmid;
+ PGShmemHeader *oldhdr;
+ IpcMemoryState state;
+
/* Try to create new segment */
memAddress = InternalIpcMemoryCreate(NextShmemSegID, sysvsize);
if (memAddress)
@@ -600,58 +653,71 @@ PGSharedMemoryCreate(Size size, bool makePrivate, int port,
/* Check shared memory and possibly remove and recreate */
- if (makePrivate) /* a standalone backend shouldn't do this */
- continue;
-
- if ((memAddress = PGSharedMemoryAttach(NextShmemSegID, &shmid)) == NULL)
- continue; /* can't attach, not one of mine */
-
/*
- * If I am not the creator and it belongs to an extant process,
- * continue.
+ * shmget() failure is typically EACCES, hence SHMSTATE_FOREIGN.
+ * ENOENT, a narrow possibility, implies SHMSTATE_ENOENT, but one can
+ * safely treat SHMSTATE_ENOENT like SHMSTATE_FOREIGN.
*/
- hdr = (PGShmemHeader *) memAddress;
- if (hdr->creatorPID != getpid())
+ shmid = shmget(NextShmemSegID, sizeof(PGShmemHeader), 0);
+ if (shmid < 0)
{
- if (kill(hdr->creatorPID, 0) == 0 || errno != ESRCH)
- {
- shmdt(memAddress);
- continue; /* segment belongs to a live process */
- }
+ oldhdr = NULL;
+ state = SHMSTATE_FOREIGN;
}
+ else
+ state = PGSharedMemoryAttach(shmid, &oldhdr);
- /*
- * The segment appears to be from a dead Postgres process, or from a
- * previous cycle of life in this same process. Zap it, if possible,
- * and any associated dynamic shared memory segments, as well. This
- * probably shouldn't fail, but if it does, assume the segment belongs
- * to someone else after all, and continue quietly.
- */
- if (hdr->dsm_control != 0)
- dsm_cleanup_using_control_segment(hdr->dsm_control);
- shmdt(memAddress);
- if (shmctl(shmid, IPC_RMID, NULL) < 0)
- continue;
+ switch (state)
+ {
+ case SHMSTATE_ANALYSIS_FAILURE:
+ case SHMSTATE_ATTACHED:
+ ereport(FATAL,
+ (errcode(ERRCODE_LOCK_FILE_EXISTS),
+ errmsg("pre-existing shared memory block (key %lu, ID %lu) is still in use",
+ (unsigned long) NextShmemSegID,
+ (unsigned long) shmid),
+ errhint("Terminate any old server processes associated with data directory \"%s\".",
+ DataDir)));
+ break;
+ case SHMSTATE_ENOENT:
- /*
- * Now try again to create the segment.
- */
- memAddress = InternalIpcMemoryCreate(NextShmemSegID, sysvsize);
- if (memAddress)
- break; /* successful create and attach */
+ /*
+ * To our surprise, some other process deleted since our last
+ * InternalIpcMemoryCreate(). Moments earlier, we would have
+ * seen SHMSTATE_FOREIGN. Try that same ID again.
+ */
+ elog(LOG,
+ "shared memory block (key %lu, ID %lu) deleted during startup",
+ (unsigned long) NextShmemSegID,
+ (unsigned long) shmid);
+ break;
+ case SHMSTATE_FOREIGN:
+ NextShmemSegID++;
+ break;
+ case SHMSTATE_UNATTACHED:
- /*
- * Can only get here if some other process managed to create the same
- * shmem key before we did. Let him have that one, loop around to try
- * next key.
- */
+ /*
+ * The segment pertains to DataDir, and every process that had
+ * used it has died or detached. Zap it, if possible, and any
+ * associated dynamic shared memory segments, as well. This
+ * shouldn't fail, but if it does, assume the segment belongs
+ * to someone else after all, and try the next candidate.
+ * Otherwise, try again to create the segment. That may fail
+ * if some other process creates the same shmem key before we
+ * do, in which case we'll try the next key.
+ */
+ if (oldhdr->dsm_control != 0)
+ dsm_cleanup_using_control_segment(oldhdr->dsm_control);
+ if (shmctl(shmid, IPC_RMID, NULL) < 0)
+ NextShmemSegID++;
+ break;
+ }
+
+ if (oldhdr && shmdt(oldhdr) < 0)
+ elog(LOG, "shmdt(%p) failed: %m", oldhdr);
}
- /*
- * OK, we created a new segment. Mark it as created by this process. The
- * order of assignments here is critical so that another Postgres process
- * can't see the header as valid but belonging to an invalid PID!
- */
+ /* Initialize new segment. */
hdr = (PGShmemHeader *) memAddress;
hdr->creatorPID = getpid();
hdr->magic = PGShmemMagic;
@@ -707,7 +773,8 @@ void
PGSharedMemoryReAttach(void)
{
IpcMemoryId shmid;
- void *hdr;
+ PGShmemHeader *hdr;
+ IpcMemoryState state;
void *origUsedShmemSegAddr = UsedShmemSegAddr;
Assert(UsedShmemSegAddr != NULL);
@@ -720,14 +787,18 @@ PGSharedMemoryReAttach(void)
#endif
elog(DEBUG3, "attaching to %p", UsedShmemSegAddr);
- hdr = (void *) PGSharedMemoryAttach((IpcMemoryKey) UsedShmemSegID, &shmid);
- if (hdr == NULL)
+ shmid = shmget(UsedShmemSegID, sizeof(PGShmemHeader), 0);
+ if (shmid < 0)
+ state = SHMSTATE_FOREIGN;
+ else
+ state = PGSharedMemoryAttach(shmid, &hdr);
+ if (state != SHMSTATE_ATTACHED)
elog(FATAL, "could not reattach to shared memory (key=%d, addr=%p): %m",
(int) UsedShmemSegID, UsedShmemSegAddr);
if (hdr != origUsedShmemSegAddr)
elog(FATAL, "reattaching to shared memory returned unexpected address (got %p, expected %p)",
hdr, origUsedShmemSegAddr);
- dsm_set_control_handle(((PGShmemHeader *) hdr)->dsm_control);
+ dsm_set_control_handle(hdr->dsm_control);
UsedShmemSegAddr = hdr; /* probably redundant */
}
@@ -801,31 +872,3 @@ PGSharedMemoryDetach(void)
AnonymousShmem = NULL;
}
}
-
-
-/*
- * Attach to shared memory and make sure it has a Postgres header
- *
- * Returns attach address if OK, else NULL
- */
-static PGShmemHeader *
-PGSharedMemoryAttach(IpcMemoryKey key, IpcMemoryId *shmid)
-{
- PGShmemHeader *hdr;
-
- if ((*shmid = shmget(key, sizeof(PGShmemHeader), 0)) < 0)
- return NULL;
-
- hdr = (PGShmemHeader *) shmat(*shmid, UsedShmemSegAddr, PG_SHMAT_FLAGS);
-
- if (hdr == (PGShmemHeader *) -1)
- return NULL; /* failed: must be some other app's */
-
- if (hdr->magic != PGShmemMagic)
- {
- shmdt((void *) hdr);
- return NULL; /* segment belongs to a non-Postgres app */
- }
-
- return hdr;
-}
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index bc1e946..b583166 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -170,14 +170,9 @@ EnableLockPagesPrivilege(int elevel)
*
* Create a shared memory segment of the given size and initialize its
* standard header.
- *
- * makePrivate means to always create a new segment, rather than attach to
- * or recycle any existing segment. On win32, we always create a new segment,
- * since there is no need for recycling (segments go away automatically
- * when the last backend exits)
*/
PGShmemHeader *
-PGSharedMemoryCreate(Size size, bool makePrivate, int port,
+PGSharedMemoryCreate(Size size, int port,
PGShmemHeader **shim)
{
void *memAddress;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 067487f..fcc175d 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -2604,7 +2604,7 @@ reset_shared(int port)
* determine IPC keys. This helps ensure that we will clean up dead IPC
* objects if the postmaster crashes and is restarted.
*/
- CreateSharedMemoryAndSemaphores(false, port);
+ CreateSharedMemoryAndSemaphores(port);
}
@@ -4945,7 +4945,7 @@ SubPostmasterMain(int argc, char *argv[])
InitProcess();
/* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores(false, 0);
+ CreateSharedMemoryAndSemaphores(0);
/* And run the backend */
BackendRun(&port); /* does not return */
@@ -4959,7 +4959,7 @@ SubPostmasterMain(int argc, char *argv[])
InitAuxiliaryProcess();
/* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores(false, 0);
+ CreateSharedMemoryAndSemaphores(0);
AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
}
@@ -4972,7 +4972,7 @@ SubPostmasterMain(int argc, char *argv[])
InitProcess();
/* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores(false, 0);
+ CreateSharedMemoryAndSemaphores(0);
AutoVacLauncherMain(argc - 2, argv + 2); /* does not return */
}
@@ -4985,7 +4985,7 @@ SubPostmasterMain(int argc, char *argv[])
InitProcess();
/* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores(false, 0);
+ CreateSharedMemoryAndSemaphores(0);
AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
}
@@ -5003,7 +5003,7 @@ SubPostmasterMain(int argc, char *argv[])
InitProcess();
/* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores(false, 0);
+ CreateSharedMemoryAndSemaphores(0);
/* Fetch MyBgworkerEntry from shared memory */
shmem_slot = atoi(argv[1] + 15);
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 5965d36..d7d7335 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -89,12 +89,9 @@ RequestAddinShmemSpace(Size size)
* through the same code as before. (Note that the called routines mostly
* check IsUnderPostmaster, rather than EXEC_BACKEND, to detect this case.
* This is a bit code-wasteful and could be cleaned up.)
- *
- * If "makePrivate" is true then we only need private memory, not shared
- * memory. This is true for a standalone backend, false for a postmaster.
*/
void
-CreateSharedMemoryAndSemaphores(bool makePrivate, int port)
+CreateSharedMemoryAndSemaphores(int port)
{
PGShmemHeader *shim = NULL;
@@ -166,7 +163,7 @@ CreateSharedMemoryAndSemaphores(bool makePrivate, int port)
/*
* Create the shmem segment
*/
- seghdr = PGSharedMemoryCreate(size, makePrivate, port, &shim);
+ seghdr = PGSharedMemoryCreate(size, port, &shim);
InitShmemAccess(seghdr);
@@ -187,12 +184,9 @@ CreateSharedMemoryAndSemaphores(bool makePrivate, int port)
{
/*
* We are reattaching to an existing shared memory segment. This
- * should only be reached in the EXEC_BACKEND case, and even then only
- * with makePrivate == false.
+ * should only be reached in the EXEC_BACKEND case.
*/
-#ifdef EXEC_BACKEND
- Assert(!makePrivate);
-#else
+#ifndef EXEC_BACKEND
elog(PANIC, "should be attached to shared memory already");
#endif
}
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index 1c2a99c..e9f72b5 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -446,9 +446,11 @@ InitCommunication(void)
{
/*
* We're running a postgres bootstrap process or a standalone backend.
- * Create private "shmem" and semaphores.
+ * Though we won't listen on PostPortNumber, use it to select a shmem
+ * key. This increases the chance of detecting a leftover live
+ * backend of this DataDir.
*/
- CreateSharedMemoryAndSemaphores(true, 0);
+ CreateSharedMemoryAndSemaphores(PostPortNumber);
}
}
diff --git a/src/include/storage/ipc.h b/src/include/storage/ipc.h
index 0660252..e9b243f 100644
--- a/src/include/storage/ipc.h
+++ b/src/include/storage/ipc.h
@@ -76,6 +76,6 @@ extern void on_exit_reset(void);
/* ipci.c */
extern PGDLLIMPORT shmem_startup_hook_type shmem_startup_hook;
-extern void CreateSharedMemoryAndSemaphores(bool makePrivate, int port);
+extern void CreateSharedMemoryAndSemaphores(int port);
#endif /* IPC_H */
diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h
index 57d3107..50bf9f0 100644
--- a/src/include/storage/pg_shmem.h
+++ b/src/include/storage/pg_shmem.h
@@ -30,7 +30,7 @@ typedef struct PGShmemHeader /* standard header for all Postgres shmem */
{
int32 magic; /* magic # to identify Postgres segments */
#define PGShmemMagic 679834894
- pid_t creatorPID; /* PID of creating process */
+ pid_t creatorPID; /* PID of creating process (set but unread) */
Size totalsize; /* total size of segment */
Size freeoffset; /* offset to first free space */
dsm_handle dsm_control; /* ID of dynamic shared memory control seg */
@@ -81,8 +81,8 @@ extern void PGSharedMemoryReAttach(void);
extern void PGSharedMemoryNoReAttach(void);
#endif
-extern PGShmemHeader *PGSharedMemoryCreate(Size size, bool makePrivate,
- int port, PGShmemHeader **shim);
+extern PGShmemHeader *PGSharedMemoryCreate(Size size, int port,
+ PGShmemHeader **shim);
extern bool PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2);
extern void PGSharedMemoryDetach(void);
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 81deed9..f657063 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -104,7 +104,8 @@ our @EXPORT = qw(
get_new_node
);
-our ($test_localhost, $test_pghost, $last_port_assigned, @all_nodes, $died);
+our ($use_tcp, $test_localhost, $test_pghost, $last_host_assigned,
+ $last_port_assigned, @all_nodes, $died);
# Windows path to virtual file system root
@@ -118,13 +119,14 @@ if ($Config{osname} eq 'msys')
INIT
{
- # PGHOST is set once and for all through a single series of tests when
- # this module is loaded.
- $test_localhost = "127.0.0.1";
- $test_pghost =
- $TestLib::windows_os ? $test_localhost : TestLib::tempdir_short;
- $ENV{PGHOST} = $test_pghost;
- $ENV{PGDATABASE} = 'postgres';
+ # Set PGHOST for backward compatibility. This doesn't work for own_host
+ # nodes, so prefer to not rely on this when writing new tests.
+ $use_tcp = $TestLib::windows_os;
+ $test_localhost = "127.0.0.1";
+ $last_host_assigned = 1;
+ $test_pghost = $use_tcp ? $test_localhost : TestLib::tempdir_short;
+ $ENV{PGHOST} = $test_pghost;
+ $ENV{PGDATABASE} = 'postgres';
# Tracking of last port value assigned to accelerate free port lookup.
$last_port_assigned = int(rand() * 16384) + 49152;
@@ -155,7 +157,9 @@ sub new
_host => $pghost,
_basedir => "$TestLib::tmp_check/t_${testname}_${name}_data",
_name => $name,
- _logfile => "$TestLib::log_path/${testname}_${name}.log"
+ _logfile_generation => 0,
+ _logfile_base => "$TestLib::log_path/${testname}_${name}",
+ _logfile => "$TestLib::log_path/${testname}_${name}.log"
};
bless $self, $class;
@@ -473,8 +477,9 @@ sub init
print $conf "max_wal_senders = 0\n";
}
- if ($TestLib::windows_os)
+ if ($use_tcp)
{
+ print $conf "unix_socket_directories = ''\n";
print $conf "listen_addresses = '$host'\n";
}
else
@@ -536,12 +541,11 @@ sub backup
{
my ($self, $backup_name) = @_;
my $backup_path = $self->backup_dir . '/' . $backup_name;
- my $port = $self->port;
my $name = $self->name;
print "# Taking pg_basebackup $backup_name from node \"$name\"\n";
- TestLib::system_or_bail('pg_basebackup', '-D', $backup_path, '-p', $port,
- '--no-sync');
+ TestLib::system_or_bail('pg_basebackup', '-D', $backup_path, '-h',
+ $self->host, '-p', $self->port, '--no-sync');
print "# Backup finished\n";
return;
}
@@ -651,6 +655,7 @@ sub init_from_backup
{
my ($self, $root_node, $backup_name, %params) = @_;
my $backup_path = $root_node->backup_dir . '/' . $backup_name;
+ my $host = $self->host;
my $port = $self->port;
my $node_name = $self->name;
my $root_name = $root_node->name;
@@ -677,6 +682,15 @@ sub init_from_backup
qq(
port = $port
));
+ if ($use_tcp)
+ {
+ $self->append_conf('postgresql.conf', "listen_addresses = '$host'");
+ }
+ else
+ {
+ $self->append_conf('postgresql.conf',
+ "unix_socket_directories = '$host'");
+ }
$self->enable_streaming($root_node) if $params{has_streaming};
$self->enable_restoring($root_node) if $params{has_restoring};
return;
@@ -684,17 +698,45 @@ port = $port
=pod
-=item $node->start()
+=item $node->rotate_logfile()
+
+Switch to a new PostgreSQL log file. This does not alter any running
+PostgreSQL process. Subsequent method calls, including pg_ctl invocations,
+will use the new name. Return the new name.
+
+=cut
+
+sub rotate_logfile
+{
+ my ($self) = @_;
+ $self->{_logfile} = sprintf('%s_%d.log',
+ $self->{_logfile_base},
+ ++$self->{_logfile_generation});
+ return $self->{_logfile};
+}
+
+=pod
+
+=item $node->start(%params) => success_or_failure
Wrapper for pg_ctl start
Start the node and wait until it is ready to accept connections.
+=over
+
+=item fail_ok => 1
+
+By default, failure terminates the entire F<prove> invocation. If given,
+instead return a true or false value to indicate success or failure.
+
+=back
+
=cut
sub start
{
- my ($self) = @_;
+ my ($self, %params) = @_;
my $port = $self->port;
my $pgdata = $self->data_dir;
my $name = $self->name;
@@ -721,10 +763,34 @@ sub start
{
print "# pg_ctl start failed; logfile:\n";
print TestLib::slurp_file($self->logfile);
- BAIL_OUT("pg_ctl start failed");
+ BAIL_OUT("pg_ctl start failed") unless $params{fail_ok};
+ return 0;
}
$self->_update_pid(1);
+ return 1;
+}
+
+=pod
+
+=item $node->kill9()
+
+Send SIGKILL (signal 9) to the postmaster.
+
+Note: if the node is already known stopped, this does nothing.
+However, if we think it's running and it's not, it's important for
+this to fail. Otherwise, tests might fail to detect server crashes.
+
+=cut
+
+sub kill9
+{
+ my ($self) = @_;
+ my $name = $self->name;
+ return unless defined $self->{_pid};
+ print "### Killing node \"$name\" using signal 9\n";
+ kill(9, $self->{_pid}) or BAIL_OUT("kill(9, $self->{_pid}) failed");
+ $self->{_pid} = undef;
return;
}
@@ -965,7 +1031,7 @@ sub _update_pid
=pod
-=item PostgresNode->get_new_node(node_name)
+=item PostgresNode->get_new_node(node_name, %params)
Build a new object of class C<PostgresNode> (or of a subclass, if you have
one), assigning a free port number. Remembers the node, to prevent its port
@@ -974,6 +1040,22 @@ shut down when the test script exits.
You should generally use this instead of C<PostgresNode::new(...)>.
+=over
+
+=item port => [1,65535]
+
+By default, this function assigns a port number to each node. Specify this to
+force a particular port number. The caller is responsible for evaluating
+potential conflicts and privilege requirements.
+
+=item own_host => 1
+
+By default, all nodes use the same PGHOST value. If specified, generate a
+PGHOST specific to this node. This allows multiple nodes to use the same
+port.
+
+=back
+
For backwards compatibility, it is also exported as a standalone function,
which can only create objects of class C<PostgresNode>.
@@ -982,10 +1064,11 @@ which can only create objects of class C<PostgresNode>.
sub get_new_node
{
my $class = 'PostgresNode';
- $class = shift if 1 < scalar @_;
- my $name = shift;
- my $found = 0;
- my $port = $last_port_assigned;
+ $class = shift if scalar(@_) % 2 != 1;
+ my ($name, %params) = @_;
+ my $port_is_forced = defined $params{port};
+ my $found = $port_is_forced;
+ my $port = $port_is_forced ? $params{port} : $last_port_assigned;
while ($found == 0)
{
@@ -1002,13 +1085,15 @@ sub get_new_node
$found = 0 if ($node->port == $port);
}
- # Check to see if anything else is listening on this TCP port.
- # This is *necessary* on Windows, and seems like a good idea
- # on Unixen as well, even though we don't ask the postmaster
- # to open a TCP port on Unix.
+ # Check to see if anything else is listening on this TCP port. Accept
+ # only ports available for all possible listen_addresses values, so
+ # the caller can harness this port for the widest range of purposes.
+ # This is *necessary* on Windows, and seems like a good idea on Unixen
+ # as well, even though we don't ask the postmaster to open a TCP port
+ # on Unix.
if ($found == 1)
{
- my $iaddr = inet_aton($test_localhost);
+ my $iaddr = inet_aton('0.0.0.0');
my $paddr = sockaddr_in($port, $iaddr);
my $proto = getprotobyname("tcp");
@@ -1024,16 +1109,35 @@ sub get_new_node
}
}
- print "# Found free port $port\n";
+ print "# Found port $port\n";
+
+ # Select a host.
+ my $host = $test_pghost;
+ if ($params{own_host})
+ {
+ if ($use_tcp)
+ {
+ # This assumes $use_tcp platforms treat every address in
+ # 127.0.0.1/24, not just 127.0.0.1, as a usable loopback.
+ $last_host_assigned++;
+ $last_host_assigned > 254 and BAIL_OUT("too many own_host nodes");
+ $host = '127.0.0.' . $last_host_assigned;
+ }
+ else
+ {
+ $host = "$test_pghost/$name"; # Assume $name =~ /^[-_a-zA-Z0-9]+$/
+ mkdir $host;
+ }
+ }
# Lock port number found by creating a new node
- my $node = $class->new($name, $test_pghost, $port);
+ my $node = $class->new($name, $host, $port);
# Add node to list of nodes
push(@all_nodes, $node);
# And update port for next time
- $last_port_assigned = $port;
+ $port_is_forced or $last_port_assigned = $port;
return $node;
}
@@ -1424,9 +1528,8 @@ $stderr);
=item $node->command_ok(...)
-Runs a shell command like TestLib::command_ok, but with PGPORT
-set so that the command will default to connecting to this
-PostgresNode.
+Runs a shell command like TestLib::command_ok, but with PGHOST and PGPORT set
+so that the command will default to connecting to this PostgresNode.
=cut
@@ -1436,6 +1539,7 @@ sub command_ok
my $self = shift;
+ local $ENV{PGHOST} = $self->host;
local $ENV{PGPORT} = $self->port;
TestLib::command_ok(@_);
@@ -1446,7 +1550,7 @@ sub command_ok
=item $node->command_fails(...)
-TestLib::command_fails with our PGPORT. See command_ok(...)
+TestLib::command_fails with our connection parameters. See command_ok(...)
=cut
@@ -1456,6 +1560,7 @@ sub command_fails
my $self = shift;
+ local $ENV{PGHOST} = $self->host;
local $ENV{PGPORT} = $self->port;
TestLib::command_fails(@_);
@@ -1466,7 +1571,7 @@ sub command_fails
=item $node->command_like(...)
-TestLib::command_like with our PGPORT. See command_ok(...)
+TestLib::command_like with our connection parameters. See command_ok(...)
=cut
@@ -1476,6 +1581,7 @@ sub command_like
my $self = shift;
+ local $ENV{PGHOST} = $self->host;
local $ENV{PGPORT} = $self->port;
TestLib::command_like(@_);
@@ -1486,7 +1592,8 @@ sub command_like
=item $node->command_checks_all(...)
-TestLib::command_checks_all with our PGPORT. See command_ok(...)
+TestLib::command_checks_all with our connection parameters. See
+command_ok(...)
=cut
@@ -1496,6 +1603,7 @@ sub command_checks_all
my $self = shift;
+ local $ENV{PGHOST} = $self->host;
local $ENV{PGPORT} = $self->port;
TestLib::command_checks_all(@_);
@@ -1520,6 +1628,7 @@ sub issues_sql_like
my ($self, $cmd, $expected_sql, $test_name) = @_;
+ local $ENV{PGHOST} = $self->host;
local $ENV{PGPORT} = $self->port;
truncate $self->logfile, 0;
@@ -1534,8 +1643,8 @@ sub issues_sql_like
=item $node->run_log(...)
-Runs a shell command like TestLib::run_log, but with PGPORT set so
-that the command will default to connecting to this PostgresNode.
+Runs a shell command like TestLib::run_log, but with connection parameters set
+so that the command will default to connecting to this PostgresNode.
=cut
@@ -1543,6 +1652,7 @@ sub run_log
{
my $self = shift;
+ local $ENV{PGHOST} = $self->host;
local $ENV{PGPORT} = $self->port;
TestLib::run_log(@_);
diff --git a/src/test/recovery/t/017_shm.pl b/src/test/recovery/t/017_shm.pl
new file mode 100644
index 0000000..0969d4a
--- /dev/null
+++ b/src/test/recovery/t/017_shm.pl
@@ -0,0 +1,198 @@
+#
+# Tests of pg_shmem.h functions
+#
+use strict;
+use warnings;
+use IPC::Run 'run';
+use PostgresNode;
+use Test::More;
+use TestLib;
+use Time::HiRes qw(usleep);
+
+plan tests => 5;
+
+my $tempdir = TestLib::tempdir;
+my $port;
+
+# Log "ipcs" diffs on a best-effort basis, swallowing any error.
+my $ipcs_before = "$tempdir/ipcs_before";
+eval { run_log [ 'ipcs', '-am' ], '>', $ipcs_before; };
+
+sub log_ipcs
+{
+ eval { run_log [ 'ipcs', '-am' ], '|', [ 'diff', $ipcs_before, '-' ] };
+ return;
+}
+
+# These tests need a $port such that nothing creates or removes a segment in
+# $port's IpcMemoryKey range while this test script runs. While there's no
+# way to ensure that in general, we do ensure that if PostgreSQL tests are the
+# only actors. With TCP, the first get_new_node picks a port number. With
+# Unix sockets, use a postmaster, $port_holder, to represent a key space
+# reservation. $port_holder holds a reservation on the key space of port
+# 1+$port_holder->port if it created the first IpcMemoryKey of its own port's
+# key space. If multiple copies of this test script run concurrently, they
+# will pick different ports. $port_holder postmasters use odd-numbered ports,
+# and tests use even-numbered ports. In the absence of collisions from other
+# shmget() activity, gnat starts with key 0x7d001 (512001), and flea starts
+# with key 0x7d002 (512002).
+my $port_holder;
+if (!$PostgresNode::use_tcp)
+{
+ my $lock_port;
+ for ($lock_port = 511; $lock_port < 711; $lock_port += 2)
+ {
+ $port_holder = PostgresNode->get_new_node(
+ "port${lock_port}_holder",
+ port => $lock_port,
+ own_host => 1);
+ $port_holder->init;
+ $port_holder->append_conf('postgresql.conf', 'max_connections = 5');
+ $port_holder->start;
+ # Match the AddToDataDirLockFile() call in sysv_shmem.c. Assume all
+ # systems not using sysv_shmem.c do use TCP.
+ my $shmem_key_line_prefix = sprintf("%9lu ", 1 + $lock_port * 1000);
+ last
+ if slurp_file($port_holder->data_dir . '/postmaster.pid') =~
+ /^$shmem_key_line_prefix/m;
+ $port_holder->stop;
+ }
+ $port = $lock_port + 1;
+}
+
+# Node setup.
+sub init_start
+{
+ my $name = shift;
+ my $ret = PostgresNode->get_new_node($name, port => $port, own_host => 1);
+ defined($port) or $port = $ret->port; # same port for all nodes
+ $ret->init;
+ # Limit semaphore consumption, since we run several nodes concurrently.
+ $ret->append_conf('postgresql.conf', 'max_connections = 5');
+ $ret->start;
+ log_ipcs();
+ return $ret;
+}
+my $gnat = init_start 'gnat';
+my $flea = init_start 'flea';
+
+# Upon postmaster death, postmaster children exit automatically.
+$gnat->kill9;
+log_ipcs();
+$flea->restart; # flea ignores the shm key gnat abandoned.
+log_ipcs();
+poll_start($gnat); # gnat recycles its former shm key.
+log_ipcs();
+
+# After clean shutdown, the nodes swap shm keys.
+$gnat->stop;
+$flea->restart;
+log_ipcs();
+$gnat->start;
+log_ipcs();
+
+# Scenarios involving no postmaster.pid, dead postmaster, and a live backend.
+# Use a regress.c function to emulate the responsiveness of a backend working
+# through a CPU-intensive task.
+$gnat->safe_psql('postgres', <<EOSQL);
+CREATE FUNCTION wait_pid(int)
+ RETURNS void
+ AS '$ENV{REGRESS_SHLIB}'
+ LANGUAGE C STRICT;
+EOSQL
+my $slow_query = 'SELECT wait_pid(pg_backend_pid())';
+my ($stdout, $stderr);
+my $slow_client = IPC::Run::start(
+ [
+ 'psql', '-X', '-qAt', '-d', $gnat->connstr('postgres'),
+ '-c', $slow_query
+ ],
+ '<',
+ \undef,
+ '>',
+ \$stdout,
+ '2>',
+ \$stderr,
+ IPC::Run::timeout(900)); # five times the poll_query_until timeout
+ok( $gnat->poll_query_until(
+ 'postgres',
+ "SELECT 1 FROM pg_stat_activity WHERE query = '$slow_query'", '1'),
+ 'slow query started');
+my $slow_pid = $gnat->safe_psql('postgres',
+ "SELECT pid FROM pg_stat_activity WHERE query = '$slow_query'");
+$gnat->kill9;
+unlink($gnat->data_dir . '/postmaster.pid');
+$gnat->rotate_logfile; # on Windows, can't open old log for writing
+log_ipcs();
+# Reject ordinary startup. Retry for the same reasons poll_start() does.
+my $pre_existing_msg = qr/pre-existing shared memory block/;
+{
+ my $max_attempts = 180 * 10; # Retry every 0.1s for at least 180s.
+ my $attempts = 0;
+ while ($attempts < $max_attempts)
+ {
+ last
+ if $gnat->start(fail_ok => 1)
+ || slurp_file($gnat->logfile) =~ $pre_existing_msg;
+ usleep(100_000);
+ $attempts++;
+ }
+}
+like(slurp_file($gnat->logfile),
+ $pre_existing_msg, 'detected live backend via shared memory');
+# Reject single-user startup.
+my $single_stderr;
+ok( !run_log(
+ [ 'postgres', '--single', '-D', $gnat->data_dir, 'template1' ],
+ '<', \('SELECT 1 + 1'), '2>', \$single_stderr),
+ 'live query blocks --single');
+print STDERR $single_stderr;
+like($single_stderr, $pre_existing_msg,
+ 'single-user mode detected live backend via shared memory');
+log_ipcs();
+# Fail to reject startup if shm key N has become available and we crash while
+# using key N+1. This is unwanted, but expected. Windows is immune, because
+# its GetSharedMemName() use DataDir strings, not numeric keys.
+$flea->stop; # release first key
+is( $gnat->start(fail_ok => 1),
+ $TestLib::windows_os ? 0 : 1,
+ 'key turnover fools only sysv_shmem.c');
+$gnat->stop; # release first key (no-op on $TestLib::windows_os)
+$flea->start; # grab first key
+# cleanup
+TestLib::system_log('pg_ctl', 'kill', 'QUIT', $slow_pid);
+$slow_client->finish; # client has detected backend termination
+log_ipcs();
+poll_start($gnat); # recycle second key
+
+$gnat->stop;
+$flea->stop;
+$port_holder->stop if $port_holder;
+log_ipcs();
+
+
+# When the kernel is slow to deliver SIGKILL, a postmaster child is slow to
+# exit in response to SIGQUIT, or a postmaster child is slow to exit after
+# postmaster death, we may need retries to start a new postmaster.
+sub poll_start
+{
+ my ($node) = @_;
+
+ my $max_attempts = 180 * 10;
+ my $attempts = 0;
+
+ while ($attempts < $max_attempts)
+ {
+ $node->start(fail_ok => 1) && return 1;
+
+ # Wait 0.1 second before retrying.
+ usleep(100_000);
+
+ $attempts++;
+ }
+
+ # No success within 180 seconds. Try one last time without fail_ok, which
+ # will BAIL_OUT unless it succeeds.
+ $node->start && return 1;
+ return 0;
+}
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index d0e8c68..2aa29ab 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -205,6 +205,7 @@ sub tap_check
local %ENV = %ENV;
$ENV{PERL5LIB} = "$topdir/src/test/perl;$ENV{PERL5LIB}";
$ENV{PG_REGRESS} = "$topdir/$Config/pg_regress/pg_regress";
+ $ENV{REGRESS_SHLIB} = "$topdir/src/test/regress/regress.dll";
$ENV{TESTDIR} = "$dir";
PGSharedMemoryCreate-safety-v4-incr.patchtext/plain; charset=us-asciiDownload
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index df844ff..a9d7bf9 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -352,9 +352,9 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
return SHMSTATE_ENOENT;
/*
- * EACCES implies that the segment belongs to some other userid, which
- * means it is not a Postgres shmem segment (or at least, not one that
- * is relevant to our data directory).
+ * EACCES implies we have no read permission, which means it is not a
+ * Postgres shmem segment (or at least, not one that is relevant to
+ * our data directory).
*/
if (errno == EACCES)
return SHMSTATE_FOREIGN;
@@ -388,13 +388,19 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
return SHMSTATE_ANALYSIS_FAILURE; /* can't stat; be conservative */
/*
- * If we can't attach, be conservative. This may fail if postmaster.pid
- * furnished the shmId and another user created a world-readable segment
- * of the same shmId.
+ * Attachment fails if we have no write permission. Since that will never
+ * happen with Postgres IPCProtection, such a failure shows the segment is
+ * not a Postgres segment. If attachment fails for some other reason, be
+ * conservative.
*/
hdr = (PGShmemHeader *) shmat(shmId, UsedShmemSegAddr, PG_SHMAT_FLAGS);
if (hdr == (PGShmemHeader *) -1)
- return SHMSTATE_ANALYSIS_FAILURE;
+ {
+ if (errno == EACCES)
+ return SHMSTATE_FOREIGN;
+ else
+ return SHMSTATE_ANALYSIS_FAILURE;
+ }
*addr = hdr;
if (hdr->magic != PGShmemMagic ||
diff --git a/src/test/recovery/t/017_shm.pl b/src/test/recovery/t/017_shm.pl
index 8e85d6b..0969d4a 100644
--- a/src/test/recovery/t/017_shm.pl
+++ b/src/test/recovery/t/017_shm.pl
@@ -7,8 +7,9 @@ use IPC::Run 'run';
use PostgresNode;
use Test::More;
use TestLib;
+use Time::HiRes qw(usleep);
-plan tests => 6;
+plan tests => 5;
my $tempdir = TestLib::tempdir;
my $port;
@@ -23,31 +24,40 @@ sub log_ipcs
return;
}
-# With Unix sockets, choose a port number such that the port number's first
-# IpcMemoryKey candidate is available. If multiple copies of this test run
-# concurrently, they will pick different ports. In the absence of collisions
-# from other shmget() activity, gnat starts with key 0x7d001 (512001), and
-# flea starts with key 0x7d002 (512002). With TCP, the first get_new_node
-# picks a port number.
+# These tests need a $port such that nothing creates or removes a segment in
+# $port's IpcMemoryKey range while this test script runs. While there's no
+# way to ensure that in general, we do ensure that if PostgreSQL tests are the
+# only actors. With TCP, the first get_new_node picks a port number. With
+# Unix sockets, use a postmaster, $port_holder, to represent a key space
+# reservation. $port_holder holds a reservation on the key space of port
+# 1+$port_holder->port if it created the first IpcMemoryKey of its own port's
+# key space. If multiple copies of this test script run concurrently, they
+# will pick different ports. $port_holder postmasters use odd-numbered ports,
+# and tests use even-numbered ports. In the absence of collisions from other
+# shmget() activity, gnat starts with key 0x7d001 (512001), and flea starts
+# with key 0x7d002 (512002).
my $port_holder;
if (!$PostgresNode::use_tcp)
{
- for ($port = 512; $port < 612; ++$port)
+ my $lock_port;
+ for ($lock_port = 511; $lock_port < 711; $lock_port += 2)
{
$port_holder = PostgresNode->get_new_node(
- "port${port}_holder",
- port => $port,
+ "port${lock_port}_holder",
+ port => $lock_port,
own_host => 1);
$port_holder->init;
+ $port_holder->append_conf('postgresql.conf', 'max_connections = 5');
$port_holder->start;
# Match the AddToDataDirLockFile() call in sysv_shmem.c. Assume all
# systems not using sysv_shmem.c do use TCP.
- my $shmem_key_line_prefix = sprintf("%9lu ", 1 + $port * 1000);
+ my $shmem_key_line_prefix = sprintf("%9lu ", 1 + $lock_port * 1000);
last
if slurp_file($port_holder->data_dir . '/postmaster.pid') =~
/^$shmem_key_line_prefix/m;
$port_holder->stop;
}
+ $port = $lock_port + 1;
}
# Node setup.
@@ -57,6 +67,8 @@ sub init_start
my $ret = PostgresNode->get_new_node($name, port => $port, own_host => 1);
defined($port) or $port = $ret->port; # same port for all nodes
$ret->init;
+ # Limit semaphore consumption, since we run several nodes concurrently.
+ $ret->append_conf('postgresql.conf', 'max_connections = 5');
$ret->start;
log_ipcs();
return $ret;
@@ -112,12 +124,22 @@ $gnat->kill9;
unlink($gnat->data_dir . '/postmaster.pid');
$gnat->rotate_logfile; # on Windows, can't open old log for writing
log_ipcs();
-# Reject ordinary startup.
-ok(!$gnat->start(fail_ok => 1), 'live query blocks restart');
-like(
- slurp_file($gnat->logfile),
- qr/pre-existing shared memory block/,
- 'detected live backend via shared memory');
+# Reject ordinary startup. Retry for the same reasons poll_start() does.
+my $pre_existing_msg = qr/pre-existing shared memory block/;
+{
+ my $max_attempts = 180 * 10; # Retry every 0.1s for at least 180s.
+ my $attempts = 0;
+ while ($attempts < $max_attempts)
+ {
+ last
+ if $gnat->start(fail_ok => 1)
+ || slurp_file($gnat->logfile) =~ $pre_existing_msg;
+ usleep(100_000);
+ $attempts++;
+ }
+}
+like(slurp_file($gnat->logfile),
+ $pre_existing_msg, 'detected live backend via shared memory');
# Reject single-user startup.
my $single_stderr;
ok( !run_log(
@@ -125,9 +147,7 @@ ok( !run_log(
'<', \('SELECT 1 + 1'), '2>', \$single_stderr),
'live query blocks --single');
print STDERR $single_stderr;
-like(
- $single_stderr,
- qr/pre-existing shared memory block/,
+like($single_stderr, $pre_existing_msg,
'single-user mode detected live backend via shared memory');
log_ipcs();
# Fail to reject startup if shm key N has become available and we crash while
@@ -151,8 +171,9 @@ $port_holder->stop if $port_holder;
log_ipcs();
-# When postmaster children are slow to exit after postmaster death, we may
-# need retries to start a new postmaster.
+# When the kernel is slow to deliver SIGKILL, a postmaster child is slow to
+# exit in response to SIGQUIT, or a postmaster child is slow to exit after
+# postmaster death, we may need retries to start a new postmaster.
sub poll_start
{
my ($node) = @_;
On Mon, Apr 8, 2019 at 6:42 PM Noah Misch <noah@leadboat.com> wrote:
- lorikeet's FailedAssertion("!(vmq->mq_sender == ((void *)0))" looked
suspicious, but this happened six other times in the past year[2], always on
v10 lorikeet.
It happens on v11 too:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=lorikeet&dt=2018-09-25%2010%3A06%3A31
The text changed slightly because we dropped an unnecessary extra
pointer-to-volatile:
FailedAssertion("!(mq->mq_sender == ((void *)0))"
So either two workers started with the same parallel worker number, or
something unexpectedly overwrote the shm_mq struct?
--
Thomas Munro
https://enterprisedb.com
On Mon, Apr 08, 2019 at 08:07:28PM +1200, Thomas Munro wrote:
On Mon, Apr 8, 2019 at 6:42 PM Noah Misch <noah@leadboat.com> wrote:
- lorikeet's FailedAssertion("!(vmq->mq_sender == ((void *)0))" looked
suspicious, but this happened six other times in the past year[2], always on
v10 lorikeet.It happens on v11 too:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=lorikeet&dt=2018-09-25%2010%3A06%3A31
The text changed slightly because we dropped an unnecessary extra
pointer-to-volatile:FailedAssertion("!(mq->mq_sender == ((void *)0))"
So either two workers started with the same parallel worker number, or
something unexpectedly overwrote the shm_mq struct?
Ahh. In that case, it's a duplicate of
/messages/by-id/flat/136712b0-0619-5619-4634-0f0286acaef7@2ndQuadrant.com
On Mon, Apr 8, 2019 at 6:42 PM Noah Misch <noah@leadboat.com> wrote:
- AIX animals failed two ways. First, I missed a "use" statement such that
poll_start() would fail if it needed more than one attempt. Second, I
assumed $pid would be gone as soon as kill(9, $pid) returned[1].
[1] POSIX says "sig or at least one pending unblocked signal shall be
delivered to the sending thread before kill() returns." I doubt the
postmaster had another signal pending often enough to explain the failures, so
AIX probably doesn't follow POSIX in this respect.
It looks like you fixed this, but I was curious about this obversation
as someone interested in learning more about kernel stuff and
portability... Maybe I misunderstood, but isn't POSIX referring to
kill(sig, $YOUR_OWN_PID) there? That is, if you signal *yourself*,
and no other thread exists that could handle the signal, it will be
handled by the sending thread, and in the case of SIGKILL it will
therefore never return. But here, you were talking about a perl
script that kills the postmaster, no? If so, that passage doesn't
seem to apply. In any case, regardless of whether the signal handler
has run to completion when kill() returns, doesn't the pid have to
continue to exist in the process table until it is reaped by its
parent (possibly in response to SIGCHLD), with one of the wait*()
family of system calls?
--
Thomas Munro
https://enterprisedb.com
On Thu, Apr 11, 2019 at 12:48:35PM +1200, Thomas Munro wrote:
On Mon, Apr 8, 2019 at 6:42 PM Noah Misch <noah@leadboat.com> wrote:
- AIX animals failed two ways. First, I missed a "use" statement such that
poll_start() would fail if it needed more than one attempt. Second, I
assumed $pid would be gone as soon as kill(9, $pid) returned[1].[1] POSIX says "sig or at least one pending unblocked signal shall be
delivered to the sending thread before kill() returns." I doubt the
postmaster had another signal pending often enough to explain the failures, so
AIX probably doesn't follow POSIX in this respect.It looks like you fixed this, but I was curious about this obversation
as someone interested in learning more about kernel stuff and
portability... Maybe I misunderstood, but isn't POSIX referring to
kill(sig, $YOUR_OWN_PID) there? That is, if you signal *yourself*,
and no other thread exists that could handle the signal, it will be
handled by the sending thread, and in the case of SIGKILL it will
therefore never return. But here, you were talking about a perl
script that kills the postmaster, no? If so, that passage doesn't
seem to apply.
You're right. I revoke the footnote.
In any case, regardless of whether the signal handler
has run to completion when kill() returns, doesn't the pid have to
continue to exist in the process table until it is reaped by its
parent (possibly in response to SIGCHLD), with one of the wait*()
family of system calls?
True. I'll add that to the code comment.
On Thu, Apr 11, 2019 at 6:22 PM Noah Misch <noah@leadboat.com> wrote:
- my $iaddr = inet_aton($test_localhost); + my $iaddr = inet_aton('0.0.0.0');
This causes make check-world to deliver a flurry of pop-ups from
macOS's built-in Firewall asking if perl should be allowed to listen
to all interfaces (well I didn't catch the exact message, but that's
probably the drift). Not sure if they'd go away permanently if I
managed to click OK before they disappear, but it's fun trying. The
silly firewall facility is not actually enabled by default on this OS,
but unfortunately this company-issued machine has it forced to on.
This isn't really an objection to the code, it's more of a bemused
anecdote about a computer that can't decide whether it's a Unix
workstation or a Fisher Price My First Computer.
--
Thomas Munro
https://enterprisedb.com
On Thu, Apr 18, 2019 at 04:30:46PM +1200, Thomas Munro wrote:
On Thu, Apr 11, 2019 at 6:22 PM Noah Misch <noah@leadboat.com> wrote:
- my $iaddr = inet_aton($test_localhost); + my $iaddr = inet_aton('0.0.0.0');This causes make check-world to deliver a flurry of pop-ups from
macOS's built-in Firewall asking if perl should be allowed to listen
to all interfaces (well I didn't catch the exact message, but that's
probably the drift). Not sure if they'd go away permanently if I
managed to click OK before they disappear, but it's fun trying. The
silly firewall facility is not actually enabled by default on this OS,
but unfortunately this company-issued machine has it forced to on.
This isn't really an objection to the code, it's more of a bemused
anecdote about a computer that can't decide whether it's a Unix
workstation or a Fisher Price My First Computer.
That is unfortunate. The "Allowing specific applications" section of
https://support.apple.com/en-us/HT201642 appears to offer a way to allow perl
permanently. Separately, it wouldn't cost much for us to abandon that check
on !$use_tcp (non-Windows) configurations.
On Fri, Mar 29, 2019 at 09:53:51AM +0000, Daniel Gustafsson wrote:
On Saturday, March 9, 2019 8:16 AM, Noah Misch <noah@leadboat.com> wrote:
I tested on Red Hat and on Windows Server 2016; I won't be shocked
if the test (not the code under test) breaks on other Windows configurations.IIRC there are Windows versions where Win32::Process::KillProcess is required
for this, but thats admittedly somewhat dated knowledge. If the buildfarm goes
red on older Windows animals it might be something to look at perhaps.
Since my second attempt at committing this (commit c098509), I fixed these
bugs in the new test file:
- MSYS-orchestrated mingw32 (buildfarm member jacana) failed the Perl kill(9,
...) calls[1]https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=jacana&dt=2019-04-13%2014%3A39%3A17. For HEAD and v11, using "pg_ctl kill KILL <pid>" fixed that.
For v10 and v9.6, I disabled the test under msys. I can reproduce this with
Perl 5.28 from msys2. Its kill(9, ...) fails for any non-msys2 process (any
ordinary Windows process). [commits 947a350, 2bc0474]
- The regress.dll path needed MSYS-to-w32 path translation. [commit 9daefff]
- The changes to port selection in get_new_node() made Linux-specific
assumptions, so Windows builds had much less protection against port
conflict. [commit 4ab02e8]
- Got EPIPE when writing to stdin of a child process that exited too quickly.
[commit e12a472]
Things have been stable on the buildfarm for the last twelve days, so I think
this one is over.
[1]: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=jacana&dt=2019-04-13%2014%3A39%3A17
On Tue, Apr 30, 2019 at 10:47:37PM +1200, Thomas Munro wrote:
On Mon, Apr 22, 2019 at 4:59 AM Noah Misch <noah@leadboat.com> wrote:
On Thu, Apr 18, 2019 at 04:30:46PM +1200, Thomas Munro wrote:
This causes make check-world to deliver a flurry of pop-ups from
macOS's built-in Firewall asking if perl should be allowed to listen
to all interfaces [...].That is unfortunate. The "Allowing specific applications" section of
https://support.apple.com/en-us/HT201642 appears to offer a way to allow perl
permanently. Separately, it wouldn't cost much for us to abandon that check
on !$use_tcp (non-Windows) configurations.My system is set up not to allow that and I suppose I could go and
argue with my IT department about that, but I'm interested in your
second suggestion if the test is in fact not serving any useful
purpose for non-Windows systems anyway. Do you mean like this?--- a/src/test/perl/PostgresNode.pm +++ b/src/test/perl/PostgresNode.pm @@ -1098,17 +1098,12 @@ sub get_new_node # native Perl (https://stackoverflow.com/a/14388707), so we also test # individual addresses. # - # This seems like a good idea on Unixen as well, even though we don't - # ask the postmaster to open a TCP port on Unix. On Non-Linux, - # non-Windows kernels, binding to 127.0.0.1/24 addresses other than - # 127.0.0.1 fails with EADDRNOTAVAIL. - #
Deleting that comment paragraph isn't quite right, since we're still testing
127.0.0.1 everywhere. The paragraph does have cause to change.
# XXX A port available now may become unavailable by the time we start # the postmaster. if ($found == 1) { - foreach my $addr (qw(127.0.0.1 0.0.0.0), - $use_tcp ? qw(127.0.0.2 127.0.0.3) : ()) + foreach my $addr (qw(127.0.0.1), + $use_tcp ? qw(0.0.0.0 127.0.0.2 127.0.0.3) : ())
This is what I meant, yes.
Show quoted text
{
can_bind($addr, $port) or $found = 0;
}
Import Notes
Reply to msg id not found: CA+hUKGLEsjr_nbGtbHbaPnv3ryvhDM37psz9YCxdkKLxfi15Q@mail.gmail.com
On Wed, May 1, 2019 at 12:57 PM Noah Misch <noah@leadboat.com> wrote:
On Tue, Apr 30, 2019 at 10:47:37PM +1200, Thomas Munro wrote:
--- a/src/test/perl/PostgresNode.pm +++ b/src/test/perl/PostgresNode.pm @@ -1098,17 +1098,12 @@ sub get_new_node # native Perl (https://stackoverflow.com/a/14388707), so we also test # individual addresses. # - # This seems like a good idea on Unixen as well, even though we don't - # ask the postmaster to open a TCP port on Unix. On Non-Linux, - # non-Windows kernels, binding to 127.0.0.1/24 addresses other than - # 127.0.0.1 fails with EADDRNOTAVAIL. - #Deleting that comment paragraph isn't quite right, since we're still testing
127.0.0.1 everywhere. The paragraph does have cause to change.
Hi Noah,
I put the second sentence back and tweaked it thus: s/fails/might
fail/. Maybe I'm being too pedantic here, but binding to 127.0.0.2
works on other OSes too, as long as you've configured an interface or
alias for it (and it's not terribly uncommon to do so). Here's a
version with a commit message. Please let me know if you want to
tweak the comments further.
--
Thomas Munro
https://enterprisedb.com
Attachments:
0001-Probe-only-127.0.0.1-when-looking-for-ports-on-Unix.patchapplication/octet-stream; name=0001-Probe-only-127.0.0.1-when-looking-for-ports-on-Unix.patchDownload
From 9557fe3fa9579302e0a23af0e912ebf9b7b3eba0 Mon Sep 17 00:00:00 2001
From: Thomas Munro <tmunro@postgresql.org>
Date: Mon, 6 May 2019 15:02:41 +1200
Subject: [PATCH] Probe only 127.0.0.1 when looking for ports on Unix.
Commit c0985099, later adjusted by commit 4ab02e81, probed 0.0.0.0
in addition to 127.0.0.1, for the benefit of Windows build farm
animals. It isn't really useful on Unix systems, and turned out to
be a bit inconvenient to users of some corporate firewall software.
Switch back to probing just 127.0.0.1 on non-Windows systems.
Back-patch to 9.4 like the other commits.
Discussion: https://postgr.es/m/CA%2BhUKG%2B21EPwfgs4m%2BtqyRtbVqkOUvP8QQ8sWk9%2Bh55Aub1H3A%40mail.gmail.com
---
src/test/perl/PostgresNode.pm | 10 ++++------
1 file changed, 4 insertions(+), 6 deletions(-)
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 76874141c5..f00e81ba74 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -1098,17 +1098,15 @@ sub get_new_node
# native Perl (https://stackoverflow.com/a/14388707), so we also test
# individual addresses.
#
- # This seems like a good idea on Unixen as well, even though we don't
- # ask the postmaster to open a TCP port on Unix. On Non-Linux,
- # non-Windows kernels, binding to 127.0.0.1/24 addresses other than
- # 127.0.0.1 fails with EADDRNOTAVAIL.
+ # On Non-Linux, non-Windows kernels, binding to 127.0.0.1/24 addresses
+ # other than 127.0.0.1 might fail with EADDRNOTAVAIL.
#
# XXX A port available now may become unavailable by the time we start
# the postmaster.
if ($found == 1)
{
- foreach my $addr (qw(127.0.0.1 0.0.0.0),
- $use_tcp ? qw(127.0.0.2 127.0.0.3) : ())
+ foreach my $addr (qw(127.0.0.1),
+ $use_tcp ? qw(127.0.0.2 127.0.0.3 0.0.0.0) : ())
{
can_bind($addr, $port) or $found = 0;
}
--
2.21.0
On Mon, May 6, 2019 at 3:30 PM Thomas Munro <thomas.munro@gmail.com> wrote:
I put the second sentence back and tweaked it thus: s/fails/might
fail/. Maybe I'm being too pedantic here, but binding to 127.0.0.2
works on other OSes too, as long as you've configured an interface or
alias for it (and it's not terribly uncommon to do so). Here's a
version with a commit message. Please let me know if you want to
tweak the comments further.
Pushed, with a further adjustment to the comment.
--
Thomas Munro
https://enterprisedb.com
On Wed, May 08, 2019 at 10:10:35PM +1200, Thomas Munro wrote:
On Mon, May 6, 2019 at 3:30 PM Thomas Munro <thomas.munro@gmail.com> wrote:
I put the second sentence back and tweaked it thus: s/fails/might
fail/. Maybe I'm being too pedantic here, but binding to 127.0.0.2
works on other OSes too, as long as you've configured an interface or
alias for it (and it's not terribly uncommon to do so).
Even if, say, 99% of FreeBSD systems did configure such an interface, that
would not be enough to make it okay to probe 127.0.0.2 on FreeBSD.
Pushed, with a further adjustment to the comment.
I'm fine with what you committed.