possible dsm bug in dsm_attach()
Hi,
dsm_attach() does the following:
nitems = dsm_control->nitems;
for (i = 0; i < nitems; ++i)
{
/* If the reference count is 0, the slot is actually unused. */
if (dsm_control->item[i].refcnt == 0)
continue;
/*
* If the reference count is 1, the slot is still in use, but the
* segment is in the process of going away. Treat that as if we
* didn't find a match.
*/
if (dsm_control->item[i].refcnt == 1)
break;
/* Otherwise, if the descriptor matches, we've found a match. */
if (dsm_control->item[i].handle == seg->handle)
{
dsm_control->item[i].refcnt++;
seg->control_slot = i;
break;
}
}
The break because of refcnt == 1 doesn't generally seem to be a good
idea. Why are we bailing if there's *any* segment that's in the process
of being removed? I think the check should be there *after* the
dsm_control->item[i].handle == seg->handle check?
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, May 6, 2014 at 8:43 AM, Andres Freund <andres@2ndquadrant.com> wrote:
dsm_attach() does the following:
nitems = dsm_control->nitems;
for (i = 0; i < nitems; ++i)
{
/* If the reference count is 0, the slot is actually unused. */
if (dsm_control->item[i].refcnt == 0)
continue;/*
* If the reference count is 1, the slot is still in use, but the
* segment is in the process of going away. Treat that as if we
* didn't find a match.
*/
if (dsm_control->item[i].refcnt == 1)
break;/* Otherwise, if the descriptor matches, we've found a match. */
if (dsm_control->item[i].handle == seg->handle)
{
dsm_control->item[i].refcnt++;
seg->control_slot = i;
break;
}
}The break because of refcnt == 1 doesn't generally seem to be a good
idea. Why are we bailing if there's *any* segment that's in the process
of being removed? I think the check should be there *after* the
dsm_control->item[i].handle == seg->handle check?
You are correct. Good catch.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2014-05-06 08:48:57 -0400, Robert Haas wrote:
On Tue, May 6, 2014 at 8:43 AM, Andres Freund <andres@2ndquadrant.com> wrote:
The break because of refcnt == 1 doesn't generally seem to be a good
idea. Why are we bailing if there's *any* segment that's in the process
of being removed? I think the check should be there *after* the
dsm_control->item[i].handle == seg->handle check?You are correct. Good catch.
Fix attached.
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
0001-Don-t-bail-in-dsm_attach-if-any-any-other-segment-is.patchtext/x-patch; charset=us-asciiDownload
>From 9bc05ac103c77fcd2276754a3ccbf49e66dd00b7 Mon Sep 17 00:00:00 2001
From: Andres Freund <andres@anarazel.de>
Date: Tue, 6 May 2014 19:08:07 +0200
Subject: [PATCH] Don't bail in dsm_attach() if any any other segment is about
to be destroyed.
The previous coding accidentally checked for refcnt == 1, which
signals moribund segments, before checking whether the segment we're
looking at is the one we're trying to attach to.
---
src/backend/storage/ipc/dsm.c | 15 ++++++++-------
1 file changed, 8 insertions(+), 7 deletions(-)
diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c
index d86b0c7..bde7026 100644
--- a/src/backend/storage/ipc/dsm.c
+++ b/src/backend/storage/ipc/dsm.c
@@ -570,6 +570,10 @@ dsm_attach(dsm_handle h)
if (dsm_control->item[i].refcnt == 0)
continue;
+ /* Not the segment you're looking for. */
+ if (dsm_control->item[i].handle != seg->handle)
+ continue;
+
/*
* If the reference count is 1, the slot is still in use, but the
* segment is in the process of going away. Treat that as if we
@@ -578,13 +582,10 @@ dsm_attach(dsm_handle h)
if (dsm_control->item[i].refcnt == 1)
break;
- /* Otherwise, if the descriptor matches, we've found a match. */
- if (dsm_control->item[i].handle == seg->handle)
- {
- dsm_control->item[i].refcnt++;
- seg->control_slot = i;
- break;
- }
+ /* Otherwise we've found a match. */
+ dsm_control->item[i].refcnt++;
+ seg->control_slot = i;
+ break;
}
LWLockRelease(DynamicSharedMemoryControlLock);
--
1.8.5.rc2.dirty
On Tue, May 6, 2014 at 1:14 PM, Andres Freund <andres@2ndquadrant.com> wrote:
On 2014-05-06 08:48:57 -0400, Robert Haas wrote:
On Tue, May 6, 2014 at 8:43 AM, Andres Freund <andres@2ndquadrant.com> wrote:
The break because of refcnt == 1 doesn't generally seem to be a good
idea. Why are we bailing if there's *any* segment that's in the process
of being removed? I think the check should be there *after* the
dsm_control->item[i].handle == seg->handle check?You are correct. Good catch.
Fix attached.
Committed, thanks.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2014-05-06 13:45:13 -0400, Robert Haas wrote:
On Tue, May 6, 2014 at 1:14 PM, Andres Freund <andres@2ndquadrant.com> wrote:
On 2014-05-06 08:48:57 -0400, Robert Haas wrote:
On Tue, May 6, 2014 at 8:43 AM, Andres Freund <andres@2ndquadrant.com> wrote:
The break because of refcnt == 1 doesn't generally seem to be a good
idea. Why are we bailing if there's *any* segment that's in the process
of being removed? I think the check should be there *after* the
dsm_control->item[i].handle == seg->handle check?You are correct. Good catch.
Fix attached.
Committed, thanks.
Heh. Not a fan of film references? :)
Thanks,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, May 6, 2014 at 1:46 PM, Andres Freund <andres@2ndquadrant.com> wrote:
Fix attached.
Committed, thanks.
Heh. Not a fan of film references? :)
I didn't quite put the pieces together there. I just thought the use
of "you" was awkward.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, May 6, 2014 at 11:15 PM, Robert Haas <robertmhaas@gmail.com> wrote:
On Tue, May 6, 2014 at 1:14 PM, Andres Freund <andres@2ndquadrant.com>
wrote:
On 2014-05-06 08:48:57 -0400, Robert Haas wrote:
On Tue, May 6, 2014 at 8:43 AM, Andres Freund <andres@2ndquadrant.com>
wrote:
The break because of refcnt == 1 doesn't generally seem to be a good
idea. Why are we bailing if there's *any* segment that's in the
process
of being removed? I think the check should be there *after* the
dsm_control->item[i].handle == seg->handle check?You are correct. Good catch.
Fix attached.
Committed, thanks.
dsm_create(Size size, int flags)
{
..
/* Lock the control segment so we can register the new segment. */
LWLockAcquire(DynamicSharedMemoryControlLock, LW_EXCLUSIVE);
..
/* Verify that we can support an additional mapping. */
if (nitems >= dsm_control->maxitems)
{
if ((flags & DSM_CREATE_NULL_IF_MAXSEGMENTS) != 0)
{
dsm_impl_op(DSM_OP_DESTROY, seg->handle, 0, &seg->impl_private,
&seg->mapped_address, &seg->mapped_size, WARNING);
if (seg->resowner != NULL)
ResourceOwnerForgetDSM(seg->resowner, seg);
dlist_delete(&seg->node);
pfree(seg);
return NULL;
}
..
}
Is there a reason lock is not released in case we return NULL in above
code?
I am facing an issue in case we need to create many segments for
large inheritance hierarchy. Attached patch fixes the problem for me.
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com
Attachments:
release_lock_dsm_v1.patchapplication/octet-stream; name=release_lock_dsm_v1.patchDownload
diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c
index 321bad9..29e46c2 100644
--- a/src/backend/storage/ipc/dsm.c
+++ b/src/backend/storage/ipc/dsm.c
@@ -502,6 +502,7 @@ dsm_create(Size size, int flags)
{
if ((flags & DSM_CREATE_NULL_IF_MAXSEGMENTS) != 0)
{
+ LWLockRelease(DynamicSharedMemoryControlLock);
dsm_impl_op(DSM_OP_DESTROY, seg->handle, 0, &seg->impl_private,
&seg->mapped_address, &seg->mapped_size, WARNING);
if (seg->resowner != NULL)
On Wed, Mar 25, 2015 at 6:11 AM, Amit Kapila <amit.kapila16@gmail.com> wrote:
I am facing an issue in case we need to create many segments for
large inheritance hierarchy. Attached patch fixes the problem for me.
Sigh. You'd think I'd be able to write a 30-line patch without
introducing not one but two stupid bugs. But if you did think that,
you'd be wrong.
Committed.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers