pg_shmem_allocations view
Hi,
I've more than once wanted to know what allocated shared memory in
postgres installation is used for. Especially with more an more
extensions around that's quite useful.
Thus I've written a patch to add a new SRF/VIEW
pg_get_shmem_allocations/pg_shmem_allocations that shows the contents of
the shared memory index:
postgres=# SELECT * FROM pg_shmem_allocations ORDER BY size DESC;
key | off | size | allocated
-------------------------------------+-------------+-------------+-----------
Buffer Blocks | 286242528 | 17179869184 | t
Buffer Descriptors | 152024800 | 134217728 | t
Checkpointer Data | 17584720352 | 41943080 | t
XLOG Ctl | 134234112 | 16804496 | t
CLOG Ctl | 151038624 | 525312 | t
| 17627719648 | 366624 | f
Backend Activity Buffer | 17584379168 | 272000 | t
SUBTRANS Ctl | 151563936 | 263168 | t
OldSerXid SLRU Ctl | 17584225696 | 131648 | t
MultiXactMember Ctl | 151892960 | 131648 | t
Shared Buffer Lookup Table | 17466111712 | 131184 | t
shmInvalBuffer | 17584653184 | 66104 | t
Async Ctl | 17626666048 | 65856 | t
MultiXactOffset Ctl | 151827104 | 65856 | t
Fast Path Strong Relation Lock Data | 17583882752 | 4100 | t
Backend Status Array | 17584373304 | 3672 | t
PROCLOCK hash | 17583785856 | 2160 | t
PREDICATELOCKTARGET hash | 17583886856 | 2160 | t
PREDICATELOCK hash | 17583957632 | 2160 | t
pg_stat_statements hash | 17626731952 | 2160 | t
SERIALIZABLEXID hash | 17584175184 | 2160 | t
LOCK hash | 17583684096 | 2160 | t
Background Worker Data | 17584651184 | 1992 | t
Wal Receiver Ctl | 17626663712 | 1192 | t
Backend Client Host Name Buffer | 17584378064 | 1088 | t
Backend Application Name Buffer | 17584376976 | 1088 | t
ProcSignalSlots | 17584719472 | 864 | t
Sync Scan Locations List | 17626665120 | 656 | t
Async Queue Control | 17626665776 | 244 | t
Control File | 134233856 | 240 | t
AutoVacuum Data | 17626663432 | 224 | t
BTree Vacuum State | 17626664904 | 216 | t
PMSignalState | 17584719288 | 180 | t
Shared MultiXact State | 152024608 | 176 | t
Proc Array | 17584373192 | 108 | t
PredXactList | 17584149248 | 88 | t
Proc Header | 17584357360 | 88 | t
Wal Sender Ctl | 17626663656 | 56 | t
pg_stat_statements | 17626731904 | 48 | t
Buffer Strategy Status | 17583684064 | 32 | t
RWConflictPool | 17584184832 | 24 | t
Prepared Transaction Table | 17584651168 | 16 | t
FinishedSerializableTransactions | 17584225664 | 16 | t
OldSerXidControlData | 17584357344 | 16 | t
(44 rows)
I think this is quite worthwile information. It'd possibly be better of
in an extension, but the relevant dastructures aren't public.
The attached patch doesn't contain documentation because I wasn't sure
that others would be interested in the feature.
Greetings,
Andres Freund
PS: Yes, the checkpointer's allocation is crazy. The fsync queue is
sized by NBuffers which is absurd.
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
0001-Add-support-for-wrapping-to-psql-s-extended-mode.-Th.patchtext/x-patch; charset=us-asciiDownload+1222-24
Hi,
On 2014-05-04 13:44:17 +0200, Andres Freund wrote:
postgres=# SELECT * FROM pg_shmem_allocations ORDER BY size DESC;
key | off | size | allocated
-------------------------------------+-------------+-------------+-----------
Buffer Blocks | 286242528 | 17179869184 | t
Buffer Descriptors | 152024800 | 134217728 | t
...
OldSerXidControlData | 17584357344 | 16 | t
(44 rows)
Thinking about this, I think it was a mistake to not add a 'name' field
to dynamic shared memory's dsm_control_item. Right now it's very hard to
figure out which extension allocated a dsm segment. Imo we should change
that before 9.4 is out. I am not suggesting to use it to identify
segments, but just as an identifier, passed in into dsm_create().
Imo there should be a corresponding pg_dynshmem_allocations to
pg_shmem_allocations.
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Hi,
On 2014-05-04 13:44:17 +0200, Andres Freund wrote:
postgres=# SELECT * FROM pg_shmem_allocations ORDER BY size DESC;
key | off | size | allocated
-------------------------------------+-------------+-------------+-----------
Buffer Blocks | 286242528 | 17179869184 | t
Buffer Descriptors | 152024800 | 134217728 | t
Abhijit notified me that I've attached the wrong patch. Corrected.
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
0001-Add-pg_shmem_allocations-view.patchtext/x-patch; charset=us-asciiDownload+110-1
On 04-05-2014 08:44, Andres Freund wrote:
I've more than once wanted to know what allocated shared memory in
postgres installation is used for. Especially with more an more
extensions around that's quite useful.
A few years ago I had to provide such information an did something
similar. Is it useful? Yes. However, it is a developer's feature.
On 2014-05-04 13:44:17 +0200, Andres Freund wrote:
Thinking about this, I think it was a mistake to not add a 'name' field
to dynamic shared memory's dsm_control_item. Right now it's very hard to
figure out which extension allocated a dsm segment. Imo we should change
that before 9.4 is out. I am not suggesting to use it to identify
segments, but just as an identifier, passed in into dsm_create().
+1.
Imo there should be a corresponding pg_dynshmem_allocations to
pg_shmem_allocations.
... or another boolean column (say 'dynamic') and just one view.
--
Euler Taveira Timbira - http://www.timbira.com.br/
PostgreSQL: Consultoria, Desenvolvimento, Suporte 24x7 e Treinamento
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Sun, May 4, 2014 at 7:50 AM, Andres Freund <andres@2ndquadrant.com> wrote:
On 2014-05-04 13:44:17 +0200, Andres Freund wrote:
postgres=# SELECT * FROM pg_shmem_allocations ORDER BY size DESC;
key | off | size | allocated
-------------------------------------+-------------+-------------+-----------
Buffer Blocks | 286242528 | 17179869184 | t
Buffer Descriptors | 152024800 | 134217728 | t
...
OldSerXidControlData | 17584357344 | 16 | t
(44 rows)Thinking about this, I think it was a mistake to not add a 'name' field
to dynamic shared memory's dsm_control_item. Right now it's very hard to
figure out which extension allocated a dsm segment. Imo we should change
that before 9.4 is out. I am not suggesting to use it to identify
segments, but just as an identifier, passed in into dsm_create().Imo there should be a corresponding pg_dynshmem_allocations to
pg_shmem_allocations.
Well, right now a dsm_control_item is 8 bytes. If we add a name field
of our usual 64 bytes, they'll each be 9 times bigger. We're not
talking about a lot of bytes in absolute terms, but I guess I'm not in
favor of an 800% size increase without somewhat more justification
than you've provided here. Who is using dynamic shared memory for
enough different things at this point to get confused?
I'm quite in favor of having something like this for the main shared
memory segment, but I think that's 9.5 material at this point.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Robert Haas <robertmhaas@gmail.com> writes:
On Sun, May 4, 2014 at 7:50 AM, Andres Freund <andres@2ndquadrant.com> wrote:
Thinking about this, I think it was a mistake to not add a 'name' field
to dynamic shared memory's dsm_control_item.
Well, right now a dsm_control_item is 8 bytes. If we add a name field
of our usual 64 bytes, they'll each be 9 times bigger.
And the controlled shared segment is likely to be how big exactly? It's
probably not even possible for it to be smaller than a page size, 4K or
so depending on the OS. I agree with Andres that a name would be a good
idea; complaining about the space needed to hold it is penny-wise and
pound-foolish.
I'm quite in favor of having something like this for the main shared
memory segment, but I think that's 9.5 material at this point.
If you're prepared to break the current APIs later to add a name parameter
(which would have to be required, if it's to be useful at all), then sure,
put the question off till 9.5.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2014-05-05 15:04:07 -0400, Robert Haas wrote:
On Sun, May 4, 2014 at 7:50 AM, Andres Freund <andres@2ndquadrant.com> wrote:
On 2014-05-04 13:44:17 +0200, Andres Freund wrote:
postgres=# SELECT * FROM pg_shmem_allocations ORDER BY size DESC;
key | off | size | allocated
-------------------------------------+-------------+-------------+-----------
Buffer Blocks | 286242528 | 17179869184 | t
Buffer Descriptors | 152024800 | 134217728 | t
...
OldSerXidControlData | 17584357344 | 16 | t
(44 rows)Thinking about this, I think it was a mistake to not add a 'name' field
to dynamic shared memory's dsm_control_item. Right now it's very hard to
figure out which extension allocated a dsm segment. Imo we should change
that before 9.4 is out. I am not suggesting to use it to identify
segments, but just as an identifier, passed in into dsm_create().Imo there should be a corresponding pg_dynshmem_allocations to
pg_shmem_allocations.Well, right now a dsm_control_item is 8 bytes. If we add a name field
of our usual 64 bytes, they'll each be 9 times bigger. We're not
talking about a lot of bytes in absolute terms, but I guess I'm not in
favor of an 800% size increase without somewhat more justification
than you've provided here. Who is using dynamic shared memory for
enough different things at this point to get confused?
The kernel side overhead of creating a shared memory segment are so much
higher that this really isn't a meaningful saving. Also, are you really
considering a couple hundred bytes to be a problem?
I think it's quite a sensible thing for an administrator to ask where
all the memory has gone. The more users for dsm there the more important
that'll get. Right now pretty much the only thing a admin could do is to
poke around in /proc to see which backend has mapped the segment and try
to figure out via the logs what caused it to do so. Not nice.
I'm quite in favor of having something like this for the main shared
memory segment, but I think that's 9.5 material at this point.
Clearly. For one the version I posted here missed allocations which
aren't done via ShmemInitStruct (lwlock main array and hash table
allocations primarily). For another it's too late ;)
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2014-05-05 15:09:02 -0400, Tom Lane wrote:
I'm quite in favor of having something like this for the main shared
memory segment, but I think that's 9.5 material at this point.If you're prepared to break the current APIs later to add a name parameter
(which would have to be required, if it's to be useful at all), then sure,
put the question off till 9.5.
I understood Robert to mean that it's too late for my proposed view for
9.4 - and I agree - but I wholeheartedly agree with you that we should
add a name parameter to the dsm API *now*. We can just Assert() that it's
nonzero if we don't think it's useful for now.
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Mon, May 5, 2014 at 3:09 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Robert Haas <robertmhaas@gmail.com> writes:
On Sun, May 4, 2014 at 7:50 AM, Andres Freund <andres@2ndquadrant.com> wrote:
Thinking about this, I think it was a mistake to not add a 'name' field
to dynamic shared memory's dsm_control_item.Well, right now a dsm_control_item is 8 bytes. If we add a name field
of our usual 64 bytes, they'll each be 9 times bigger.And the controlled shared segment is likely to be how big exactly? It's
probably not even possible for it to be smaller than a page size, 4K or
so depending on the OS. I agree with Andres that a name would be a good
idea; complaining about the space needed to hold it is penny-wise and
pound-foolish.
The control segment is sized to support a number of dynamic shared
memory segments not exceeding 64 + 2 *MaxBackends. With default
settings, that currently works out to 288 segments, or 2306 bytes.
So, adding a 64-byte name to each of those structures would increase
the size from 2k to about 20k.
So, sure, that's not a lot of memory. But I'm still not convinced
that's it's very useful. What I think is going to happen is that (1)
most people won't be used dynamic shared memory at all, so they won't
have any use for this; (2) those people who do run an extension that
uses dynamic shared memory will most likely only be running one such
extension, so they won't need a name to know what the segments are
being used for; and (3) if and when we eventually get parallel query,
dynamic shared memory segments will be widely used, but a bunch of
segments that are all named "parallel_query" or "parallel_query.$PID"
isn't going to be too informative.
Now, all that having been said, I recognize that human-readable names
are a generally useful thing, so I'm not going to hold my breath until
I turn blue if other people really want this, and it may turn out to
be useful someday. But if anyone is curious whether I'm *confident*
that it will be useful someday: at this point, no.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Robert Haas <robertmhaas@gmail.com> writes:
On Mon, May 5, 2014 at 3:09 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
And the controlled shared segment is likely to be how big exactly? It's
probably not even possible for it to be smaller than a page size, 4K or
so depending on the OS. I agree with Andres that a name would be a good
idea; complaining about the space needed to hold it is penny-wise and
pound-foolish....
Now, all that having been said, I recognize that human-readable names
are a generally useful thing, so I'm not going to hold my breath until
I turn blue if other people really want this, and it may turn out to
be useful someday. But if anyone is curious whether I'm *confident*
that it will be useful someday: at this point, no.
I'm not confident that it'll be useful either. But I am confident that
if we don't put it in now, and decide we want it later, there will be
complaints when we change the API. Better to have an ignored parameter
than no parameter.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Mon, May 5, 2014 at 6:54 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Robert Haas <robertmhaas@gmail.com> writes:
On Mon, May 5, 2014 at 3:09 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
And the controlled shared segment is likely to be how big exactly? It's
probably not even possible for it to be smaller than a page size, 4K or
so depending on the OS. I agree with Andres that a name would be a good
idea; complaining about the space needed to hold it is penny-wise and
pound-foolish....
Now, all that having been said, I recognize that human-readable names
are a generally useful thing, so I'm not going to hold my breath until
I turn blue if other people really want this, and it may turn out to
be useful someday. But if anyone is curious whether I'm *confident*
that it will be useful someday: at this point, no.I'm not confident that it'll be useful either. But I am confident that
if we don't put it in now, and decide we want it later, there will be
complaints when we change the API. Better to have an ignored parameter
than no parameter.
I'm generally skeptical of that philosophy. If we put in an ignored
parameter, people may pass pointers to NULL or to garbage or to an
overly-long string, and they won't know it's broken until we make it
do something; at which point their code will begin to fail without
warning. Speaking as an employee of a company that maintains several
PostgreSQL extensions that sometimes need to be updated for newer
server versions, I'd rather have a clean API break that makes the
build fail than a "soft" break that supposedly lets things continue
working but maybe breaks them in subtler ways. Another problem with
this idea is that we might never get around to making it do anything,
and then the dead parameter is just a stupid and unnecessary wart.
If we're going to do anything at all here for 9.4, I recommend
ignoring the fact we're in feature freeze and going whole hog: add the
name, add the monitoring view, and add the monitoring view for the
main shared memory segment just for good measure. That way, if we get
the design wrong or something, we have a chance of getting some
feedback. If we're not going to do that, then I vote for doing
nothing and considering later whether to break it for 9.5, by which
time we may have some evidence as to whether and how this code is
really being used. Anyone who expects PostgreSQL's C API to be
completely stable is going to be regularly disappointed, as most
recently demonstrated by the Enormous Header Churn of the 9.3 era. I
don't particularly mind being the cause of further disappointment; as
long as the breakage is obvious rather than subtle, the fix usually
takes about 10 minutes.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 5 May 2014 21:54, Robert Haas <robertmhaas@gmail.com> wrote:
On Mon, May 5, 2014 at 3:09 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Robert Haas <robertmhaas@gmail.com> writes:
On Sun, May 4, 2014 at 7:50 AM, Andres Freund <andres@2ndquadrant.com> wrote:
Thinking about this, I think it was a mistake to not add a 'name' field
to dynamic shared memory's dsm_control_item.Well, right now a dsm_control_item is 8 bytes. If we add a name field
of our usual 64 bytes, they'll each be 9 times bigger.And the controlled shared segment is likely to be how big exactly? It's
probably not even possible for it to be smaller than a page size, 4K or
so depending on the OS. I agree with Andres that a name would be a good
idea; complaining about the space needed to hold it is penny-wise and
pound-foolish.The control segment is sized to support a number of dynamic shared
memory segments not exceeding 64 + 2 *MaxBackends. With default
settings, that currently works out to 288 segments, or 2306 bytes.
So, adding a 64-byte name to each of those structures would increase
the size from 2k to about 20k.So, sure, that's not a lot of memory. But I'm still not convinced
that's it's very useful. What I think is going to happen is that (1)
most people won't be used dynamic shared memory at all, so they won't
have any use for this; (2) those people who do run an extension that
uses dynamic shared memory will most likely only be running one such
extension, so they won't need a name to know what the segments are
being used for; and (3) if and when we eventually get parallel query,
dynamic shared memory segments will be widely used, but a bunch of
segments that are all named "parallel_query" or "parallel_query.$PID"
isn't going to be too informative.
Not sure your arguments hold any water.
Most people don't use most features... and so we're not allowed
features that can be debugged?
How do you know people will only use one extension that uses dshmem?
Why would we call multiple segments the same thing??
If names are a problem, lets give them numbers. Seems a minor point.
Perhaps we can allocate space for names dynamically??
Not being able to tell segments apart from each other is just daft, if
we are trying to supply bug free software for the world to use.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2014-05-05 23:20:43 -0400, Robert Haas wrote:
On Mon, May 5, 2014 at 6:54 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
I'm not confident that it'll be useful either. But I am confident that
if we don't put it in now, and decide we want it later, there will be
complaints when we change the API. Better to have an ignored parameter
than no parameter.I'm generally skeptical of that philosophy. If we put in an ignored
parameter, people may pass pointers to NULL or to garbage or to an
overly-long string, and they won't know it's broken until we make it
do something; at which point their code will begin to fail without
warning.
If it were a complex change, maybe. But I don't think that's likely
here.
Assert(name != NULL && strlen(name) > 0 && strlen(name) < NAMEDATALEN);
should perfectly do the trick.
If we're going to do anything at all here for 9.4, I recommend
ignoring the fact we're in feature freeze and going whole hog: add the
name, add the monitoring view, and add the monitoring view for the
main shared memory segment just for good measure.
We can do that as well. If there's agreement on that path I'll update
the patch to also show dynamic statements.
Anyone who expects PostgreSQL's C API to be
completely stable is going to be regularly disappointed, as most
recently demonstrated by the Enormous Header Churn of the 9.3 era. I
don't particularly mind being the cause of further disappointment; as
long as the breakage is obvious rather than subtle, the fix usually
takes about 10 minutes.
Didn't you complain rather loudly about that change?
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, May 6, 2014 at 7:45 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
The control segment is sized to support a number of dynamic shared
memory segments not exceeding 64 + 2 *MaxBackends. With default
settings, that currently works out to 288 segments, or 2306 bytes.
So, adding a 64-byte name to each of those structures would increase
the size from 2k to about 20k.So, sure, that's not a lot of memory. But I'm still not convinced
that's it's very useful. What I think is going to happen is that (1)
most people won't be used dynamic shared memory at all, so they won't
have any use for this; (2) those people who do run an extension that
uses dynamic shared memory will most likely only be running one such
extension, so they won't need a name to know what the segments are
being used for; and (3) if and when we eventually get parallel query,
dynamic shared memory segments will be widely used, but a bunch of
segments that are all named "parallel_query" or "parallel_query.$PID"
isn't going to be too informative.Not sure your arguments hold any water.
I'm not, either.
Most people don't use most features... and so we're not allowed
features that can be debugged?
I certainly didn't say that.
How do you know people will only use one extension that uses dshmem?
I don't. If they do, that's a good argument for adding this.
Why would we call multiple segments the same thing??
It's not clear to me how someone is going to intelligently name
multiple segments used by the same extension. Maybe they'll give them
all the same name. Maybe they'll name them all extension_name.pid.
More than likely, different extensions will use different conventions.
:-(
It might be sensible to add a "creator PID" field to the DSM control
items. Of course, that PID might have exited, but it could still
possibly be useful for debugging purposes.
If names are a problem, lets give them numbers. Seems a minor point.
Perhaps we can allocate space for names dynamically??
A static buffer, as proposed by Andres, seems a lot simper.
Not being able to tell segments apart from each other is just daft, if
we are trying to supply bug free software for the world to use.
I can see I'm losing this argument.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 05/06/2014 02:59 PM, Robert Haas wrote:
Why would we call multiple segments the same thing??
It's not clear to me how someone is going to intelligently name
multiple segments used by the same extension. Maybe they'll give them
all the same name. Maybe they'll name them all extension_name.pid.
More than likely, different extensions will use different conventions.
:-(
That seems sensible to me. The best scheme will depend on how the
segments are used. Best to leave it to the extension author.
- Heikki
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 6 May 2014 13:06, Heikki Linnakangas <hlinnakangas@vmware.com> wrote:
The best scheme will depend on how the segments
are used. Best to leave it to the extension author.
+1
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Hi,
On 2014-05-06 13:56:41 +0200, Andres Freund wrote:
On 2014-05-05 23:20:43 -0400, Robert Haas wrote:
On Mon, May 5, 2014 at 6:54 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
I'm not confident that it'll be useful either. But I am confident that
if we don't put it in now, and decide we want it later, there will be
complaints when we change the API. Better to have an ignored parameter
than no parameter.I'm generally skeptical of that philosophy. If we put in an ignored
parameter, people may pass pointers to NULL or to garbage or to an
overly-long string, and they won't know it's broken until we make it
do something; at which point their code will begin to fail without
warning.If it were a complex change, maybe. But I don't think that's likely
here.
Assert(name != NULL && strlen(name) > 0 && strlen(name) < NAMEDATALEN);
should perfectly do the trick.
Attached are two patches:
a) Patch addin a 'name' parameter to dsm_create(). I think we should
apply this to 9.4.
b) pg_dynamic_shmem_allocations and pg_static_shmem_allocations
views. The previous version didn't include dsm support and didn't
take the required lock.
I am not so sure whether b) should be applied together with a) in 9.4,
but I'd be happy enough to add docs if people agree with the naming.
FWIW, I like dsm_create()'s internals more after this patch...
postgres=# \d pg_dynamic_shmem_allocations
View "pg_catalog.pg_dynamic_shmem_allocations"
Column | Type | Modifiers
--------+--------+-----------
handle | bigint |
name | text |
size | bigint |
refcnt | bigint |
postgres=# \d pg_static_shmem_allocations
View "pg_catalog.pg_static_shmem_allocations"
Column | Type | Modifiers
-----------+---------+-----------
key | text |
off | bigint |
size | bigint |
allocated | boolean |
postgres=# SELECT * FROM pg_dynamic_shmem_allocations;
handle | name | size | refcnt
------------+-------------+-------+--------
1120921036 | test_shm_mq | 65656 | 1
(1 row)
postgres=# SELECT * FROM pg_static_shmem_allocations ORDER BY key NULLS FIRST;
key | off | size | allocated
-------------------------------------+------------+------------+-----------
| 2222605024 | 1727776 | f
| | 34844752 | t
Async Ctl | 2222539168 | 65856 | t
Async Queue Control | 2222537784 | 1384 | t
AutoVacuum Data | 2222533576 | 224 | t
Backend Activity Buffer | 2217099552 | 114688 | t
Backend Application Name Buffer | 2217085216 | 7168 | t
Backend Client Host Name Buffer | 2217092384 | 7168 | t
Backend Status Array | 2217061024 | 24192 | t
Background Worker Data | 2217214256 | 1992 | t
BTree Vacuum State | 2222535768 | 1356 | t
Buffer Blocks | 51365312 | 2147483648 | t
Buffer Descriptors | 34588096 | 16777216 | t
Buffer Strategy Status | 2213546176 | 32 | t
Checkpointer Data | 2217290656 | 5242920 | t
CLOG Ctl | 33601152 | 525312 | t
Control File | 16796384 | 240 | t
Fast Path Strong Relation Lock Data | 2214767072 | 4100 | t
FinishedSerializableTransactions | 2216841952 | 16 | t
LOCK hash | 2213546208 | 2160 | t
MultiXactMember Ctl | 34455488 | 131648 | t
MultiXactOffset Ctl | 34389632 | 65856 | t
OldSerXidControlData | 2216973632 | 16 | t
OldSerXid SLRU Ctl | 2216841984 | 131648 | t
PMSignalState | 2217285400 | 940 | t
PREDICATELOCK hash | 2215182944 | 2160 | t
PREDICATELOCKTARGET hash | 2214771176 | 2160 | t
PredXactList | 2216348384 | 88 | t
Prepared Transaction Table | 2217214240 | 16 | t
Proc Array | 2217060536 | 488 | t
Proc Header | 2216973648 | 88 | t
PROCLOCK hash | 2214183264 | 2160 | t
ProcSignalSlots | 2217286344 | 4284 | t
RWConflictPool | 2216573120 | 24 | t
SERIALIZABLEXID hash | 2216518720 | 2160 | t
Shared Buffer Lookup Table | 2198848960 | 16496 | t
Shared MultiXact State | 34587136 | 936 | t
shmInvalBuffer | 2217216256 | 69144 | t
SUBTRANS Ctl | 34126464 | 263168 | t
Sync Scan Locations List | 2222537128 | 656 | t
Wal Receiver Ctl | 2222534576 | 1192 | t
Wal Sender Ctl | 2222533800 | 776 | t
XLOG Ctl | 16796640 | 16804496 | t
(43 rows)
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Andres Freund <andres@2ndquadrant.com> writes:
Attached are two patches:
a) Patch addin a 'name' parameter to dsm_create(). I think we should
apply this to 9.4.
b) pg_dynamic_shmem_allocations and pg_static_shmem_allocations
views. The previous version didn't include dsm support and didn't
take the required lock.
I am not so sure whether b) should be applied together with a) in 9.4,
but I'd be happy enough to add docs if people agree with the naming.
FWIW, I vote for fixing (a) now but holding (b) for 9.5.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, May 6, 2014 at 2:34 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Andres Freund <andres@2ndquadrant.com> writes:
Attached are two patches:
a) Patch addin a 'name' parameter to dsm_create(). I think we should
apply this to 9.4.
b) pg_dynamic_shmem_allocations and pg_static_shmem_allocations
views. The previous version didn't include dsm support and didn't
take the required lock.I am not so sure whether b) should be applied together with a) in 9.4,
but I'd be happy enough to add docs if people agree with the naming.FWIW, I vote for fixing (a) now but holding (b) for 9.5.
I guess I'll vote for applying both. I don't see a lot of risk, and I
think doing one with out the other is somewhat pointless.
Regarding patch 0002, I don't think we're using the term "static
shmem" anywhere else, so I vote for dropping the word static, so that
we have pg_get_shmem_allocations() and
pg_get_dynamic_shmem_allocations(). Also, I think using the
"allocated" column is pretty weird. There are always exactly two
entries with allocated = false, one of which is for actual free memory
and the other of which is for memory that actually IS allocated but
without using ShmemIndex (maybe the latter was supposed to have
allocated = true but still key = null?). I guess I'd vote for
ditching the allocated column completely and outputting the memory
allocated without ShmemIndex using some fixed tag (like "ShmemIndex"
or "Bootstrap" or "Overhead" or something).
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Robert Haas <robertmhaas@gmail.com> writes:
On Tue, May 6, 2014 at 2:34 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
FWIW, I vote for fixing (a) now but holding (b) for 9.5.
I guess I'll vote for applying both. I don't see a lot of risk, and I
think doing one with out the other is somewhat pointless.
The difference is that there's not consensus about the details of the
views ... as borne out by your next paragraph.
Now admittedly, we could always redefine the views in 9.5, but
I'd rather not be doing this sort of thing in haste. Something
as user-visible as a system view really ought to have baked awhile
before we ship it. Patch (a) is merely institutionalizing the
expectation that DSM segments should have names, which is a much
lower-risk bet.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
