Add pg_stat_kind_info system view
Hey hackers,
In 7949d959458[0]https://github.com/postgres/postgres/commit/7949d9594582ab49dee221e1db1aa5401ace49d4, we introduced pluggable cumulative statistics.
Extensions could now register their own custom statistics. However,
there was no way for an extension author to introspect their custom
statistics through SQL, and I don't believe there is currently a way to
introspect builtin statistics either.
Additionally, there was no way to know how much shared memory each
statistics kind was using. We could only know the total shared memory
used by the statistics subsystem.
dbltap@postgres=# SELECT * FROM pg_shmem_allocations WHERE name = 'Shared Memory Stats';
name | off | size | allocated_size
---------------------+-----------+--------+----------------
Shared Memory Stats | 153456640 | 321976 | 321976
In this patch, I have added a new system view: pg_stat_kind_info built
on a new function pg_stat_get_kind_info(). The view has the following
columns:
- id: id of the statistics kind
- name: name of the statistics kind
- count: number of entries for the statistics kind
- builtin: whether the statistics kind of builtin or not
- shared_size: shared memory size of each entry
dbltap@postgres=# SELECT * FROM pg_stat_kind_info;
id | name | count | builtin | shared_size
----+-------------------------+--------+---------+-------------
1 | database | (null) | t | 288
2 | relation | (null) | t | 248
3 | function | (null) | t | 56
4 | replslot | (null) | t | 120
5 | subscription | (null) | t | 120
6 | backend | (null) | t | 2952
7 | archiver | 1 | t | 0
8 | bgwriter | 1 | t | 0
9 | checkpointer | 1 | t | 0
10 | io | 1 | t | 0
11 | lock | 1 | t | 0
12 | slru | 1 | t | 0
13 | wal | 1 | t | 0
25 | test_custom_var_stats | 0 | f | 40
26 | test_custom_fixed_stats | 1 | f | 56
(15 rows)
I am not sure that shared_size is a good column name, and I needed to
include pgstat_internal.h in pgstatfuncs.c to get everything working.
I'm also curious to hear if anyone thinks there is other valuable
information to expose.
[0]: https://github.com/postgres/postgres/commit/7949d9594582ab49dee221e1db1aa5401ace49d4
--
Tristan Partin
PostgreSQL Contributors Team
AWS (https://aws.amazon.com)
Of course I forgot to attach the patch :D.
--
Tristan Partin
PostgreSQL Contributors Team
AWS (https://aws.amazon.com)
Attachments:
v1-0001-Add-pg_state_kind_info-view-and-pg_stat_get_kind_.patchtext/x-patch; charset=utf-8; name=v1-0001-Add-pg_state_kind_info-view-and-pg_stat_get_kind_.patchDownload+229-2
On Thu, Apr 30, 2026 at 07:38:57PM +0000, Tristan Partin wrote:
Of course I forgot to attach the patch :D.
Thanks for the patch. That sounds like a good idea particularly for
track_entry_count, because we have no real way to provide this
information, which would be more valuable if an in-core stats kind has
the idea to switch this flag to true in the future, and I don't really
want all extensions to re-implement the same thing to access this
information.
+ if (info->track_entry_count)
+ {
+ values[2] = Int64GetDatum(pgstat_get_entry_count(kind));
+ }
+ else
+ {
+ nulls[2] = true;
+ }
Returning NULL if the flag is not set sounds sensible. For
fixed-numbered, fine by me for 1, so as it is possible to aggregate
the full size counting with the fixed shmem size of each stats kind.
+SELECT name, builtin FROM pg_stat_kind_info
+ ORDER BY name COLLATE "C";
This is not going to be stable if running installcheck on an instance
where a custom kind is loaded, so let's restrict the query to check
for built-in kinds.
I'd suggest to expand the data published to more fields and not only
what's presented here, so as it becomes possible to do more SQL sanity
checks with the stats kind info (same attribute name as the fields):
- fixed_amount, where shared_size > 0 does not make sense.
- snapshot_ctl_off and shared_ctl_off, that only makes sense under
fixed_amount. These should not be set for !fixed_amount.
- pending_size, that should not be set for fixed_amount.
- existence of flush_pending_cb, delete_pending_cb,
reset_timestamp_cb, to_serialized_name, from_serialized_name,
to_serialized_data, from_serialized_data (should be booleans), fine
for !fixed_amount, never for fixed_amount.
- existence of init_shmem_cb, reset_all_cb, snapshot_cb (should be
booleans), fine for fixed_amount, never for !fixed_amount.
I may be missing one or two things. pgstat_internal.h documents all
these requirements, the idea is to translate these requirements at SQL
level. We should definitely apply these checks for both custom and
built-in stats kinds, which should save time for developers of pgstats
in core and outside of core by detecting inconsistent patterns
beforehand.
--
Michael
On Thu May 14, 2026 at 5:30 AM UTC, Michael Paquier wrote:
On Thu, Apr 30, 2026 at 07:38:57PM +0000, Tristan Partin wrote:
Of course I forgot to attach the patch :D.
Thanks for the patch. That sounds like a good idea particularly for
track_entry_count, because we have no real way to provide this
information, which would be more valuable if an in-core stats kind has
the idea to switch this flag to true in the future, and I don't really
want all extensions to re-implement the same thing to access this
information.+ if (info->track_entry_count) + { + values[2] = Int64GetDatum(pgstat_get_entry_count(kind)); + } + else + { + nulls[2] = true; + }Returning NULL if the flag is not set sounds sensible. For
fixed-numbered, fine by me for 1, so as it is possible to aggregate
the full size counting with the fixed shmem size of each stats kind.
Awesome.
+SELECT name, builtin FROM pg_stat_kind_info + ORDER BY name COLLATE "C";This is not going to be stable if running installcheck on an instance
where a custom kind is loaded, so let's restrict the query to check
for built-in kinds.
I'll fix this in v2, which I will send after resolving more discussion.
I had not considered this as a potential problem. I'll remove the
builtin column from the SELECT and filter on builtin as suggested.
I'd suggest to expand the data published to more fields and not only
what's presented here, so as it becomes possible to do more SQL sanity
checks with the stats kind info (same attribute name as the fields):
- fixed_amount, where shared_size > 0 does not make sense.
- snapshot_ctl_off and shared_ctl_off, that only makes sense under
fixed_amount. These should not be set for !fixed_amount.
- pending_size, that should not be set for fixed_amount.
- existence of flush_pending_cb, delete_pending_cb,
reset_timestamp_cb, to_serialized_name, from_serialized_name,
to_serialized_data, from_serialized_data (should be booleans), fine
for !fixed_amount, never for fixed_amount.
- existence of init_shmem_cb, reset_all_cb, snapshot_cb (should be
booleans), fine for fixed_amount, never for !fixed_amount.
Can you share how someone might use this additional information? I can
see some inherent value for additionally exposing:
- fixed_amount
- accessed_across_databases
- write_to_file
- snapshot_ctl_off
- shared_ctl_off
- shared_data_off
- shared_data_len
- pending_size
I may be missing one or two things. pgstat_internal.h documents all
these requirements, the idea is to translate these requirements at SQL
level. We should definitely apply these checks for both custom and
built-in stats kinds, which should save time for developers of pgstats
in core and outside of core by detecting inconsistent patterns
beforehand.
Not sure I understand why we would want to expose the existence of the
callbacks to make assertions at the SQL level. I see that in
pgstat_register_kind(), we have the following code:
/*
* Check some data for fixed-numbered stats.
*/
if (kind_info->fixed_amount)
{
if (kind_info->shared_size == 0)
ereport(ERROR,
(errmsg("custom cumulative statistics property is invalid"),
errhint("Custom cumulative statistics require a shared memory size for fixed-numbered objects.")));
if (kind_info->track_entry_count)
ereport(ERROR,
(errmsg("custom cumulative statistics property is invalid"),
errhint("Custom cumulative statistics cannot use entry count tracking for fixed-numbered objects.")));
}
We could extend these invariant checks to make sure that the callbacks
are only set when fixed_amount is true for instance. I am very much open
to having my mind changed.
--
Tristan Partin
PostgreSQL Contributors Team
AWS (https://aws.amazon.com)
On Thu, May 28, 2026 at 05:11:42PM +0000, Tristan Partin wrote:
Can you share how someone might use this additional information?
Not sure I understand why we would want to expose the existence of the
callbacks to make assertions at the SQL level. I see that in
pgstat_register_kind(), we have the following code:We could extend these invariant checks to make sure that the callbacks
are only set when fixed_amount is true for instance. I am very much open
to having my mind changed.
There is currently no internal mechanism to make sure that the
built-in stats kinds have a consistent setup in terms of flags and
callbacks set, so for developers we could immediately complain when
generating patches that add new stats kinds. For custom stats kinds,
loaded libraries could have more cross-checks.
Perhaps it is not worth bothering at the end, and I'd be fine to keep
the data published as minimal as you see fit. Still, fixed_amount,
write_to_file and accessed_across_databases feel like useful
additions.
If we keep shared_size, it may make sense to set it to NULL if we
don't know about it? That's the case of the built-in fixed-sized
stats kinds. We set the value for custom fixed-sized stats kinds.
--
Michael
On Mon Jun 1, 2026 at 5:41 AM UTC, Michael Paquier wrote:
On Thu, May 28, 2026 at 05:11:42PM +0000, Tristan Partin wrote:
Can you share how someone might use this additional information?
Not sure I understand why we would want to expose the existence of the
callbacks to make assertions at the SQL level. I see that in
pgstat_register_kind(), we have the following code:We could extend these invariant checks to make sure that the callbacks
are only set when fixed_amount is true for instance. I am very much open
to having my mind changed.There is currently no internal mechanism to make sure that the
built-in stats kinds have a consistent setup in terms of flags and
callbacks set, so for developers we could immediately complain when
generating patches that add new stats kinds. For custom stats kinds,
loaded libraries could have more cross-checks.
I think there is still some confusion on my end about this line of
discussion.
Perhaps it is not worth bothering at the end, and I'd be fine to keep
the data published as minimal as you see fit. Still, fixed_amount,
write_to_file and accessed_across_databases feel like useful
additions.
The additional columns are now added in v2. Note that I named
write_to_file as written_to_file for the column name. I wonder if
persisted would be a better name for the column and if persisted would
be a better name for PgStat_KindInfo::write_to_file.
If we keep shared_size, it may make sense to set it to NULL if we
don't know about it? That's the case of the built-in fixed-sized
stats kinds. We set the value for custom fixed-sized stats kinds.
I believe that I also captured this correctly in v2.
--
Tristan Partin
PostgreSQL Contributors Team
AWS (https://aws.amazon.com)
On Thu, Jun 04, 2026 at 04:59:00PM +0000, Tristan Partin wrote:
I believe that I also captured this correctly in v2.
No v2 of the patch has been attached. :D
--
Michael
Hi all,
On Wed, Jun 10, 2026 at 3:34 PM Tristan Partin <tristan@partin.io> wrote:
Of course I forgot to attach the patch :D.
--
I reviewed the v1 patch and had verified that both the view and the
function are created successfully and return identical results. The
builtin statistics kinds are exposed correctly, and the IDs and names
are unique without any duplicates. The count column behaves as
expected, returning NULL for statistics kinds that do not support
entry count tracking and 1 for fixed statistics kinds. While reviewing
the patch, I found it may be useful to expose a boolean column such as
track_entry_count in pg_stat_kind_info that would explicitly indicate
whether entry count tracking is enabled for a statistics kind and also
would help users understand why the count column is NULL for certain
statistics kinds. Exposing this internal property would make the view
more self-explanatory and improve its usefulness for debugging. Apart
from these suggestions, I did not find any issues with the current
implementation.
Looking forward to more feedback.
Regards,
Solai