[PATCH] Optionally record Plan IDs to track plan changes for a query
Hi all,
Inspired by a prior proposal by Sami Imseih for tracking Plan IDs [0]/messages/by-id/604E3199-2DD2-47DD-AC47-774A6F97DCA9@amazon.com, as
well as extensions like pg_stat_plans [1]https://github.com/2ndQuadrant/pg_stat_plans (unmaintained), pg_store_plans
[2]: https://ossc-db.github.io/pg_store_plans/
(enabled by default on AWS), this proposed patch set adds:
1. An updated in-core facility to optionally track Plan IDs based on
hashing the plan nodes during the existing treewalk in setrefs.c -
controlled by the new "compute_plan_id" GUC
2. An example user of plan IDs with a new pg_stat_plans extension in
contrib, that also records the first plan text with EXPLAIN (COSTS OFF)
My overall perspective is that (1) is best done in-core to keep overhead
low, whilst (2) could be done outside of core (or merged with a future
pg_stat_statements) and is included here mainly for illustration purposes.
Notes including what constitutes a plan ID follow, after a quick example:
## Example
Having the planid + an extension that records it, plus the first plan text,
lets you track different plans for the same query:
bench=# SELECT * FROM pgbench_accounts WHERE aid = 123;
bench=# SET enable_indexscan = off;
bench=# SELECT * FROM pgbench_accounts WHERE aid = 123;
bench=# SELECT queryid, planid, plan FROM pg_stat_plans WHERE plan LIKE
'%pgbench%';
queryid | planid |
plan
----------------------+----------------------+------------------------------------------------------------
-5986989572677096226 | -2057350818695327558 | Index Scan using
pgbench_accounts_pkey on pgbench_accounts+
| | Index Cond: (aid = 123)
-5986989572677096226 | 2815444815385882663 | Bitmap Heap Scan on
pgbench_accounts +
| | Recheck Cond: (aid = 123)
+
| | -> Bitmap Index Scan on
pgbench_accounts_pkey +
| | Index Cond: (aid =
123)
And this also supports showing the plan for a currently running query (call
count is zero in such cases):
session 1:
bench# SELECT pg_sleep(100), COUNT(*) FROM pgbench_accounts;
session 2:
bench=# SELECT query, plan FROM pg_stat_activity
JOIN pg_stat_plans ON (usesysid = userid AND datid = dbid AND query_id =
queryid AND plan_id = planid)
WHERE query LIKE 'SELECT pg_sleep%';
query |
plan
-------------------------------------------------------+------------------------------------
SELECT pg_sleep(100), COUNT(*) FROM pgbench_accounts; | Aggregate
+
| -> Seq Scan on
pgbench_accounts
## What is a plan ID?
My overall hypothesis here is that identifying different plan shapes for
the same normalized query (i.e. queryid) is useful, because it lets you
detect use of different plan choices such as which join order or index was
used based on different input parameters (or different column statistics
due to a recent ANALYZE) for the same normalized query.
You can get this individually for a given query with EXPLAIN of course, but
if you want to track this over time the only workable mechanism in my
experience is auto_explain, which is good for sampling outliers, but bad
for getting a comprehensive view of which plans where used and how often.
To me the closest to what I consider a "plan shape" is the output of
EXPLAIN (COSTS OFF), that is, the plan nodes and their filters/conditions,
but discarding the exact costs as well as ignoring any execution
statistics. The idea behind the proposed plan ID implementation is trying
to match that by hashing plan nodes, similar to how query IDs hash
post-parse analysis query nodes.
One notable edge case are plans that involve partitions - those could of
course lead to a lot of different planids for a given queryid, based on how
many partitions were pruned. We could consider special casing this, e.g. by
trying to be smart about declarative partitioning, and considering plans to
be identical if they scan the same number of partitions with the same scan
methods. However this could also be done by an out-of-core extension,
either by defining a better planid mechanism, or maintaining a grouped
planid of sorts based on the internal planid.
The partitions problem reminds me a bit of the IN list problem with
pg_stat_statements (which we still haven't resolved) - despite the problem
the extension has been successfully used for many years by many Postgres
users, even for those workloads where you have thousands of entries for the
same query with different IN list lengths.
## Why does this need to be in core?
Unfortunately both existing open-source extensions I'm familiar with are
not suitable for production use. Out of the two, only pg_store_plans [2]https://ossc-db.github.io/pg_store_plans/ is
being maintained, however it carries significant overhead because it
calculates the plan ID by hashing the EXPLAIN text output every time a
query is executed.
My colleague Marko (CCed) and I evaluated whether pg_store_plans could be
modified to instead calculate the planid by hashing the plan tree, and ran
into three issues:
1. The existing node jumbling in core is not usable by extensions, and it
is necessary to have something like it for hashing Filters/Conds
(ultimately requiring us to duplicate all of it in the extension, and keep
maintaining that for every major release)
2. Whilst its cheap enough, it seems unnecessary to do an additional tree
walk when setrefs.c already walks the plan tree in a near-final state
3. It seems useful to enable showing the plan shape of a currently running
query (e.g. to identify whether a plan regression causes the query to run
forever), and this is much easier to do by adding planid to
pg_stat_activity, like the queryid
I also suspect that Aurora's implementation in [3]https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora_stat_plans.html had some in-core
modifications to enable it work efficiently, but I'm not familiar with any
implementation details beyond what's in the public documentation.
## Implementation notes
The attached patch set includes two preparatory patches that could be
committed independently if deemed useful:
The first patch allows use of node jumbling by other unit files /
extensions, which would help an out-of-core extension avoid duplicating all
the node jumbling code.
The second patch adds a function for the extensible cumulative statistics
system to drop all entries for a given statistics kind. This already exists
for resetting, but in case of a dynamic list of entries its more useful to
be able to drop all of them when "reset" is called.
The third patch adds plan ID tracking in core. This is turned off by
default, and can be enabled by setting "compute_plan_id" to "on". Plan IDs
are shown in pg_stat_activity, as well as EXPLAIN and auto_explain output,
to allow matching a given plan ID to a plan text, without requiring the use
of an extension. There are some minor TODOs in the plan jumbling logic that
I haven't finalized yet. There is also an open question whether we should
use the node attribute mechanism instead of custom jumbling logic?
The fourth patch adds the pg_stat_plans contrib extension, for illustrative
purposes. This is inspired by pg_stat_statements, but intentionally kept
separate for easier review and since it does not use an external file and
could technically be used independently. We may want to develop this into a
unified pg_stat_statements+plans in-core mechanism in the future, but I
think that is best kept for a separate discussion.
The pg_stat_plans extension utilizes the cumulative statistics system for
tracking statistics (extensible thanks to recent changes!), as well as
dynamic shared memory to track plan texts up to a given limit (2kB by
default). As a side note, managing extra allocations with the new
extensible stats is a bit cumbersome - it would be helpful to have a hook
for cleaning up data associated to entries (like a DSA allocation).
Thanks,
Lukas
[0]: /messages/by-id/604E3199-2DD2-47DD-AC47-774A6F97DCA9@amazon.com
/messages/by-id/604E3199-2DD2-47DD-AC47-774A6F97DCA9@amazon.com
[1]: https://github.com/2ndQuadrant/pg_stat_plans
[2]: https://ossc-db.github.io/pg_store_plans/
[3]: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora_stat_plans.html
https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora_stat_plans.html
--
Lukas Fittl
Attachments:
0002-Cumulative-statistics-Add-pgstat_drop_entries_of_kin.patchapplication/octet-stream; name=0002-Cumulative-statistics-Add-pgstat_drop_entries_of_kin.patchDownload
From d61a9c94325f980c76d548be6db6f6e86f96e0cd Mon Sep 17 00:00:00 2001
From: Lukas Fittl <lukas@fittl.com>
Date: Thu, 2 Jan 2025 10:46:30 -0800
Subject: [PATCH 2/4] Cumulative statistics: Add pgstat_drop_entries_of_kind
helper
This allows users of the cumulative statistics systems to drop all
entries for a given kind, similar to how pgstat_reset_entries_of_kind
allows resetting all entreis for a given statistics kind.
---
src/backend/utils/activity/pgstat_shmem.c | 33 +++++++++++++++++++++++
src/include/utils/pgstat_internal.h | 1 +
2 files changed, 34 insertions(+)
diff --git a/src/backend/utils/activity/pgstat_shmem.c b/src/backend/utils/activity/pgstat_shmem.c
index bcc8b2eb4f..43f2dfd695 100644
--- a/src/backend/utils/activity/pgstat_shmem.c
+++ b/src/backend/utils/activity/pgstat_shmem.c
@@ -1015,6 +1015,39 @@ pgstat_drop_all_entries(void)
pgstat_request_entry_refs_gc();
}
+void
+pgstat_drop_entries_of_kind(PgStat_Kind kind)
+{
+ dshash_seq_status hstat;
+ PgStatShared_HashEntry *ps;
+ uint64 not_freed_count = 0;
+
+ dshash_seq_init(&hstat, pgStatLocal.shared_hash, true);
+ while ((ps = dshash_seq_next(&hstat)) != NULL)
+ {
+ if (ps->dropped || ps->key.kind != kind)
+ continue;
+
+ /* delete local reference */
+ if (pgStatEntryRefHash)
+ {
+ PgStat_EntryRefHashEntry *lohashent =
+ pgstat_entry_ref_hash_lookup(pgStatEntryRefHash, ps->key);
+
+ if (lohashent)
+ pgstat_release_entry_ref(lohashent->key, lohashent->entry_ref,
+ true);
+ }
+
+ if (!pgstat_drop_entry_internal(ps, &hstat))
+ not_freed_count++;
+ }
+ dshash_seq_term(&hstat);
+
+ if (not_freed_count > 0)
+ pgstat_request_entry_refs_gc();
+}
+
static void
shared_stat_reset_contents(PgStat_Kind kind, PgStatShared_Common *header,
TimestampTz ts)
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index 811ed9b005..900f71c033 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -707,6 +707,7 @@ extern bool pgstat_lock_entry_shared(PgStat_EntryRef *entry_ref, bool nowait);
extern void pgstat_unlock_entry(PgStat_EntryRef *entry_ref);
extern bool pgstat_drop_entry(PgStat_Kind kind, Oid dboid, uint64 objid);
extern void pgstat_drop_all_entries(void);
+extern void pgstat_drop_entries_of_kind(PgStat_Kind kind);
extern PgStat_EntryRef *pgstat_get_entry_ref_locked(PgStat_Kind kind, Oid dboid, uint64 objid,
bool nowait);
extern void pgstat_reset_entry(PgStat_Kind kind, Oid dboid, uint64 objid, TimestampTz ts);
--
2.47.1
0003-Optionally-record-a-plan_id-in-PlannedStmt-to-identi.patchapplication/octet-stream; name=0003-Optionally-record-a-plan_id-in-PlannedStmt-to-identi.patchDownload
From 5b5d84f3d5a25ef47fa9042985a8dfe33c76473c Mon Sep 17 00:00:00 2001
From: Lukas Fittl <lukas@fittl.com>
Date: Tue, 31 Dec 2024 15:16:10 -0800
Subject: [PATCH 3/4] Optionally record a plan_id in PlannedStmt to identify
plan shape
When enabled via the new compute_plan_id GUC (default off), this utilizes
the existing treewalk in setrefs.c after planning to calculate a hash
(the "plan_id", or plan identifier) that can be used to identify
which plan was chosen.
The plan_id generally intends to be the same if a given EXPLAIN (without
ANALYZE) output is the same. The plan_id includes both the top-level plan
as well as all subplans. Execution statistics are excluded.
If enabled, the plan_id is shown for currently running queries in
pg_stat_activity, as well as recorded in EXPLAIN and auto_explain output.
Other in core users or extensions can use this facility to show or
accumulate statistics about the plans used by queries, to help identify
plan regressions, or drive plan management decisions.
Note that this commit intentionally does not include a facility to map
a given plan_id to the EXPLAIN text output - it is a assumed that users
can utilize the auto_explain extension to establish this mapping as
needed, or extensions can record this via the existing planner hook.
---
doc/src/sgml/config.sgml | 32 ++
doc/src/sgml/monitoring.sgml | 16 +
src/backend/catalog/system_views.sql | 1 +
src/backend/commands/explain.c | 17 +
src/backend/executor/execMain.c | 8 +-
src/backend/executor/execParallel.c | 1 +
src/backend/nodes/Makefile | 1 +
src/backend/nodes/meson.build | 1 +
src/backend/nodes/planjumble.c | 451 ++++++++++++++++++
src/backend/optimizer/plan/planner.c | 17 +
src/backend/optimizer/plan/setrefs.c | 8 +
src/backend/tcop/postgres.c | 1 +
src/backend/utils/activity/backend_status.c | 70 ++-
src/backend/utils/adt/pgstatfuncs.c | 7 +-
src/backend/utils/misc/guc_tables.c | 28 ++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/catalog/pg_proc.dat | 6 +-
src/include/nodes/pathnodes.h | 3 +
src/include/nodes/planjumble.h | 33 ++
src/include/nodes/plannodes.h | 4 +-
src/include/nodes/primnodes.h | 6 +-
src/include/utils/backend_status.h | 5 +
src/test/regress/expected/explain.out | 11 +
src/test/regress/expected/rules.out | 9 +-
src/test/regress/sql/explain.sql | 4 +
25 files changed, 722 insertions(+), 19 deletions(-)
create mode 100644 src/backend/nodes/planjumble.c
create mode 100644 src/include/nodes/planjumble.h
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index fbdd6ce574..196d64efd8 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -8546,6 +8546,38 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-compute-plan-id" xreflabel="compute_plan_id">
+ <term><varname>compute_plan_id</varname> (<type>enum</type>)
+ <indexterm>
+ <primary><varname>compute_plan_id</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Enables in-core computation of a plan identifier.
+ Plan identifiers can be displayed in the <link
+ linkend="monitoring-pg-stat-activity-view"><structname>pg_stat_activity</structname></link>
+ view or using <command>EXPLAIN</command>.
+ Note that an external module can alternatively be used if the
+ in-core plan identifier computation method is not acceptable.
+ In this case, in-core computation must be always disabled.
+ Valid values are <literal>off</literal> (always disabled),
+ <literal>on</literal> (always enabled) and <literal>regress</literal> which
+ has the same effect as <literal>on</literal>, except that the
+ query identifier is not shown in the <literal>EXPLAIN</literal> output
+ in order to facilitate automated regression testing.
+ The default is <literal>off</literal>.
+ </para>
+ <note>
+ <para>
+ To ensure that only one plan identifier is calculated and
+ displayed, extensions that calculate plan identifiers should
+ throw an error if a plan identifier has already been computed.
+ </para>
+ </note>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-log-statement-stats">
<term><varname>log_statement_stats</varname> (<type>boolean</type>)
<indexterm>
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index d0d176cc54..85b38ee642 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -972,6 +972,22 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>plan_id</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Identifier of this backend's most recent query plan. If
+ <structfield>state</structfield> is <literal>active</literal> this
+ field shows the identifier of the currently executing query plan. In
+ all other states, it shows the identifier of last query plan that
+ was executed. Plan identifiers are not computed by default so this
+ field will be null unless <xref linkend="guc-compute-plan-id"/>
+ parameter is enabled or a third-party module that computes plan
+ identifiers is configured.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>query</structfield> <type>text</type>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index da9a8fe99f..a26e00a792 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -880,6 +880,7 @@ CREATE VIEW pg_stat_activity AS
S.backend_xid,
s.backend_xmin,
S.query_id,
+ S.plan_id,
S.query,
S.backend_type
FROM pg_stat_get_activity(NULL) AS S
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index a201ed3082..deeedf70cc 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -25,6 +25,7 @@
#include "nodes/extensible.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
+#include "nodes/planjumble.h"
#include "parser/analyze.h"
#include "parser/parsetree.h"
#include "rewrite/rewriteHandler.h"
@@ -966,6 +967,22 @@ ExplainPrintPlan(ExplainState *es, QueryDesc *queryDesc)
ExplainPropertyInteger("Query Identifier", NULL, (int64)
queryDesc->plannedstmt->queryId, es);
}
+
+ /*
+ * COMPUTE_PLAN_ID_REGRESS means COMPUTE_PLAN_ID_YES, but we don't show
+ * the queryid in any of the EXPLAIN plans to keep stable the results
+ * generated by regression test suites.
+ */
+ if (es->verbose && queryDesc->plannedstmt->planId != UINT64CONST(0) &&
+ compute_plan_id != COMPUTE_PLAN_ID_REGRESS)
+ {
+ /*
+ * Output the queryid as an int64 rather than a uint64 so we match
+ * what would be seen in the BIGINT pg_stat_activity.plan_id column.
+ */
+ ExplainPropertyInteger("Plan Identifier", NULL, (int64)
+ queryDesc->plannedstmt->planId, es);
+ }
}
/*
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 1c12d6ebff..19becdd66c 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -120,13 +120,15 @@ ExecutorStart(QueryDesc *queryDesc, int eflags)
{
/*
* In some cases (e.g. an EXECUTE statement or an execute message with the
- * extended query protocol) the query_id won't be reported, so do it now.
+ * extended query protocol) the query_id and plan_id won't be reported,
+ * so do it now.
*
- * Note that it's harmless to report the query_id multiple times, as the
- * call will be ignored if the top level query_id has already been
+ * Note that it's harmless to report the identifiers multiple times, as the
+ * call will be ignored if the top level query_id / plan_id has already been
* reported.
*/
pgstat_report_query_id(queryDesc->plannedstmt->queryId, false);
+ pgstat_report_plan_id(queryDesc->plannedstmt->planId, queryDesc->plannedstmt->queryId, false);
if (ExecutorStart_hook)
(*ExecutorStart_hook) (queryDesc, eflags);
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 846ec727de..54cb22967e 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -174,6 +174,7 @@ ExecSerializePlan(Plan *plan, EState *estate)
pstmt = makeNode(PlannedStmt);
pstmt->commandType = CMD_SELECT;
pstmt->queryId = pgstat_get_my_query_id();
+ pstmt->planId = pgstat_get_my_plan_id();
pstmt->hasReturning = false;
pstmt->hasModifyingCTE = false;
pstmt->canSetTag = true;
diff --git a/src/backend/nodes/Makefile b/src/backend/nodes/Makefile
index 66bbad8e6e..1ab9345d28 100644
--- a/src/backend/nodes/Makefile
+++ b/src/backend/nodes/Makefile
@@ -25,6 +25,7 @@ OBJS = \
nodeFuncs.o \
outfuncs.o \
params.o \
+ planjumble.o \
print.o \
queryjumblefuncs.o \
read.o \
diff --git a/src/backend/nodes/meson.build b/src/backend/nodes/meson.build
index 49626f160e..4640e7a361 100644
--- a/src/backend/nodes/meson.build
+++ b/src/backend/nodes/meson.build
@@ -8,6 +8,7 @@ backend_sources += files(
'multibitmapset.c',
'nodeFuncs.c',
'params.c',
+ 'planjumble.c',
'print.c',
'read.c',
'tidbitmap.c',
diff --git a/src/backend/nodes/planjumble.c b/src/backend/nodes/planjumble.c
new file mode 100644
index 0000000000..a2f580d771
--- /dev/null
+++ b/src/backend/nodes/planjumble.c
@@ -0,0 +1,451 @@
+/*-------------------------------------------------------------------------
+ *
+ * planjumble.c
+ * Plan fingerprinting.
+ *
+ * Calculates the plan fingerprint for a given plan tree. Note this works
+ * in combination with the planner's setrefs functionality in order to
+ * walk the tree.
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/nodes/planjumble.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "nodes/planjumble.h"
+#include "parser/parsetree.h"
+
+#define JUMBLE_SIZE 1024 /* query serialization buffer size */
+
+/* GUC parameters */
+int compute_plan_id = COMPUTE_PLAN_ID_OFF;
+
+#define JUMBLE_VALUE(item) \
+ AppendJumble(jstate, (const unsigned char *) &(item), sizeof(item))
+#define JUMBLE_STRING(str) \
+do { \
+ if (str) \
+ AppendJumble(jstate, (const unsigned char *) (str), strlen(str) + 1); \
+} while(0)
+
+/*
+ * Jumble the target relation of a scan or modify node
+ *
+ * This functions similarly to ExplainTargetRel.
+ */
+static void
+JumbleTargetRel(JumbleState *jstate, List* rtable, Plan *plan, Index rti)
+{
+ RangeTblEntry *rte = rt_fetch(rti, rtable);
+
+ switch (nodeTag(plan))
+ {
+ case T_SeqScan:
+ case T_SampleScan:
+ case T_IndexScan:
+ case T_IndexOnlyScan:
+ case T_BitmapHeapScan:
+ case T_TidScan:
+ case T_TidRangeScan:
+ case T_ForeignScan:
+ case T_CustomScan:
+ case T_ModifyTable:
+ /* Assert it's on a real relation */
+ Assert(rte->rtekind == RTE_RELATION);
+ JUMBLE_VALUE(rte->relid);
+ break;
+ case T_TableFuncScan:
+ {
+ TableFunc *tablefunc = ((TableFuncScan *) plan)->tablefunc;
+
+ Assert(rte->rtekind == RTE_TABLEFUNC);
+ JUMBLE_VALUE(tablefunc->functype);
+ }
+ break;
+ case T_ValuesScan:
+ Assert(rte->rtekind == RTE_VALUES);
+ break;
+ case T_CteScan:
+ /* Assert it's on a non-self-reference CTE */
+ Assert(rte->rtekind == RTE_CTE);
+ Assert(!rte->self_reference);
+ JUMBLE_STRING(rte->ctename);
+ break;
+ case T_NamedTuplestoreScan:
+ Assert(rte->rtekind == RTE_NAMEDTUPLESTORE);
+ JUMBLE_STRING(rte->enrname);
+ break;
+ case T_WorkTableScan:
+ /* Assert it's on a self-reference CTE */
+ Assert(rte->rtekind == RTE_CTE);
+ Assert(rte->self_reference);
+ JUMBLE_STRING(rte->ctename);
+ break;
+ default:
+ break;
+ }
+}
+
+/*
+ * Jumble the target of a Scan node
+ */
+static void
+JumbleScanTarget(JumbleState *jstate, List* rtable, Scan *scan)
+{
+ JumbleTargetRel(jstate, rtable, (Plan *) scan, scan->scanrelid);
+}
+
+/*
+ * JumblePlanNode: Append significant information to the plan identifier jumble
+ *
+ * Note this intentionally doesn't descend into child plan nodes, since the caller
+ * already takes care of that.
+ */
+void
+JumblePlanNode(JumbleState *jstate, List* rtable, Plan *plan)
+{
+ JUMBLE_VALUE(nodeTag(plan));
+ JumbleNode(jstate, (Node *) plan->qual);
+ JumbleNode(jstate, (Node *) plan->targetlist);
+
+ /*
+ * Plan-type-specific fixes
+ */
+ switch (nodeTag(plan))
+ {
+ case T_SeqScan:
+ {
+ SeqScan *splan = (SeqScan *) plan;
+ JumbleScanTarget(jstate, rtable, &splan->scan);
+ }
+ break;
+ case T_SampleScan:
+ {
+ SampleScan *splan = (SampleScan *) plan;
+ JumbleScanTarget(jstate, rtable, &splan->scan);
+ /* TODO: It may be worth jumbling the properties of splan->tablesample */
+ }
+ break;
+ case T_IndexScan:
+ {
+ IndexScan *splan = (IndexScan *) plan;
+ JumbleScanTarget(jstate, rtable, &splan->scan);
+ JUMBLE_VALUE(splan->indexid);
+ JumbleNode(jstate, (Node *) splan->indexqual);
+ /* Skip splan->indexqualorig */
+ JumbleNode(jstate, (Node *) splan->indexorderby);
+ /* Skip splan->indexorderbyorig */
+ JUMBLE_VALUE(splan->indexorderdir);
+ }
+ break;
+ case T_IndexOnlyScan:
+ {
+ IndexOnlyScan *splan = (IndexOnlyScan *) plan;
+ JumbleScanTarget(jstate, rtable, &splan->scan);
+ JUMBLE_VALUE(splan->indexid);
+ JumbleNode(jstate, (Node *) splan->indexqual);
+ /* Skip splan->recheckqual */
+ JumbleNode(jstate, (Node *) splan->indexorderby);
+ /* Skip splan->indextlist */
+ JUMBLE_VALUE(splan->indexorderdir);
+ }
+ break;
+ case T_BitmapIndexScan:
+ {
+ BitmapIndexScan *splan = (BitmapIndexScan *) plan;
+ JumbleScanTarget(jstate, rtable, &splan->scan);
+ JUMBLE_VALUE(splan->indexid);
+ JumbleNode(jstate, (Node *) splan->indexqual);
+ /* Skip splan->indexqualorig */
+ }
+ break;
+ case T_BitmapHeapScan:
+ {
+ BitmapHeapScan *splan = (BitmapHeapScan *) plan;
+ JumbleScanTarget(jstate, rtable, &splan->scan);
+ /* Skip splan->bitmapqualorig */
+ }
+ break;
+ case T_TidScan:
+ {
+ TidScan *splan = (TidScan *) plan;
+ JumbleScanTarget(jstate, rtable, &splan->scan);
+ JumbleNode(jstate, (Node *) splan->tidquals);
+ }
+ break;
+ case T_TidRangeScan:
+ {
+ TidRangeScan *splan = (TidRangeScan *) plan;
+ JumbleScanTarget(jstate, rtable, &splan->scan);
+ JumbleNode(jstate, (Node *) splan->tidrangequals);
+ }
+ break;
+ case T_SubqueryScan:
+ {
+ SubqueryScan *splan = (SubqueryScan *) plan;
+ /* TODO: JumbleScanHeader currently doesn't jumble the subplan name */
+ JumbleScanTarget(jstate, rtable, &splan->scan);
+ /* We rely on the caller to descend into the actual subplans */
+ }
+ break;
+ case T_FunctionScan:
+ {
+ FunctionScan *splan = (FunctionScan *) plan;
+
+ /*
+ * If the expression is still a call of a single function,
+ * we can jumble the OID of the function. Otherwise, punt.
+ * (Even if it was a single function call originally, the
+ * optimizer could have simplified it away.)
+ */
+ if (list_length(splan->functions) == 1)
+ {
+ RangeTblFunction *rtfunc = (RangeTblFunction *) linitial(splan->functions);
+
+ if (IsA(rtfunc->funcexpr, FuncExpr))
+ {
+ FuncExpr *funcexpr = (FuncExpr *) rtfunc->funcexpr;
+ Oid funcid = funcexpr->funcid;
+ JUMBLE_VALUE(funcid);
+ }
+ }
+ }
+ break;
+ case T_TableFuncScan:
+ {
+ TableFuncScan *splan = (TableFuncScan *) plan;
+ JumbleScanTarget(jstate, rtable, &splan->scan);
+ /* TODO: Should we jumble splan->tablefunc? */
+ }
+ break;
+ case T_ValuesScan:
+ {
+ ValuesScan *splan = (ValuesScan *) plan;
+ JumbleScanTarget(jstate, rtable, &splan->scan);
+ JumbleNode(jstate, (Node *) splan->values_lists);
+ }
+ break;
+ case T_CteScan:
+ {
+ CteScan *splan = (CteScan *) plan;
+ JumbleScanTarget(jstate, rtable, &splan->scan);
+ }
+ break;
+ case T_NamedTuplestoreScan:
+ {
+ NamedTuplestoreScan *splan = (NamedTuplestoreScan *) plan;
+ JumbleScanTarget(jstate, rtable, &splan->scan);
+ }
+ break;
+ case T_WorkTableScan:
+ {
+ WorkTableScan *splan = (WorkTableScan *) plan;
+ JumbleScanTarget(jstate, rtable, &splan->scan);
+ }
+ break;
+ case T_ForeignScan:
+ {
+ ForeignScan *splan = (ForeignScan *) plan;
+ JumbleScanTarget(jstate, rtable, &splan->scan);
+ /* TODO: Should we jumble any FDW-specific information here, like EXPLAIN? */
+ }
+ break;
+ case T_CustomScan:
+ {
+ ForeignScan *splan = (ForeignScan *) plan;
+ JumbleScanTarget(jstate, rtable, &splan->scan);
+ }
+ break;
+ case T_NestLoop:
+ {
+ NestLoop *jplan = (NestLoop *) plan;
+ JUMBLE_VALUE(jplan->join.jointype);
+ /* Skip jplan->join.inner_unique */
+ JumbleNode(jstate, (Node *) jplan->join.joinqual);
+ JumbleNode(jstate, (Node *) jplan->nestParams);
+ }
+ break;
+ case T_MergeJoin:
+ {
+ MergeJoin *jplan = (MergeJoin *) plan;
+ JUMBLE_VALUE(jplan->join.jointype);
+ /* Skip jplan->join.inner_unique */
+ JumbleNode(jstate, (Node *) jplan->join.joinqual);
+ JumbleNode(jstate, (Node *) jplan->mergeclauses);
+ JumbleNode(jstate, (Node *) jplan->mergeFamilies);
+ JumbleNode(jstate, (Node *) jplan->mergeCollations);
+ JumbleNode(jstate, (Node *) jplan->mergeReversals);
+ JumbleNode(jstate, (Node *) jplan->mergeNullsFirst);
+ }
+ break;
+ case T_HashJoin:
+ {
+ HashJoin *jplan = (HashJoin *) plan;
+ JUMBLE_VALUE(jplan->join.jointype);
+ /* Skip jplan->join.inner_unique */
+ JumbleNode(jstate, (Node *) jplan->join.joinqual);
+ JumbleNode(jstate, (Node *) jplan->hashclauses);
+ JumbleNode(jstate, (Node *) jplan->hashoperators);
+ JumbleNode(jstate, (Node *) jplan->hashcollations);
+ JumbleNode(jstate, (Node *) jplan->hashkeys);
+ }
+ break;
+ case T_Gather:
+ {
+ Gather *gplan = (Gather *) plan;
+ JUMBLE_VALUE(gplan->num_workers);
+ /* Skip all other fields */
+ }
+ break;
+ case T_GatherMerge:
+ {
+ GatherMerge *gplan = (GatherMerge *) plan;
+ JUMBLE_VALUE(gplan->num_workers);
+ /* Skip all other fields */
+ }
+ break;
+ case T_Hash:
+ {
+ Hash *hplan = (Hash *) plan;
+ JumbleNode(jstate, (Node *) hplan->hashkeys);
+ /* Skip all other fields */
+ break;
+ }
+ break;
+ case T_Memoize:
+ {
+ Memoize *mplan = (Memoize *) plan;
+ JumbleNode(jstate, (Node *) mplan->param_exprs);
+ JUMBLE_VALUE(mplan->binary_mode);
+ /* Skip all other fields */
+ }
+ break;
+ case T_Material:
+ /* Materialize node has no fields of its own */
+ break;
+ case T_Sort:
+ {
+ Sort *splan = (Sort *) plan;
+ int i;
+ for (i = 0; i < splan->numCols; i++)
+ {
+ JUMBLE_VALUE(splan->sortColIdx[i]);
+ JUMBLE_VALUE(splan->sortOperators[i]);
+ JUMBLE_VALUE(splan->collations[i]);
+ JUMBLE_VALUE(splan->nullsFirst[i]);
+ }
+ }
+ break;
+ case T_IncrementalSort:
+ {
+ IncrementalSort *splan = (IncrementalSort *) plan;
+ int i;
+ for (i = 0; i < splan->sort.numCols; i++)
+ {
+ JUMBLE_VALUE(splan->sort.sortColIdx[i]);
+ JUMBLE_VALUE(splan->sort.sortOperators[i]);
+ JUMBLE_VALUE(splan->sort.collations[i]);
+ JUMBLE_VALUE(splan->sort.nullsFirst[i]);
+ }
+ JUMBLE_VALUE(splan->nPresortedCols);
+ }
+ break;
+ case T_Unique:
+ /* Skip all Unique node fields since EXPLAIN does not show them either */
+ break;
+ case T_SetOp:
+ {
+ SetOp *splan = (SetOp *) plan;
+ JUMBLE_VALUE(splan->cmd);
+ JUMBLE_VALUE(splan->strategy);
+ /* Skip all other fields since EXPLAIN does not show them either */
+ }
+ break;
+ case T_LockRows:
+ /* Skip all LockRows node fields since EXPLAIN does not show them either */
+ break;
+ case T_Limit:
+ /* Skip all Limit node fields since EXPLAIN does not show them either */
+ break;
+ case T_Agg:
+ {
+ Agg *agg = (Agg *) plan;
+ JUMBLE_VALUE(agg->aggstrategy);
+ JUMBLE_VALUE(agg->aggsplit);
+ /* Skip all other fields since EXPLAIN does not show them either */
+ }
+ break;
+ case T_Group:
+ {
+ Group *gplan = (Group *) plan;
+ int i;
+ for (i = 0; i < gplan->numCols; i++)
+ {
+ JUMBLE_VALUE(gplan->grpColIdx[i]);
+ JUMBLE_VALUE(gplan->grpOperators[i]);
+ JUMBLE_VALUE(gplan->grpCollations[i]);
+ }
+ }
+ break;
+ case T_WindowAgg:
+ {
+ WindowAgg *wplan = (WindowAgg *) plan;
+ JumbleNode(jstate, (Node *) wplan->runConditionOrig);
+ /* Skip all other fields since EXPLAIN does not show them either */
+ }
+ break;
+ case T_Result:
+ {
+ Result *splan = (Result *) plan;
+ JumbleNode(jstate, splan->resconstantqual);
+ }
+ break;
+ case T_ProjectSet:
+ /* ProjectSet node has no fields of its own */
+ break;
+ case T_ModifyTable:
+ {
+ ModifyTable *splan = (ModifyTable *) plan;
+ ListCell *lc;
+ JUMBLE_VALUE(splan->operation);
+ foreach (lc, splan->resultRelations)
+ {
+ JumbleTargetRel(jstate, rtable, plan, lfirst_int(lc));
+ }
+ JUMBLE_VALUE(splan->onConflictAction);
+ foreach (lc, splan->arbiterIndexes)
+ {
+ JumbleTargetRel(jstate, rtable, plan, lfirst_int(lc));
+ }
+ JumbleNode(jstate, splan->onConflictWhere);
+ /* Skip all other fields since EXPLAIN does not show them either */
+ }
+ break;
+ case T_Append:
+ /* Descending into Append node children is handled by the caller */
+ break;
+ case T_MergeAppend:
+ /* Descending into MergeAppend node children is handled by the caller */
+ break;
+ case T_RecursiveUnion:
+ /* Skip all RecursiveUnion node fields since EXPLAIN does not show them either */
+ break;
+ case T_BitmapAnd:
+ /* Descending into BitmapAnd node children is handled by the caller */
+ break;
+ case T_BitmapOr:
+ /* Descending into BitmapOr node children is handled by the caller */
+ break;
+ default:
+ elog(ERROR, "unrecognized node type: %d",
+ (int) nodeTag(plan));
+ break;
+ }
+}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 7468961b01..16888d152e 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -34,6 +34,7 @@
#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
+#include "nodes/planjumble.h"
#ifdef OPTIMIZER_DEBUG
#include "nodes/print.h"
#endif
@@ -532,6 +533,15 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions,
Assert(glob->finalrowmarks == NIL);
Assert(glob->resultRelations == NIL);
Assert(glob->appendRelations == NIL);
+
+ /*
+ * Initialize plan identifier jumble if needed
+ *
+ * Note the actual jumbling is done in the tree walk in set_plan_references
+ */
+ if (compute_plan_id == COMPUTE_PLAN_ID_ON)
+ glob->plan_jumble_state = InitializeJumbleState(false);
+
top_plan = set_plan_references(root, top_plan);
/* ... and the subplans (both regular subplans and initplans) */
Assert(list_length(glob->subplans) == list_length(glob->subroots));
@@ -570,6 +580,13 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions,
result->stmt_location = parse->stmt_location;
result->stmt_len = parse->stmt_len;
+ if (compute_plan_id == COMPUTE_PLAN_ID_ON)
+ {
+ result->planId = HashJumbleState(glob->plan_jumble_state);
+ pfree(glob->plan_jumble_state->jumble);
+ pfree(glob->plan_jumble_state);
+ }
+
result->jitFlags = PGJIT_NONE;
if (jit_enabled && jit_above_cost >= 0 &&
top_plan->total_cost > jit_above_cost)
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 6d23df108d..c02f939690 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -19,6 +19,7 @@
#include "catalog/pg_type.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
+#include "nodes/planjumble.h"
#include "optimizer/optimizer.h"
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
@@ -1295,6 +1296,13 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset)
break;
}
+ /*
+ * If enabled, append significant information to the plan identifier jumble
+ * (we do this here since we're already walking the tree in a near-final state)
+ */
+ if (compute_plan_id == COMPUTE_PLAN_ID_ON)
+ JumblePlanNode(root->glob->plan_jumble_state, root->glob->finalrtable, plan);
+
/*
* Now recurse into child plans, if any
*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 8590278818..03a04a198e 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -1106,6 +1106,7 @@ exec_simple_query(const char *query_string)
size_t cmdtaglen;
pgstat_report_query_id(0, true);
+ pgstat_report_plan_id(0, 0, true);
/*
* Get the command name for use in status display (it also becomes the
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index bf33e33a4e..77f3ca14ff 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -379,6 +379,7 @@ pgstat_bestart(void)
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = UINT64CONST(0);
+ lbeentry.st_plan_id = UINT64CONST(0);
/*
* we don't zero st_progress_param here to save cycles; nobody should
@@ -533,6 +534,7 @@ pgstat_report_activity(BackendState state, const char *cmd_str)
/* st_xact_start_timestamp and wait_event_info are also disabled */
beentry->st_xact_start_timestamp = 0;
beentry->st_query_id = UINT64CONST(0);
+ beentry->st_plan_id = UINT64CONST(0);
proc->wait_event_info = 0;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -588,12 +590,15 @@ pgstat_report_activity(BackendState state, const char *cmd_str)
beentry->st_state_start_timestamp = current_timestamp;
/*
- * If a new query is started, we reset the query identifier as it'll only
- * be known after parse analysis, to avoid reporting last query's
- * identifier.
+ * If a new query is started, we reset the query and plan identifier as it'll only
+ * be known after parse analysis / planning, to avoid reporting last query's
+ * identifiers.
*/
if (state == STATE_RUNNING)
+ {
beentry->st_query_id = UINT64CONST(0);
+ beentry->st_plan_id = UINT64CONST(0);
+ }
if (cmd_str != NULL)
{
@@ -644,6 +649,45 @@ pgstat_report_query_id(uint64 query_id, bool force)
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
+/* --------
+ * pgstat_report_plan_id() -
+ *
+ * Called to update top-level plan identifier.
+ * --------
+ */
+void
+pgstat_report_plan_id(uint64 plan_id, uint64 query_id, bool force)
+{
+ volatile PgBackendStatus *beentry = MyBEEntry;
+
+ /*
+ * if track_activities is disabled, st_plan_id should already have been
+ * reset
+ */
+ if (!beentry || !pgstat_track_activities)
+ return;
+
+ /*
+ * We only report the top-level plan identifiers. The stored plan_id is
+ * reset when a backend calls pgstat_report_activity(STATE_RUNNING), or
+ * with an explicit call to this function using the force flag. If the
+ * saved plan identifier is not zero or the query identifier is 0,
+ * it means that it's not a top-level command, so ignore the one provided
+ * unless it's an explicit call to reset the identifier.
+ */
+ if ((beentry->st_plan_id != 0 || query_id == 0) && !force)
+ return;
+
+ /*
+ * Update my status entry, following the protocol of bumping
+ * st_changecount before and after. We use a volatile pointer here to
+ * ensure the compiler doesn't try to get cute.
+ */
+ PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
+ beentry->st_plan_id = plan_id;
+ PGSTAT_END_WRITE_ACTIVITY(beentry);
+}
+
/* ----------
* pgstat_report_appname() -
@@ -1040,6 +1084,26 @@ pgstat_get_my_query_id(void)
return MyBEEntry->st_query_id;
}
+/* ----------
+ * pgstat_get_my_plan_id() -
+ *
+ * Return current backend's plan identifier.
+ */
+uint64
+pgstat_get_my_plan_id(void)
+{
+ if (!MyBEEntry)
+ return 0;
+
+ /*
+ * There's no need for a lock around pgstat_begin_read_activity /
+ * pgstat_end_read_activity here as it's only called from
+ * pg_stat_get_activity which is already protected, or from the same
+ * backend which means that there won't be concurrent writes.
+ */
+ return MyBEEntry->st_plan_id;
+}
+
/* ----------
* pgstat_get_backend_type_by_proc_number() -
*
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6fc34f7494..4a5fc94877 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -302,7 +302,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum
pg_stat_get_activity(PG_FUNCTION_ARGS)
{
-#define PG_STAT_GET_ACTIVITY_COLS 31
+#define PG_STAT_GET_ACTIVITY_COLS 32
int num_backends = pgstat_fetch_stat_numbackends();
int curr_backend;
int pid = PG_ARGISNULL(0) ? -1 : PG_GETARG_INT32(0);
@@ -613,6 +613,10 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
nulls[30] = true;
else
values[30] = UInt64GetDatum(beentry->st_query_id);
+ if (beentry->st_plan_id == 0)
+ nulls[31] = true;
+ else
+ values[31] = UInt64GetDatum(beentry->st_plan_id);
}
else
{
@@ -642,6 +646,7 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
nulls[28] = true;
nulls[29] = true;
nulls[30] = true;
+ nulls[31] = true;
}
tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 8cf1afbad2..5139ba094a 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -50,6 +50,7 @@
#include "libpq/auth.h"
#include "libpq/libpq.h"
#include "libpq/scram.h"
+#include "nodes/planjumble.h"
#include "nodes/queryjumble.h"
#include "optimizer/cost.h"
#include "optimizer/geqo.h"
@@ -309,6 +310,23 @@ static const struct config_enum_entry compute_query_id_options[] = {
{NULL, 0, false}
};
+/*
+ * Although only "on" and "off" are documented, we accept
+ * all the likely variants of "on" and "off".
+ */
+static const struct config_enum_entry compute_plan_id_options[] = {
+ {"regress", COMPUTE_PLAN_ID_REGRESS, false},
+ {"on", COMPUTE_PLAN_ID_ON, false},
+ {"off", COMPUTE_PLAN_ID_OFF, false},
+ {"true", COMPUTE_PLAN_ID_ON, true},
+ {"false", COMPUTE_PLAN_ID_OFF, true},
+ {"yes", COMPUTE_PLAN_ID_ON, true},
+ {"no", COMPUTE_PLAN_ID_OFF, true},
+ {"1", COMPUTE_PLAN_ID_ON, true},
+ {"0", COMPUTE_PLAN_ID_OFF, true},
+ {NULL, 0, false}
+};
+
/*
* Although only "on", "off", and "partition" are documented, we
* accept all the likely variants of "on" and "off".
@@ -4863,6 +4881,16 @@ struct config_enum ConfigureNamesEnum[] =
NULL, NULL, NULL
},
+ {
+ {"compute_plan_id", PGC_SUSET, STATS_MONITORING,
+ gettext_noop("Enables in-core computation of plan identifiers."),
+ NULL
+ },
+ &compute_plan_id,
+ COMPUTE_PLAN_ID_OFF, compute_plan_id_options,
+ NULL, NULL, NULL
+ },
+
{
{"constraint_exclusion", PGC_USERSET, QUERY_TUNING_OTHER,
gettext_noop("Enables the planner to use constraints to optimize queries."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index a2ac7575ca..70fa7f886f 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -649,6 +649,7 @@
# - Monitoring -
#compute_query_id = auto
+#compute_plan_id = off
#log_statement_stats = off
#log_parser_stats = off
#log_planner_stats = off
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 2dcc2d42da..56feae2583 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5549,9 +5549,9 @@
proname => 'pg_stat_get_activity', prorows => '100', proisstrict => 'f',
proretset => 't', provolatile => 's', proparallel => 'r',
prorettype => 'record', proargtypes => 'int4',
- proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,bool,int4,int8}',
- proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,gss_delegation,leader_pid,query_id}',
+ proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,bool,int4,int8,int8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,gss_delegation,leader_pid,query_id,plan_id}',
prosrc => 'pg_stat_get_activity' },
{ oid => '6318', descr => 'describe wait events',
proname => 'pg_get_wait_events', procost => '10', prorows => '250',
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 58748d2ca6..9068cb826e 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -163,6 +163,9 @@ typedef struct PlannerGlobal
/* partition descriptors */
PartitionDirectory partition_directory pg_node_attr(read_write_ignore);
+
+ /* optional jumble state for plan identifier claculation */
+ struct JumbleState *plan_jumble_state pg_node_attr(read_write_ignore);
} PlannerGlobal;
/* macro for fetching the Plan associated with a SubPlan node */
diff --git a/src/include/nodes/planjumble.h b/src/include/nodes/planjumble.h
new file mode 100644
index 0000000000..d35b66aec4
--- /dev/null
+++ b/src/include/nodes/planjumble.h
@@ -0,0 +1,33 @@
+/*-------------------------------------------------------------------------
+ *
+ * planjumble.h
+ * Plan fingerprinting.
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/include/nodes/planjumble.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PLANJUMBLE_H
+#define PLANJUMBLE_H
+
+#include "nodes/plannodes.h"
+#include "nodes/queryjumble.h"
+
+/* Values for the compute_plan_id GUC */
+enum ComputePlanIdType
+{
+ COMPUTE_PLAN_ID_OFF,
+ COMPUTE_PLAN_ID_ON,
+ COMPUTE_PLAN_ID_REGRESS,
+};
+
+/* GUC parameters */
+extern PGDLLIMPORT int compute_plan_id;
+
+extern void JumblePlanNode(JumbleState *jumble, List* rtable, Plan *plan);
+
+#endif /* PLANJUMBLE_H */
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 4633121689..2f06d1d496 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -53,6 +53,8 @@ typedef struct PlannedStmt
uint64 queryId; /* query identifier (copied from Query) */
+ uint64 planId; /* plan identifier (calculated if compute_plan_id is enabled, can also be set by plugins) */
+
bool hasReturning; /* is it insert|update|delete|merge RETURNING? */
bool hasModifyingCTE; /* has insert|update|delete|merge in WITH? */
@@ -813,7 +815,7 @@ typedef struct NestLoop
typedef struct NestLoopParam
{
- pg_node_attr(no_equal, no_query_jumble)
+ pg_node_attr(no_equal)
NodeTag type;
int paramno; /* number of the PARAM_EXEC Param to set */
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
index b0ef1952e8..eac6511f81 100644
--- a/src/include/nodes/primnodes.h
+++ b/src/include/nodes/primnodes.h
@@ -1058,8 +1058,6 @@ typedef struct SubLink
*/
typedef struct SubPlan
{
- pg_node_attr(no_query_jumble)
-
Expr xpr;
/* Fields copied from original SubLink: */
SubLinkType subLinkType; /* see above */
@@ -1090,8 +1088,8 @@ typedef struct SubPlan
List *parParam; /* indices of input Params from parent plan */
List *args; /* exprs to pass as parParam values */
/* Estimated execution costs: */
- Cost startup_cost; /* one-time setup cost */
- Cost per_call_cost; /* cost for each subplan evaluation */
+ Cost startup_cost pg_node_attr(query_jumble_ignore); /* one-time setup cost */
+ Cost per_call_cost pg_node_attr(query_jumble_ignore); /* cost for each subplan evaluation */
} SubPlan;
/*
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index 4e8b39a66d..8bb01256dd 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -170,6 +170,9 @@ typedef struct PgBackendStatus
/* query identifier, optionally computed using post_parse_analyze_hook */
uint64 st_query_id;
+
+ /* plan identifier, optionally computed after planning */
+ uint64 st_plan_id;
} PgBackendStatus;
@@ -316,6 +319,7 @@ extern void pgstat_clear_backend_activity_snapshot(void);
/* Activity reporting functions */
extern void pgstat_report_activity(BackendState state, const char *cmd_str);
extern void pgstat_report_query_id(uint64 query_id, bool force);
+extern void pgstat_report_plan_id(uint64 query_id, uint64 plan_id, bool force);
extern void pgstat_report_tempfile(size_t filesize);
extern void pgstat_report_appname(const char *appname);
extern void pgstat_report_xact_timestamp(TimestampTz tstamp);
@@ -323,6 +327,7 @@ extern const char *pgstat_get_backend_current_activity(int pid, bool checkUser);
extern const char *pgstat_get_crashed_backend_activity(int pid, char *buffer,
int buflen);
extern uint64 pgstat_get_my_query_id(void);
+extern uint64 pgstat_get_my_plan_id(void);
extern BackendType pgstat_get_backend_type_by_proc_number(ProcNumber procNumber);
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index ee31e41d50..8bfa3c1a5f 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -699,6 +699,17 @@ select explain_filter('explain (verbose) create table test_ctas as select 1');
Query Identifier: N
(3 rows)
+-- Test compute_plan_id
+set compute_plan_id = on;
+select explain_filter('explain (verbose) select * from int8_tbl i8');
+ explain_filter
+----------------------------------------------------------------
+ Seq Scan on public.int8_tbl i8 (cost=N.N..N.N rows=N width=N)
+ Output: q1, q2
+ Query Identifier: N
+ Plan Identifier: N
+(4 rows)
+
-- Test SERIALIZE option
select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
explain_filter
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 3014d047fe..c041a49dea 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1760,9 +1760,10 @@ pg_stat_activity| SELECT s.datid,
s.backend_xid,
s.backend_xmin,
s.query_id,
+ s.plan_id,
s.query,
s.backend_type
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
LEFT JOIN pg_database d ON ((s.datid = d.oid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_all_indexes| SELECT c.oid AS relid,
@@ -1886,7 +1887,7 @@ pg_stat_gssapi| SELECT pid,
gss_princ AS principal,
gss_enc AS encrypted,
gss_delegation AS credentials_delegated
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
WHERE (client_port IS NOT NULL);
pg_stat_io| SELECT backend_type,
object,
@@ -2090,7 +2091,7 @@ pg_stat_replication| SELECT s.pid,
w.sync_priority,
w.sync_state,
w.reply_time
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
JOIN pg_stat_get_wal_senders() w(pid, state, sent_lsn, write_lsn, flush_lsn, replay_lsn, write_lag, flush_lag, replay_lag, sync_priority, sync_state, reply_time) ON ((s.pid = w.pid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_replication_slots| SELECT s.slot_name,
@@ -2124,7 +2125,7 @@ pg_stat_ssl| SELECT pid,
ssl_client_dn AS client_dn,
ssl_client_serial AS client_serial,
ssl_issuer_dn AS issuer_dn
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
WHERE (client_port IS NOT NULL);
pg_stat_subscription| SELECT su.oid AS subid,
su.subname,
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index 0bafa87049..d787ad2cda 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -167,6 +167,10 @@ select explain_filter('explain (verbose) select * from int8_tbl i8');
select explain_filter('explain (verbose) declare test_cur cursor for select * from int8_tbl');
select explain_filter('explain (verbose) create table test_ctas as select 1');
+-- Test compute_plan_id
+set compute_plan_id = on;
+select explain_filter('explain (verbose) select * from int8_tbl i8');
+
-- Test SERIALIZE option
select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
select explain_filter('explain (analyze,serialize text,buffers,timing off) select * from int8_tbl i8');
--
2.47.1
0001-Allow-using-jumbling-logic-outside-of-query-jumble-u.patchapplication/octet-stream; name=0001-Allow-using-jumbling-logic-outside-of-query-jumble-u.patchDownload
From 23911df5bf9e9841b02a3008402c870a8b757258 Mon Sep 17 00:00:00 2001
From: Lukas Fittl <lukas@fittl.com>
Date: Tue, 31 Dec 2024 15:05:39 -0800
Subject: [PATCH 1/4] Allow using jumbling logic outside of query jumble unit
file
This can be useful either for jumbling expressions in other contexts
(e.g. to calculate a plan jumble), or to allow extensions to use
a modified jumbling logic more easily.
This intentionally supports the use case where a separate jumbling logic
does not care about recording constants, as the query jumble does.
---
src/backend/nodes/queryjumblefuncs.c | 59 +++++++++++++++++-----------
src/include/nodes/queryjumble.h | 6 +++
2 files changed, 41 insertions(+), 24 deletions(-)
diff --git a/src/backend/nodes/queryjumblefuncs.c b/src/backend/nodes/queryjumblefuncs.c
index e8bf95690b..0b3af4169a 100644
--- a/src/backend/nodes/queryjumblefuncs.c
+++ b/src/backend/nodes/queryjumblefuncs.c
@@ -51,10 +51,7 @@ int compute_query_id = COMPUTE_QUERY_ID_AUTO;
*/
bool query_id_enabled = false;
-static void AppendJumble(JumbleState *jstate,
- const unsigned char *item, Size size);
static void RecordConstLocation(JumbleState *jstate, int location);
-static void _jumbleNode(JumbleState *jstate, Node *node);
static void _jumbleA_Const(JumbleState *jstate, Node *node);
static void _jumbleList(JumbleState *jstate, Node *node);
static void _jumbleVariableSetStmt(JumbleState *jstate, Node *node);
@@ -109,28 +106,42 @@ CleanQuerytext(const char *query, int *location, int *len)
}
JumbleState *
-JumbleQuery(Query *query)
+InitializeJumbleState(bool record_clocations)
{
- JumbleState *jstate = NULL;
-
- Assert(IsQueryIdEnabled());
-
- jstate = (JumbleState *) palloc(sizeof(JumbleState));
+ JumbleState *jstate = (JumbleState *) palloc0(sizeof(JumbleState));
/* Set up workspace for query jumbling */
jstate->jumble = (unsigned char *) palloc(JUMBLE_SIZE);
jstate->jumble_len = 0;
- jstate->clocations_buf_size = 32;
- jstate->clocations = (LocationLen *)
- palloc(jstate->clocations_buf_size * sizeof(LocationLen));
- jstate->clocations_count = 0;
- jstate->highest_extern_param_id = 0;
+
+ if (record_clocations)
+ {
+ jstate->clocations_buf_size = 32;
+ jstate->clocations = (LocationLen *)
+ palloc(jstate->clocations_buf_size * sizeof(LocationLen));
+ }
+
+ return jstate;
+}
+
+uint64
+HashJumbleState(JumbleState *jstate)
+{
+ return DatumGetUInt64(hash_any_extended(jstate->jumble,
+ jstate->jumble_len,
+ 0));
+}
+
+JumbleState *
+JumbleQuery(Query *query)
+{
+ JumbleState *jstate = InitializeJumbleState(true);
+
+ Assert(IsQueryIdEnabled());
/* Compute query ID and mark the Query node with it */
- _jumbleNode(jstate, (Node *) query);
- query->queryId = DatumGetUInt64(hash_any_extended(jstate->jumble,
- jstate->jumble_len,
- 0));
+ JumbleNode(jstate, (Node *) query);
+ query->queryId = HashJumbleState(jstate);
/*
* If we are unlucky enough to get a hash of zero, use 1 instead for
@@ -164,7 +175,7 @@ EnableQueryId(void)
* AppendJumble: Append a value that is substantive in a given query to
* the current jumble.
*/
-static void
+void
AppendJumble(JumbleState *jstate, const unsigned char *item, Size size)
{
unsigned char *jumble = jstate->jumble;
@@ -205,7 +216,7 @@ static void
RecordConstLocation(JumbleState *jstate, int location)
{
/* -1 indicates unknown or undefined location */
- if (location >= 0)
+ if (location >= 0 && jstate->clocations_buf_size > 0)
{
/* enlarge array if needed */
if (jstate->clocations_count >= jstate->clocations_buf_size)
@@ -224,7 +235,7 @@ RecordConstLocation(JumbleState *jstate, int location)
}
#define JUMBLE_NODE(item) \
- _jumbleNode(jstate, (Node *) expr->item)
+ JumbleNode(jstate, (Node *) expr->item)
#define JUMBLE_LOCATION(location) \
RecordConstLocation(jstate, expr->location)
#define JUMBLE_FIELD(item) \
@@ -239,8 +250,8 @@ do { \
#include "queryjumblefuncs.funcs.c"
-static void
-_jumbleNode(JumbleState *jstate, Node *node)
+void
+JumbleNode(JumbleState *jstate, Node *node)
{
Node *expr = node;
@@ -305,7 +316,7 @@ _jumbleList(JumbleState *jstate, Node *node)
{
case T_List:
foreach(l, expr)
- _jumbleNode(jstate, lfirst(l));
+ JumbleNode(jstate, lfirst(l));
break;
case T_IntList:
foreach(l, expr)
diff --git a/src/include/nodes/queryjumble.h b/src/include/nodes/queryjumble.h
index f1c55c8067..7f36dd025d 100644
--- a/src/include/nodes/queryjumble.h
+++ b/src/include/nodes/queryjumble.h
@@ -83,4 +83,10 @@ IsQueryIdEnabled(void)
return query_id_enabled;
}
+/* Functions intended for other users of jumbling (e.g. plan jumbling) */
+extern JumbleState *InitializeJumbleState(bool record_clocations);
+extern void AppendJumble(JumbleState *jstate, const unsigned char *item, Size size);
+extern void JumbleNode(JumbleState *jstate, Node *node);
+extern uint64 HashJumbleState(JumbleState *jstate);
+
#endif /* QUERYJUMBLE_H */
--
2.47.1
0004-Add-pg_stat_plans-contrib-extension.patchapplication/octet-stream; name=0004-Add-pg_stat_plans-contrib-extension.patchDownload
From 9840edbbfb22e4c6ffbbf6a5eb80aa863a95edf6 Mon Sep 17 00:00:00 2001
From: Lukas Fittl <lukas@fittl.com>
Date: Thu, 2 Jan 2025 10:47:50 -0800
Subject: [PATCH 4/4] Add pg_stat_plans contrib extension
This extension allows tracking per-plan call counts and execution time,
as well as capturing the plan text, aka EXPLAIN (COSTS OFF), for the
first execution of a given plan. This utilize the compute_plan_id
functionality for tracking different plans.
---
contrib/Makefile | 1 +
contrib/meson.build | 1 +
contrib/pg_stat_plans/Makefile | 29 +
contrib/pg_stat_plans/expected/cleanup.out | 1 +
contrib/pg_stat_plans/expected/privileges.out | 125 +++
contrib/pg_stat_plans/expected/select.out | 262 ++++++
contrib/pg_stat_plans/meson.build | 43 +
contrib/pg_stat_plans/pg_stat_plans--1.0.sql | 32 +
contrib/pg_stat_plans/pg_stat_plans.c | 743 ++++++++++++++++++
contrib/pg_stat_plans/pg_stat_plans.conf | 3 +
contrib/pg_stat_plans/pg_stat_plans.control | 5 +
contrib/pg_stat_plans/sql/cleanup.sql | 1 +
contrib/pg_stat_plans/sql/privileges.sql | 59 ++
contrib/pg_stat_plans/sql/select.sql | 67 ++
doc/src/sgml/contrib.sgml | 1 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/pgstatplans.sgml | 413 ++++++++++
17 files changed, 1787 insertions(+)
create mode 100644 contrib/pg_stat_plans/Makefile
create mode 100644 contrib/pg_stat_plans/expected/cleanup.out
create mode 100644 contrib/pg_stat_plans/expected/privileges.out
create mode 100644 contrib/pg_stat_plans/expected/select.out
create mode 100644 contrib/pg_stat_plans/meson.build
create mode 100644 contrib/pg_stat_plans/pg_stat_plans--1.0.sql
create mode 100644 contrib/pg_stat_plans/pg_stat_plans.c
create mode 100644 contrib/pg_stat_plans/pg_stat_plans.conf
create mode 100644 contrib/pg_stat_plans/pg_stat_plans.control
create mode 100644 contrib/pg_stat_plans/sql/cleanup.sql
create mode 100644 contrib/pg_stat_plans/sql/privileges.sql
create mode 100644 contrib/pg_stat_plans/sql/select.sql
create mode 100644 doc/src/sgml/pgstatplans.sgml
diff --git a/contrib/Makefile b/contrib/Makefile
index 952855d9b6..8de010afde 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -34,6 +34,7 @@ SUBDIRS = \
pg_freespacemap \
pg_logicalinspect \
pg_prewarm \
+ pg_stat_plans \
pg_stat_statements \
pg_surgery \
pg_trgm \
diff --git a/contrib/meson.build b/contrib/meson.build
index 159ff41555..430910fba4 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -49,6 +49,7 @@ subdir('pg_freespacemap')
subdir('pg_logicalinspect')
subdir('pg_prewarm')
subdir('pgrowlocks')
+subdir('pg_stat_plans')
subdir('pg_stat_statements')
subdir('pgstattuple')
subdir('pg_surgery')
diff --git a/contrib/pg_stat_plans/Makefile b/contrib/pg_stat_plans/Makefile
new file mode 100644
index 0000000000..e073db95ed
--- /dev/null
+++ b/contrib/pg_stat_plans/Makefile
@@ -0,0 +1,29 @@
+# contrib/pg_stat_plans/Makefile
+
+MODULE_big = pg_stat_plans
+OBJS = \
+ $(WIN32RES) \
+ pg_stat_plans.o
+
+EXTENSION = pg_stat_plans
+DATA = pg_stat_plans--1.0.sql
+PGFILEDESC = "pg_stat_plans - track per-plan call counts, execution times and EXPLAIN texts"
+
+LDFLAGS_SL += $(filter -lm, $(LIBS))
+
+REGRESS_OPTS = --temp-config $(top_srcdir)/contrib/pg_stat_plans/pg_stat_plans.conf
+REGRESS = select privileges cleanup
+# Disabled because these tests require "shared_preload_libraries=pg_stat_plans",
+# which typical installcheck users do not have (e.g. buildfarm clients).
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/pg_stat_plans
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/pg_stat_plans/expected/cleanup.out b/contrib/pg_stat_plans/expected/cleanup.out
new file mode 100644
index 0000000000..51565617ce
--- /dev/null
+++ b/contrib/pg_stat_plans/expected/cleanup.out
@@ -0,0 +1 @@
+DROP EXTENSION pg_stat_plans;
diff --git a/contrib/pg_stat_plans/expected/privileges.out b/contrib/pg_stat_plans/expected/privileges.out
new file mode 100644
index 0000000000..3e21d6d701
--- /dev/null
+++ b/contrib/pg_stat_plans/expected/privileges.out
@@ -0,0 +1,125 @@
+--
+-- Only superusers and roles with privileges of the pg_read_all_stats role
+-- are allowed to see the plan text, queryid and planid of queries executed by
+-- other users. Other users can see the statistics.
+--
+CREATE ROLE regress_stats_superuser SUPERUSER;
+CREATE ROLE regress_stats_user1;
+CREATE ROLE regress_stats_user2;
+GRANT pg_read_all_stats TO regress_stats_user2;
+SET ROLE regress_stats_superuser;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
+SELECT 1 AS "ONE";
+ ONE
+-----
+ 1
+(1 row)
+
+SET ROLE regress_stats_user1;
+SELECT 1+1 AS "TWO";
+ TWO
+-----
+ 2
+(1 row)
+
+--
+-- A superuser can read all columns of queries executed by others,
+-- including plan text, queryid and planid.
+--
+SET ROLE regress_stats_superuser;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+ rolname | queryid_bool | planid_bool | plan | calls
+-------------------------+--------------+-------------+------------------------------------------------------------------------------------+-------
+ regress_stats_superuser | t | t | Result | 1
+ regress_stats_superuser | t | t | Result | 1
+ regress_stats_superuser | t | t | Sort +| 0
+ | | | Sort Key: pg_authid.rolname, pg_stat_plans.plan COLLATE "C", pg_stat_plans.calls+|
+ | | | -> Hash Join +|
+ | | | Hash Cond: (pg_stat_plans.userid = pg_authid.oid) +|
+ | | | -> Function Scan on pg_stat_plans +|
+ | | | -> Hash +|
+ | | | -> Seq Scan on pg_authid |
+ regress_stats_user1 | t | t | Result | 1
+(4 rows)
+
+--
+-- regress_stats_user1 has no privileges to read the plan text, queryid
+-- or planid of queries executed by others but can see statistics
+-- like calls and rows.
+--
+SET ROLE regress_stats_user1;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+ rolname | queryid_bool | planid_bool | plan | calls
+-------------------------+--------------+-------------+------------------------------------------------------------------------------------+-------
+ regress_stats_superuser | | | <insufficient privilege> | 1
+ regress_stats_superuser | | | <insufficient privilege> | 1
+ regress_stats_superuser | | | <insufficient privilege> | 1
+ regress_stats_user1 | t | t | Result | 1
+ regress_stats_user1 | t | t | Sort +| 0
+ | | | Sort Key: pg_authid.rolname, pg_stat_plans.plan COLLATE "C", pg_stat_plans.calls+|
+ | | | -> Hash Join +|
+ | | | Hash Cond: (pg_stat_plans.userid = pg_authid.oid) +|
+ | | | -> Function Scan on pg_stat_plans +|
+ | | | -> Hash +|
+ | | | -> Seq Scan on pg_authid |
+(5 rows)
+
+--
+-- regress_stats_user2, with pg_read_all_stats role privileges, can
+-- read all columns, including plan text, queryid and planid, of queries
+-- executed by others.
+--
+SET ROLE regress_stats_user2;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+ rolname | queryid_bool | planid_bool | plan | calls
+-------------------------+--------------+-------------+------------------------------------------------------------------------------------+-------
+ regress_stats_superuser | t | t | Result | 1
+ regress_stats_superuser | t | t | Result | 1
+ regress_stats_superuser | t | t | Sort +| 1
+ | | | Sort Key: pg_authid.rolname, pg_stat_plans.plan COLLATE "C", pg_stat_plans.calls+|
+ | | | -> Hash Join +|
+ | | | Hash Cond: (pg_stat_plans.userid = pg_authid.oid) +|
+ | | | -> Function Scan on pg_stat_plans +|
+ | | | -> Hash +|
+ | | | -> Seq Scan on pg_authid |
+ regress_stats_user1 | t | t | Result | 1
+ regress_stats_user1 | t | t | Sort +| 1
+ | | | Sort Key: pg_authid.rolname, pg_stat_plans.plan COLLATE "C", pg_stat_plans.calls+|
+ | | | -> Hash Join +|
+ | | | Hash Cond: (pg_stat_plans.userid = pg_authid.oid) +|
+ | | | -> Function Scan on pg_stat_plans +|
+ | | | -> Hash +|
+ | | | -> Seq Scan on pg_authid |
+ regress_stats_user2 | t | t | Sort +| 0
+ | | | Sort Key: pg_authid.rolname, pg_stat_plans.plan COLLATE "C", pg_stat_plans.calls+|
+ | | | -> Hash Join +|
+ | | | Hash Cond: (pg_stat_plans.userid = pg_authid.oid) +|
+ | | | -> Function Scan on pg_stat_plans +|
+ | | | -> Hash +|
+ | | | -> Seq Scan on pg_authid |
+(6 rows)
+
+--
+-- cleanup
+--
+RESET ROLE;
+DROP ROLE regress_stats_superuser;
+DROP ROLE regress_stats_user1;
+DROP ROLE regress_stats_user2;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
diff --git a/contrib/pg_stat_plans/expected/select.out b/contrib/pg_stat_plans/expected/select.out
new file mode 100644
index 0000000000..906d8ce90d
--- /dev/null
+++ b/contrib/pg_stat_plans/expected/select.out
@@ -0,0 +1,262 @@
+--
+-- SELECT statements
+--
+CREATE EXTENSION pg_stat_plans;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
+--
+-- simple statements
+--
+SELECT 1 FROM pg_class LIMIT 1;
+ ?column?
+----------
+ 1
+(1 row)
+
+SELECT 1 FROM pg_class WHERE relname = 'pg_class';
+ ?column?
+----------
+ 1
+(1 row)
+
+SET enable_indexscan = off;
+SELECT 1 FROM pg_class WHERE relname = 'pg_class';
+ ?column?
+----------
+ 1
+(1 row)
+
+SET enable_indexscan = on;
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+ plan | calls
+--------------------------------------------------------------+-------
+ Bitmap Heap Scan on pg_class +| 1
+ Recheck Cond: (relname = 'pg_class'::name) +|
+ -> Bitmap Index Scan on pg_class_relname_nsp_index +|
+ Index Cond: (relname = 'pg_class'::name) |
+ Index Only Scan using pg_class_relname_nsp_index on pg_class+| 1
+ Index Cond: (relname = 'pg_class'::name) |
+ Limit +| 1
+ -> Seq Scan on pg_class |
+ Result | 1
+ Sort +| 0
+ Sort Key: pg_stat_plans.plan COLLATE "C" +|
+ -> Function Scan on pg_stat_plans |
+(5 rows)
+
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
+--
+-- subplans and CTEs
+--
+WITH x AS MATERIALIZED (SELECT 1)
+SELECT * FROM x;
+ ?column?
+----------
+ 1
+(1 row)
+
+SELECT a.attname,
+ (SELECT pg_catalog.pg_get_expr(d.adbin, d.adrelid)
+ FROM pg_catalog.pg_attrdef d
+ WHERE d.adrelid = a.attrelid AND d.adnum = a.attnum AND a.atthasdef)
+ FROM pg_catalog.pg_attribute a
+ WHERE a.attrelid = 'pg_class'::regclass
+ ORDER BY attnum LIMIT 1;
+ attname | pg_get_expr
+----------+-------------
+ tableoid |
+(1 row)
+
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+ plan | calls
+-------------------------------------------------------------------------------+-------
+ CTE Scan on x +| 1
+ CTE x +|
+ -> Result |
+ Limit +| 1
+ -> Index Scan using pg_attribute_relid_attnum_index on pg_attribute a +|
+ Index Cond: (attrelid = '1259'::oid) +|
+ SubPlan 1 +|
+ -> Result +|
+ One-Time Filter: a.atthasdef +|
+ -> Seq Scan on pg_attrdef d +|
+ Filter: ((adrelid = a.attrelid) AND (adnum = a.attnum)) |
+ Result | 1
+ Sort +| 0
+ Sort Key: pg_stat_plans.plan COLLATE "C" +|
+ -> Function Scan on pg_stat_plans |
+(4 rows)
+
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
+--
+-- partitoning
+--
+create table lp (a char) partition by list (a);
+create table lp_default partition of lp default;
+create table lp_ef partition of lp for values in ('e', 'f');
+create table lp_ad partition of lp for values in ('a', 'd');
+create table lp_bc partition of lp for values in ('b', 'c');
+create table lp_g partition of lp for values in ('g');
+create table lp_null partition of lp for values in (null);
+select * from lp;
+ a
+---
+(0 rows)
+
+select * from lp where a > 'a' and a < 'd';
+ a
+---
+(0 rows)
+
+select * from lp where a > 'a' and a <= 'd';
+ a
+---
+(0 rows)
+
+select * from lp where a = 'a';
+ a
+---
+(0 rows)
+
+select * from lp where 'a' = a; /* commuted */
+ a
+---
+(0 rows)
+
+select * from lp where a is not null;
+ a
+---
+(0 rows)
+
+select * from lp where a is null;
+ a
+---
+(0 rows)
+
+select * from lp where a = 'a' or a = 'c';
+ a
+---
+(0 rows)
+
+select * from lp where a is not null and (a = 'a' or a = 'c');
+ a
+---
+(0 rows)
+
+select * from lp where a <> 'g';
+ a
+---
+(0 rows)
+
+select * from lp where a <> 'a' and a <> 'd';
+ a
+---
+(0 rows)
+
+select * from lp where a not in ('a', 'd');
+ a
+---
+(0 rows)
+
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+ plan | calls
+--------------------------------------------------------------------------------+-------
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ Filter: ((a = 'a'::bpchar) OR (a = 'c'::bpchar)) +|
+ -> Seq Scan on lp_bc lp_2 +|
+ Filter: ((a = 'a'::bpchar) OR (a = 'c'::bpchar)) |
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ Filter: ((a > 'a'::bpchar) AND (a <= 'd'::bpchar)) +|
+ -> Seq Scan on lp_bc lp_2 +|
+ Filter: ((a > 'a'::bpchar) AND (a <= 'd'::bpchar)) +|
+ -> Seq Scan on lp_default lp_3 +|
+ Filter: ((a > 'a'::bpchar) AND (a <= 'd'::bpchar)) |
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ Filter: ((a IS NOT NULL) AND ((a = 'a'::bpchar) OR (a = 'c'::bpchar)))+|
+ -> Seq Scan on lp_bc lp_2 +|
+ Filter: ((a IS NOT NULL) AND ((a = 'a'::bpchar) OR (a = 'c'::bpchar))) |
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ Filter: (a <> 'g'::bpchar) +|
+ -> Seq Scan on lp_bc lp_2 +|
+ Filter: (a <> 'g'::bpchar) +|
+ -> Seq Scan on lp_ef lp_3 +|
+ Filter: (a <> 'g'::bpchar) +|
+ -> Seq Scan on lp_default lp_4 +|
+ Filter: (a <> 'g'::bpchar) |
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ Filter: (a IS NOT NULL) +|
+ -> Seq Scan on lp_bc lp_2 +|
+ Filter: (a IS NOT NULL) +|
+ -> Seq Scan on lp_ef lp_3 +|
+ Filter: (a IS NOT NULL) +|
+ -> Seq Scan on lp_g lp_4 +|
+ Filter: (a IS NOT NULL) +|
+ -> Seq Scan on lp_default lp_5 +|
+ Filter: (a IS NOT NULL) |
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ -> Seq Scan on lp_bc lp_2 +|
+ -> Seq Scan on lp_ef lp_3 +|
+ -> Seq Scan on lp_g lp_4 +|
+ -> Seq Scan on lp_null lp_5 +|
+ -> Seq Scan on lp_default lp_6 |
+ Append +| 1
+ -> Seq Scan on lp_bc lp_1 +|
+ Filter: ((a <> 'a'::bpchar) AND (a <> 'd'::bpchar)) +|
+ -> Seq Scan on lp_ef lp_2 +|
+ Filter: ((a <> 'a'::bpchar) AND (a <> 'd'::bpchar)) +|
+ -> Seq Scan on lp_g lp_3 +|
+ Filter: ((a <> 'a'::bpchar) AND (a <> 'd'::bpchar)) +|
+ -> Seq Scan on lp_default lp_4 +|
+ Filter: ((a <> 'a'::bpchar) AND (a <> 'd'::bpchar)) |
+ Append +| 1
+ -> Seq Scan on lp_bc lp_1 +|
+ Filter: ((a > 'a'::bpchar) AND (a < 'd'::bpchar)) +|
+ -> Seq Scan on lp_default lp_2 +|
+ Filter: ((a > 'a'::bpchar) AND (a < 'd'::bpchar)) |
+ Append +| 1
+ -> Seq Scan on lp_bc lp_1 +|
+ Filter: (a <> ALL ('{a,d}'::bpchar[])) +|
+ -> Seq Scan on lp_ef lp_2 +|
+ Filter: (a <> ALL ('{a,d}'::bpchar[])) +|
+ -> Seq Scan on lp_g lp_3 +|
+ Filter: (a <> ALL ('{a,d}'::bpchar[])) +|
+ -> Seq Scan on lp_default lp_4 +|
+ Filter: (a <> ALL ('{a,d}'::bpchar[])) |
+ Result | 1
+ Seq Scan on lp_ad lp +| 1
+ Filter: ('a'::bpchar = a) |
+ Seq Scan on lp_ad lp +| 1
+ Filter: (a = 'a'::bpchar) |
+ Seq Scan on lp_null lp +| 1
+ Filter: (a IS NULL) |
+ Sort +| 0
+ Sort Key: pg_stat_plans.plan COLLATE "C" +|
+ -> Function Scan on pg_stat_plans |
+(14 rows)
+
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
diff --git a/contrib/pg_stat_plans/meson.build b/contrib/pg_stat_plans/meson.build
new file mode 100644
index 0000000000..3bd884d960
--- /dev/null
+++ b/contrib/pg_stat_plans/meson.build
@@ -0,0 +1,43 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+pg_stat_plans_sources = files(
+ 'pg_stat_plans.c',
+)
+
+if host_system == 'windows'
+ pg_stat_plans_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'pg_stat_plans',
+ '--FILEDESC', 'pg_stat_plans - track per-plan call counts, execution times and EXPLAIN texts',])
+endif
+
+pg_stat_plans = shared_module('pg_stat_plans',
+ pg_stat_plans_sources,
+ kwargs: contrib_mod_args + {
+ 'dependencies': contrib_mod_args['dependencies'],
+ },
+)
+contrib_targets += pg_stat_plans
+
+install_data(
+ 'pg_stat_plans.control',
+ 'pg_stat_plans--1.0.sql',
+ kwargs: contrib_data_args,
+)
+
+tests += {
+ 'name': 'pg_stat_plans',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'regress': {
+ 'sql': [
+ 'select',
+ 'privileges',
+ 'cleanup',
+ ],
+ 'regress_args': ['--temp-config', files('pg_stat_plans.conf')],
+ # Disabled because these tests require
+ # "shared_preload_libraries=pg_stat_plans", which typical
+ # runningcheck users do not have (e.g. buildfarm clients).
+ 'runningcheck': false,
+ }
+}
diff --git a/contrib/pg_stat_plans/pg_stat_plans--1.0.sql b/contrib/pg_stat_plans/pg_stat_plans--1.0.sql
new file mode 100644
index 0000000000..412d9e73ae
--- /dev/null
+++ b/contrib/pg_stat_plans/pg_stat_plans--1.0.sql
@@ -0,0 +1,32 @@
+/* contrib/pg_stat_plans/pg_stat_plans--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_stat_plans" to load this file. \quit
+
+-- Register functions.
+CREATE FUNCTION pg_stat_plans_reset()
+RETURNS void
+AS 'MODULE_PATHNAME'
+LANGUAGE C PARALLEL SAFE;
+
+CREATE FUNCTION pg_stat_plans(IN showplan boolean,
+ OUT userid oid,
+ OUT dbid oid,
+ OUT queryid bigint,
+ OUT planid bigint,
+ OUT calls int8,
+ OUT total_exec_time float8,
+ OUT plan text
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stat_plans_1_0'
+LANGUAGE C STRICT VOLATILE PARALLEL SAFE;
+
+-- Register a view on the function for ease of use.
+CREATE VIEW pg_stat_plans AS
+ SELECT * FROM pg_stat_plans(true);
+
+GRANT SELECT ON pg_stat_plans TO PUBLIC;
+
+-- Don't want this to be available to non-superusers.
+REVOKE ALL ON FUNCTION pg_stat_plans_reset() FROM PUBLIC;
diff --git a/contrib/pg_stat_plans/pg_stat_plans.c b/contrib/pg_stat_plans/pg_stat_plans.c
new file mode 100644
index 0000000000..318a7cddc7
--- /dev/null
+++ b/contrib/pg_stat_plans/pg_stat_plans.c
@@ -0,0 +1,743 @@
+/*--------------------------------------------------------------------------
+ *
+ * pg_stat_plans.c
+ * Track per-plan call counts, execution times and EXPLAIN texts
+ * across a whole database cluster.
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * contrib/pg_stat_plans/pg_stat_plans.c
+ *
+ * -------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/parallel.h"
+#include "catalog/pg_authid.h"
+#include "commands/explain.h"
+#include "common/hashfn.h"
+#include "funcapi.h"
+#include "mb/pg_wchar.h"
+#include "nodes/planjumble.h"
+#include "pgstat.h"
+#include "optimizer/planner.h"
+#include "utils/acl.h"
+#include "utils/builtins.h"
+#include "utils/guc.h"
+#include "utils/pgstat_internal.h"
+#include "utils/snapmgr.h"
+
+PG_MODULE_MAGIC;
+
+/* Current nesting depth of planner/ExecutorRun/ProcessUtility calls */
+static int nesting_level = 0;
+
+/* Saved hook values */
+static planner_hook_type prev_planner_hook = NULL;
+static ExecutorStart_hook_type prev_ExecutorStart = NULL;
+static ExecutorRun_hook_type prev_ExecutorRun = NULL;
+static ExecutorFinish_hook_type prev_ExecutorFinish = NULL;
+static ExecutorEnd_hook_type prev_ExecutorEnd = NULL;
+
+/*---- GUC variables ----*/
+
+typedef enum
+{
+ PGSP_TRACK_NONE, /* track no plans */
+ PGSP_TRACK_TOP, /* only plans for top level statements */
+ PGSP_TRACK_ALL, /* all plans, including for nested statements */
+} PGSPTrackLevel;
+
+static const struct config_enum_entry track_options[] =
+{
+ {"none", PGSP_TRACK_NONE, false},
+ {"top", PGSP_TRACK_TOP, false},
+ {"all", PGSP_TRACK_ALL, false},
+ {NULL, 0, false}
+};
+
+static int pgsp_max = 5000; /* max # plans to track */
+static int pgsp_max_size = 2048; /* max size of plan text to track (in bytes) */
+static int pgsp_track = PGSP_TRACK_TOP; /* tracking level */
+
+#define pgsp_enabled(level) \
+ (!IsParallelWorker() && \
+ (compute_plan_id != COMPUTE_PLAN_ID_OFF) && \
+ (pgsp_track == PGSP_TRACK_ALL || \
+ (pgsp_track == PGSP_TRACK_TOP && (level) == 0)))
+
+#define USAGE_INCREASE 0.5 /* increase by this each time we report stats */
+#define USAGE_DECREASE_FACTOR (0.99) /* decreased every pgstat_dealloc_plans */
+#define USAGE_DEALLOC_PERCENT 5 /* free this % of entries at once */
+
+/*---- Function declarations ----*/
+
+PG_FUNCTION_INFO_V1(pg_stat_plans_reset);
+PG_FUNCTION_INFO_V1(pg_stat_plans_1_0);
+
+/* Structures for statistics of plans */
+typedef struct PgStatShared_PlanInfo
+{
+ /* key elements that identify a plan (together with the dboid) */
+ uint64 planid;
+ uint64 queryid;
+ Oid userid; /* userid is tracked to allow users to see their own query plans */
+
+ dsa_pointer plan_text; /* pointer to DSA memory containing plan text */
+ int plan_encoding; /* plan text encoding */
+} PgStatShared_PlanInfo;
+
+typedef struct PgStat_StatPlanEntry
+{
+ PgStat_Counter exec_count;
+ double exec_time;
+ double usage; /* Usage factor of the entry, used to prioritize which plans to age out */
+
+ /* Only used in shared structure, not in local pending stats */
+ PgStatShared_PlanInfo info;
+} PgStat_StatPlanEntry;
+
+typedef struct PgStatShared_Plan
+{
+ PgStatShared_Common header;
+ PgStat_StatPlanEntry stats;
+} PgStatShared_Plan;
+
+static bool plan_stats_flush_cb(PgStat_EntryRef *entry_ref, bool nowait);
+
+static const PgStat_KindInfo plan_stats = {
+ .name = "plan_stats",
+ .fixed_amount = false,
+
+ /* We currently don't write to a file since plan texts would get lost (and just the stats on their own aren't that useful) */
+ .write_to_file = false,
+
+ /* Plan statistics are available system-wide to simplify monitoring scripts */
+ .accessed_across_databases = true,
+
+ .shared_size = sizeof(PgStatShared_Plan),
+ .shared_data_off = offsetof(PgStatShared_Plan, stats),
+ .shared_data_len = sizeof(((PgStatShared_Plan *) 0)->stats),
+ .pending_size = sizeof(PgStat_StatPlanEntry),
+ .flush_pending_cb = plan_stats_flush_cb,
+};
+
+/*
+ * Compute stats entry idx from query ID and plan ID with an 8-byte hash.
+ *
+ * Whilst we could theorically just use the plan ID here, we intentionally
+ * add the query ID into the mix to ease interpreting the data in combination
+ * with pg_stat_statements.
+ */
+#define PGSTAT_PLAN_IDX(query_id, plan_id, user_id) hash_combine64(query_id, hash_combine64(plan_id, user_id))
+
+/*
+ * Kind ID reserved for statistics of plans.
+ */
+#define PGSTAT_KIND_PLANS PGSTAT_KIND_EXPERIMENTAL /* TODO: Assign */
+
+/*
+ * Callback for stats handling
+ */
+static bool
+plan_stats_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
+{
+ PgStat_StatPlanEntry *localent;
+ PgStatShared_Plan *shfuncent;
+
+ localent = (PgStat_StatPlanEntry *) entry_ref->pending;
+ shfuncent = (PgStatShared_Plan *) entry_ref->shared_stats;
+
+ if (!pgstat_lock_entry(entry_ref, nowait))
+ return false;
+
+ shfuncent->stats.exec_count += localent->exec_count;
+ shfuncent->stats.exec_time += localent->exec_time;
+ shfuncent->stats.usage += localent->usage;
+
+ pgstat_unlock_entry(entry_ref);
+
+ return true;
+}
+
+static char *
+pgsp_explain_plan(QueryDesc *queryDesc)
+{
+ ExplainState *es;
+ StringInfo es_str;
+
+ es = NewExplainState();
+ es_str = es->str;
+
+ /*
+ * We turn off COSTS since identical planids may have very different costs,
+ * and it could be misleading to only show the first recorded plan's costs.
+ */
+ es->costs = false;
+ es->format = EXPLAIN_FORMAT_TEXT;
+
+ ExplainBeginOutput(es);
+ ExplainPrintPlan(es, queryDesc);
+ ExplainEndOutput(es);
+
+ return es_str->data;
+}
+
+static void
+pgstat_gc_plan_memory()
+{
+ dshash_seq_status hstat;
+ PgStatShared_HashEntry *p;
+
+ /* dshash entry is not modified, take shared lock */
+ dshash_seq_init(&hstat, pgStatLocal.shared_hash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ PgStatShared_Common *header;
+ PgStat_StatPlanEntry *statent;
+
+ if (!p->dropped || p->key.kind != PGSTAT_KIND_PLANS)
+ continue;
+
+ header = dsa_get_address(pgStatLocal.dsa, p->body);
+
+ if (!LWLockConditionalAcquire(&header->lock, LW_EXCLUSIVE))
+ continue;
+
+ statent = (PgStat_StatPlanEntry *) pgstat_get_entry_data(PGSTAT_KIND_PLANS, header);
+
+ /* Clean up this entry's plan text allocation, if we haven't done so already */
+ if (DsaPointerIsValid(statent->info.plan_text))
+ {
+ dsa_free(pgStatLocal.dsa, statent->info.plan_text);
+ statent->info.plan_text = InvalidDsaPointer;
+
+ /* Allow removal of the shared stats entry */
+ pg_atomic_fetch_sub_u32(&p->refcount, 1);
+ }
+
+ LWLockRelease(&header->lock);
+ }
+ dshash_seq_term(&hstat);
+
+ // Encourage other backends to clean up dropped entry refs
+ pgstat_request_entry_refs_gc();
+}
+
+typedef struct PlanDeallocEntry
+{
+ PgStat_HashKey key;
+ double usage;
+} PlanDeallocEntry;
+
+/*
+ * list sort comparator for sorting into decreasing usage order
+ */
+static int
+entry_cmp_lru(const union ListCell *lhs, const union ListCell *rhs)
+{
+ double l_usage = ((PlanDeallocEntry *) lfirst(lhs))->usage;
+ double r_usage = ((PlanDeallocEntry *) lfirst(rhs))->usage;
+
+ if (l_usage > r_usage)
+ return -1;
+ else if (l_usage < r_usage)
+ return +1;
+ else
+ return 0;
+}
+
+static void
+pgstat_dealloc_plans()
+{
+ dshash_seq_status hstat;
+ PgStatShared_HashEntry *p;
+ List *entries = NIL;
+ ListCell *lc;
+ int nvictims;
+
+ /* dshash entry is not modified, take shared lock */
+ dshash_seq_init(&hstat, pgStatLocal.shared_hash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ PgStatShared_Common *header;
+ PgStat_StatPlanEntry *statent;
+ PlanDeallocEntry *entry;
+
+ if (p->dropped || p->key.kind != PGSTAT_KIND_PLANS)
+ continue;
+
+ header = dsa_get_address(pgStatLocal.dsa, p->body);
+
+ if (!LWLockConditionalAcquire(&header->lock, LW_EXCLUSIVE))
+ continue;
+
+ statent = (PgStat_StatPlanEntry *) pgstat_get_entry_data(PGSTAT_KIND_PLANS, header);
+ statent->usage *= USAGE_DECREASE_FACTOR;
+
+ entry = palloc(sizeof(PlanDeallocEntry));
+ entry->key = p->key;
+ entry->usage = statent->usage;
+
+ LWLockRelease(&header->lock);
+
+ entries = lappend(entries, entry);
+ }
+ dshash_seq_term(&hstat);
+
+ /* Sort by usage ascending (lowest used entries are last) */
+ list_sort(entries, entry_cmp_lru);
+
+ /* At a minimum, deallocate 10 entries to make it worth our while */
+ nvictims = Max(10, list_length(entries) * USAGE_DEALLOC_PERCENT / 100);
+ nvictims = Min(nvictims, list_length(entries));
+
+ /* Actually drop the entries */
+ for_each_from(lc, entries, list_length(entries) - nvictims)
+ {
+ PlanDeallocEntry *entry = lfirst(lc);
+ pgstat_drop_entry(entry->key.kind, entry->key.dboid, entry->key.objid);
+ }
+
+ /* Clean up our working memory immediately */
+ foreach(lc, entries)
+ {
+ PlanDeallocEntry *entry = lfirst(lc);
+ pfree(entry);
+ }
+ pfree(entries);
+}
+
+static void
+pgstat_gc_plans()
+{
+ dshash_seq_status hstat;
+ PgStatShared_HashEntry *p;
+ bool have_dropped_entries = false;
+ size_t plan_entry_count = 0;
+
+ /* TODO: Prevent concurrent GC cycles - flag an active GC run somehow */
+
+ /*
+ * Count our active entries, and whether there are any dropped entries we
+ * may need to clean up at the end.
+ */
+ dshash_seq_init(&hstat, pgStatLocal.shared_hash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ if (p->key.kind != PGSTAT_KIND_PLANS)
+ continue;
+
+ if (p->dropped)
+ have_dropped_entries = true;
+ else
+ plan_entry_count++;
+ }
+ dshash_seq_term(&hstat);
+
+ /*
+ * If we're over the limit, delete entries with lowest usage factor.
+ */
+ if (plan_entry_count > pgsp_max)
+ {
+ pgstat_dealloc_plans();
+ have_dropped_entries = true; /* Assume we did some work */
+ }
+
+ /* If there are dropped entries, clean up their plan memory if needed */
+ if (have_dropped_entries)
+ pgstat_gc_plan_memory();
+}
+
+static void
+pgstat_report_plan_stats(QueryDesc *queryDesc,
+ PgStat_Counter exec_count,
+ double exec_time)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_Plan *shstatent;
+ PgStat_StatPlanEntry *statent;
+ bool newly_created;
+ uint64 queryId = queryDesc->plannedstmt->queryId;
+ uint64 planId = queryDesc->plannedstmt->planId;
+ Oid userid = GetUserId();
+
+ entry_ref = pgstat_prep_pending_entry(PGSTAT_KIND_PLANS, MyDatabaseId,
+ PGSTAT_PLAN_IDX(queryId, planId, userid), &newly_created);
+
+ shstatent = (PgStatShared_Plan *) entry_ref->shared_stats;
+ statent = &shstatent->stats;
+
+ if (newly_created)
+ {
+ char *plan = pgsp_explain_plan(queryDesc);
+ size_t plan_size = Min(strlen(plan), pgsp_max_size);
+
+ (void) pgstat_lock_entry(entry_ref, false);
+
+ /*
+ * We may be over the limit, so run GC now before saving entry
+ * (we do this whilst holding the lock on the new entry so we don't remove it by accident)
+ */
+ pgstat_gc_plans();
+
+ shstatent->stats.info.planid = planId;
+ shstatent->stats.info.queryid = queryId;
+ shstatent->stats.info.userid = userid;
+ shstatent->stats.info.plan_text = dsa_allocate(pgStatLocal.dsa, plan_size);
+ strlcpy(dsa_get_address(pgStatLocal.dsa, shstatent->stats.info.plan_text), plan, plan_size);
+
+ shstatent->stats.info.plan_encoding = GetDatabaseEncoding();
+
+ /* Increase refcount here so entry can't get released without us dropping the plan text */
+ pg_atomic_fetch_add_u32(&entry_ref->shared_entry->refcount, 1);
+
+ pgstat_unlock_entry(entry_ref);
+
+ pfree(plan);
+ }
+
+ statent->exec_count += exec_count;
+ statent->exec_time += exec_time;
+ statent->usage += USAGE_INCREASE;
+}
+
+/*
+ * Planner hook: forward to regular planner, but increase plan count and
+ * record query plan if needed.
+ */
+static PlannedStmt *
+pgsp_planner(Query *parse,
+ const char *query_string,
+ int cursorOptions,
+ ParamListInfo boundParams)
+{
+ PlannedStmt *result;
+
+ /*
+ * Increment the nesting level, to ensure that functions
+ * evaluated during planning are not seen as top-level calls.
+ */
+ nesting_level++;
+ PG_TRY();
+ {
+ if (prev_planner_hook)
+ result = prev_planner_hook(parse, query_string, cursorOptions,
+ boundParams);
+ else
+ result = standard_planner(parse, query_string, cursorOptions,
+ boundParams);
+ }
+ PG_FINALLY();
+ {
+ nesting_level--;
+ }
+ PG_END_TRY();
+
+ return result;
+}
+
+/*
+ * ExecutorStart hook: start up tracking if needed
+ */
+static void
+pgsp_ExecutorStart(QueryDesc *queryDesc, int eflags)
+{
+ uint64 queryId = queryDesc->plannedstmt->queryId;
+ uint64 planId = queryDesc->plannedstmt->planId;
+
+ if (prev_ExecutorStart)
+ prev_ExecutorStart(queryDesc, eflags);
+ else
+ standard_ExecutorStart(queryDesc, eflags);
+
+ if (queryId != UINT64CONST(0) && planId != UINT64CONST(0) &&
+ pgsp_enabled(nesting_level))
+ {
+ /* Record initial entry now, so plan text is available for currently running queries */
+ pgstat_report_plan_stats(queryDesc,
+ 0, /* executions are counted in pgsp_ExecutorEnd */
+ 0.0);
+
+ /*
+ * Set up to track total elapsed time in ExecutorRun. Make sure the
+ * space is allocated in the per-query context so it will go away at
+ * ExecutorEnd.
+ */
+ if (queryDesc->totaltime == NULL)
+ {
+ MemoryContext oldcxt;
+
+ oldcxt = MemoryContextSwitchTo(queryDesc->estate->es_query_cxt);
+ queryDesc->totaltime = InstrAlloc(1, INSTRUMENT_ALL, false);
+ MemoryContextSwitchTo(oldcxt);
+ }
+ }
+}
+
+/*
+ * ExecutorRun hook: all we need do is track nesting depth
+ */
+static void
+pgsp_ExecutorRun(QueryDesc *queryDesc, ScanDirection direction, uint64 count)
+{
+ nesting_level++;
+ PG_TRY();
+ {
+ if (prev_ExecutorRun)
+ prev_ExecutorRun(queryDesc, direction, count);
+ else
+ standard_ExecutorRun(queryDesc, direction, count);
+ }
+ PG_FINALLY();
+ {
+ nesting_level--;
+ }
+ PG_END_TRY();
+}
+
+/*
+ * ExecutorFinish hook: all we need do is track nesting depth
+ */
+static void
+pgsp_ExecutorFinish(QueryDesc *queryDesc)
+{
+ nesting_level++;
+ PG_TRY();
+ {
+ if (prev_ExecutorFinish)
+ prev_ExecutorFinish(queryDesc);
+ else
+ standard_ExecutorFinish(queryDesc);
+ }
+ PG_FINALLY();
+ {
+ nesting_level--;
+ }
+ PG_END_TRY();
+}
+
+/*
+ * ExecutorEnd hook: store results if needed
+ */
+static void
+pgsp_ExecutorEnd(QueryDesc *queryDesc)
+{
+ uint64 queryId = queryDesc->plannedstmt->queryId;
+ uint64 planId = queryDesc->plannedstmt->planId;
+
+ if (queryId != UINT64CONST(0) && planId != UINT64CONST(0) &&
+ queryDesc->totaltime && pgsp_enabled(nesting_level))
+ {
+ /*
+ * Make sure stats accumulation is done. (Note: it's okay if several
+ * levels of hook all do this.)
+ */
+ InstrEndLoop(queryDesc->totaltime);
+
+ pgstat_report_plan_stats(queryDesc,
+ 1,
+ queryDesc->totaltime->total * 1000.0 /* convert to msec */);
+ }
+
+ if (prev_ExecutorEnd)
+ prev_ExecutorEnd(queryDesc);
+ else
+ standard_ExecutorEnd(queryDesc);
+}
+
+/*
+ * Module load callback
+ */
+void
+_PG_init(void)
+{
+ /*
+ * In order to register for shared memory stats, we have to be loaded via
+ * shared_preload_libraries. If not, fall out without hooking into any of
+ * the main system. (We don't throw error here because it seems useful to
+ * allow the pg_stat_plans functions to be created even when the
+ * module isn't active. The functions must protect themselves against
+ * being called then, however.)
+ */
+ if (!process_shared_preload_libraries_in_progress)
+ return;
+
+ /*
+ * Inform the postmaster that we want to enable query_id calculation if
+ * compute_query_id is set to auto.
+ *
+ * Note that this does not apply to compute_plan_id, which must be
+ * enabled explicitly.
+ */
+ EnableQueryId();
+
+ /*
+ * Define (or redefine) custom GUC variables.
+ */
+ DefineCustomIntVariable("pg_stat_plans.max",
+ "Sets the maximum number of plans tracked by pg_stat_plans in shared memory.",
+ NULL,
+ &pgsp_max,
+ 5000,
+ 100,
+ INT_MAX / 2,
+ PGC_SIGHUP,
+ 0,
+ NULL,
+ NULL,
+ NULL);
+
+ DefineCustomIntVariable("pg_stat_plans.max_size",
+ "Sets the maximum size of plan texts tracked by pg_stat_plans in shared memory.",
+ NULL,
+ &pgsp_max_size,
+ 2048,
+ 100,
+ 1048576, /* 1MB hard limit */
+ PGC_SUSET,
+ 0,
+ NULL,
+ NULL,
+ NULL);
+
+ DefineCustomEnumVariable("pg_stat_plans.track",
+ "Selects which plans are tracked by pg_stat_plans.",
+ NULL,
+ &pgsp_track,
+ PGSP_TRACK_TOP,
+ track_options,
+ PGC_SUSET,
+ 0,
+ NULL,
+ NULL,
+ NULL);
+
+ MarkGUCPrefixReserved("pg_stat_plans");
+
+ /*
+ * Install hooks.
+ */
+ prev_planner_hook = planner_hook;
+ planner_hook = pgsp_planner;
+ prev_ExecutorStart = ExecutorStart_hook;
+ ExecutorStart_hook = pgsp_ExecutorStart;
+ prev_ExecutorRun = ExecutorRun_hook;
+ ExecutorRun_hook = pgsp_ExecutorRun;
+ prev_ExecutorFinish = ExecutorFinish_hook;
+ ExecutorFinish_hook = pgsp_ExecutorFinish;
+ prev_ExecutorEnd = ExecutorEnd_hook;
+ ExecutorEnd_hook = pgsp_ExecutorEnd;
+
+ pgstat_register_kind(PGSTAT_KIND_PLANS, &plan_stats);
+}
+
+/*
+ * Reset statement statistics.
+ */
+Datum
+pg_stat_plans_reset(PG_FUNCTION_ARGS)
+{
+ pgstat_drop_entries_of_kind(PGSTAT_KIND_PLANS);
+
+ /* Free plan text memory and allow cleanup of dropped entries */
+ pgstat_gc_plan_memory();
+
+ PG_RETURN_VOID();
+}
+
+#define PG_STAT_PLANS_COLS 7
+
+Datum
+pg_stat_plans_1_0(PG_FUNCTION_ARGS)
+{
+ bool showplan = PG_GETARG_BOOL(0);
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ Oid userid = GetUserId();
+ bool is_allowed_role = false;
+
+ dshash_seq_status hstat;
+ PgStatShared_HashEntry *p;
+
+ /*
+ * Superusers or roles with the privileges of pg_read_all_stats members
+ * are allowed
+ */
+ is_allowed_role = has_privs_of_role(userid, ROLE_PG_READ_ALL_STATS);
+
+ /* stats kind must be registered already */
+ if (!pgstat_get_kind_info(PGSTAT_KIND_PLANS))
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("pg_stat_plans must be loaded via \"shared_preload_libraries\"")));
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ /* dshash entry is not modified, take shared lock */
+ dshash_seq_init(&hstat, pgStatLocal.shared_hash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ PgStat_StatPlanEntry *statent;
+ Datum values[PG_STAT_PLANS_COLS];
+ bool nulls[PG_STAT_PLANS_COLS];
+ int i = 0;
+
+ if (p->dropped || p->key.kind != PGSTAT_KIND_PLANS)
+ continue;
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+
+ statent = pgstat_fetch_entry(p->key.kind, p->key.dboid, p->key.objid);
+
+ values[i++] = ObjectIdGetDatum(statent->info.userid);
+ values[i++] = ObjectIdGetDatum(p->key.dboid);
+ if (is_allowed_role || statent->info.userid == userid)
+ {
+ int64 queryid = statent->info.queryid;
+ int64 planid = statent->info.planid;
+ values[i++] = Int64GetDatumFast(queryid);
+ values[i++] = Int64GetDatumFast(planid);
+ }
+ else
+ {
+ nulls[i++] = true;
+ nulls[i++] = true;
+ }
+ values[i++] = Int64GetDatumFast(statent->exec_count);
+ values[i++] = Float8GetDatumFast(statent->exec_time);
+
+ if (showplan && (is_allowed_role || statent->info.userid == userid))
+ {
+ char *pstr = DsaPointerIsValid(statent->info.plan_text) ? dsa_get_address(pgStatLocal.dsa, statent->info.plan_text) : NULL;
+
+ if (pstr)
+ {
+ char *enc = pg_any_to_server(pstr, strlen(pstr), statent->info.plan_encoding);
+ values[i++] = CStringGetTextDatum(enc);
+
+ if (enc != pstr)
+ pfree(enc);
+ }
+ else
+ {
+ nulls[i++] = true;
+ }
+ }
+ else if (showplan)
+ {
+ values[i++] = CStringGetTextDatum("<insufficient privilege>");
+ }
+ else
+ {
+ nulls[i++] = true;
+ }
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
+ }
+ dshash_seq_term(&hstat);
+
+ return (Datum) 0;
+}
diff --git a/contrib/pg_stat_plans/pg_stat_plans.conf b/contrib/pg_stat_plans/pg_stat_plans.conf
new file mode 100644
index 0000000000..bfe571d547
--- /dev/null
+++ b/contrib/pg_stat_plans/pg_stat_plans.conf
@@ -0,0 +1,3 @@
+shared_preload_libraries = 'pg_stat_plans'
+compute_plan_id = on
+compute_query_id = on
diff --git a/contrib/pg_stat_plans/pg_stat_plans.control b/contrib/pg_stat_plans/pg_stat_plans.control
new file mode 100644
index 0000000000..4db3a47239
--- /dev/null
+++ b/contrib/pg_stat_plans/pg_stat_plans.control
@@ -0,0 +1,5 @@
+# pg_stat_plans extension
+comment = 'track per-plan call counts, execution times and EXPLAIN texts'
+default_version = '1.0'
+module_pathname = '$libdir/pg_stat_plans'
+relocatable = true
diff --git a/contrib/pg_stat_plans/sql/cleanup.sql b/contrib/pg_stat_plans/sql/cleanup.sql
new file mode 100644
index 0000000000..51565617ce
--- /dev/null
+++ b/contrib/pg_stat_plans/sql/cleanup.sql
@@ -0,0 +1 @@
+DROP EXTENSION pg_stat_plans;
diff --git a/contrib/pg_stat_plans/sql/privileges.sql b/contrib/pg_stat_plans/sql/privileges.sql
new file mode 100644
index 0000000000..aaad72a655
--- /dev/null
+++ b/contrib/pg_stat_plans/sql/privileges.sql
@@ -0,0 +1,59 @@
+--
+-- Only superusers and roles with privileges of the pg_read_all_stats role
+-- are allowed to see the plan text, queryid and planid of queries executed by
+-- other users. Other users can see the statistics.
+--
+
+CREATE ROLE regress_stats_superuser SUPERUSER;
+CREATE ROLE regress_stats_user1;
+CREATE ROLE regress_stats_user2;
+GRANT pg_read_all_stats TO regress_stats_user2;
+
+SET ROLE regress_stats_superuser;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+SELECT 1 AS "ONE";
+
+SET ROLE regress_stats_user1;
+SELECT 1+1 AS "TWO";
+
+--
+-- A superuser can read all columns of queries executed by others,
+-- including plan text, queryid and planid.
+--
+
+SET ROLE regress_stats_superuser;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+
+--
+-- regress_stats_user1 has no privileges to read the plan text, queryid
+-- or planid of queries executed by others but can see statistics
+-- like calls and rows.
+--
+
+SET ROLE regress_stats_user1;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+
+--
+-- regress_stats_user2, with pg_read_all_stats role privileges, can
+-- read all columns, including plan text, queryid and planid, of queries
+-- executed by others.
+--
+
+SET ROLE regress_stats_user2;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+
+--
+-- cleanup
+--
+
+RESET ROLE;
+DROP ROLE regress_stats_superuser;
+DROP ROLE regress_stats_user1;
+DROP ROLE regress_stats_user2;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
diff --git a/contrib/pg_stat_plans/sql/select.sql b/contrib/pg_stat_plans/sql/select.sql
new file mode 100644
index 0000000000..f0e803ad70
--- /dev/null
+++ b/contrib/pg_stat_plans/sql/select.sql
@@ -0,0 +1,67 @@
+--
+-- SELECT statements
+--
+
+CREATE EXTENSION pg_stat_plans;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+
+--
+-- simple statements
+--
+
+SELECT 1 FROM pg_class LIMIT 1;
+
+SELECT 1 FROM pg_class WHERE relname = 'pg_class';
+
+SET enable_indexscan = off;
+SELECT 1 FROM pg_class WHERE relname = 'pg_class';
+SET enable_indexscan = on;
+
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+
+--
+-- subplans and CTEs
+--
+
+WITH x AS MATERIALIZED (SELECT 1)
+SELECT * FROM x;
+
+SELECT a.attname,
+ (SELECT pg_catalog.pg_get_expr(d.adbin, d.adrelid)
+ FROM pg_catalog.pg_attrdef d
+ WHERE d.adrelid = a.attrelid AND d.adnum = a.attnum AND a.atthasdef)
+ FROM pg_catalog.pg_attribute a
+ WHERE a.attrelid = 'pg_class'::regclass
+ ORDER BY attnum LIMIT 1;
+
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+
+--
+-- partitoning
+--
+
+create table lp (a char) partition by list (a);
+create table lp_default partition of lp default;
+create table lp_ef partition of lp for values in ('e', 'f');
+create table lp_ad partition of lp for values in ('a', 'd');
+create table lp_bc partition of lp for values in ('b', 'c');
+create table lp_g partition of lp for values in ('g');
+create table lp_null partition of lp for values in (null);
+
+select * from lp;
+select * from lp where a > 'a' and a < 'd';
+select * from lp where a > 'a' and a <= 'd';
+select * from lp where a = 'a';
+select * from lp where 'a' = a; /* commuted */
+select * from lp where a is not null;
+select * from lp where a is null;
+select * from lp where a = 'a' or a = 'c';
+select * from lp where a is not null and (a = 'a' or a = 'c');
+select * from lp where a <> 'g';
+select * from lp where a <> 'a' and a <> 'd';
+select * from lp where a not in ('a', 'd');
+
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index 7c381949a5..4a5a02c704 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -157,6 +157,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
&pglogicalinspect;
&pgprewarm;
&pgrowlocks;
+ &pgstatplans;
&pgstatstatements;
&pgstattuple;
&pgsurgery;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 66e6dccd4c..b0afb33ce2 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -146,6 +146,7 @@
<!ENTITY pglogicalinspect SYSTEM "pglogicalinspect.sgml">
<!ENTITY pgprewarm SYSTEM "pgprewarm.sgml">
<!ENTITY pgrowlocks SYSTEM "pgrowlocks.sgml">
+<!ENTITY pgstatplans SYSTEM "pgstatplans.sgml">
<!ENTITY pgstatstatements SYSTEM "pgstatstatements.sgml">
<!ENTITY pgstattuple SYSTEM "pgstattuple.sgml">
<!ENTITY pgsurgery SYSTEM "pgsurgery.sgml">
diff --git a/doc/src/sgml/pgstatplans.sgml b/doc/src/sgml/pgstatplans.sgml
new file mode 100644
index 0000000000..100ce6b3aa
--- /dev/null
+++ b/doc/src/sgml/pgstatplans.sgml
@@ -0,0 +1,413 @@
+<!-- doc/src/sgml/pgstatplans.sgml -->
+
+<sect1 id="pgstatplans" xreflabel="pg_stat_plans">
+ <title>pg_stat_plans — track per-plan call counts, execution times and EXPLAIN texts</title>
+
+ <indexterm zone="pgstatplans">
+ <primary>pg_stat_plans</primary>
+ </indexterm>
+
+ <para>
+ The <filename>pg_stat_plans</filename> module provides a means for
+ tracking per-plan statistics and plan texts of all SQL statements executed by
+ a server.
+ </para>
+
+ <para>
+ The module must be loaded by adding <literal>pg_stat_plans</literal> to
+ <xref linkend="guc-shared-preload-libraries"/> in
+ <filename>postgresql.conf</filename>, because it requires additional shared memory.
+ This means that a server restart is needed to add or remove the module.
+ In addition, query and plan identifier calculation must be enabled in order for the
+ module to be active by setting both <xref linkend="guc-compute-plan-id"/> to <literal>on</literal>
+ and <xref linkend="guc-compute-query-id"/> to <literal>auto</literal> or <literal>on</literal>.
+ </para>
+
+ <para>
+ When <filename>pg_stat_plans</filename> is active, it tracks
+ statistics across all databases of the server. To access and manipulate
+ these statistics, the module provides the <structname>pg_stat_plans</structname>
+ view and the utility functions <function>pg_stat_plans_reset</function> and
+ <function>pg_stat_plans</function>. These are not available globally but
+ can be enabled for a specific database with
+ <command>CREATE EXTENSION pg_stat_plans</command>.
+ </para>
+
+ <sect2 id="pgstatplans-pg-stat-plans">
+ <title>The <structname>pg_stat_plans</structname> View</title>
+
+ <para>
+ The statistics gathered by the module are made available via a
+ view named <structname>pg_stat_plans</structname>. This view
+ contains one row for each distinct combination of database ID, user
+ ID, query ID and plan ID (up to the maximum number of distinct plans
+ that the module can track). The columns of the view are shown in
+ <xref linkend="pgstatplans-columns"/>.
+ </para>
+
+ <table id="pgstatplans-columns">
+ <title><structname>pg_stat_plans</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>userid</structfield> <type>oid</type>
+ (references <link linkend="catalog-pg-authid"><structname>pg_authid</structname></link>.<structfield>oid</structfield>)
+ </para>
+ <para>
+ OID of user who executed the statement
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbid</structfield> <type>oid</type>
+ (references <link linkend="catalog-pg-database"><structname>pg_database</structname></link>.<structfield>oid</structfield>)
+ </para>
+ <para>
+ OID of database in which the statement was executed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>queryid</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Hash code to identify identical normalized queries.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>planid</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Hash code to identify identical plan shapes.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>plan</structfield> <type>text</type>
+ </para>
+ <para>
+ Plan text of a representative plan. This is similar to the output of
+ <literal>EXPLAIN (COSTS OFF)</literal>. Note the plan text will contain constant
+ values of the first plan recorded, but subsequent executions of the
+ same plan hash code (<structfield>planid</structfield>) with different
+ constant values will be tracked under the same entry.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>calls</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the plan was executed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_exec_time</structfield> <type>double precision</type>
+ </para>
+ <para>
+ Total time spent executing the plan, in milliseconds
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ For security reasons, only superusers and roles with privileges of the
+ <literal>pg_read_all_stats</literal> role are allowed to see the plan text,
+ <structfield>queryid</structfield> and <structfield>planid</structfield>
+ of queries executed by other users. Other users can see the statistics,
+ however, if the view has been installed in their database.
+ </para>
+
+ <para>
+ Plannable queries (that is, <command>SELECT</command>, <command>INSERT</command>,
+ <command>UPDATE</command>, <command>DELETE</command>, and <command>MERGE</command>)
+ are combined into a single <structname>pg_stat_plans</structname> entry whenever
+ they have identical plan structures according to an internal hash calculation.
+ Typically, two plans will be considered the same for this purpose if they have
+ the same <literal>EXPLAIN (COSTS OFF)</literal> output and are semantically
+ equivalent except for the values of literal constants appearing in the query plan.
+ </para>
+
+ <para>
+ Note that queries that have not finished executing yet will show in
+ <structname>pg_stat_plans</structname> with their plan text, but without
+ the <structname>calls</structname> field being incremented. This can be
+ used to identify the query plan for a currently running statement by joining
+ <link linkend="monitoring-pg-stat-activity-view">pg_stat_activity</link>
+ with <structname>pg_stat_plans</structname>, see example usage in
+ <xref linkend="pgstatplans-sample-output"/>.
+ </para>
+
+ <para>
+ Consumers of <structname>pg_stat_plans</structname> should use
+ <structfield>planid</structfield> in combination with
+ <structfield>queryid</structfield>, <structfield>dbid</structfield>
+ and <structfield>userid</structfield> as a stable and reliable identifier
+ for each entry, instead of using its plan text. However, it is important
+ to understand that there are only limited guarantees around the stability
+ of the <structfield>planid</structfield> hash value. Since the identifier
+ is derived from the plan tree, its value is a function of, among other
+ things, the internal object identifiers appearing in this representation.
+ This has some counterintuitive implications. For example,
+ <filename>pg_stat_plans</filename> will consider two apparently-identical
+ plans to be distinct, if they reference a table that was dropped
+ and recreated between the creation of the two plans.
+ The hashing process is also sensitive to differences in
+ machine architecture and other facets of the platform.
+ Furthermore, it is not safe to assume that <structfield>planid</structfield>
+ will be stable across major versions of <productname>PostgreSQL</productname>.
+ </para>
+
+ <para>
+ Two servers participating in replication based on physical WAL replay can
+ be expected to have identical <structfield>planid</structfield> values for
+ the same plan. However, logical replication schemes do not promise to
+ keep replicas identical in all relevant details, so
+ <structfield>planid</structfield> will not be a useful identifier for
+ accumulating costs across a set of logical replicas.
+ If in doubt, direct testing is recommended.
+ </para>
+
+ <para>
+ Plan texts are stored in shared memory, and limited in length. To increase
+ the maximum length of stored plan texts you can increase
+ <varname>pg_stat_plans.max_size</varname>. This value can be changed for
+ an individual connection, or set as a server-wide setting.
+ </para>
+ </sect2>
+
+ <sect2 id="pgstatplans-funcs">
+ <title>Functions</title>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ <function>pg_stat_plans_reset() returns void</function>
+ <indexterm>
+ <primary>pg_stat_plans_reset</primary>
+ </indexterm>
+ </term>
+
+ <listitem>
+ <para>
+ <function>pg_stat_plans_reset</function> discards statistics and plan texts
+ gathered so far by <filename>pg_stat_plans</filename>.
+ By default, this function can only be executed by superusers.
+ Access may be granted to others using <command>GRANT</command>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <function>pg_stat_plans(showplan boolean) returns setof record</function>
+ <indexterm>
+ <primary>pg_stat_plans</primary>
+ <secondary>function</secondary>
+ </indexterm>
+ </term>
+
+ <listitem>
+ <para>
+ The <structname>pg_stat_plans</structname> view is defined in
+ terms of a function also named <function>pg_stat_plans</function>.
+ It is possible for clients to call
+ the <function>pg_stat_plans</function> function directly, and by
+ specifying <literal>showplan := false</literal> have plan texts be
+ omitted (that is, the <literal>OUT</literal> argument that corresponds
+ to the view's <structfield>plan</structfield> column will return nulls). This
+ feature is intended to support external tools that might wish to avoid
+ the overhead of repeatedly retrieving plan texts of indeterminate
+ length. Such tools can instead cache the first plan text observed
+ for each entry themselves, since that is
+ all <filename>pg_stat_plans</filename> itself does, and then retrieve
+ plan texts only as needed.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+
+ <sect2 id="pgstatplans-config-params">
+ <title>Configuration Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ <varname>pg_stat_plans.max</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>pg_stat_plans.max</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+
+ <listitem>
+ <para>
+ <varname>pg_stat_plans.max</varname> is the maximum number of
+ plans tracked by the module (i.e., the maximum number of rows
+ in the <structname>pg_stat_plans</structname> view). If more distinct
+ plans than that are observed, information about the least-executed
+ plans is discarded. The default value is 5000.
+ Only superusers can change this setting. Changing the setting requires
+ a reload of the server.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <varname>pg_stat_plans.max_size</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>pg_stat_plans.max_size</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+
+ <listitem>
+ <para>
+ <varname>pg_stat_plans.max_size</varname> is the maximum length of
+ each plan text tracked by the module in bytes. Longer plan texts will be truncated.
+ The default value is 2048 (2kB).
+ Only superusers can change this setting.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <varname>pg_stat_plans.track</varname> (<type>enum</type>)
+ <indexterm>
+ <primary><varname>pg_stat_plans.track</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+
+ <listitem>
+ <para>
+ <varname>pg_stat_plans.track</varname> controls which plans
+ are counted by the module.
+ Specify <literal>top</literal> to track plans by top-level statements (those issued
+ directly by clients), <literal>all</literal> to also track nested statements
+ (such as statements invoked within functions), or <literal>none</literal> to
+ disable plan statistics collection.
+ The default value is <literal>top</literal>.
+ Only superusers can change this setting.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ <para>
+ The module requires additional shared memory proportional to
+ <varname>pg_stat_plans.max</varname> for statistics, as well as
+ <varname>pg_stat_plans.max</varname> multiplied by
+ <varname>pg_stat_plans.max_size</varname> for plan texts. Note that this
+ memory is only consumed when entries are created, and not if
+ <varname>pg_stat_plans.track</varname> is set to <literal>none</literal>.
+ </para>
+
+ <para>
+ These parameters must be set in <filename>postgresql.conf</filename>.
+ Typical usage might be:
+
+<programlisting>
+# postgresql.conf
+shared_preload_libraries = 'pg_stat_plans'
+
+compute_query_id = on
+compute_plan_id = on
+pg_stat_plans.max = 10000
+pg_stat_plans.max_size = 4096
+pg_stat_plans.track = all
+</programlisting>
+ </para>
+ </sect2>
+
+ <sect2 id="pgstatplans-sample-output">
+ <title>Sample Output</title>
+
+<screen>
+bench=# SELECT pg_stat_plans_reset();
+
+$ pgbench -i bench
+$ pgbench -c10 -t300 bench
+
+bench=# \x
+bench=# SELECT plan, calls, total_exec_time
+ FROM pg_stat_plans ORDER BY total_exec_time DESC LIMIT 5;
+-[ RECORD 1 ]---+-----------------------------------------------------------------
+plan | Update on pgbench_tellers +
+ | -> Seq Scan on pgbench_tellers +
+ | Filter: (tid = 5)
+calls | 3000
+total_exec_time | 642.8880919999993
+-[ RECORD 2 ]---+-----------------------------------------------------------------
+plan | Update on pgbench_branches +
+ | -> Seq Scan on pgbench_branches +
+ | Filter: (bid = 1)
+calls | 1813
+total_exec_time | 476.64152700000005
+-[ RECORD 3 ]---+-----------------------------------------------------------------
+plan | Update on pgbench_branches +
+ | -> Index Scan using pgbench_branches_pkey on pgbench_branches+
+ | Index Cond: (bid = 1)
+calls | 1187
+total_exec_time | 326.1257549999999
+-[ RECORD 4 ]---+-----------------------------------------------------------------
+plan | Update on pgbench_accounts +
+ | -> Index Scan using pgbench_accounts_pkey on pgbench_accounts+
+ | Index Cond: (aid = 48793)
+calls | 3000
+total_exec_time | 21.664690000000093
+-[ RECORD 5 ]---+-----------------------------------------------------------------
+plan | Insert on pgbench_history +
+ | -> Result
+calls | 3000
+total_exec_time | 4.365250999999957
+
+session 1:
+
+bench# SELECT pg_sleep(100), COUNT(*) FROM pgbench_accounts;
+
+session 2:
+
+bench=# SELECT query, plan FROM pg_stat_activity
+ JOIN pg_stat_plans ON (usesysid = userid AND datid = dbid AND query_id = queryid AND plan_id = planid)
+ WHERE query LIKE 'SELECT pg_sleep%';
+ query | plan
+-------------------------------------------------------+------------------------------------
+ SELECT pg_sleep(100), COUNT(*) FROM pgbench_accounts; | Aggregate +
+ | -> Seq Scan on pgbench_accounts
+(1 row)
+
+</screen>
+ </sect2>
+
+ <sect2 id="pgstatplans-authors">
+ <title>Authors</title>
+
+ <para>
+ Lukas Fittl <email>lukas@fittl.com</email>.
+ </para>
+ </sect2>
+
+</sect1>
--
2.47.1
On Thu, 2 Jan 2025 12:46:04 -0800
Lukas Fittl <lukas@fittl.com> wrote:
this proposed patch set adds:
1. An updated in-core facility to optionally track Plan IDs based on
hashing the plan nodes during the existing treewalk in setrefs.c -
controlled by the new "compute_plan_id" GUC
2. An example user of plan IDs with a new pg_stat_plans extension in
contrib, that also records the first plan text with EXPLAIN (COSTS
OFF)My overall perspective is that (1) is best done in-core to keep
overhead low, whilst (2) could be done outside of core (or merged
with a future pg_stat_statements) and is included here mainly for
illustration purposes.
And 2025 is starting with a bang! Nice to see this email! Being able to
collect telemetry that indicates when plan changes happened would be
very useful.
The specifics of how a plan ID is generated are going to have some edge
cases (as you noted)
I concur that the ideal place for this to eventually land would be
alongside queryid in pg_stat_activity
-Jeremy
On Thu, Jan 2, 2025 at 10:47 PM Lukas Fittl <lukas@fittl.com> wrote:
The first patch allows use of node jumbling by other unit files /
extensions, which would help an out-of-core extension avoid duplicating all
the node jumbling code.The second patch adds a function for the extensible cumulative statistics
system to drop all entries for a given statistics kind. This already exists
for resetting, but in case of a dynamic list of entries its more useful to
be able to drop all of them when "reset" is called.The third patch adds plan ID tracking in core. This is turned off by
default, and can be enabled by setting "compute_plan_id" to "on". Plan IDs
are shown in pg_stat_activity, as well as EXPLAIN and auto_explain output,
to allow matching a given plan ID to a plan text, without requiring the use
of an extension. There are some minor TODOs in the plan jumbling logic that
I haven't finalized yet. There is also an open question whether we should
use the node attribute mechanism instead of custom jumbling logic?The fourth patch adds the pg_stat_plans contrib extension, for
illustrative purposes. This is inspired by pg_stat_statements, but
intentionally kept separate for easier review and since it does not use an
external file and could technically be used independently. We may want to
develop this into a unified pg_stat_statements+plans in-core mechanism in
the future, but I think that is best kept for a separate discussion.The pg_stat_plans extension utilizes the cumulative statistics system for
tracking statistics (extensible thanks to recent changes!), as well as
dynamic shared memory to track plan texts up to a given limit (2kB by
default). As a side note, managing extra allocations with the new
extensible stats is a bit cumbersome - it would be helpful to have a hook
for cleaning up data associated to entries (like a DSA allocation).Thanks,
Lukas[0]:
/messages/by-id/604E3199-2DD2-47DD-AC47-774A6F97DCA9@amazon.com
<https://url.avanan.click/v2/r01/___https://www.postgresql.org/message-id/flat/604E3199-2DD2-47DD-AC47-774A6F97DCA9*40amazon.com___.YXAzOnBlcmNvbmE6YTpnOjk5NDUyOGU1MTIwZDhhNDQxNzNiMDM0NjEwZjY1NTIxOjc6YWM0YjpjN2VmMzI5ZmVjMmM2N2RlNDg0MGVlNjJmMGFlOTQ3OGQ1NTM1ODZmZGMxNzI2NGQ4NmEwMDcxYmI1ODVjY2RjOmg6VDpO>
[1]: https://github.com/2ndQuadrant/pg_stat_plans
<https://url.avanan.click/v2/r01/___https://github.com/2ndQuadrant/pg_stat_plans___.YXAzOnBlcmNvbmE6YTpnOjk5NDUyOGU1MTIwZDhhNDQxNzNiMDM0NjEwZjY1NTIxOjc6NjM3NTowODVhZWY2OGY1MjdhYWEzY2NiMDY1NTVlNzcwYjM5YTlmOTI5ODU3ZWI5ZWY2NjY1YTljMDBmMWEyNDU0ZmMwOmg6VDpO>
[2]: https://ossc-db.github.io/pg_store_plans/
<https://url.avanan.click/v2/r01/___https://ossc-db.github.io/pg_store_plans/___.YXAzOnBlcmNvbmE6YTpnOjk5NDUyOGU1MTIwZDhhNDQxNzNiMDM0NjEwZjY1NTIxOjc6NjU3ZDo0NjA1YzQ1ZTk2ZGEzZmZiNmM5NTEyYjZiMTRmYjk3Y2RjMTE5M2ZkMTMwYTg4ZWM1NjdmMWY1N2RhZjI5YTliOmg6VDpO>
[3]:
https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora_stat_plans.html
<https://url.avanan.click/v2/r01/___https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora_stat_plans.html___.YXAzOnBlcmNvbmE6YTpnOjk5NDUyOGU1MTIwZDhhNDQxNzNiMDM0NjEwZjY1NTIxOjc6ZGNjNTozOGIyNDM2MWVhYzg1MTcyNjc5NzJlZTdkM2JkNzliMjE3NjYzODk5MGQwMTdkNDM1YzliMGU5MDA1ZmEwNzFlOmg6VDpO>--
Lukas Fittl
Hello Lukas,
We have another extension that does plan ID tracking: pg_stat_monitor. So I
think it would be great to have this functionality in core.
I tested your patch set on top of *86749ea3b76* PG revision on MacOS. All
tests successfully passed. However, pgident shows that some files are not
properly formatted.
--
Artem Gavrilov
Senior Software Engineer, Percona
artem.gavrilov@percona.com
On Thu, Jan 02, 2025 at 12:46:04PM -0800, Lukas Fittl wrote:
Inspired by a prior proposal by Sami Imseih for tracking Plan IDs [0], as
well as extensions like pg_stat_plans [1] (unmaintained), pg_store_plans
[2] (not usable on production, see notes later) and aurora_stat_plans [3]
(enabled by default on AWS), this proposed patch set adds:
0002 introduces this new routine to delete all the entries of the new
stats kind you are adding:
+void
+pgstat_drop_entries_of_kind(PgStat_Kind kind)
+{
+ dshash_seq_status hstat;
+ PgStatShared_HashEntry *ps;
+ uint64 not_freed_count = 0;
+
+ dshash_seq_init(&hstat, pgStatLocal.shared_hash, true);
This is the same as pgstat_drop_all_entries(), except for the filter
based on the stats kind and the fact that you need to take care of the
local reference for an entry of this kind, if there are any, like
pgstat_drop_entry(). Why not, that can be useful on its own depending
on the stats you are working on. May I suggest the addition of a code
path outside of your main proposal to test this API? For example
injection_stats.c with a new SQL function to reset everything.
+static void
+pgstat_gc_plan_memory()
+{
+ dshash_seq_status hstat;
+ PgStatShared_HashEntry *p;
+
+ /* dshash entry is not modified, take shared lock */
+ dshash_seq_init(&hstat, pgStatLocal.shared_hash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ PgStatShared_Common *header;
+ PgStat_StatPlanEntry *statent;
Question time: pgstat_drop_entries_of_kind() is called once in 0004,
which does a second sequential scan of pgStatLocal.shared_hash.
That's not efficient, making me question what's the reason to think
why pgstat_drop_entries_of_kind() is the best approach to use. I like
the idea of pgstat_drop_entries_of_kind(), less how it's applied in
the context of the main patch.
Mixed feelings about the choices of JumblePlanNode() in 0003 based on
its complexity as implemented. When it comes to such things, we
should keep the custom node functions short, applying node_attr
instead to the elements of the nodes so as the assumptions behind the
jumbling are documented within the structure definitions in the
headers, not the jumbling code itself.
--
Michael
Thanks for starting this thread. This is an important feature.
I am still reviewing, but wanted to share some initial comments.
== pg_stat_plans extension (0004)
1. pg_stat_plans_1_0 should not call pgstat_fetch_entry.l
This is not needed since we already have the entry with a shared lock
and it could lead to an assertion error when pgstat_fetch_entry
may conditionally call dshash_find. dshash_find asserts that the lock
is not already held. Calling pgstat_get_entry_data should be
enough here.
2. a "toplevel" bool field is missing in pg_stat_plans to indicate the
plan is for a nested query.
3. I think we should add cumulative planning_time. This
Probably should be controlled with a separate GUC as well.
4. For deallocation, I wonder if it makes more sense to zap the
plans with the lowest total execution time rather than calls; or make
this configurable. In fact, I think choosing the eviction strategy
should be done in pg_stat_statements as well ( but that belongs
in a separate discussion ). The idea is to give more priority to
plans that have the most overall database time.
5. What are your thoughts about controlling the memory by
size rather than .max and .max_size ? if a typical plan
is 2KB, a user can fit 10k plans with 20MB. A typical
user can probably allocate much more memory for this
purpose.
Also, pgstat_gc_plans is doing a loop over the
hash to get the # of entries. I don't think this
is a good idea for performance and it may not be possible to
actually enforce the .max on a dshash since
the lock is taken on a partition level.
6. I do like the idea of showing an in-flight plan.
This is so useful for a rare plan, especially on the
first capture of the plan ( calls = 0), and the planId
can be joined with pg_stat_activity to get the query
text.
/* Record initial entry now, so plan text is available for currently
running queries */
pgstat_report_plan_stats(queryDesc,
0, /* executions are counted in
pgsp_ExecutorEnd */
0.0);
We will need to be clear in the documentation
that calls being 0 is a valid scenario.
== core plan id computation (0003)
1. compute_plan_id should do exactly what compute_query_id
does. It should have an "auto" as the default which automatically
computes a plan id when pg_stat_plans is enabled.
2.
Mixed feelings about the choices of JumblePlanNode() in 0003 based on
its complexity as implemented. When it comes to such things, we
should keep the custom node functions short, applying node_attr
instead to the elements of the nodes so as the assumptions behind the
jumbling are documented within the structure definitions in the
headers, not the jumbling code itself.
+1
we should be able to control which node is considered for plan_id
computation using a node attribute such as plan_jumble_ignore.
I played around with this idea by building on top of your proposal
and attached my experiment code for this. The tricky part will be finalizing
which nodes and node fields to use for plan computation.
3. We may want to combine all the jumbling code into
a single jumble.c since the query and plan jumble will
share a lot of the same code, i.e. JumbleState.
_JumbleNode, etc.
Regards,
Sami Imseih
Amazon Web Services (AWS)
Attachments:
Experiment-with-plan-identifier-computation.patchapplication/octet-stream; name=Experiment-with-plan-identifier-computation.patchDownload
From be688ab9e3625c1b3ea855d20fbe0b4f68d9b43d Mon Sep 17 00:00:00 2001
From: Sami Imseih <simseih@amazon.com>
Date: Thu, 23 Jan 2025 13:06:42 -0600
Subject: [PATCH 1/1] Experiment with plan identifier computation
---
.../pg_stat_statements/pg_stat_statements.c | 2 +-
src/backend/access/brin/brin.c | 5 +
src/backend/access/nbtree/nbtsort.c | 7 +
src/backend/catalog/system_views.sql | 1 +
src/backend/commands/createas.c | 2 +-
src/backend/commands/explain.c | 16 ++
src/backend/commands/extension.c | 2 +-
src/backend/commands/portalcmds.c | 2 +-
src/backend/commands/vacuumparallel.c | 2 +
src/backend/executor/execMain.c | 2 +-
src/backend/executor/execParallel.c | 1 +
src/backend/nodes/Makefile | 4 +-
src/backend/nodes/gen_node_support.pl | 95 ++++++++
src/backend/nodes/planjumblefuncs.c | 220 ++++++++++++++++++
src/backend/nodes/queryjumblefuncs.c | 2 +-
src/backend/optimizer/plan/planner.c | 5 +
src/backend/parser/analyze.c | 2 +-
src/backend/postmaster/launch_backend.c | 5 +-
src/backend/tcop/postgres.c | 2 +
src/backend/utils/activity/backend_status.c | 63 +++++
src/backend/utils/adt/pgstatfuncs.c | 8 +-
src/backend/utils/error/csvlog.c | 3 +
src/backend/utils/error/elog.c | 8 +
src/backend/utils/error/jsonlog.c | 3 +
src/backend/utils/misc/guc_tables.c | 30 ++-
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/catalog/pg_proc.dat | 6 +-
src/include/nodes/bitmapset.h | 2 +-
src/include/nodes/{queryjumble.h => jumble.h} | 23 +-
src/include/nodes/meson.build | 1 +
src/include/nodes/parsenodes.h | 4 +-
src/include/nodes/plannodes.h | 54 ++---
src/include/nodes/primnodes.h | 8 +-
src/include/parser/analyze.h | 2 +-
src/include/utils/backend_status.h | 5 +
src/test/regress/expected/rules.out | 9 +-
36 files changed, 553 insertions(+), 54 deletions(-)
create mode 100644 src/backend/nodes/planjumblefuncs.c
rename src/include/nodes/{queryjumble.h => jumble.h} (80%)
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index bebf8134eb..26ef7f3e03 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -55,7 +55,7 @@
#include "jit/jit.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
-#include "nodes/queryjumble.h"
+#include "nodes/jumble.h"
#include "optimizer/planner.h"
#include "parser/analyze.h"
#include "parser/scanner.h"
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index 4289142e20..33137862a1 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -70,6 +70,9 @@ typedef struct BrinShared
/* Query ID, for report in worker processes */
uint64 queryid;
+ /* Plan ID, for report in worker processes */
+ uint64 planid;
+
/*
* workersdonecv is used to monitor the progress of workers. All parallel
* participants must indicate that they are done before leader can use
@@ -2452,6 +2455,7 @@ _brin_begin_parallel(BrinBuildState *buildstate, Relation heap, Relation index,
brinshared->scantuplesortstates = scantuplesortstates;
brinshared->pagesPerRange = buildstate->bs_pagesPerRange;
brinshared->queryid = pgstat_get_my_query_id();
+ brinshared->planid = pgstat_get_my_plan_id();
ConditionVariableInit(&brinshared->workersdonecv);
SpinLockInit(&brinshared->mutex);
@@ -2897,6 +2901,7 @@ _brin_parallel_build_main(dsm_segment *seg, shm_toc *toc)
/* Track query ID */
pgstat_report_query_id(brinshared->queryid, false);
+ pgstat_report_plan_id(brinshared->planid, false);
/* Open relations within worker */
heapRel = table_open(brinshared->heaprelid, heapLockmode);
diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c
index 7aba852db9..7d90b2a94f 100644
--- a/src/backend/access/nbtree/nbtsort.c
+++ b/src/backend/access/nbtree/nbtsort.c
@@ -107,6 +107,9 @@ typedef struct BTShared
/* Query ID, for report in worker processes */
uint64 queryid;
+ /* Plan ID, for report in worker processes */
+ uint64 planid;
+
/*
* workersdonecv is used to monitor the progress of workers. All parallel
* participants must indicate that they are done before leader can use
@@ -1508,6 +1511,7 @@ _bt_begin_parallel(BTBuildState *buildstate, bool isconcurrent, int request)
btshared->isconcurrent = isconcurrent;
btshared->scantuplesortstates = scantuplesortstates;
btshared->queryid = pgstat_get_my_query_id();
+ btshared->planid = pgstat_get_my_plan_id();
ConditionVariableInit(&btshared->workersdonecv);
SpinLockInit(&btshared->mutex);
/* Initialize mutable state */
@@ -1793,6 +1797,9 @@ _bt_parallel_build_main(dsm_segment *seg, shm_toc *toc)
/* Track query ID */
pgstat_report_query_id(btshared->queryid, false);
+ /* Track plan ID */
+ pgstat_report_plan_id(btshared->planid, false);
+
/* Open relations within worker */
heapRel = table_open(btshared->heaprelid, heapLockmode);
indexRel = index_open(btshared->indexrelid, indexLockmode);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 46868bf7e8..a49efc6332 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -885,6 +885,7 @@ CREATE VIEW pg_stat_activity AS
S.backend_xid,
s.backend_xmin,
S.query_id,
+ S.plan_id,
S.query,
S.backend_type
FROM pg_stat_get_activity(NULL) AS S
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 23cecd99c9..a8498e370c 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -37,7 +37,7 @@
#include "commands/view.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
-#include "nodes/queryjumble.h"
+#include "nodes/jumble.h"
#include "parser/analyze.h"
#include "rewrite/rewriteHandler.h"
#include "tcop/tcopprot.h"
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index c24e66f82e..d722b7048c 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -966,6 +966,22 @@ ExplainPrintPlan(ExplainState *es, QueryDesc *queryDesc)
ExplainPropertyInteger("Query Identifier", NULL, (int64)
queryDesc->plannedstmt->queryId, es);
}
+
+ /*
+ * COMPUTE_PLAN_ID_REGRESS means COMPUTE_PLAN_ID_AUTO, but we don't show
+ * the planid in any of the EXPLAIN plans to keep stable the results
+ * generated by regression test suites.
+ */
+ if (es->verbose && queryDesc->plannedstmt->planId != UINT64CONST(0) &&
+ compute_plan_id != COMPUTE_PLAN_ID_REGRESS)
+ {
+ /*
+ * Output the queryid as an int64 rather than a uint64 so we match
+ * what would be seen in the BIGINT pg_stat_plans.planid column.
+ */
+ ExplainPropertyInteger("Plan Identifier", NULL, (int64)
+ queryDesc->plannedstmt->planId, es);
+ }
}
/*
diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c
index ba540e3de5..3a462d708b 100644
--- a/src/backend/commands/extension.c
+++ b/src/backend/commands/extension.c
@@ -54,7 +54,7 @@
#include "funcapi.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
-#include "nodes/queryjumble.h"
+#include "nodes/jumble.h"
#include "storage/fd.h"
#include "tcop/utility.h"
#include "utils/acl.h"
diff --git a/src/backend/commands/portalcmds.c b/src/backend/commands/portalcmds.c
index e7c8171c10..926ec2af36 100644
--- a/src/backend/commands/portalcmds.c
+++ b/src/backend/commands/portalcmds.c
@@ -28,7 +28,7 @@
#include "executor/executor.h"
#include "executor/tstoreReceiver.h"
#include "miscadmin.h"
-#include "nodes/queryjumble.h"
+#include "nodes/jumble.h"
#include "parser/analyze.h"
#include "rewrite/rewriteHandler.h"
#include "tcop/pquery.h"
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 0d92e694d6..0de51d1a4a 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -64,6 +64,7 @@ typedef struct PVShared
Oid relid;
int elevel;
uint64 queryid;
+ uint64 planid;
/*
* Fields for both index vacuum and cleanup.
@@ -371,6 +372,7 @@ parallel_vacuum_init(Relation rel, Relation *indrels, int nindexes,
shared->relid = RelationGetRelid(rel);
shared->elevel = elevel;
shared->queryid = pgstat_get_my_query_id();
+ shared->planid = pgstat_get_my_plan_id();
shared->maintenance_work_mem_worker =
(nindexes_mwm > 0) ?
maintenance_work_mem / Min(parallel_workers, nindexes_mwm) :
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index fb8dba3ab2..9ddfa5d867 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -50,7 +50,7 @@
#include "foreign/fdwapi.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
-#include "nodes/queryjumble.h"
+#include "nodes/jumble.h"
#include "parser/parse_relation.h"
#include "pgstat.h"
#include "rewrite/rewriteHandler.h"
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index ff4d9dd1bb..20ebc23b00 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -174,6 +174,7 @@ ExecSerializePlan(Plan *plan, EState *estate)
pstmt = makeNode(PlannedStmt);
pstmt->commandType = CMD_SELECT;
pstmt->queryId = pgstat_get_my_query_id();
+ pstmt->planId = pgstat_get_my_plan_id();
pstmt->hasReturning = false;
pstmt->hasModifyingCTE = false;
pstmt->canSetTag = true;
diff --git a/src/backend/nodes/Makefile b/src/backend/nodes/Makefile
index 66bbad8e6e..55902bee29 100644
--- a/src/backend/nodes/Makefile
+++ b/src/backend/nodes/Makefile
@@ -27,6 +27,7 @@ OBJS = \
params.o \
print.o \
queryjumblefuncs.o \
+ planjumblefuncs.o \
read.o \
readfuncs.o \
tidbitmap.o \
@@ -91,7 +92,8 @@ copyfuncs.o: copyfuncs.c copyfuncs.funcs.c copyfuncs.switch.c | node-support-sta
equalfuncs.o: equalfuncs.c equalfuncs.funcs.c equalfuncs.switch.c | node-support-stamp
outfuncs.o: outfuncs.c outfuncs.funcs.c outfuncs.switch.c | node-support-stamp
queryjumblefuncs.o: queryjumblefuncs.c queryjumblefuncs.funcs.c queryjumblefuncs.switch.c | node-support-stamp
+planjumblefuncs.o: planjumblefuncs.c planjumblefuncs.funcs.c planjumblefuncs.switch.c | node-support-stamp
readfuncs.o: readfuncs.c readfuncs.funcs.c readfuncs.switch.c | node-support-stamp
clean:
- rm -f node-support-stamp $(addsuffix funcs.funcs.c,copy equal out queryjumble read) $(addsuffix funcs.switch.c,copy equal out queryjumble read) nodetags.h
+ rm -f node-support-stamp $(addsuffix funcs.funcs.c,copy equal out queryjumble planjumble read) $(addsuffix funcs.switch.c,copy equal out queryjumble read) nodetags.h
diff --git a/src/backend/nodes/gen_node_support.pl b/src/backend/nodes/gen_node_support.pl
index 7c012c27f8..77acb24e8b 100644
--- a/src/backend/nodes/gen_node_support.pl
+++ b/src/backend/nodes/gen_node_support.pl
@@ -131,6 +131,8 @@ my @no_read_write;
my @special_read_write;
# node types we don't want any support functions for, just node tags
my @nodetag_only;
+# node types we don't want plan jumble support for
+my @no_plan_jumble;
# types that are copied by straight assignment
my @scalar_types = qw(
@@ -164,6 +166,7 @@ push @node_types, qw(List);
push @no_copy, qw(List);
push @no_equal, qw(List);
push @no_query_jumble, qw(List);
+push @no_plan_jumble, qw(List);
push @special_read_write, qw(List);
# Nodes with custom copy/equal implementations are skipped from
@@ -176,6 +179,9 @@ my @custom_read_write;
# Similarly for custom query jumble implementation.
my @custom_query_jumble;
+# Similarly for custom plan jumble implementation.
+my @custom_plan_jumble;
+
# Track node types with manually assigned NodeTag numbers.
my %manual_nodetag_number;
@@ -346,6 +352,10 @@ foreach my $infile (@ARGV)
{
push @no_query_jumble, $in_struct;
}
+ elsif ($attr eq 'no_plan_jumble')
+ {
+ push @no_plan_jumble, $in_struct;
+ }
elsif ($attr eq 'no_read')
{
push @no_read, $in_struct;
@@ -424,6 +434,8 @@ foreach my $infile (@ARGV)
if elem $supertype, @no_read;
push @no_query_jumble, $in_struct
if elem $supertype, @no_query_jumble;
+ push @no_plan_jumble, $in_struct
+ if elem $supertype, @no_plan_jumble;
}
}
@@ -475,6 +487,7 @@ foreach my $infile (@ARGV)
equal_ignore_if_zero
query_jumble_ignore
query_jumble_location
+ plan_jumble_ignore
read_write_ignore
write_only_relids
write_only_nondefault_pathtarget
@@ -1339,6 +1352,88 @@ _jumble${n}(JumbleState *jstate, Node *node)
close $jff;
close $jfs;
+# planjumblefuncs.c
+
+push @output_files, 'planjumblefuncs.funcs.c';
+open my $pjff, '>', "$output_path/planjumblefuncs.funcs.c$tmpext" or die $!;
+push @output_files, 'planjumblefuncs.switch.c';
+open my $pjfs, '>', "$output_path/planjumblefuncs.switch.c$tmpext" or die $!;
+
+printf $pjff $header_comment, 'planjumblefuncs.funcs.c';
+printf $pjfs $header_comment, 'planjumblefuncs.switch.c';
+
+print $pjff $node_includes;
+
+foreach my $n (@node_types)
+{
+ next if elem $n, @abstract_types;
+ next if elem $n, @nodetag_only;
+ my $struct_no_plan_jumble = (elem $n, @no_plan_jumble);
+
+ print $pjfs "\t\t\tcase T_${n}:\n"
+ . "\t\t\t\t_jumble${n}(jstate, expr);\n"
+ . "\t\t\t\tbreak;\n"
+ unless $struct_no_plan_jumble;
+
+ next if elem $n, @custom_plan_jumble;
+
+ print $pjff "
+static void
+_jumble${n}(JumbleState *jstate, Node *node)
+{
+\t${n} *expr = (${n} *) node;\n
+" unless $struct_no_plan_jumble;
+
+ # print instructions for each field
+ foreach my $f (@{ $node_type_info{$n}->{fields} })
+ {
+ my $t = $node_type_info{$n}->{field_types}{$f};
+ my @a = @{ $node_type_info{$n}->{field_attrs}{$f} };
+ my $plan_jumble_ignore = $struct_no_plan_jumble;
+
+ # extract per-field attributes
+ foreach my $a (@a)
+ {
+ if ($a eq 'plan_jumble_ignore')
+ {
+ $plan_jumble_ignore = 1;
+ }
+ }
+
+ # node type
+ if (($t =~ /^(\w+)\*$/ or $t =~ /^struct\s+(\w+)\*$/)
+ and elem $1, @node_types)
+ {
+ print $pjff "\tJUMBLE_NODE($f);\n"
+ unless $plan_jumble_ignore;
+ }
+ elsif ($t eq 'char*')
+ {
+ print $pjff "\tJUMBLE_STRING($f);\n"
+ unless $plan_jumble_ignore;
+ }
+ else
+ {
+ print $pjff "\tJUMBLE_FIELD($f);\n"
+ unless $plan_jumble_ignore;
+ }
+ }
+
+ # Some nodes have no attributes like CheckPointStmt,
+ # so tweak things for empty contents.
+ if (scalar(@{ $node_type_info{$n}->{fields} }) == 0)
+ {
+ print $pjff "\t(void) expr;\n"
+ unless $struct_no_plan_jumble;
+ }
+
+ print $pjff "}
+" unless $struct_no_plan_jumble;
+}
+
+close $pjff;
+close $pjfs;
+
# now rename the temporary files to their final names
foreach my $file (@output_files)
{
diff --git a/src/backend/nodes/planjumblefuncs.c b/src/backend/nodes/planjumblefuncs.c
new file mode 100644
index 0000000000..6080382ce7
--- /dev/null
+++ b/src/backend/nodes/planjumblefuncs.c
@@ -0,0 +1,220 @@
+/*-------------------------------------------------------------------------
+ *
+ * planjumblefuncs.c
+ * Plan fingerprinting.
+ *
+ * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/nodes/planjumblefuncs.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "common/hashfn.h"
+#include "miscadmin.h"
+#include "nodes/jumble.h"
+#include "parser/scansup.h"
+#include "utils/backend_status.h"
+
+#define JUMBLE_SIZE 1024 /* query serialization buffer size */
+
+/* GUC parameters */
+int compute_plan_id = COMPUTE_PLAN_ID_AUTO;
+
+/*
+ * True when compute_plan_id is ON or AUTO, and a module requests them.
+ *
+ * Note that IsPlanIdEnabled() should be used instead of checking
+ * plan_id_enabled or compute_plan_id directly when we want to know
+ * whether plan identifiers are computed in the core or not.
+ */
+bool plan_id_enabled = false;
+
+static void AppendJumble(JumbleState *jstate,
+ const unsigned char *item, Size size);
+static void _jumbleNode(JumbleState *jstate, Node *node);
+static void _jumbleList(JumbleState *jstate, Node *node);
+
+JumbleState *
+JumblePlan(PlannedStmt *ps)
+{
+ JumbleState *jstate = NULL;
+
+ Assert(IsPlanIdEnabled());
+
+ jstate = (JumbleState *) palloc(sizeof(JumbleState));
+
+ /* Set up workspace for query jumbling */
+ jstate->jumble = (unsigned char *) palloc(JUMBLE_SIZE);
+ jstate->jumble_len = 0;
+ jstate->clocations_buf_size = 32;
+ jstate->clocations = (LocationLen *)
+ palloc(jstate->clocations_buf_size * sizeof(LocationLen));
+ jstate->clocations_count = 0;
+ jstate->highest_extern_param_id = 0;
+
+ /* Compute query ID and mark the Query node with it */
+ _jumbleNode(jstate, (Node *) ps->planTree);
+ ps->planId = DatumGetUInt64(hash_any_extended(jstate->jumble,
+ jstate->jumble_len,
+ 0));
+ pgstat_report_plan_id(ps->planId, false);
+
+ return jstate;
+}
+
+/*
+ * Enables plan identifier computation.
+ *
+ * Third-party plugins can use this function to inform core that they require
+ * a plan identifier to be computed.
+ */
+void
+EnablePlanId(void)
+{
+ if (compute_plan_id != COMPUTE_PLAN_ID_OFF)
+ plan_id_enabled = true;
+}
+
+/*
+ * AppendJumble: Append a value that is substantive in a given query to
+ * the current jumble.
+ */
+static void
+AppendJumble(JumbleState *jstate, const unsigned char *item, Size size)
+{
+ unsigned char *jumble = jstate->jumble;
+ Size jumble_len = jstate->jumble_len;
+
+ /*
+ * Whenever the jumble buffer is full, we hash the current contents and
+ * reset the buffer to contain just that hash value, thus relying on the
+ * hash to summarize everything so far.
+ */
+ while (size > 0)
+ {
+ Size part_size;
+
+ if (jumble_len >= JUMBLE_SIZE)
+ {
+ uint64 start_hash;
+
+ start_hash = DatumGetUInt64(hash_any_extended(jumble,
+ JUMBLE_SIZE, 0));
+ memcpy(jumble, &start_hash, sizeof(start_hash));
+ jumble_len = sizeof(start_hash);
+ }
+ part_size = Min(size, JUMBLE_SIZE - jumble_len);
+ memcpy(jumble + jumble_len, item, part_size);
+ jumble_len += part_size;
+ item += part_size;
+ size -= part_size;
+ }
+ jstate->jumble_len = jumble_len;
+}
+
+#define JUMBLE_NODE(item) \
+ _jumbleNode(jstate, (Node *) expr->item)
+#define JUMBLE_LOCATION(location) \
+ RecordConstLocation(jstate, expr->location)
+#define JUMBLE_FIELD(item) \
+ AppendJumble(jstate, (const unsigned char *) &(expr->item), sizeof(expr->item))
+#define JUMBLE_FIELD_SINGLE(item) \
+ AppendJumble(jstate, (const unsigned char *) &(item), sizeof(item))
+#define JUMBLE_STRING(str) \
+do { \
+ if (expr->str) \
+ AppendJumble(jstate, (const unsigned char *) (expr->str), strlen(expr->str) + 1); \
+} while(0)
+
+#include "planjumblefuncs.funcs.c"
+
+static void
+_jumbleNode(JumbleState *jstate, Node *node)
+{
+ Node *expr = node;
+
+ if (expr == NULL)
+ return;
+
+ /* Guard against stack overflow due to overly complex expressions */
+ check_stack_depth();
+
+ /*
+ * We always emit the node's NodeTag, then any additional fields that are
+ * considered significant, and then we recurse to any child nodes.
+ */
+ JUMBLE_FIELD(type);
+
+ switch (nodeTag(expr))
+ {
+#include "planjumblefuncs.switch.c"
+
+ case T_List:
+ case T_IntList:
+ case T_OidList:
+ case T_XidList:
+ _jumbleList(jstate, expr);
+ break;
+ default:
+ /* Only a warning, since we can stumble along anyway */
+ elog(WARNING, "unrecognized node type: %d",
+ (int) nodeTag(expr));
+ break;
+ }
+
+ /* Special cases to handle outside the automated code */
+ switch (nodeTag(expr))
+ {
+ case T_Param:
+ {
+ Param *p = (Param *) node;
+
+ /*
+ * Update the highest Param id seen, in order to start
+ * normalization correctly.
+ */
+ if (p->paramkind == PARAM_EXTERN &&
+ p->paramid > jstate->highest_extern_param_id)
+ jstate->highest_extern_param_id = p->paramid;
+ }
+ break;
+ default:
+ break;
+ }
+}
+
+static void
+_jumbleList(JumbleState *jstate, Node *node)
+{
+ List *expr = (List *) node;
+ ListCell *l;
+
+ switch (expr->type)
+ {
+ case T_List:
+ foreach(l, expr)
+ _jumbleNode(jstate, lfirst(l));
+ break;
+ case T_IntList:
+ foreach(l, expr)
+ JUMBLE_FIELD_SINGLE(lfirst_int(l));
+ break;
+ case T_OidList:
+ foreach(l, expr)
+ JUMBLE_FIELD_SINGLE(lfirst_oid(l));
+ break;
+ case T_XidList:
+ foreach(l, expr)
+ JUMBLE_FIELD_SINGLE(lfirst_xid(l));
+ break;
+ default:
+ elog(ERROR, "unrecognized list node type: %d",
+ (int) expr->type);
+ return;
+ }
+}
diff --git a/src/backend/nodes/queryjumblefuncs.c b/src/backend/nodes/queryjumblefuncs.c
index b103a28193..9a3fc0478d 100644
--- a/src/backend/nodes/queryjumblefuncs.c
+++ b/src/backend/nodes/queryjumblefuncs.c
@@ -34,7 +34,7 @@
#include "common/hashfn.h"
#include "miscadmin.h"
-#include "nodes/queryjumble.h"
+#include "nodes/jumble.h"
#include "parser/scansup.h"
#define JUMBLE_SIZE 1024 /* query serialization buffer size */
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 6803edd085..e0df15fe0d 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -37,6 +37,7 @@
#ifdef OPTIMIZER_DEBUG
#include "nodes/print.h"
#endif
+#include "nodes/jumble.h"
#include "nodes/supportnodes.h"
#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
@@ -569,6 +570,7 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions,
result->utilityStmt = parse->utilityStmt;
result->stmt_location = parse->stmt_location;
result->stmt_len = parse->stmt_len;
+ result->planId = UINT64CONST(0);
result->jitFlags = PGJIT_NONE;
if (jit_enabled && jit_above_cost >= 0 &&
@@ -598,6 +600,9 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions,
if (glob->partition_directory != NULL)
DestroyPartitionDirectory(glob->partition_directory);
+ if (IsPlanIdEnabled())
+ JumblePlan(result);
+
return result;
}
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 76f58b3aca..cd235921b4 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -31,7 +31,7 @@
#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
-#include "nodes/queryjumble.h"
+#include "nodes/jumble.h"
#include "optimizer/optimizer.h"
#include "parser/analyze.h"
#include "parser/parse_agg.h"
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index a97a1eda6d..3d34784c04 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -53,7 +53,7 @@
#include "utils/memutils.h"
#ifdef EXEC_BACKEND
-#include "nodes/queryjumble.h"
+#include "nodes/jumble.h"
#include "storage/pg_shmem.h"
#include "storage/spin.h"
#endif
@@ -115,6 +115,7 @@ typedef struct
bool redirection_done;
bool IsBinaryUpgrade;
bool query_id_enabled;
+ bool plan_id_enabled;
int max_safe_fds;
int MaxBackends;
int num_pmchild_slots;
@@ -744,6 +745,7 @@ save_backend_variables(BackendParameters *param,
param->redirection_done = redirection_done;
param->IsBinaryUpgrade = IsBinaryUpgrade;
param->query_id_enabled = query_id_enabled;
+ param->plan_id_enabled = plan_id_enabled;
param->max_safe_fds = max_safe_fds;
param->MaxBackends = MaxBackends;
@@ -1004,6 +1006,7 @@ restore_backend_variables(BackendParameters *param)
redirection_done = param->redirection_done;
IsBinaryUpgrade = param->IsBinaryUpgrade;
query_id_enabled = param->query_id_enabled;
+ plan_id_enabled = param->plan_id_enabled;
max_safe_fds = param->max_safe_fds;
MaxBackends = param->MaxBackends;
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 5655348a2e..587f164b55 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -1106,6 +1106,7 @@ exec_simple_query(const char *query_string)
size_t cmdtaglen;
pgstat_report_query_id(0, true);
+ pgstat_report_plan_id(0, true);
/*
* Get the command name for use in status display (it also becomes the
@@ -2163,6 +2164,7 @@ exec_execute_message(const char *portal_name, long max_rows)
if (stmt->queryId != UINT64CONST(0))
{
pgstat_report_query_id(stmt->queryId, false);
+ pgstat_report_plan_id(stmt->planId, false);
break;
}
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 731342799a..7ccb2c6c6c 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -379,6 +379,7 @@ pgstat_bestart(void)
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = UINT64CONST(0);
+ lbeentry.st_plan_id = UINT64CONST(0);
/*
* we don't zero st_progress_param here to save cycles; nobody should
@@ -533,6 +534,7 @@ pgstat_report_activity(BackendState state, const char *cmd_str)
/* st_xact_start_timestamp and wait_event_info are also disabled */
beentry->st_xact_start_timestamp = 0;
beentry->st_query_id = UINT64CONST(0);
+ beentry->st_plan_id = UINT64CONST(0);
proc->wait_event_info = 0;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -593,7 +595,10 @@ pgstat_report_activity(BackendState state, const char *cmd_str)
* identifier.
*/
if (state == STATE_RUNNING)
+ {
beentry->st_query_id = UINT64CONST(0);
+ beentry->st_plan_id = UINT64CONST(0);
+ }
if (cmd_str != NULL)
{
@@ -644,6 +649,44 @@ pgstat_report_query_id(uint64 query_id, bool force)
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
+/* --------
+ * pgstat_report_plan_id() -
+ *
+ * Called to update top-level plan identifier.
+ * --------
+ */
+void
+pgstat_report_plan_id(uint64 plan_id, bool force)
+{
+ volatile PgBackendStatus *beentry = MyBEEntry;
+
+ /*
+ * if track_activities is disabled, st_plan_id should already have been
+ * reset
+ */
+ if (!beentry || !pgstat_track_activities)
+ return;
+
+ /*
+ * We only report the top-level plan identifiers. The stored plan_id is
+ * reset when a backend calls pgstat_report_activity(STATE_RUNNING), or
+ * with an explicit call to this function using the force flag. If the
+ * saved plan identifier is not zero it means that it's not a top-level
+ * command, so ignore the one provided unless it's an explicit call to
+ * reset the identifier.
+ */
+ if (beentry->st_plan_id != 0 && !force)
+ return;
+
+ /*
+ * Update my status entry, following the protocol of bumping
+ * st_changecount before and after. We use a volatile pointer here to
+ * ensure the compiler doesn't try to get cute.
+ */
+ PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
+ beentry->st_plan_id = plan_id;
+ PGSTAT_END_WRITE_ACTIVITY(beentry);
+}
/* ----------
* pgstat_report_appname() -
@@ -1040,6 +1083,26 @@ pgstat_get_my_query_id(void)
return MyBEEntry->st_query_id;
}
+/* ----------
+ * pgstat_get_my_plan_id() -
+ *
+ * Return current backend's query identifier.
+ */
+uint64
+pgstat_get_my_plan_id(void)
+{
+ if (!MyBEEntry)
+ return 0;
+
+ /*
+ * There's no need for a lock around pgstat_begin_read_activity /
+ * pgstat_end_read_activity here as it's only called from
+ * pg_stat_get_activity which is already protected, or from the same
+ * backend which means that there won't be concurrent writes.
+ */
+ return MyBEEntry->st_plan_id;
+}
+
/* ----------
* pgstat_get_backend_type_by_proc_number() -
*
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 0f5e0a9778..c033dd0778 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -302,7 +302,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum
pg_stat_get_activity(PG_FUNCTION_ARGS)
{
-#define PG_STAT_GET_ACTIVITY_COLS 31
+#define PG_STAT_GET_ACTIVITY_COLS 32
int num_backends = pgstat_fetch_stat_numbackends();
int curr_backend;
int pid = PG_ARGISNULL(0) ? -1 : PG_GETARG_INT32(0);
@@ -613,6 +613,11 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
nulls[30] = true;
else
values[30] = UInt64GetDatum(beentry->st_query_id);
+
+ if (beentry->st_plan_id == 0)
+ nulls[31] = true;
+ else
+ values[31] = UInt64GetDatum(beentry->st_plan_id);
}
else
{
@@ -642,6 +647,7 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
nulls[28] = true;
nulls[29] = true;
nulls[30] = true;
+ nulls[31] = true;
}
tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
diff --git a/src/backend/utils/error/csvlog.c b/src/backend/utils/error/csvlog.c
index 8e9fbcb999..86650b02ff 100644
--- a/src/backend/utils/error/csvlog.c
+++ b/src/backend/utils/error/csvlog.c
@@ -250,6 +250,9 @@ write_csvlog(ErrorData *edata)
/* query id */
appendStringInfo(&buf, "%lld", (long long) pgstat_get_my_query_id());
+ /* plan id */
+ appendStringInfo(&buf, "%lld", (long long) pgstat_get_my_plan_id());
+
appendStringInfoChar(&buf, '\n');
/* If in the syslogger process, try to write messages direct to file */
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 860bbd40d4..c8861b5090 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -3154,6 +3154,14 @@ log_status_format(StringInfo buf, const char *format, ErrorData *edata)
appendStringInfo(buf, "%lld",
(long long) pgstat_get_my_query_id());
break;
+ case 'X':
+ if (padding != 0)
+ appendStringInfo(buf, "%*lld", padding,
+ (long long) pgstat_get_my_plan_id());
+ else
+ appendStringInfo(buf, "%lld",
+ (long long) pgstat_get_my_plan_id());
+ break;
default:
/* format error - ignore it */
break;
diff --git a/src/backend/utils/error/jsonlog.c b/src/backend/utils/error/jsonlog.c
index 6533f1d688..a88bd4339e 100644
--- a/src/backend/utils/error/jsonlog.c
+++ b/src/backend/utils/error/jsonlog.c
@@ -287,6 +287,9 @@ write_jsonlog(ErrorData *edata)
appendJSONKeyValueFmt(&buf, "query_id", false, "%lld",
(long long) pgstat_get_my_query_id());
+ appendJSONKeyValueFmt(&buf, "plan_id", false, "%lld",
+ (long long) pgstat_get_my_plan_id());
+
/* Finish string */
appendStringInfoChar(&buf, '}');
appendStringInfoChar(&buf, '\n');
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 38cb9e970d..bc02e7c103 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -50,7 +50,7 @@
#include "libpq/auth.h"
#include "libpq/libpq.h"
#include "libpq/scram.h"
-#include "nodes/queryjumble.h"
+#include "nodes/jumble.h"
#include "optimizer/cost.h"
#include "optimizer/geqo.h"
#include "optimizer/optimizer.h"
@@ -309,6 +309,24 @@ static const struct config_enum_entry compute_query_id_options[] = {
{NULL, 0, false}
};
+/*
+ * Although only "on", "off", and "auto" are documented, we accept
+ * all the likely variants of "on" and "off".
+ */
+static const struct config_enum_entry compute_plan_id_options[] = {
+ {"auto", COMPUTE_PLAN_ID_AUTO, false},
+ {"regress", COMPUTE_PLAN_ID_REGRESS, false},
+ {"on", COMPUTE_PLAN_ID_ON, false},
+ {"off", COMPUTE_PLAN_ID_OFF, false},
+ {"true", COMPUTE_PLAN_ID_ON, true},
+ {"false", COMPUTE_PLAN_ID_OFF, true},
+ {"yes", COMPUTE_PLAN_ID_ON, true},
+ {"no", COMPUTE_PLAN_ID_OFF, true},
+ {"1", COMPUTE_PLAN_ID_ON, true},
+ {"0", COMPUTE_PLAN_ID_OFF, true},
+ {NULL, 0, false}
+};
+
/*
* Although only "on", "off", and "partition" are documented, we
* accept all the likely variants of "on" and "off".
@@ -4873,6 +4891,16 @@ struct config_enum ConfigureNamesEnum[] =
NULL, NULL, NULL
},
+ {
+ {"compute_plan_id", PGC_SUSET, STATS_MONITORING,
+ gettext_noop("Enables in-core computation of the plan tree."),
+ NULL
+ },
+ &compute_plan_id,
+ COMPUTE_PLAN_ID_AUTO, compute_plan_id_options,
+ NULL, NULL, NULL
+ },
+
{
{"constraint_exclusion", PGC_USERSET, QUERY_TUNING_OTHER,
gettext_noop("Enables the planner to use constraints to optimize queries."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 079efa1baa..0634ae90dd 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -641,6 +641,7 @@
# - Monitoring -
#compute_query_id = auto
+#compute_plan_id = auto
#log_statement_stats = off
#log_parser_stats = off
#log_planner_stats = off
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 18560755d2..de341e2d02 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5549,9 +5549,9 @@
proname => 'pg_stat_get_activity', prorows => '100', proisstrict => 'f',
proretset => 't', provolatile => 's', proparallel => 'r',
prorettype => 'record', proargtypes => 'int4',
- proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,bool,int4,int8}',
- proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,gss_delegation,leader_pid,query_id}',
+ proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,bool,int4,int8,int8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,gss_delegation,leader_pid,query_id,plan_id}',
prosrc => 'pg_stat_get_activity' },
{ oid => '6318', descr => 'describe wait events',
proname => 'pg_get_wait_events', procost => '10', prorows => '250',
diff --git a/src/include/nodes/bitmapset.h b/src/include/nodes/bitmapset.h
index 03faca9308..a8694d646a 100644
--- a/src/include/nodes/bitmapset.h
+++ b/src/include/nodes/bitmapset.h
@@ -48,7 +48,7 @@ typedef int32 signedbitmapword; /* must be the matching signed type */
typedef struct Bitmapset
{
- pg_node_attr(custom_copy_equal, special_read_write, no_query_jumble)
+ pg_node_attr(custom_copy_equal, special_read_write, no_query_jumble, no_plan_jumble)
NodeTag type;
int nwords; /* number of words in array */
diff --git a/src/include/nodes/queryjumble.h b/src/include/nodes/jumble.h
similarity index 80%
rename from src/include/nodes/queryjumble.h
rename to src/include/nodes/jumble.h
index 50eb956658..3cd35f145f 100644
--- a/src/include/nodes/queryjumble.h
+++ b/src/include/nodes/jumble.h
@@ -1,13 +1,13 @@
/*-------------------------------------------------------------------------
*
- * queryjumble.h
+ * jumble.h
* Query normalization and fingerprinting.
*
* Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* IDENTIFICATION
- * src/include/nodes/queryjumble.h
+ * src/include/nodes/jumble.h
*
*-------------------------------------------------------------------------
*/
@@ -15,6 +15,7 @@
#define QUERYJUMBLE_H
#include "nodes/parsenodes.h"
+#include "nodes/plannodes.h"
/*
* Struct for tracking locations/lengths of constants during normalization
@@ -57,17 +58,25 @@ enum ComputeQueryIdType
COMPUTE_QUERY_ID_ON,
COMPUTE_QUERY_ID_AUTO,
COMPUTE_QUERY_ID_REGRESS,
+ COMPUTE_PLAN_ID_OFF,
+ COMPUTE_PLAN_ID_ON,
+ COMPUTE_PLAN_ID_AUTO,
+ COMPUTE_PLAN_ID_REGRESS,
};
/* GUC parameters */
extern PGDLLIMPORT int compute_query_id;
+extern PGDLLIMPORT int compute_plan_id;
extern const char *CleanQuerytext(const char *query, int *location, int *len);
extern JumbleState *JumbleQuery(Query *query);
+extern JumbleState *JumblePlan(PlannedStmt *ps);
extern void EnableQueryId(void);
+extern void EnablePlanId(void);
extern PGDLLIMPORT bool query_id_enabled;
+extern PGDLLIMPORT bool plan_id_enabled;
/*
* Returns whether query identifier computation has been enabled, either
@@ -83,4 +92,14 @@ IsQueryIdEnabled(void)
return query_id_enabled;
}
+static inline bool
+IsPlanIdEnabled(void)
+{
+ if (compute_plan_id == COMPUTE_PLAN_ID_OFF)
+ return false;
+ if (compute_plan_id == COMPUTE_PLAN_ID_ON)
+ return true;
+ return plan_id_enabled;
+}
+
#endif /* QUERYJUMBLE_H */
diff --git a/src/include/nodes/meson.build b/src/include/nodes/meson.build
index f3dd5461fe..2e73b39640 100644
--- a/src/include/nodes/meson.build
+++ b/src/include/nodes/meson.build
@@ -37,6 +37,7 @@ node_support_output = [
'copyfuncs.funcs.c', 'copyfuncs.switch.c',
'equalfuncs.funcs.c', 'equalfuncs.switch.c',
'queryjumblefuncs.funcs.c', 'queryjumblefuncs.switch.c',
+ 'planjumblefuncs.funcs.c', 'planjumblefuncs.switch.c',
]
node_support_install = [
dir_include_server / 'nodes',
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index ffe155ee20..8bf2b11f8b 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -367,7 +367,7 @@ union ValUnion
typedef struct A_Const
{
- pg_node_attr(custom_copy_equal, custom_read_write, custom_query_jumble)
+ pg_node_attr(custom_copy_equal, custom_read_write, custom_query_jumble, no_plan_jumble)
NodeTag type;
union ValUnion val;
@@ -2670,7 +2670,7 @@ typedef enum VariableSetKind
typedef struct VariableSetStmt
{
- pg_node_attr(custom_query_jumble)
+ pg_node_attr(custom_query_jumble, no_plan_jumble)
NodeTag type;
VariableSetKind kind;
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 9e19cdd284..d3eb40e5ea 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -53,6 +53,8 @@ typedef struct PlannedStmt
uint64 queryId; /* query identifier (copied from Query) */
+ uint64 planId; /* plan identifier (copied from PlannedStmt) */
+
bool hasReturning; /* is it insert|update|delete|merge RETURNING? */
bool hasModifyingCTE; /* has insert|update|delete|merge in WITH? */
@@ -125,33 +127,33 @@ typedef struct Plan
/*
* estimated execution costs for plan (see costsize.c for more info)
*/
- int disabled_nodes; /* count of disabled nodes */
- Cost startup_cost; /* cost expended before fetching any tuples */
- Cost total_cost; /* total cost (assuming all tuples fetched) */
+ int disabled_nodes pg_node_attr(plan_jumble_ignore); /* count of disabled nodes */
+ Cost startup_cost pg_node_attr(plan_jumble_ignore); /* cost expended before fetching any tuples */
+ Cost total_cost pg_node_attr(plan_jumble_ignore); /* total cost (assuming all tuples fetched) */
/*
* planner's estimate of result size of this plan step
*/
- Cardinality plan_rows; /* number of rows plan is expected to emit */
- int plan_width; /* average row width in bytes */
+ Cardinality plan_rows pg_node_attr(plan_jumble_ignore); /* number of rows plan is expected to emit */
+ int plan_width pg_node_attr(plan_jumble_ignore); /* average row width in bytes */
/*
* information needed for parallel query
*/
- bool parallel_aware; /* engage parallel-aware logic? */
- bool parallel_safe; /* OK to use as part of parallel plan? */
+ bool parallel_aware pg_node_attr(plan_jumble_ignore); /* engage parallel-aware logic? */
+ bool parallel_safe pg_node_attr(plan_jumble_ignore); /* OK to use as part of parallel plan? */
/*
* information needed for asynchronous execution
*/
- bool async_capable; /* engage asynchronous-capable logic? */
+ bool async_capable pg_node_attr(plan_jumble_ignore); /* engage asynchronous-capable logic? */
/*
* Common structural data for all Plan types.
*/
- int plan_node_id; /* unique across entire final plan tree */
+ int plan_node_id pg_node_attr(plan_jumble_ignore); /* unique across entire final plan tree */
List *targetlist; /* target list to be computed at this node */
- List *qual; /* implicitly-ANDed qual conditions */
+ List *qual pg_node_attr(plan_jumble_ignore); /* implicitly-ANDed qual conditions */
struct Plan *lefttree; /* input plan tree(s) */
struct Plan *righttree;
List *initPlan; /* Init Plan nodes (un-correlated expr
@@ -168,8 +170,8 @@ typedef struct Plan
* params that affect the node (i.e., the setParams of its initplans).
* These are _all_ the PARAM_EXEC params that affect this node.
*/
- Bitmapset *extParam;
- Bitmapset *allParam;
+ Bitmapset *extParam pg_node_attr(plan_jumble_ignore);
+ Bitmapset *allParam pg_node_attr(plan_jumble_ignore);
} Plan;
/* ----------------
@@ -268,7 +270,7 @@ struct PartitionPruneInfo; /* forward reference to struct below */
typedef struct Append
{
Plan plan;
- Bitmapset *apprelids; /* RTIs of appendrel(s) formed by this node */
+ Bitmapset *apprelids pg_node_attr(plan_jumble_ignore); /* RTIs of appendrel(s) formed by this node */
List *appendplans;
int nasyncplans; /* # of asynchronous plans */
@@ -292,7 +294,7 @@ typedef struct MergeAppend
Plan plan;
/* RTIs of appendrel(s) formed by this node */
- Bitmapset *apprelids;
+ Bitmapset *apprelids pg_node_attr(plan_jumble_ignore);
List *mergeplans;
@@ -389,7 +391,7 @@ typedef struct Scan
pg_node_attr(abstract)
Plan plan;
- Index scanrelid; /* relid is index into the range table */
+ Index scanrelid pg_node_attr(plan_jumble_ignore); /* relid is index into the range table */
} Scan;
/* ----------------
@@ -719,8 +721,8 @@ typedef struct ForeignScan
List *fdw_private; /* private data for FDW */
List *fdw_scan_tlist; /* optional tlist describing scan tuple */
List *fdw_recheck_quals; /* original quals not in scan.plan.qual */
- Bitmapset *fs_relids; /* base+OJ RTIs generated by this scan */
- Bitmapset *fs_base_relids; /* base RTIs generated by this scan */
+ Bitmapset *fs_relids pg_node_attr(plan_jumble_ignore); /* base+OJ RTIs generated by this scan */
+ Bitmapset *fs_base_relids pg_node_attr(plan_jumble_ignore); /* base RTIs generated by this scan */
bool fsSystemCol; /* true if any "system column" is needed */
} ForeignScan;
@@ -748,7 +750,7 @@ typedef struct CustomScan
List *custom_exprs; /* expressions that custom code may evaluate */
List *custom_private; /* private data for custom code */
List *custom_scan_tlist; /* optional tlist describing scan tuple */
- Bitmapset *custom_relids; /* RTIs generated by this scan */
+ Bitmapset *custom_relids pg_node_attr(plan_jumble_ignore); /* RTIs generated by this scan */
/*
* NOTE: The method field of CustomScan is required to be a pointer to a
@@ -924,7 +926,7 @@ typedef struct Memoize
uint32 est_entries;
/* paramids from param_exprs */
- Bitmapset *keyparamids;
+ Bitmapset *keyparamids pg_node_attr(plan_jumble_ignore);
} Memoize;
/* ----------------
@@ -1023,7 +1025,7 @@ typedef struct Agg
uint64 transitionSpace;
/* IDs of Params used in Aggref inputs */
- Bitmapset *aggParams;
+ Bitmapset *aggParams pg_node_attr(plan_jumble_ignore);
/* Note: planner provides numGroups & aggParams only in HASHED/MIXED case */
@@ -1147,7 +1149,7 @@ typedef struct Gather
int rescan_param; /* ID of Param that signals a rescan, or -1 */
bool single_copy; /* don't execute plan more than once */
bool invisible; /* suppress EXPLAIN display (for testing)? */
- Bitmapset *initParam; /* param id's of initplans which are referred
+ Bitmapset *initParam pg_node_attr(plan_jumble_ignore); /* param id's of initplans which are referred
* at gather or one of it's child node */
} Gather;
@@ -1186,7 +1188,7 @@ typedef struct GatherMerge
* param id's of initplans which are referred at gather merge or one of
* it's child node
*/
- Bitmapset *initParam;
+ Bitmapset *initParam pg_node_attr(plan_jumble_ignore);
} GatherMerge;
/* ----------------
@@ -1426,7 +1428,7 @@ typedef struct PartitionPruneInfo
NodeTag type;
List *prune_infos;
- Bitmapset *other_subplans;
+ Bitmapset *other_subplans pg_node_attr(plan_jumble_ignore);
} PartitionPruneInfo;
/*
@@ -1456,7 +1458,7 @@ typedef struct PartitionedRelPruneInfo
Index rtindex;
/* Indexes of all partitions which subplans or subparts are present for */
- Bitmapset *present_parts;
+ Bitmapset *present_parts pg_node_attr(plan_jumble_ignore);
/* Length of the following arrays: */
int nparts;
@@ -1480,7 +1482,7 @@ typedef struct PartitionedRelPruneInfo
List *exec_pruning_steps; /* List of PartitionPruneStep */
/* All PARAM_EXEC Param IDs in exec_pruning_steps */
- Bitmapset *execparamids;
+ Bitmapset *execparamids pg_node_attr(plan_jumble_ignore);
} PartitionedRelPruneInfo;
/*
@@ -1531,7 +1533,7 @@ typedef struct PartitionPruneStepOp
StrategyNumber opstrategy;
List *exprs;
List *cmpfns;
- Bitmapset *nullkeys;
+ Bitmapset *nullkeys pg_node_attr(plan_jumble_ignore);
} PartitionPruneStepOp;
/*
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
index 59e7bb26bb..8eb9959496 100644
--- a/src/include/nodes/primnodes.h
+++ b/src/include/nodes/primnodes.h
@@ -2246,15 +2246,15 @@ typedef struct TargetEntry
/* attribute number (see notes above) */
AttrNumber resno;
/* name of the column (could be NULL) */
- char *resname pg_node_attr(query_jumble_ignore);
+ char *resname pg_node_attr(query_jumble_ignore, plan_jumble_ignore);
/* nonzero if referenced by a sort/group clause */
Index ressortgroupref;
/* OID of column's source table */
- Oid resorigtbl pg_node_attr(query_jumble_ignore);
+ Oid resorigtbl pg_node_attr(query_jumble_ignore, plan_jumble_ignore);
/* column's number in source table */
- AttrNumber resorigcol pg_node_attr(query_jumble_ignore);
+ AttrNumber resorigcol pg_node_attr(query_jumble_ignore, plan_jumble_ignore);
/* set to true to eliminate the attribute from final target list */
- bool resjunk pg_node_attr(query_jumble_ignore);
+ bool resjunk pg_node_attr(query_jumble_ignore, plan_jumble_ignore);
} TargetEntry;
diff --git a/src/include/parser/analyze.h b/src/include/parser/analyze.h
index f1bd18c49f..e414b630bc 100644
--- a/src/include/parser/analyze.h
+++ b/src/include/parser/analyze.h
@@ -15,7 +15,7 @@
#define ANALYZE_H
#include "nodes/params.h"
-#include "nodes/queryjumble.h"
+#include "nodes/jumble.h"
#include "parser/parse_node.h"
/* Hook for plugins to get control at end of parse analysis */
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index d3d4ff6c5c..e074bc5b45 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -170,6 +170,9 @@ typedef struct PgBackendStatus
/* query identifier, optionally computed using post_parse_analyze_hook */
uint64 st_query_id;
+
+ /* plan identifier, optionally computed using planner_hook */
+ uint64 st_plan_id;
} PgBackendStatus;
@@ -316,6 +319,7 @@ extern void pgstat_clear_backend_activity_snapshot(void);
/* Activity reporting functions */
extern void pgstat_report_activity(BackendState state, const char *cmd_str);
extern void pgstat_report_query_id(uint64 query_id, bool force);
+extern void pgstat_report_plan_id(uint64 query_id, bool force);
extern void pgstat_report_tempfile(size_t filesize);
extern void pgstat_report_appname(const char *appname);
extern void pgstat_report_xact_timestamp(TimestampTz tstamp);
@@ -323,6 +327,7 @@ extern const char *pgstat_get_backend_current_activity(int pid, bool checkUser);
extern const char *pgstat_get_crashed_backend_activity(int pid, char *buffer,
int buflen);
extern uint64 pgstat_get_my_query_id(void);
+extern uint64 pgstat_get_my_plan_id(void);
extern BackendType pgstat_get_backend_type_by_proc_number(ProcNumber procNumber);
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 856a8349c5..e20cc7d28c 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1760,9 +1760,10 @@ pg_stat_activity| SELECT s.datid,
s.backend_xid,
s.backend_xmin,
s.query_id,
+ s.plan_id,
s.query,
s.backend_type
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
LEFT JOIN pg_database d ON ((s.datid = d.oid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_all_indexes| SELECT c.oid AS relid,
@@ -1886,7 +1887,7 @@ pg_stat_gssapi| SELECT pid,
gss_princ AS principal,
gss_enc AS encrypted,
gss_delegation AS credentials_delegated
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
WHERE (client_port IS NOT NULL);
pg_stat_io| SELECT backend_type,
object,
@@ -2092,7 +2093,7 @@ pg_stat_replication| SELECT s.pid,
w.sync_priority,
w.sync_state,
w.reply_time
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
JOIN pg_stat_get_wal_senders() w(pid, state, sent_lsn, write_lsn, flush_lsn, replay_lsn, write_lag, flush_lag, replay_lag, sync_priority, sync_state, reply_time) ON ((s.pid = w.pid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_replication_slots| SELECT s.slot_name,
@@ -2126,7 +2127,7 @@ pg_stat_ssl| SELECT pid,
ssl_client_dn AS client_dn,
ssl_client_serial AS client_serial,
ssl_issuer_dn AS issuer_dn
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
WHERE (client_port IS NOT NULL);
pg_stat_subscription| SELECT su.oid AS subid,
su.subname,
--
2.39.5 (Apple Git-154)
On Tue, Jan 21, 2025 at 10:47 AM Artem Gavrilov <artem.gavrilov@percona.com>
wrote:
We have another extension that does plan ID tracking: pg_stat_monitor. So
I think it would be great to have this functionality in core.
Thanks! I had forgotten that pg_stat_monitor can optionally track plan
statistics. Its actually another data point for why the plan ID calculation
should be in core:
Like pg_store_plans, pg_stat_monitor is hashing the plan text to calculate
the plan ID [0]https://github.com/percona/pg_stat_monitor/blob/main/pg_stat_monitor.c#L730, which can have measurable overhead (judging from our
benchmarks of pg_store_plans). It also utilizes EXPLAIN (COSTS OFF) for
getting the plan text [1]https://github.com/percona/pg_stat_monitor/blob/main/pg_stat_monitor.c#L678, which tracks with my thinking as to what should
be considered significant for the plan ID jumbling.
I tested your patch set on top of *86749ea3b76* PG revision on MacOS. All
tests successfully passed. However, pgident shows that some files are not
properly formatted.
Thanks, appreciate the test and note re: pgident, taking care of that in
the next patch refresh.
Thanks,
Lukas
[0]: https://github.com/percona/pg_stat_monitor/blob/main/pg_stat_monitor.c#L730
https://github.com/percona/pg_stat_monitor/blob/main/pg_stat_monitor.c#L730
[1]: https://github.com/percona/pg_stat_monitor/blob/main/pg_stat_monitor.c#L678
https://github.com/percona/pg_stat_monitor/blob/main/pg_stat_monitor.c#L678
--
Lukas Fittl
On 1/3/25 03:46, Lukas Fittl wrote:
My overall perspective is that (1) is best done in-core to keep overhead
low, whilst (2) could be done outside of core (or merged with a future
pg_stat_statements) and is included here mainly for illustration purposes.
Thank you for the patch and your attention to this issue!
I am pleased with the export of the jumbling functions and their
generalisation.
I may not be close to the task monitoring area, but I utilise queryId
and other tools to differ plan nodes inside extensions. Initially, like
queryId serves as a class identifier for queries, plan_id identifies a
class of nodes, not a single node. In the implementation provided here,
nodes with the same hash can represent different subtrees. For example,
JOIN(A, JOIN(B,C)) and JOIN(JOIN(B,C),A) may have the same ID.
Moreover, I wonder if this version of plan_id reacts to the join level
change. It appears that only a change of the join clause alters the
plan_id hash value, which means you would end up with a single hash for
very different plan nodes. Is that acceptable? To address this, we
should consider the hashes of the left and right subtrees and the hashes
of each subplan (especially in the case of Append).
Overall, similar to discussions on queryId, various extensions may want
different logic for generating plan_id (more or less unique guarantees,
for example). Hence, it would be beneficial to separate this logic and
allow extensions to provide different plan_ids. IMO, What we need is a
'List *ext' field in each of the Plan, Path, PlanStmt, and Query
structures. Such 'ext' field may contain different stuff that extensions
want to push without interference between them - specific plan_id as an
example.
Additionally, we could bridge the gap between the cloud of paths and the
plan by adding a hook at the end of the create_plan_recurse routine.
This may facilitate the transfer of information regarding optimiser
decisions that could be influenced by an extension into the plan.
--
regards, Andrei Lepikhov
Thanks for the reviews! Attached an updated v2 patch set, notes inline
below.
On Tue, Jan 21, 2025 at 4:47 PM Michael Paquier <michael@paquier.xyz> wrote:
May I suggest the addition of a code
path outside of your main proposal to test this API? For example
injection_stats.c with a new SQL function to reset everything.
Good idea - added an example use of this to injection_stats.c in the
attached 0002.
Question time: pgstat_drop_entries_of_kind() is called once in 0004,
which does a second sequential scan of pgStatLocal.shared_hash.
That's not efficient, making me question what's the reason to think
why pgstat_drop_entries_of_kind() is the best approach to use. I like
the idea of pgstat_drop_entries_of_kind(), less how it's applied in
the context of the main patch.
My motivation for doing two scans here, one in pgstat_drop_entries_of_kind
and one in pgstat_gc_plan_memory (both called from the reset function) was
that the first time through we hold an exclusive lock on
pgStatLocal.shared_hash, vs the second time (when we free plan texts) we
hold a share lock.
Maybe that doesn't matter, since "dsa_free" is fast anyway, and we can just
do this all in one go whilst holding an exclusive lock?
Overall, I also do wonder if it wouldn't be better to have a callback
mechanism in the shared memory stats, so stats plugins can do extra work
when an entry gets dropped (like freeing the DSA memory for the plan text),
vs having to add all this extra logic to do it.
On Thu, Jan 23, 2025 at 5:25 PM Sami Imseih <samimseih@gmail.com> wrote:
Thanks for starting this thread. This is an important feature.
I am still reviewing, but wanted to share some initial comments.
Thanks for taking the time! I had started on a v2 patch based on Michael's
note before I saw your email and experiment, so apologies for sending this
in the middle of your review :)
== pg_stat_plans extension (0004)
1. pg_stat_plans_1_0 should not call pgstat_fetch_entry.l
This is not needed since we already have the entry with a shared lock
and it could lead to an assertion error when pgstat_fetch_entry
may conditionally call dshash_find. dshash_find asserts that the lock
is not already held. Calling pgstat_get_entry_data should be
enough here.
Fixed.
2. a "toplevel" bool field is missing in pg_stat_plans to indicate the
plan is for a nested query.
Good point - added.
3. I think we should add cumulative planning_time. This
Probably should be controlled with a separate GUC as well.
Hmm. Don't we already have that in pg_stat_statements?
Though, in practice I see that turned off most of the time (due to its
overhead?), not sure if we could do better if it this was done here instead?
4. For deallocation, I wonder if it makes more sense to zap the
plans with the lowest total execution time rather than calls; or make
this configurable. In fact, I think choosing the eviction strategy
should be done in pg_stat_statements as well ( but that belongs
in a separate discussion ). The idea is to give more priority to
plans that have the most overall database time.
Yeah, that's a good point, its likely people would be most interested in
slow plans, vs those that were called a lot.
Happy to adjust it that way - I don't think we need to make it configurable
to be honest.
5. What are your thoughts about controlling the memory by
size rather than .max and .max_size ? if a typical plan
is 2KB, a user can fit 10k plans with 20MB. A typical
user can probably allocate much more memory for this
purpose.
Interesting idea! I'd be curious to get more feedback on the overall
approach here before digging into this, but I like this as it'd be more
intuitive from an end-user perspective.
6. I do like the idea of showing an in-flight plan.
This is so useful for a rare plan, especially on the
first capture of the plan ( calls = 0), and the planId
can be joined with pg_stat_activity to get the query
text.
Yes indeed, I think it would be a miss if we didn't allow looking at
in-flight plan IDs.
For the record, I can't take credit for the idea, I think I got this either
from your earlier plan ID patch, or from talking with you at PGConf NYC
last year.
== core plan id computation (0003)
1. compute_plan_id should do exactly what compute_query_id
does. It should have an "auto" as the default which automatically
computes a plan id when pg_stat_plans is enabled.
Yep, it does seem better to be consistent here. I added "auto" in v2 and
made it the default.
Mixed feelings about the choices of JumblePlanNode() in 0003 based on
its complexity as implemented. When it comes to such things, we
should keep the custom node functions short, applying node_attr
instead to the elements of the nodes so as the assumptions behind the
jumbling are documented within the structure definitions in the
headers, not the jumbling code itself.+1
we should be able to control which node is considered for plan_id
computation using a node attribute such as plan_jumble_ignore.
I played around with this idea by building on top of your proposal
and attached my experiment code for this. The tricky part will be
finalizing
which nodes and node fields to use for plan computation.
Agreed, its better to do this via the node_attr infrastructure. I've done
this in the attached before I saw your experiment code, so it may be worth
comparing the approaches.
Generally, I tried to stay closer to the idea of "only jumble what EXPLAIN
(COSTS OFF) would show", vs jumbling most plan fields by default.
That does mean we have a lot of extra "node_attr(query_jumble_ignore)" tags
in the plan node structs. We could potentially invent a new way of only
jumbling what's marked vs the current jumbling all by default + ignoring
some fields, but not sure if that's worth it.
3. We may want to combine all the jumbling code into
a single jumble.c since the query and plan jumble will
share a lot of the same code, i.e. JumbleState.
_JumbleNode, etc.
Agreed, that's what I ended up doing in v2. I think we can state that plan
jumbling is a super set of query jumbling, so it seems best to not have two
copies of very similar jumbling conds/funcs.
I retained the "query" prefix for now to not generate a big diff, but we
should maybe consider dropping that in both the source file names and the
node attributes?
Thanks,
Lukas
--
Lukas Fittl
Attachments:
v2-0003-Optionally-record-a-plan_id-in-PlannedStmt-to-ide.patchapplication/x-patch; name=v2-0003-Optionally-record-a-plan_id-in-PlannedStmt-to-ide.patchDownload
From b258a0b94d5881f5faa1cd9baae6feb184ad7a56 Mon Sep 17 00:00:00 2001
From: Lukas Fittl <lukas@fittl.com>
Date: Tue, 31 Dec 2024 15:16:10 -0800
Subject: [PATCH v2 3/4] Optionally record a plan_id in PlannedStmt to identify
plan shape
When enabled via the new compute_plan_id GUC (default off), this utilizes
the existing treewalk in setrefs.c after planning to calculate a hash
(the "plan_id", or plan identifier) that can be used to identify
which plan was chosen.
The plan_id generally intends to be the same if a given EXPLAIN (without
ANALYZE) output is the same. The plan_id includes both the top-level plan
as well as all subplans. Execution statistics are excluded.
If enabled, the plan_id is shown for currently running queries in
pg_stat_activity, as well as recorded in EXPLAIN and auto_explain output.
Other in core users or extensions can use this facility to show or
accumulate statistics about the plans used by queries, to help identify
plan regressions, or drive plan management decisions.
Note that this commit intentionally does not include a facility to map
a given plan_id to the EXPLAIN text output - it is a assumed that users
can utilize the auto_explain extension to establish this mapping as
needed, or extensions can record this via the existing planner hook.
---
doc/src/sgml/config.sgml | 34 ++
doc/src/sgml/monitoring.sgml | 16 +
src/backend/catalog/system_views.sql | 1 +
src/backend/commands/explain.c | 16 +
src/backend/executor/execMain.c | 10 +-
src/backend/executor/execParallel.c | 1 +
src/backend/nodes/gen_node_support.pl | 50 ++-
src/backend/nodes/queryjumblefuncs.c | 78 +++-
src/backend/optimizer/plan/planner.c | 18 +
src/backend/optimizer/plan/setrefs.c | 9 +
src/backend/postmaster/launch_backend.c | 3 +
src/backend/tcop/postgres.c | 1 +
src/backend/utils/activity/backend_status.c | 70 ++-
src/backend/utils/adt/pgstatfuncs.c | 7 +-
src/backend/utils/misc/guc_tables.c | 28 ++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/catalog/pg_proc.dat | 6 +-
src/include/nodes/pathnodes.h | 3 +
src/include/nodes/plannodes.h | 398 +++++++++++-------
src/include/nodes/primnodes.h | 7 +-
src/include/nodes/queryjumble.h | 34 +-
src/include/utils/backend_status.h | 5 +
src/test/regress/expected/explain.out | 11 +
src/test/regress/expected/rules.out | 9 +-
src/test/regress/sql/explain.sql | 4 +
25 files changed, 635 insertions(+), 185 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index a782f10998..85a1e0d8d4 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -8406,6 +8406,40 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-compute-plan-id" xreflabel="compute_plan_id">
+ <term><varname>compute_plan_id</varname> (<type>enum</type>)
+ <indexterm>
+ <primary><varname>compute_plan_id</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Enables in-core computation of a plan identifier.
+ Plan identifiers can be displayed in the <link
+ linkend="monitoring-pg-stat-activity-view"><structname>pg_stat_activity</structname></link>
+ view or using <command>EXPLAIN</command>.
+ Note that an external module can alternatively be used if the
+ in-core plan identifier computation method is not acceptable.
+ In this case, in-core computation must be always disabled.
+ Valid values are <literal>off</literal> (always disabled),
+ <literal>on</literal> (always enabled), <literal>auto</literal>,
+ which lets modules that utilize plan identifiers enable
+ it automatically, and <literal>regress</literal> which
+ has the same effect as <literal>on</literal>, except that the
+ query identifier is not shown in the <literal>EXPLAIN</literal> output
+ in order to facilitate automated regression testing.
+ The default is <literal>auto</literal>.
+ </para>
+ <note>
+ <para>
+ To ensure that only one plan identifier is calculated and
+ displayed, extensions that calculate plan identifiers should
+ throw an error if a plan identifier has already been computed.
+ </para>
+ </note>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-log-statement-stats">
<term><varname>log_statement_stats</varname> (<type>boolean</type>)
<indexterm>
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index e5888fae2b..a276807052 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -972,6 +972,22 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>plan_id</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Identifier of this backend's most recent query plan. If
+ <structfield>state</structfield> is <literal>active</literal> this
+ field shows the identifier of the currently executing query plan. In
+ all other states, it shows the identifier of last query plan that
+ was executed. Plan identifiers are not computed by default so this
+ field will be null unless <xref linkend="guc-compute-plan-id"/>
+ parameter is enabled or a third-party module that computes plan
+ identifiers is configured.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>query</structfield> <type>text</type>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 46868bf7e8..a49efc6332 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -885,6 +885,7 @@ CREATE VIEW pg_stat_activity AS
S.backend_xid,
s.backend_xmin,
S.query_id,
+ S.plan_id,
S.query,
S.backend_type
FROM pg_stat_get_activity(NULL) AS S
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index c24e66f82e..31a1761f53 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -966,6 +966,22 @@ ExplainPrintPlan(ExplainState *es, QueryDesc *queryDesc)
ExplainPropertyInteger("Query Identifier", NULL, (int64)
queryDesc->plannedstmt->queryId, es);
}
+
+ /*
+ * COMPUTE_PLAN_ID_REGRESS means COMPUTE_PLAN_ID_YES, but we don't show
+ * the queryid in any of the EXPLAIN plans to keep stable the results
+ * generated by regression test suites.
+ */
+ if (es->verbose && queryDesc->plannedstmt->planId != UINT64CONST(0) &&
+ compute_plan_id != COMPUTE_PLAN_ID_REGRESS)
+ {
+ /*
+ * Output the queryid as an int64 rather than a uint64 so we match
+ * what would be seen in the BIGINT pg_stat_activity.plan_id column.
+ */
+ ExplainPropertyInteger("Plan Identifier", NULL, (int64)
+ queryDesc->plannedstmt->planId, es);
+ }
}
/*
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index fb8dba3ab2..0b334fa2c6 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -120,13 +120,15 @@ ExecutorStart(QueryDesc *queryDesc, int eflags)
{
/*
* In some cases (e.g. an EXECUTE statement or an execute message with the
- * extended query protocol) the query_id won't be reported, so do it now.
+ * extended query protocol) the query_id and plan_id won't be reported, so
+ * do it now.
*
- * Note that it's harmless to report the query_id multiple times, as the
- * call will be ignored if the top level query_id has already been
- * reported.
+ * Note that it's harmless to report the identifiers multiple times, as
+ * the call will be ignored if the top level query_id / plan_id has
+ * already been reported.
*/
pgstat_report_query_id(queryDesc->plannedstmt->queryId, false);
+ pgstat_report_plan_id(queryDesc->plannedstmt->planId, queryDesc->plannedstmt->queryId, false);
if (ExecutorStart_hook)
(*ExecutorStart_hook) (queryDesc, eflags);
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index ff4d9dd1bb..20ebc23b00 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -174,6 +174,7 @@ ExecSerializePlan(Plan *plan, EState *estate)
pstmt = makeNode(PlannedStmt);
pstmt->commandType = CMD_SELECT;
pstmt->queryId = pgstat_get_my_query_id();
+ pstmt->planId = pgstat_get_my_plan_id();
pstmt->hasReturning = false;
pstmt->hasModifyingCTE = false;
pstmt->canSetTag = true;
diff --git a/src/backend/nodes/gen_node_support.pl b/src/backend/nodes/gen_node_support.pl
index 7c012c27f8..e7b04678e0 100644
--- a/src/backend/nodes/gen_node_support.pl
+++ b/src/backend/nodes/gen_node_support.pl
@@ -475,6 +475,7 @@ foreach my $infile (@ARGV)
equal_ignore_if_zero
query_jumble_ignore
query_jumble_location
+ query_jumble_rt_index
read_write_ignore
write_only_relids
write_only_nondefault_pathtarget
@@ -1280,13 +1281,19 @@ _jumble${n}(JumbleState *jstate, Node *node)
{
my $t = $node_type_info{$n}->{field_types}{$f};
my @a = @{ $node_type_info{$n}->{field_attrs}{$f} };
+ my $array_size_field;
my $query_jumble_ignore = $struct_no_query_jumble;
my $query_jumble_location = 0;
+ my $query_jumble_rt_index = 0;
# extract per-field attributes
foreach my $a (@a)
{
- if ($a eq 'query_jumble_ignore')
+ if ($a =~ /^array_size\(([\w.]+)\)$/)
+ {
+ $array_size_field = $1;
+ }
+ elsif ($a eq 'query_jumble_ignore')
{
$query_jumble_ignore = 1;
}
@@ -1294,10 +1301,29 @@ _jumble${n}(JumbleState *jstate, Node *node)
{
$query_jumble_location = 1;
}
+ elsif ($a eq 'query_jumble_rt_index')
+ {
+ $query_jumble_rt_index = 1;
+ }
}
+ next if $query_jumble_ignore;
+
+ if ($query_jumble_rt_index)
+ {
+ if ($t eq 'List*')
+ {
+ print $jff "\tJUMBLE_RT_INDEX_LIST($f);\n"
+ unless $query_jumble_ignore;
+ }
+ else
+ {
+ print $jff "\tJUMBLE_RT_INDEX($f);\n"
+ unless $query_jumble_ignore;
+ }
+ }
# node type
- if (($t =~ /^(\w+)\*$/ or $t =~ /^struct\s+(\w+)\*$/)
+ elsif (($t =~ /^(\w+)\*$/ or $t =~ /^struct\s+(\w+)\*$/)
and elem $1, @node_types)
{
print $jff "\tJUMBLE_NODE($f);\n"
@@ -1317,6 +1343,26 @@ _jumble${n}(JumbleState *jstate, Node *node)
print $jff "\tJUMBLE_STRING($f);\n"
unless $query_jumble_ignore;
}
+ elsif ($t =~ /^(\w+)(\*|\[\w+\])$/ and elem $1, @scalar_types)
+ {
+ if (!defined $array_size_field)
+ {
+ die "no array size defined for $n.$f of type $t\n";
+ }
+ if ($node_type_info{$n}->{field_types}{$array_size_field} eq
+ 'List*')
+ {
+ print $jff
+ "\tJUMBLE_ARRAY($f, list_length(expr->$array_size_field));\n"
+ unless $query_jumble_ignore;
+ }
+ else
+ {
+ print $jff
+ "\tJUMBLE_ARRAY($f, expr->$array_size_field);\n"
+ unless $query_jumble_ignore;
+ }
+ }
else
{
print $jff "\tJUMBLE_FIELD($f);\n"
diff --git a/src/backend/nodes/queryjumblefuncs.c b/src/backend/nodes/queryjumblefuncs.c
index 545d8edcae..482d1ea828 100644
--- a/src/backend/nodes/queryjumblefuncs.c
+++ b/src/backend/nodes/queryjumblefuncs.c
@@ -35,12 +35,14 @@
#include "common/hashfn.h"
#include "miscadmin.h"
#include "nodes/queryjumble.h"
+#include "parser/parsetree.h"
#include "parser/scansup.h"
#define JUMBLE_SIZE 1024 /* query serialization buffer size */
/* GUC parameters */
int compute_query_id = COMPUTE_QUERY_ID_AUTO;
+int compute_plan_id = COMPUTE_PLAN_ID_AUTO;
/*
* True when compute_query_id is ON or AUTO, and a module requests them.
@@ -51,7 +53,18 @@ int compute_query_id = COMPUTE_QUERY_ID_AUTO;
*/
bool query_id_enabled = false;
+/*
+ * True when compute_plan_id is ON or AUTO, and a module requests them.
+ *
+ * Note that IsPlanIdEnabled() should be used instead of checking
+ * plan_id_enabled or plan_query_id directly when we want to know
+ * whether plan identifiers are computed in the core or not.
+ */
+bool plan_id_enabled = false;
+
static void RecordConstLocation(JumbleState *jstate, int location);
+static void JumbleRangeTableIndex(JumbleState *jstate, Index rti);
+static void JumbleRangeTableIndexList(JumbleState *jstate, List *l);
static void _jumbleA_Const(JumbleState *jstate, Node *node);
static void _jumbleList(JumbleState *jstate, Node *node);
static void _jumbleVariableSetStmt(JumbleState *jstate, Node *node);
@@ -106,7 +119,7 @@ CleanQuerytext(const char *query, int *location, int *len)
}
JumbleState *
-InitializeJumbleState(bool record_clocations)
+InitializeJumbleState(bool record_clocations, PlannerGlobal *glob)
{
JumbleState *jstate = (JumbleState *) palloc0(sizeof(JumbleState));
@@ -121,6 +134,8 @@ InitializeJumbleState(bool record_clocations)
palloc(jstate->clocations_buf_size * sizeof(LocationLen));
}
+ jstate->glob = glob;
+
return jstate;
}
@@ -135,7 +150,7 @@ HashJumbleState(JumbleState *jstate)
JumbleState *
JumbleQuery(Query *query)
{
- JumbleState *jstate = InitializeJumbleState(true);
+ JumbleState *jstate = InitializeJumbleState(true, NULL);
Assert(IsQueryIdEnabled());
@@ -171,6 +186,19 @@ EnableQueryId(void)
query_id_enabled = true;
}
+/*
+ * Enables plan identifier computation.
+ *
+ * Third-party plugins can use this function to inform core that they require
+ * a query identifier to be computed.
+ */
+void
+EnablePlanId(void)
+{
+ if (compute_plan_id != COMPUTE_PLAN_ID_OFF)
+ plan_id_enabled = true;
+}
+
/*
* AppendJumble: Append a value that is substantive in a given query to
* the current jumble.
@@ -238,8 +266,17 @@ RecordConstLocation(JumbleState *jstate, int location)
JumbleNode(jstate, (Node *) expr->item)
#define JUMBLE_LOCATION(location) \
RecordConstLocation(jstate, expr->location)
+#define JUMBLE_RT_INDEX(item) \
+do { \
+ if (expr->item) \
+ JumbleRangeTableIndex(jstate, expr->item); \
+} while(0)
+#define JUMBLE_RT_INDEX_LIST(item) \
+ JumbleRangeTableIndexList(jstate, expr->item)
#define JUMBLE_FIELD(item) \
AppendJumble(jstate, (const unsigned char *) &(expr->item), sizeof(expr->item))
+#define JUMBLE_ARRAY(item, len) \
+ AppendJumble(jstate, (const unsigned char *) &(expr->item), sizeof(*(expr->item)) * len)
#define JUMBLE_FIELD_SINGLE(item) \
AppendJumble(jstate, (const unsigned char *) &(item), sizeof(item))
#define JUMBLE_STRING(str) \
@@ -388,3 +425,40 @@ _jumbleVariableSetStmt(JumbleState *jstate, Node *node)
JUMBLE_FIELD(is_local);
JUMBLE_LOCATION(location);
}
+
+/*
+ * Jumble the target of a rangle table index, e.g. in a Scan or Modify node
+ */
+static void
+JumbleRangeTableIndex(JumbleState *jstate, Index rti)
+{
+ RangeTblEntry *expr = rt_fetch(rti, jstate->glob->finalrtable);
+
+ switch (expr->rtekind)
+ {
+ case RTE_RELATION:
+ JUMBLE_FIELD(relid);
+ break;
+ case RTE_CTE:
+ JUMBLE_STRING(ctename);
+ break;
+ default:
+
+ /*
+ * Ignore other targets, the jumble includes something identifying
+ * about them already
+ */
+ break;
+ }
+}
+
+static void
+JumbleRangeTableIndexList(JumbleState *jstate, List *l)
+{
+ ListCell *lc;
+
+ Assert(l->type == T_IntList);
+
+ foreach(lc, l)
+ JumbleRangeTableIndex(jstate, lfirst_int(lc));
+}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 6803edd085..70500441d8 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -37,6 +37,7 @@
#ifdef OPTIMIZER_DEBUG
#include "nodes/print.h"
#endif
+#include "nodes/queryjumble.h"
#include "nodes/supportnodes.h"
#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
@@ -532,6 +533,16 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions,
Assert(glob->finalrowmarks == NIL);
Assert(glob->resultRelations == NIL);
Assert(glob->appendRelations == NIL);
+
+ /*
+ * Initialize plan identifier jumble if needed
+ *
+ * Note the actual jumbling is done in the tree walk in
+ * set_plan_references
+ */
+ if (IsPlanIdEnabled())
+ glob->plan_jumble_state = InitializeJumbleState(false, glob);
+
top_plan = set_plan_references(root, top_plan);
/* ... and the subplans (both regular subplans and initplans) */
Assert(list_length(glob->subplans) == list_length(glob->subroots));
@@ -570,6 +581,13 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions,
result->stmt_location = parse->stmt_location;
result->stmt_len = parse->stmt_len;
+ if (IsPlanIdEnabled())
+ {
+ result->planId = HashJumbleState(glob->plan_jumble_state);
+ pfree(glob->plan_jumble_state->jumble);
+ pfree(glob->plan_jumble_state);
+ }
+
result->jitFlags = PGJIT_NONE;
if (jit_enabled && jit_above_cost >= 0 &&
top_plan->total_cost > jit_above_cost)
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 1e7b7bc6ff..f494a33810 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -19,6 +19,7 @@
#include "catalog/pg_type.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
+#include "nodes/queryjumble.h"
#include "optimizer/optimizer.h"
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
@@ -1295,6 +1296,14 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset)
break;
}
+ /*
+ * If enabled, append significant information to the plan identifier
+ * jumble (we do this here since we're already walking the tree in a
+ * near-final state)
+ */
+ if (IsPlanIdEnabled())
+ JumbleNode(root->glob->plan_jumble_state, (Node *) plan);
+
/*
* Now recurse into child plans, if any
*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index a97a1eda6d..654acf5bf0 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -115,6 +115,7 @@ typedef struct
bool redirection_done;
bool IsBinaryUpgrade;
bool query_id_enabled;
+ bool plan_id_enabled;
int max_safe_fds;
int MaxBackends;
int num_pmchild_slots;
@@ -744,6 +745,7 @@ save_backend_variables(BackendParameters *param,
param->redirection_done = redirection_done;
param->IsBinaryUpgrade = IsBinaryUpgrade;
param->query_id_enabled = query_id_enabled;
+ param->plan_id_enabled = plan_id_enabled;
param->max_safe_fds = max_safe_fds;
param->MaxBackends = MaxBackends;
@@ -1004,6 +1006,7 @@ restore_backend_variables(BackendParameters *param)
redirection_done = param->redirection_done;
IsBinaryUpgrade = param->IsBinaryUpgrade;
query_id_enabled = param->query_id_enabled;
+ plan_id_enabled = param->plan_id_enabled;
max_safe_fds = param->max_safe_fds;
MaxBackends = param->MaxBackends;
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 5655348a2e..6d8947bae9 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -1106,6 +1106,7 @@ exec_simple_query(const char *query_string)
size_t cmdtaglen;
pgstat_report_query_id(0, true);
+ pgstat_report_plan_id(0, 0, true);
/*
* Get the command name for use in status display (it also becomes the
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 731342799a..1dfb7a58f8 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -379,6 +379,7 @@ pgstat_bestart(void)
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = UINT64CONST(0);
+ lbeentry.st_plan_id = UINT64CONST(0);
/*
* we don't zero st_progress_param here to save cycles; nobody should
@@ -533,6 +534,7 @@ pgstat_report_activity(BackendState state, const char *cmd_str)
/* st_xact_start_timestamp and wait_event_info are also disabled */
beentry->st_xact_start_timestamp = 0;
beentry->st_query_id = UINT64CONST(0);
+ beentry->st_plan_id = UINT64CONST(0);
proc->wait_event_info = 0;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -588,12 +590,15 @@ pgstat_report_activity(BackendState state, const char *cmd_str)
beentry->st_state_start_timestamp = current_timestamp;
/*
- * If a new query is started, we reset the query identifier as it'll only
- * be known after parse analysis, to avoid reporting last query's
- * identifier.
+ * If a new query is started, we reset the query and plan identifier as
+ * it'll only be known after parse analysis / planning, to avoid reporting
+ * last query's identifiers.
*/
if (state == STATE_RUNNING)
+ {
beentry->st_query_id = UINT64CONST(0);
+ beentry->st_plan_id = UINT64CONST(0);
+ }
if (cmd_str != NULL)
{
@@ -644,6 +649,45 @@ pgstat_report_query_id(uint64 query_id, bool force)
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
+/* --------
+ * pgstat_report_plan_id() -
+ *
+ * Called to update top-level plan identifier.
+ * --------
+ */
+void
+pgstat_report_plan_id(uint64 plan_id, uint64 query_id, bool force)
+{
+ volatile PgBackendStatus *beentry = MyBEEntry;
+
+ /*
+ * if track_activities is disabled, st_plan_id should already have been
+ * reset
+ */
+ if (!beentry || !pgstat_track_activities)
+ return;
+
+ /*
+ * We only report the top-level plan identifiers. The stored plan_id is
+ * reset when a backend calls pgstat_report_activity(STATE_RUNNING), or
+ * with an explicit call to this function using the force flag. If the
+ * saved plan identifier is not zero or the query identifier is 0, it
+ * means that it's not a top-level command, so ignore the one provided
+ * unless it's an explicit call to reset the identifier.
+ */
+ if ((beentry->st_plan_id != 0 || query_id == 0) && !force)
+ return;
+
+ /*
+ * Update my status entry, following the protocol of bumping
+ * st_changecount before and after. We use a volatile pointer here to
+ * ensure the compiler doesn't try to get cute.
+ */
+ PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
+ beentry->st_plan_id = plan_id;
+ PGSTAT_END_WRITE_ACTIVITY(beentry);
+}
+
/* ----------
* pgstat_report_appname() -
@@ -1040,6 +1084,26 @@ pgstat_get_my_query_id(void)
return MyBEEntry->st_query_id;
}
+/* ----------
+ * pgstat_get_my_plan_id() -
+ *
+ * Return current backend's plan identifier.
+ */
+uint64
+pgstat_get_my_plan_id(void)
+{
+ if (!MyBEEntry)
+ return 0;
+
+ /*
+ * There's no need for a lock around pgstat_begin_read_activity /
+ * pgstat_end_read_activity here as it's only called from
+ * pg_stat_get_activity which is already protected, or from the same
+ * backend which means that there won't be concurrent writes.
+ */
+ return MyBEEntry->st_plan_id;
+}
+
/* ----------
* pgstat_get_backend_type_by_proc_number() -
*
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 0f5e0a9778..3c3bf2cb47 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -302,7 +302,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum
pg_stat_get_activity(PG_FUNCTION_ARGS)
{
-#define PG_STAT_GET_ACTIVITY_COLS 31
+#define PG_STAT_GET_ACTIVITY_COLS 32
int num_backends = pgstat_fetch_stat_numbackends();
int curr_backend;
int pid = PG_ARGISNULL(0) ? -1 : PG_GETARG_INT32(0);
@@ -613,6 +613,10 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
nulls[30] = true;
else
values[30] = UInt64GetDatum(beentry->st_query_id);
+ if (beentry->st_plan_id == 0)
+ nulls[31] = true;
+ else
+ values[31] = UInt64GetDatum(beentry->st_plan_id);
}
else
{
@@ -642,6 +646,7 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
nulls[28] = true;
nulls[29] = true;
nulls[30] = true;
+ nulls[31] = true;
}
tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 38cb9e970d..9ddb8e9731 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -309,6 +309,24 @@ static const struct config_enum_entry compute_query_id_options[] = {
{NULL, 0, false}
};
+/*
+ * Although only "on" and "off" are documented, we accept
+ * all the likely variants of "on" and "off".
+ */
+static const struct config_enum_entry compute_plan_id_options[] = {
+ {"auto", COMPUTE_PLAN_ID_AUTO, false},
+ {"regress", COMPUTE_PLAN_ID_REGRESS, false},
+ {"on", COMPUTE_PLAN_ID_ON, false},
+ {"off", COMPUTE_PLAN_ID_OFF, false},
+ {"true", COMPUTE_PLAN_ID_ON, true},
+ {"false", COMPUTE_PLAN_ID_OFF, true},
+ {"yes", COMPUTE_PLAN_ID_ON, true},
+ {"no", COMPUTE_PLAN_ID_OFF, true},
+ {"1", COMPUTE_PLAN_ID_ON, true},
+ {"0", COMPUTE_PLAN_ID_OFF, true},
+ {NULL, 0, false}
+};
+
/*
* Although only "on", "off", and "partition" are documented, we
* accept all the likely variants of "on" and "off".
@@ -4873,6 +4891,16 @@ struct config_enum ConfigureNamesEnum[] =
NULL, NULL, NULL
},
+ {
+ {"compute_plan_id", PGC_SUSET, STATS_MONITORING,
+ gettext_noop("Enables in-core computation of plan identifiers."),
+ NULL
+ },
+ &compute_plan_id,
+ COMPUTE_PLAN_ID_AUTO, compute_plan_id_options,
+ NULL, NULL, NULL
+ },
+
{
{"constraint_exclusion", PGC_USERSET, QUERY_TUNING_OTHER,
gettext_noop("Enables the planner to use constraints to optimize queries."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 079efa1baa..0634ae90dd 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -641,6 +641,7 @@
# - Monitoring -
#compute_query_id = auto
+#compute_plan_id = auto
#log_statement_stats = off
#log_parser_stats = off
#log_planner_stats = off
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 18560755d2..de341e2d02 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5549,9 +5549,9 @@
proname => 'pg_stat_get_activity', prorows => '100', proisstrict => 'f',
proretset => 't', provolatile => 's', proparallel => 'r',
prorettype => 'record', proargtypes => 'int4',
- proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,bool,int4,int8}',
- proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,gss_delegation,leader_pid,query_id}',
+ proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,bool,int4,int8,int8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,gss_delegation,leader_pid,query_id,plan_id}',
prosrc => 'pg_stat_get_activity' },
{ oid => '6318', descr => 'describe wait events',
proname => 'pg_get_wait_events', procost => '10', prorows => '250',
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 54ee17697e..05899d7d8d 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -163,6 +163,9 @@ typedef struct PlannerGlobal
/* partition descriptors */
PartitionDirectory partition_directory pg_node_attr(read_write_ignore);
+
+ /* optional jumble state for plan identifier claculation */
+ struct JumbleState *plan_jumble_state pg_node_attr(read_write_ignore);
} PlannerGlobal;
/* macro for fetching the Plan associated with a SubPlan node */
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 9e19cdd284..dded07841c 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -53,6 +53,10 @@ typedef struct PlannedStmt
uint64 queryId; /* query identifier (copied from Query) */
+ uint64 planId; /* plan identifier (calculated if
+ * compute_plan_id is enabled, can also be set
+ * by plugins) */
+
bool hasReturning; /* is it insert|update|delete|merge RETURNING? */
bool hasModifyingCTE; /* has insert|update|delete|merge in WITH? */
@@ -118,44 +122,55 @@ typedef struct PlannedStmt
*/
typedef struct Plan
{
- pg_node_attr(abstract, no_equal, no_query_jumble)
+ pg_node_attr(abstract, no_equal)
NodeTag type;
/*
* estimated execution costs for plan (see costsize.c for more info)
*/
- int disabled_nodes; /* count of disabled nodes */
- Cost startup_cost; /* cost expended before fetching any tuples */
- Cost total_cost; /* total cost (assuming all tuples fetched) */
+ int disabled_nodes pg_node_attr(query_jumble_ignore); /* count of disabled
+ * nodes */
+ Cost startup_cost pg_node_attr(query_jumble_ignore); /* cost expended before
+ * fetching any tuples */
+ Cost total_cost pg_node_attr(query_jumble_ignore); /* total cost (assuming
+ * all tuples fetched) */
/*
* planner's estimate of result size of this plan step
*/
- Cardinality plan_rows; /* number of rows plan is expected to emit */
- int plan_width; /* average row width in bytes */
+ Cardinality plan_rows pg_node_attr(query_jumble_ignore); /* number of rows plan
+ * is expected to emit */
+ int plan_width pg_node_attr(query_jumble_ignore); /* average row width in
+ * bytes */
/*
* information needed for parallel query
*/
- bool parallel_aware; /* engage parallel-aware logic? */
- bool parallel_safe; /* OK to use as part of parallel plan? */
+ bool parallel_aware pg_node_attr(query_jumble_ignore); /* engage parallel-aware
+ * logic? */
+ bool parallel_safe pg_node_attr(query_jumble_ignore); /* OK to use as part of
+ * parallel plan? */
/*
* information needed for asynchronous execution
*/
- bool async_capable; /* engage asynchronous-capable logic? */
+ bool async_capable pg_node_attr(query_jumble_ignore); /* engage
+ * asynchronous-capable
+ * logic? */
/*
* Common structural data for all Plan types.
*/
- int plan_node_id; /* unique across entire final plan tree */
+ int plan_node_id pg_node_attr(query_jumble_ignore); /* unique across entire
+ * final plan tree */
List *targetlist; /* target list to be computed at this node */
List *qual; /* implicitly-ANDed qual conditions */
- struct Plan *lefttree; /* input plan tree(s) */
- struct Plan *righttree;
- List *initPlan; /* Init Plan nodes (un-correlated expr
- * subselects) */
+ struct Plan *lefttree pg_node_attr(query_jumble_ignore); /* input plan tree(s) */
+ struct Plan *righttree pg_node_attr(query_jumble_ignore);
+ List *initPlan pg_node_attr(query_jumble_ignore); /* Init Plan nodes
+ * (un-correlated expr
+ * subselects) */
/*
* Information for management of parameter-change-driven rescanning
@@ -168,8 +183,8 @@ typedef struct Plan
* params that affect the node (i.e., the setParams of its initplans).
* These are _all_ the PARAM_EXEC params that affect this node.
*/
- Bitmapset *extParam;
- Bitmapset *allParam;
+ Bitmapset *extParam pg_node_attr(query_jumble_ignore);
+ Bitmapset *allParam pg_node_attr(query_jumble_ignore);
} Plan;
/* ----------------
@@ -231,31 +246,47 @@ typedef struct ModifyTable
{
Plan plan;
CmdType operation; /* INSERT, UPDATE, DELETE, or MERGE */
- bool canSetTag; /* do we set the command tag/es_processed? */
- Index nominalRelation; /* Parent RT index for use of EXPLAIN */
- Index rootRelation; /* Root RT index, if partitioned/inherited */
+ bool canSetTag pg_node_attr(query_jumble_ignore); /* do we set the command
+ * tag/es_processed? */
+ Index nominalRelation pg_node_attr(query_jumble_ignore); /* Parent RT index for
+ * use of EXPLAIN */
+ Index rootRelation pg_node_attr(query_jumble_rt_index); /* Root RT index, if
+ * partitioned/inherited */
bool partColsUpdated; /* some part key in hierarchy updated? */
- List *resultRelations; /* integer list of RT indexes */
- List *updateColnosLists; /* per-target-table update_colnos lists */
- List *withCheckOptionLists; /* per-target-table WCO lists */
- char *returningOldAlias; /* alias for OLD in RETURNING lists */
- char *returningNewAlias; /* alias for NEW in RETURNING lists */
- List *returningLists; /* per-target-table RETURNING tlists */
- List *fdwPrivLists; /* per-target-table FDW private data lists */
- Bitmapset *fdwDirectModifyPlans; /* indices of FDW DM plans */
- List *rowMarks; /* PlanRowMarks (non-locking only) */
- int epqParam; /* ID of Param for EvalPlanQual re-eval */
+ List *resultRelations pg_node_attr(query_jumble_rt_index); /* integer list of RT
+ * indexes */
+ List *updateColnosLists pg_node_attr(query_jumble_ignore); /* per-target-table
+ * update_colnos lists */
+ List *withCheckOptionLists pg_node_attr(query_jumble_ignore); /* per-target-table WCO
+ * lists */
+ char *returningOldAlias pg_node_attr(query_jumble_ignore); /* alias for OLD in
+ * RETURNING lists */
+ char *returningNewAlias pg_node_attr(query_jumble_ignore); /* alias for NEW in
+ * RETURNING lists */
+ List *returningLists pg_node_attr(query_jumble_ignore); /* per-target-table
+ * RETURNING tlists */
+ List *fdwPrivLists pg_node_attr(query_jumble_ignore); /* per-target-table FDW
+ * private data lists */
+ Bitmapset *fdwDirectModifyPlans pg_node_attr(query_jumble_ignore); /* indices of FDW DM
+ * plans */
+ List *rowMarks pg_node_attr(query_jumble_ignore); /* PlanRowMarks
+ * (non-locking only) */
+ int epqParam pg_node_attr(query_jumble_ignore); /* ID of Param for
+ * EvalPlanQual re-eval */
OnConflictAction onConflictAction; /* ON CONFLICT action */
List *arbiterIndexes; /* List of ON CONFLICT arbiter index OIDs */
List *onConflictSet; /* INSERT ON CONFLICT DO UPDATE targetlist */
List *onConflictCols; /* target column numbers for onConflictSet */
Node *onConflictWhere; /* WHERE for ON CONFLICT UPDATE */
- Index exclRelRTI; /* RTI of the EXCLUDED pseudo relation */
- List *exclRelTlist; /* tlist of the EXCLUDED pseudo relation */
- List *mergeActionLists; /* per-target-table lists of actions for
- * MERGE */
- List *mergeJoinConditions; /* per-target-table join conditions
- * for MERGE */
+ Index exclRelRTI pg_node_attr(query_jumble_ignore); /* RTI of the EXCLUDED
+ * pseudo relation */
+ List *exclRelTlist pg_node_attr(query_jumble_ignore); /* tlist of the EXCLUDED
+ * pseudo relation */
+ List *mergeActionLists pg_node_attr(query_jumble_ignore); /* per-target-table
+ * lists of actions for
+ * MERGE */
+ List *mergeJoinConditions pg_node_attr(query_jumble_ignore); /* per-target-table join
+ * conditions for MERGE */
} ModifyTable;
struct PartitionPruneInfo; /* forward reference to struct below */
@@ -268,18 +299,20 @@ struct PartitionPruneInfo; /* forward reference to struct below */
typedef struct Append
{
Plan plan;
- Bitmapset *apprelids; /* RTIs of appendrel(s) formed by this node */
- List *appendplans;
- int nasyncplans; /* # of asynchronous plans */
+ Bitmapset *apprelids pg_node_attr(query_jumble_ignore); /* RTIs of appendrel(s)
+ * formed by this node */
+ List *appendplans pg_node_attr(query_jumble_ignore);
+ int nasyncplans pg_node_attr(query_jumble_ignore); /* # of asynchronous
+ * plans */
/*
* All 'appendplans' preceding this index are non-partial plans. All
* 'appendplans' from this index onwards are partial plans.
*/
- int first_partial_plan;
+ int first_partial_plan pg_node_attr(query_jumble_ignore);
/* Info for run-time subplan pruning; NULL if we're not doing that */
- struct PartitionPruneInfo *part_prune_info;
+ struct PartitionPruneInfo *part_prune_info pg_node_attr(query_jumble_ignore);
} Append;
/* ----------------
@@ -292,29 +325,29 @@ typedef struct MergeAppend
Plan plan;
/* RTIs of appendrel(s) formed by this node */
- Bitmapset *apprelids;
+ Bitmapset *apprelids pg_node_attr(query_jumble_ignore);
- List *mergeplans;
+ List *mergeplans pg_node_attr(query_jumble_ignore);
/* these fields are just like the sort-key info in struct Sort: */
/* number of sort-key columns */
- int numCols;
+ int numCols pg_node_attr(query_jumble_ignore);
/* their indexes in the target list */
- AttrNumber *sortColIdx pg_node_attr(array_size(numCols));
+ AttrNumber *sortColIdx pg_node_attr(array_size(numCols), query_jumble_ignore);
/* OIDs of operators to sort them by */
- Oid *sortOperators pg_node_attr(array_size(numCols));
+ Oid *sortOperators pg_node_attr(array_size(numCols), query_jumble_ignore);
/* OIDs of collations */
- Oid *collations pg_node_attr(array_size(numCols));
+ Oid *collations pg_node_attr(array_size(numCols), query_jumble_ignore);
/* NULLS FIRST/LAST directions */
- bool *nullsFirst pg_node_attr(array_size(numCols));
+ bool *nullsFirst pg_node_attr(array_size(numCols), query_jumble_ignore);
/* Info for run-time subplan pruning; NULL if we're not doing that */
- struct PartitionPruneInfo *part_prune_info;
+ struct PartitionPruneInfo *part_prune_info pg_node_attr(query_jumble_ignore);
} MergeAppend;
/* ----------------
@@ -330,22 +363,22 @@ typedef struct RecursiveUnion
Plan plan;
/* ID of Param representing work table */
- int wtParam;
+ int wtParam pg_node_attr(query_jumble_ignore);
/* Remaining fields are zero/null in UNION ALL case */
/* number of columns to check for duplicate-ness */
- int numCols;
+ int numCols pg_node_attr(query_jumble_ignore);
/* their indexes in the target list */
- AttrNumber *dupColIdx pg_node_attr(array_size(numCols));
+ AttrNumber *dupColIdx pg_node_attr(array_size(numCols), query_jumble_ignore);
/* equality operators to compare with */
- Oid *dupOperators pg_node_attr(array_size(numCols));
- Oid *dupCollations pg_node_attr(array_size(numCols));
+ Oid *dupOperators pg_node_attr(array_size(numCols), query_jumble_ignore);
+ Oid *dupCollations pg_node_attr(array_size(numCols), query_jumble_ignore);
/* estimated number of groups in input */
- long numGroups;
+ long numGroups pg_node_attr(query_jumble_ignore);
} RecursiveUnion;
/* ----------------
@@ -359,7 +392,7 @@ typedef struct RecursiveUnion
typedef struct BitmapAnd
{
Plan plan;
- List *bitmapplans;
+ List *bitmapplans pg_node_attr(query_jumble_ignore);
} BitmapAnd;
/* ----------------
@@ -373,8 +406,8 @@ typedef struct BitmapAnd
typedef struct BitmapOr
{
Plan plan;
- bool isshared;
- List *bitmapplans;
+ bool isshared pg_node_attr(query_jumble_ignore);
+ List *bitmapplans pg_node_attr(query_jumble_ignore);
} BitmapOr;
/*
@@ -389,7 +422,8 @@ typedef struct Scan
pg_node_attr(abstract)
Plan plan;
- Index scanrelid; /* relid is index into the range table */
+ Index scanrelid pg_node_attr(query_jumble_rt_index); /* relid is index into
+ * the range table */
} Scan;
/* ----------------
@@ -454,9 +488,11 @@ typedef struct IndexScan
Scan scan;
Oid indexid; /* OID of index to scan */
List *indexqual; /* list of index quals (usually OpExprs) */
- List *indexqualorig; /* the same in original form */
+ List *indexqualorig pg_node_attr(query_jumble_ignore); /* the same in original
+ * form */
List *indexorderby; /* list of index ORDER BY exprs */
- List *indexorderbyorig; /* the same in original form */
+ List *indexorderbyorig pg_node_attr(query_jumble_ignore); /* the same in original
+ * form */
List *indexorderbyops; /* OIDs of sort ops for ORDER BY exprs */
ScanDirection indexorderdir; /* forward or backward or don't care */
} IndexScan;
@@ -497,9 +533,12 @@ typedef struct IndexOnlyScan
Scan scan;
Oid indexid; /* OID of index to scan */
List *indexqual; /* list of index quals (usually OpExprs) */
- List *recheckqual; /* index quals in recheckable form */
+ List *recheckqual pg_node_attr(query_jumble_ignore); /* index quals in
+ * recheckable form */
List *indexorderby; /* list of index ORDER BY exprs */
- List *indextlist; /* TargetEntry list describing index's cols */
+ List *indextlist pg_node_attr(query_jumble_ignore); /* TargetEntry list
+ * describing index's
+ * cols */
ScanDirection indexorderdir; /* forward or backward or don't care */
} IndexOnlyScan;
@@ -524,9 +563,11 @@ typedef struct BitmapIndexScan
{
Scan scan;
Oid indexid; /* OID of index to scan */
- bool isshared; /* Create shared bitmap if set */
+ bool isshared pg_node_attr(query_jumble_ignore); /* Create shared bitmap
+ * if set */
List *indexqual; /* list of index quals (OpExprs) */
- List *indexqualorig; /* the same in original form */
+ List *indexqualorig pg_node_attr(query_jumble_ignore); /* the same in original
+ * form */
} BitmapIndexScan;
/* ----------------
@@ -541,7 +582,8 @@ typedef struct BitmapIndexScan
typedef struct BitmapHeapScan
{
Scan scan;
- List *bitmapqualorig; /* index quals, in standard expr form */
+ List *bitmapqualorig pg_node_attr(query_jumble_ignore); /* index quals, in
+ * standard expr form */
} BitmapHeapScan;
/* ----------------
@@ -601,8 +643,8 @@ typedef enum SubqueryScanStatus
typedef struct SubqueryScan
{
Scan scan;
- Plan *subplan;
- SubqueryScanStatus scanstatus;
+ Plan *subplan pg_node_attr(query_jumble_ignore);
+ SubqueryScanStatus scanstatus pg_node_attr(query_jumble_ignore);
} SubqueryScan;
/* ----------------
@@ -643,8 +685,11 @@ typedef struct TableFuncScan
typedef struct CteScan
{
Scan scan;
- int ctePlanId; /* ID of init SubPlan for CTE */
- int cteParam; /* ID of Param representing CTE output */
+ int ctePlanId pg_node_attr(query_jumble_ignore); /* ID of init SubPlan
+ * for CTE */
+ int cteParam pg_node_attr(query_jumble_ignore); /* ID of Param
+ * representing CTE
+ * output */
} CteScan;
/* ----------------
@@ -664,7 +709,9 @@ typedef struct NamedTuplestoreScan
typedef struct WorkTableScan
{
Scan scan;
- int wtParam; /* ID of Param representing work table */
+ int wtParam pg_node_attr(query_jumble_ignore); /* ID of Param
+ * representing work
+ * table */
} WorkTableScan;
/* ----------------
@@ -711,17 +758,26 @@ typedef struct ForeignScan
{
Scan scan;
CmdType operation; /* SELECT/INSERT/UPDATE/DELETE */
- Index resultRelation; /* direct modification target's RT index */
- Oid checkAsUser; /* user to perform the scan as; 0 means to
- * check as current user */
+ Index resultRelation pg_node_attr(query_jumble_ignore); /* direct modification
+ * target's RT index */
+ Oid checkAsUser pg_node_attr(query_jumble_ignore); /* user to perform the
+ * scan as; 0 means to
+ * check as current user */
Oid fs_server; /* OID of foreign server */
- List *fdw_exprs; /* expressions that FDW may evaluate */
- List *fdw_private; /* private data for FDW */
- List *fdw_scan_tlist; /* optional tlist describing scan tuple */
- List *fdw_recheck_quals; /* original quals not in scan.plan.qual */
- Bitmapset *fs_relids; /* base+OJ RTIs generated by this scan */
- Bitmapset *fs_base_relids; /* base RTIs generated by this scan */
- bool fsSystemCol; /* true if any "system column" is needed */
+ List *fdw_exprs pg_node_attr(query_jumble_ignore); /* expressions that FDW
+ * may evaluate */
+ List *fdw_private pg_node_attr(query_jumble_ignore); /* private data for FDW */
+ List *fdw_scan_tlist pg_node_attr(query_jumble_ignore); /* optional tlist
+ * describing scan tuple */
+ List *fdw_recheck_quals pg_node_attr(query_jumble_ignore); /* original quals not in
+ * scan.plan.qual */
+ Bitmapset *fs_relids pg_node_attr(query_jumble_ignore); /* base+OJ RTIs
+ * generated by this
+ * scan */
+ Bitmapset *fs_base_relids pg_node_attr(query_jumble_ignore); /* base RTIs generated
+ * by this scan */
+ bool fsSystemCol pg_node_attr(query_jumble_ignore); /* true if any "system
+ * column" is needed */
} ForeignScan;
/* ----------------
@@ -742,20 +798,27 @@ struct CustomScanMethods;
typedef struct CustomScan
{
Scan scan;
- uint32 flags; /* mask of CUSTOMPATH_* flags, see
- * nodes/extensible.h */
- List *custom_plans; /* list of Plan nodes, if any */
- List *custom_exprs; /* expressions that custom code may evaluate */
- List *custom_private; /* private data for custom code */
- List *custom_scan_tlist; /* optional tlist describing scan tuple */
- Bitmapset *custom_relids; /* RTIs generated by this scan */
+ uint32 flags pg_node_attr(query_jumble_ignore); /* mask of CUSTOMPATH_*
+ * flags, see
+ * nodes/extensible.h */
+ List *custom_plans pg_node_attr(query_jumble_ignore); /* list of Plan nodes,
+ * if any */
+ List *custom_exprs pg_node_attr(query_jumble_ignore); /* expressions that
+ * custom code may
+ * evaluate */
+ List *custom_private pg_node_attr(query_jumble_ignore); /* private data for
+ * custom code */
+ List *custom_scan_tlist pg_node_attr(query_jumble_ignore); /* optional tlist
+ * describing scan tuple */
+ Bitmapset *custom_relids pg_node_attr(query_jumble_ignore); /* RTIs generated by
+ * this scan */
/*
* NOTE: The method field of CustomScan is required to be a pointer to a
* static table of callback functions. So we don't copy the table itself,
* just reference the original one.
*/
- const struct CustomScanMethods *methods;
+ const struct CustomScanMethods *methods pg_node_attr(query_jumble_ignore);
} CustomScan;
/*
@@ -792,7 +855,7 @@ typedef struct Join
Plan plan;
JoinType jointype;
- bool inner_unique;
+ bool inner_unique pg_node_attr(query_jumble_ignore);
List *joinqual; /* JOIN quals (in addition to plan.qual) */
} Join;
@@ -815,7 +878,7 @@ typedef struct NestLoop
typedef struct NestLoopParam
{
- pg_node_attr(no_equal, no_query_jumble)
+ pg_node_attr(no_equal)
NodeTag type;
int paramno; /* number of the PARAM_EXEC Param to set */
@@ -838,7 +901,7 @@ typedef struct MergeJoin
Join join;
/* Can we skip mark/restore calls? */
- bool skip_mark_restore;
+ bool skip_mark_restore pg_node_attr(query_jumble_ignore);
/* mergeclauses as expression trees */
List *mergeclauses;
@@ -894,13 +957,13 @@ typedef struct Memoize
Plan plan;
/* size of the two arrays below */
- int numKeys;
+ int numKeys pg_node_attr(query_jumble_ignore);
/* hash operators for each key */
- Oid *hashOperators pg_node_attr(array_size(numKeys));
+ Oid *hashOperators pg_node_attr(array_size(numKeys), query_jumble_ignore);
/* collations for each key */
- Oid *collations pg_node_attr(array_size(numKeys));
+ Oid *collations pg_node_attr(array_size(numKeys), query_jumble_ignore);
/* cache keys in the form of exprs containing parameters */
List *param_exprs;
@@ -909,7 +972,7 @@ typedef struct Memoize
* true if the cache entry should be marked as complete after we store the
* first tuple in it.
*/
- bool singlerow;
+ bool singlerow pg_node_attr(query_jumble_ignore);
/*
* true when cache key should be compared bit by bit, false when using
@@ -921,10 +984,10 @@ typedef struct Memoize
* The maximum number of entries that the planner expects will fit in the
* cache, or 0 if unknown
*/
- uint32 est_entries;
+ uint32 est_entries pg_node_attr(query_jumble_ignore);
/* paramids from param_exprs */
- Bitmapset *keyparamids;
+ Bitmapset *keyparamids pg_node_attr(query_jumble_ignore);
} Memoize;
/* ----------------
@@ -1007,31 +1070,31 @@ typedef struct Agg
AggSplit aggsplit;
/* number of grouping columns */
- int numCols;
+ int numCols pg_node_attr(query_jumble_ignore);
/* their indexes in the target list */
- AttrNumber *grpColIdx pg_node_attr(array_size(numCols));
+ AttrNumber *grpColIdx pg_node_attr(array_size(numCols), query_jumble_ignore);
/* equality operators to compare with */
- Oid *grpOperators pg_node_attr(array_size(numCols));
- Oid *grpCollations pg_node_attr(array_size(numCols));
+ Oid *grpOperators pg_node_attr(array_size(numCols), query_jumble_ignore);
+ Oid *grpCollations pg_node_attr(array_size(numCols), query_jumble_ignore);
/* estimated number of groups in input */
- long numGroups;
+ long numGroups pg_node_attr(query_jumble_ignore);
/* for pass-by-ref transition data */
- uint64 transitionSpace;
+ uint64 transitionSpace pg_node_attr(query_jumble_ignore);
/* IDs of Params used in Aggref inputs */
- Bitmapset *aggParams;
+ Bitmapset *aggParams pg_node_attr(query_jumble_ignore);
/* Note: planner provides numGroups & aggParams only in HASHED/MIXED case */
/* grouping sets to use */
- List *groupingSets;
+ List *groupingSets pg_node_attr(query_jumble_ignore);
/* chained Agg/Sort nodes */
- List *chain;
+ List *chain pg_node_attr(query_jumble_ignore);
} Agg;
/* ----------------
@@ -1043,43 +1106,43 @@ typedef struct WindowAgg
Plan plan;
/* ID referenced by window functions */
- Index winref;
+ Index winref pg_node_attr(query_jumble_ignore);
/* number of columns in partition clause */
- int partNumCols;
+ int partNumCols pg_node_attr(query_jumble_ignore);
/* their indexes in the target list */
- AttrNumber *partColIdx pg_node_attr(array_size(partNumCols));
+ AttrNumber *partColIdx pg_node_attr(array_size(partNumCols), query_jumble_ignore);
/* equality operators for partition columns */
- Oid *partOperators pg_node_attr(array_size(partNumCols));
+ Oid *partOperators pg_node_attr(array_size(partNumCols), query_jumble_ignore);
/* collations for partition columns */
- Oid *partCollations pg_node_attr(array_size(partNumCols));
+ Oid *partCollations pg_node_attr(array_size(partNumCols), query_jumble_ignore);
/* number of columns in ordering clause */
- int ordNumCols;
+ int ordNumCols pg_node_attr(query_jumble_ignore);
/* their indexes in the target list */
- AttrNumber *ordColIdx pg_node_attr(array_size(ordNumCols));
+ AttrNumber *ordColIdx pg_node_attr(array_size(ordNumCols), query_jumble_ignore);
/* equality operators for ordering columns */
- Oid *ordOperators pg_node_attr(array_size(ordNumCols));
+ Oid *ordOperators pg_node_attr(array_size(ordNumCols), query_jumble_ignore);
/* collations for ordering columns */
- Oid *ordCollations pg_node_attr(array_size(ordNumCols));
+ Oid *ordCollations pg_node_attr(array_size(ordNumCols), query_jumble_ignore);
/* frame_clause options, see WindowDef */
- int frameOptions;
+ int frameOptions pg_node_attr(query_jumble_ignore);
/* expression for starting bound, if any */
- Node *startOffset;
+ Node *startOffset pg_node_attr(query_jumble_ignore);
/* expression for ending bound, if any */
- Node *endOffset;
+ Node *endOffset pg_node_attr(query_jumble_ignore);
/* qual to help short-circuit execution */
- List *runCondition;
+ List *runCondition pg_node_attr(query_jumble_ignore);
/* runCondition for display in EXPLAIN */
List *runConditionOrig;
@@ -1087,25 +1150,25 @@ typedef struct WindowAgg
/* these fields are used with RANGE offset PRECEDING/FOLLOWING: */
/* in_range function for startOffset */
- Oid startInRangeFunc;
+ Oid startInRangeFunc pg_node_attr(query_jumble_ignore);
/* in_range function for endOffset */
- Oid endInRangeFunc;
+ Oid endInRangeFunc pg_node_attr(query_jumble_ignore);
/* collation for in_range tests */
- Oid inRangeColl;
+ Oid inRangeColl pg_node_attr(query_jumble_ignore);
/* use ASC sort order for in_range tests? */
- bool inRangeAsc;
+ bool inRangeAsc pg_node_attr(query_jumble_ignore);
/* nulls sort first for in_range tests? */
- bool inRangeNullsFirst;
+ bool inRangeNullsFirst pg_node_attr(query_jumble_ignore);
/*
* false for all apart from the WindowAgg that's closest to the root of
* the plan
*/
- bool topWindow;
+ bool topWindow pg_node_attr(query_jumble_ignore);
} WindowAgg;
/* ----------------
@@ -1117,16 +1180,16 @@ typedef struct Unique
Plan plan;
/* number of columns to check for uniqueness */
- int numCols;
+ int numCols pg_node_attr(query_jumble_ignore);
/* their indexes in the target list */
- AttrNumber *uniqColIdx pg_node_attr(array_size(numCols));
+ AttrNumber *uniqColIdx pg_node_attr(array_size(numCols), query_jumble_ignore);
/* equality operators to compare with */
- Oid *uniqOperators pg_node_attr(array_size(numCols));
+ Oid *uniqOperators pg_node_attr(array_size(numCols), query_jumble_ignore);
/* collations for equality comparisons */
- Oid *uniqCollations pg_node_attr(array_size(numCols));
+ Oid *uniqCollations pg_node_attr(array_size(numCols), query_jumble_ignore);
} Unique;
/* ------------
@@ -1144,11 +1207,19 @@ typedef struct Gather
{
Plan plan;
int num_workers; /* planned number of worker processes */
- int rescan_param; /* ID of Param that signals a rescan, or -1 */
- bool single_copy; /* don't execute plan more than once */
- bool invisible; /* suppress EXPLAIN display (for testing)? */
- Bitmapset *initParam; /* param id's of initplans which are referred
- * at gather or one of it's child node */
+ int rescan_param pg_node_attr(query_jumble_ignore); /* ID of Param that
+ * signals a rescan, or
+ * -1 */
+ bool single_copy pg_node_attr(query_jumble_ignore); /* don't execute plan
+ * more than once */
+ bool invisible pg_node_attr(query_jumble_ignore); /* suppress EXPLAIN
+ * display (for
+ * testing)? */
+ Bitmapset *initParam pg_node_attr(query_jumble_ignore); /* param id's of
+ * initplans which are
+ * referred at gather or
+ * one of it's child
+ * node */
} Gather;
/* ------------
@@ -1163,30 +1234,30 @@ typedef struct GatherMerge
int num_workers;
/* ID of Param that signals a rescan, or -1 */
- int rescan_param;
+ int rescan_param pg_node_attr(query_jumble_ignore);
/* remaining fields are just like the sort-key info in struct Sort */
/* number of sort-key columns */
- int numCols;
+ int numCols pg_node_attr(query_jumble_ignore);
/* their indexes in the target list */
- AttrNumber *sortColIdx pg_node_attr(array_size(numCols));
+ AttrNumber *sortColIdx pg_node_attr(array_size(numCols), query_jumble_ignore);
/* OIDs of operators to sort them by */
- Oid *sortOperators pg_node_attr(array_size(numCols));
+ Oid *sortOperators pg_node_attr(array_size(numCols), query_jumble_ignore);
/* OIDs of collations */
- Oid *collations pg_node_attr(array_size(numCols));
+ Oid *collations pg_node_attr(array_size(numCols), query_jumble_ignore);
/* NULLS FIRST/LAST directions */
- bool *nullsFirst pg_node_attr(array_size(numCols));
+ bool *nullsFirst pg_node_attr(array_size(numCols), query_jumble_ignore);
/*
* param id's of initplans which are referred at gather merge or one of
* it's child node
*/
- Bitmapset *initParam;
+ Bitmapset *initParam pg_node_attr(query_jumble_ignore);
} GatherMerge;
/* ----------------
@@ -1206,11 +1277,16 @@ typedef struct Hash
* needed to put them into the hashtable.
*/
List *hashkeys; /* hash keys for the hashjoin condition */
- Oid skewTable; /* outer join key's table OID, or InvalidOid */
- AttrNumber skewColumn; /* outer join key's column #, or zero */
- bool skewInherit; /* is outer join rel an inheritance tree? */
+ Oid skewTable pg_node_attr(query_jumble_ignore); /* outer join key's
+ * table OID, or
+ * InvalidOid */
+ AttrNumber skewColumn pg_node_attr(query_jumble_ignore); /* outer join key's
+ * column #, or zero */
+ bool skewInherit pg_node_attr(query_jumble_ignore); /* is outer join rel an
+ * inheritance tree? */
/* all other info is in the parent HashJoin node */
- Cardinality rows_total; /* estimate total rows if parallel_aware */
+ Cardinality rows_total pg_node_attr(query_jumble_ignore); /* estimate total rows
+ * if parallel_aware */
} Hash;
/* ----------------
@@ -1228,20 +1304,20 @@ typedef struct SetOp
SetOpStrategy strategy;
/* number of columns to compare */
- int numCols;
+ int numCols pg_node_attr(query_jumble_ignore);
/* their indexes in the target list */
- AttrNumber *cmpColIdx pg_node_attr(array_size(numCols));
+ AttrNumber *cmpColIdx pg_node_attr(array_size(numCols), query_jumble_ignore);
/* comparison operators (either equality operators or sort operators) */
- Oid *cmpOperators pg_node_attr(array_size(numCols));
- Oid *cmpCollations pg_node_attr(array_size(numCols));
+ Oid *cmpOperators pg_node_attr(array_size(numCols), query_jumble_ignore);
+ Oid *cmpCollations pg_node_attr(array_size(numCols), query_jumble_ignore);
/* nulls-first flags if sorting, otherwise not interesting */
- bool *cmpNullsFirst pg_node_attr(array_size(numCols));
+ bool *cmpNullsFirst pg_node_attr(array_size(numCols), query_jumble_ignore);
/* estimated number of groups in left input */
- long numGroups;
+ long numGroups pg_node_attr(query_jumble_ignore);
} SetOp;
/* ----------------
@@ -1256,8 +1332,10 @@ typedef struct SetOp
typedef struct LockRows
{
Plan plan;
- List *rowMarks; /* a list of PlanRowMark's */
- int epqParam; /* ID of Param for EvalPlanQual re-eval */
+ List *rowMarks pg_node_attr(query_jumble_ignore); /* a list of
+ * PlanRowMark's */
+ int epqParam pg_node_attr(query_jumble_ignore); /* ID of Param for
+ * EvalPlanQual re-eval */
} LockRows;
/* ----------------
@@ -1272,25 +1350,25 @@ typedef struct Limit
Plan plan;
/* OFFSET parameter, or NULL if none */
- Node *limitOffset;
+ Node *limitOffset pg_node_attr(query_jumble_ignore);
/* COUNT parameter, or NULL if none */
- Node *limitCount;
+ Node *limitCount pg_node_attr(query_jumble_ignore);
/* limit type */
- LimitOption limitOption;
+ LimitOption limitOption pg_node_attr(query_jumble_ignore);
/* number of columns to check for similarity */
- int uniqNumCols;
+ int uniqNumCols pg_node_attr(query_jumble_ignore);
/* their indexes in the target list */
- AttrNumber *uniqColIdx pg_node_attr(array_size(uniqNumCols));
+ AttrNumber *uniqColIdx pg_node_attr(array_size(uniqNumCols), query_jumble_ignore);
/* equality operators to compare with */
- Oid *uniqOperators pg_node_attr(array_size(uniqNumCols));
+ Oid *uniqOperators pg_node_attr(array_size(uniqNumCols), query_jumble_ignore);
/* collations for equality comparisons */
- Oid *uniqCollations pg_node_attr(array_size(uniqNumCols));
+ Oid *uniqCollations pg_node_attr(array_size(uniqNumCols), query_jumble_ignore);
} Limit;
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
index 59e7bb26bb..c09785a072 100644
--- a/src/include/nodes/primnodes.h
+++ b/src/include/nodes/primnodes.h
@@ -1074,8 +1074,6 @@ typedef struct SubLink
*/
typedef struct SubPlan
{
- pg_node_attr(no_query_jumble)
-
Expr xpr;
/* Fields copied from original SubLink: */
SubLinkType subLinkType; /* see above */
@@ -1106,8 +1104,9 @@ typedef struct SubPlan
List *parParam; /* indices of input Params from parent plan */
List *args; /* exprs to pass as parParam values */
/* Estimated execution costs: */
- Cost startup_cost; /* one-time setup cost */
- Cost per_call_cost; /* cost for each subplan evaluation */
+ Cost startup_cost pg_node_attr(query_jumble_ignore); /* one-time setup cost */
+ Cost per_call_cost pg_node_attr(query_jumble_ignore); /* cost for each subplan
+ * evaluation */
} SubPlan;
/*
diff --git a/src/include/nodes/queryjumble.h b/src/include/nodes/queryjumble.h
index 5afa6f3605..6c3b787b46 100644
--- a/src/include/nodes/queryjumble.h
+++ b/src/include/nodes/queryjumble.h
@@ -15,6 +15,7 @@
#define QUERYJUMBLE_H
#include "nodes/parsenodes.h"
+#include "nodes/pathnodes.h"
/*
* Struct for tracking locations/lengths of constants during normalization
@@ -48,6 +49,9 @@ typedef struct JumbleState
/* highest Param id we've seen, in order to start normalization correctly */
int highest_extern_param_id;
+
+ /* planner global info for resolving RT indexes when plan jumbling */
+ PlannerGlobal *glob;
} JumbleState;
/* Values for the compute_query_id GUC */
@@ -59,15 +63,27 @@ enum ComputeQueryIdType
COMPUTE_QUERY_ID_REGRESS,
};
+/* Values for the compute_plan_id GUC */
+enum ComputePlanIdType
+{
+ COMPUTE_PLAN_ID_OFF,
+ COMPUTE_PLAN_ID_ON,
+ COMPUTE_PLAN_ID_AUTO,
+ COMPUTE_PLAN_ID_REGRESS,
+};
+
/* GUC parameters */
extern PGDLLIMPORT int compute_query_id;
+extern PGDLLIMPORT int compute_plan_id;
extern const char *CleanQuerytext(const char *query, int *location, int *len);
extern JumbleState *JumbleQuery(Query *query);
extern void EnableQueryId(void);
+extern void EnablePlanId(void);
extern PGDLLIMPORT bool query_id_enabled;
+extern PGDLLIMPORT bool plan_id_enabled;
/*
* Returns whether query identifier computation has been enabled, either
@@ -83,8 +99,22 @@ IsQueryIdEnabled(void)
return query_id_enabled;
}
-/* Functions intended for other users of jumbling (e.g. plan jumbling) */
-extern JumbleState *InitializeJumbleState(bool record_clocations);
+/*
+ * Returns whether plan identifier computation has been enabled, either
+ * directly in the GUC or by a module when the setting is 'auto'.
+ */
+static inline bool
+IsPlanIdEnabled(void)
+{
+ if (compute_plan_id == COMPUTE_PLAN_ID_OFF)
+ return false;
+ if (compute_plan_id == COMPUTE_PLAN_ID_ON)
+ return true;
+ return plan_id_enabled;
+}
+
+/* Functions called for plan jumbling or extensions doing their own jumbling */
+extern JumbleState *InitializeJumbleState(bool record_clocations, PlannerGlobal *glob);
extern void AppendJumble(JumbleState *jstate, const unsigned char *item, Size size);
extern void JumbleNode(JumbleState *jstate, Node *node);
extern uint64 HashJumbleState(JumbleState *jstate);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index d3d4ff6c5c..437a4cec5b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -170,6 +170,9 @@ typedef struct PgBackendStatus
/* query identifier, optionally computed using post_parse_analyze_hook */
uint64 st_query_id;
+
+ /* plan identifier, optionally computed after planning */
+ uint64 st_plan_id;
} PgBackendStatus;
@@ -316,6 +319,7 @@ extern void pgstat_clear_backend_activity_snapshot(void);
/* Activity reporting functions */
extern void pgstat_report_activity(BackendState state, const char *cmd_str);
extern void pgstat_report_query_id(uint64 query_id, bool force);
+extern void pgstat_report_plan_id(uint64 query_id, uint64 plan_id, bool force);
extern void pgstat_report_tempfile(size_t filesize);
extern void pgstat_report_appname(const char *appname);
extern void pgstat_report_xact_timestamp(TimestampTz tstamp);
@@ -323,6 +327,7 @@ extern const char *pgstat_get_backend_current_activity(int pid, bool checkUser);
extern const char *pgstat_get_crashed_backend_activity(int pid, char *buffer,
int buflen);
extern uint64 pgstat_get_my_query_id(void);
+extern uint64 pgstat_get_my_plan_id(void);
extern BackendType pgstat_get_backend_type_by_proc_number(ProcNumber procNumber);
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index ee31e41d50..8bfa3c1a5f 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -699,6 +699,17 @@ select explain_filter('explain (verbose) create table test_ctas as select 1');
Query Identifier: N
(3 rows)
+-- Test compute_plan_id
+set compute_plan_id = on;
+select explain_filter('explain (verbose) select * from int8_tbl i8');
+ explain_filter
+----------------------------------------------------------------
+ Seq Scan on public.int8_tbl i8 (cost=N.N..N.N rows=N width=N)
+ Output: q1, q2
+ Query Identifier: N
+ Plan Identifier: N
+(4 rows)
+
-- Test SERIALIZE option
select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
explain_filter
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 856a8349c5..e20cc7d28c 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1760,9 +1760,10 @@ pg_stat_activity| SELECT s.datid,
s.backend_xid,
s.backend_xmin,
s.query_id,
+ s.plan_id,
s.query,
s.backend_type
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
LEFT JOIN pg_database d ON ((s.datid = d.oid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_all_indexes| SELECT c.oid AS relid,
@@ -1886,7 +1887,7 @@ pg_stat_gssapi| SELECT pid,
gss_princ AS principal,
gss_enc AS encrypted,
gss_delegation AS credentials_delegated
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
WHERE (client_port IS NOT NULL);
pg_stat_io| SELECT backend_type,
object,
@@ -2092,7 +2093,7 @@ pg_stat_replication| SELECT s.pid,
w.sync_priority,
w.sync_state,
w.reply_time
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
JOIN pg_stat_get_wal_senders() w(pid, state, sent_lsn, write_lsn, flush_lsn, replay_lsn, write_lag, flush_lag, replay_lag, sync_priority, sync_state, reply_time) ON ((s.pid = w.pid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_replication_slots| SELECT s.slot_name,
@@ -2126,7 +2127,7 @@ pg_stat_ssl| SELECT pid,
ssl_client_dn AS client_dn,
ssl_client_serial AS client_serial,
ssl_issuer_dn AS issuer_dn
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
WHERE (client_port IS NOT NULL);
pg_stat_subscription| SELECT su.oid AS subid,
su.subname,
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index 0bafa87049..d787ad2cda 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -167,6 +167,10 @@ select explain_filter('explain (verbose) select * from int8_tbl i8');
select explain_filter('explain (verbose) declare test_cur cursor for select * from int8_tbl');
select explain_filter('explain (verbose) create table test_ctas as select 1');
+-- Test compute_plan_id
+set compute_plan_id = on;
+select explain_filter('explain (verbose) select * from int8_tbl i8');
+
-- Test SERIALIZE option
select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
select explain_filter('explain (analyze,serialize text,buffers,timing off) select * from int8_tbl i8');
--
2.47.1
v2-0001-Allow-using-jumbling-logic-outside-of-query-jumbl.patchapplication/x-patch; name=v2-0001-Allow-using-jumbling-logic-outside-of-query-jumbl.patchDownload
From 95b05ca1dcd86823833e2b0661781fb04e8859a8 Mon Sep 17 00:00:00 2001
From: Lukas Fittl <lukas@fittl.com>
Date: Tue, 31 Dec 2024 15:05:39 -0800
Subject: [PATCH v2 1/4] Allow using jumbling logic outside of query jumble
unit file
This can be useful either for jumbling expressions in other contexts
(e.g. to calculate a plan jumble), or to allow extensions to use
a modified jumbling logic more easily.
This intentionally supports the use case where a separate jumbling logic
does not care about recording constants, as the query jumble does.
---
src/backend/nodes/queryjumblefuncs.c | 59 +++++++++++++++++-----------
src/include/nodes/queryjumble.h | 6 +++
2 files changed, 41 insertions(+), 24 deletions(-)
diff --git a/src/backend/nodes/queryjumblefuncs.c b/src/backend/nodes/queryjumblefuncs.c
index b103a28193..545d8edcae 100644
--- a/src/backend/nodes/queryjumblefuncs.c
+++ b/src/backend/nodes/queryjumblefuncs.c
@@ -51,10 +51,7 @@ int compute_query_id = COMPUTE_QUERY_ID_AUTO;
*/
bool query_id_enabled = false;
-static void AppendJumble(JumbleState *jstate,
- const unsigned char *item, Size size);
static void RecordConstLocation(JumbleState *jstate, int location);
-static void _jumbleNode(JumbleState *jstate, Node *node);
static void _jumbleA_Const(JumbleState *jstate, Node *node);
static void _jumbleList(JumbleState *jstate, Node *node);
static void _jumbleVariableSetStmt(JumbleState *jstate, Node *node);
@@ -109,28 +106,42 @@ CleanQuerytext(const char *query, int *location, int *len)
}
JumbleState *
-JumbleQuery(Query *query)
+InitializeJumbleState(bool record_clocations)
{
- JumbleState *jstate = NULL;
-
- Assert(IsQueryIdEnabled());
-
- jstate = (JumbleState *) palloc(sizeof(JumbleState));
+ JumbleState *jstate = (JumbleState *) palloc0(sizeof(JumbleState));
/* Set up workspace for query jumbling */
jstate->jumble = (unsigned char *) palloc(JUMBLE_SIZE);
jstate->jumble_len = 0;
- jstate->clocations_buf_size = 32;
- jstate->clocations = (LocationLen *)
- palloc(jstate->clocations_buf_size * sizeof(LocationLen));
- jstate->clocations_count = 0;
- jstate->highest_extern_param_id = 0;
+
+ if (record_clocations)
+ {
+ jstate->clocations_buf_size = 32;
+ jstate->clocations = (LocationLen *)
+ palloc(jstate->clocations_buf_size * sizeof(LocationLen));
+ }
+
+ return jstate;
+}
+
+uint64
+HashJumbleState(JumbleState *jstate)
+{
+ return DatumGetUInt64(hash_any_extended(jstate->jumble,
+ jstate->jumble_len,
+ 0));
+}
+
+JumbleState *
+JumbleQuery(Query *query)
+{
+ JumbleState *jstate = InitializeJumbleState(true);
+
+ Assert(IsQueryIdEnabled());
/* Compute query ID and mark the Query node with it */
- _jumbleNode(jstate, (Node *) query);
- query->queryId = DatumGetUInt64(hash_any_extended(jstate->jumble,
- jstate->jumble_len,
- 0));
+ JumbleNode(jstate, (Node *) query);
+ query->queryId = HashJumbleState(jstate);
/*
* If we are unlucky enough to get a hash of zero, use 1 instead for
@@ -164,7 +175,7 @@ EnableQueryId(void)
* AppendJumble: Append a value that is substantive in a given query to
* the current jumble.
*/
-static void
+void
AppendJumble(JumbleState *jstate, const unsigned char *item, Size size)
{
unsigned char *jumble = jstate->jumble;
@@ -205,7 +216,7 @@ static void
RecordConstLocation(JumbleState *jstate, int location)
{
/* -1 indicates unknown or undefined location */
- if (location >= 0)
+ if (location >= 0 && jstate->clocations_buf_size > 0)
{
/* enlarge array if needed */
if (jstate->clocations_count >= jstate->clocations_buf_size)
@@ -224,7 +235,7 @@ RecordConstLocation(JumbleState *jstate, int location)
}
#define JUMBLE_NODE(item) \
- _jumbleNode(jstate, (Node *) expr->item)
+ JumbleNode(jstate, (Node *) expr->item)
#define JUMBLE_LOCATION(location) \
RecordConstLocation(jstate, expr->location)
#define JUMBLE_FIELD(item) \
@@ -239,8 +250,8 @@ do { \
#include "queryjumblefuncs.funcs.c"
-static void
-_jumbleNode(JumbleState *jstate, Node *node)
+void
+JumbleNode(JumbleState *jstate, Node *node)
{
Node *expr = node;
@@ -305,7 +316,7 @@ _jumbleList(JumbleState *jstate, Node *node)
{
case T_List:
foreach(l, expr)
- _jumbleNode(jstate, lfirst(l));
+ JumbleNode(jstate, lfirst(l));
break;
case T_IntList:
foreach(l, expr)
diff --git a/src/include/nodes/queryjumble.h b/src/include/nodes/queryjumble.h
index 50eb956658..5afa6f3605 100644
--- a/src/include/nodes/queryjumble.h
+++ b/src/include/nodes/queryjumble.h
@@ -83,4 +83,10 @@ IsQueryIdEnabled(void)
return query_id_enabled;
}
+/* Functions intended for other users of jumbling (e.g. plan jumbling) */
+extern JumbleState *InitializeJumbleState(bool record_clocations);
+extern void AppendJumble(JumbleState *jstate, const unsigned char *item, Size size);
+extern void JumbleNode(JumbleState *jstate, Node *node);
+extern uint64 HashJumbleState(JumbleState *jstate);
+
#endif /* QUERYJUMBLE_H */
--
2.47.1
v2-0002-Cumulative-statistics-Add-pgstat_drop_entries_of_.patchapplication/x-patch; name=v2-0002-Cumulative-statistics-Add-pgstat_drop_entries_of_.patchDownload
From c3272a0012d897eaadc72f50e3e336e76671f92e Mon Sep 17 00:00:00 2001
From: Lukas Fittl <lukas@fittl.com>
Date: Thu, 2 Jan 2025 10:46:30 -0800
Subject: [PATCH v2 2/4] Cumulative statistics: Add pgstat_drop_entries_of_kind
helper
This allows users of the cumulative statistics systems to drop all
entries for a given kind, similar to how pgstat_reset_entries_of_kind
allows resetting all entries for a given statistics kind.
Add an example use of pgstat_drop_entries_of_kind in the injection
points module by adding a injection_points_stats_reset function.
---
src/backend/utils/activity/pgstat_shmem.c | 33 +++++++++++++++++++
src/include/utils/pgstat_internal.h | 1 +
.../injection_points--1.0.sql | 10 ++++++
.../injection_points/injection_stats.c | 12 +++++++
.../modules/injection_points/t/001_stats.pl | 13 ++++++++
5 files changed, 69 insertions(+)
diff --git a/src/backend/utils/activity/pgstat_shmem.c b/src/backend/utils/activity/pgstat_shmem.c
index 342586397d..7b63bb3903 100644
--- a/src/backend/utils/activity/pgstat_shmem.c
+++ b/src/backend/utils/activity/pgstat_shmem.c
@@ -1015,6 +1015,39 @@ pgstat_drop_all_entries(void)
pgstat_request_entry_refs_gc();
}
+void
+pgstat_drop_entries_of_kind(PgStat_Kind kind)
+{
+ dshash_seq_status hstat;
+ PgStatShared_HashEntry *ps;
+ uint64 not_freed_count = 0;
+
+ dshash_seq_init(&hstat, pgStatLocal.shared_hash, true);
+ while ((ps = dshash_seq_next(&hstat)) != NULL)
+ {
+ if (ps->dropped || ps->key.kind != kind)
+ continue;
+
+ /* delete local reference */
+ if (pgStatEntryRefHash)
+ {
+ PgStat_EntryRefHashEntry *lohashent =
+ pgstat_entry_ref_hash_lookup(pgStatEntryRefHash, ps->key);
+
+ if (lohashent)
+ pgstat_release_entry_ref(lohashent->key, lohashent->entry_ref,
+ true);
+ }
+
+ if (!pgstat_drop_entry_internal(ps, &hstat))
+ not_freed_count++;
+ }
+ dshash_seq_term(&hstat);
+
+ if (not_freed_count > 0)
+ pgstat_request_entry_refs_gc();
+}
+
static void
shared_stat_reset_contents(PgStat_Kind kind, PgStatShared_Common *header,
TimestampTz ts)
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index a3d39d2b72..fe3e891257 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -718,6 +718,7 @@ extern bool pgstat_lock_entry_shared(PgStat_EntryRef *entry_ref, bool nowait);
extern void pgstat_unlock_entry(PgStat_EntryRef *entry_ref);
extern bool pgstat_drop_entry(PgStat_Kind kind, Oid dboid, uint64 objid);
extern void pgstat_drop_all_entries(void);
+extern void pgstat_drop_entries_of_kind(PgStat_Kind kind);
extern PgStat_EntryRef *pgstat_get_entry_ref_locked(PgStat_Kind kind, Oid dboid, uint64 objid,
bool nowait);
extern void pgstat_reset_entry(PgStat_Kind kind, Oid dboid, uint64 objid, TimestampTz ts);
diff --git a/src/test/modules/injection_points/injection_points--1.0.sql b/src/test/modules/injection_points/injection_points--1.0.sql
index 6c81d55e0d..1d8a085c81 100644
--- a/src/test/modules/injection_points/injection_points--1.0.sql
+++ b/src/test/modules/injection_points/injection_points--1.0.sql
@@ -85,6 +85,16 @@ RETURNS bigint
AS 'MODULE_PATHNAME', 'injection_points_stats_numcalls'
LANGUAGE C STRICT;
+--
+-- injection_points_stats_reset()
+--
+-- Reports statistics, if any, related to the given injection point.
+--
+CREATE FUNCTION injection_points_stats_reset()
+RETURNS void
+AS 'MODULE_PATHNAME', 'injection_points_stats_reset'
+LANGUAGE C STRICT;
+
--
-- injection_points_stats_fixed()
--
diff --git a/src/test/modules/injection_points/injection_stats.c b/src/test/modules/injection_points/injection_stats.c
index 5db62bca66..99468d4b7e 100644
--- a/src/test/modules/injection_points/injection_stats.c
+++ b/src/test/modules/injection_points/injection_stats.c
@@ -197,3 +197,15 @@ injection_points_stats_numcalls(PG_FUNCTION_ARGS)
PG_RETURN_INT64(entry->numcalls);
}
+
+/*
+ * SQL function that resets injection point statistics.
+ */
+PG_FUNCTION_INFO_V1(injection_points_stats_reset);
+Datum
+injection_points_stats_reset(PG_FUNCTION_ARGS)
+{
+ pgstat_drop_entries_of_kind(PGSTAT_KIND_INJECTION);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/test/modules/injection_points/t/001_stats.pl b/src/test/modules/injection_points/t/001_stats.pl
index d4539fe872..dc83367958 100644
--- a/src/test/modules/injection_points/t/001_stats.pl
+++ b/src/test/modules/injection_points/t/001_stats.pl
@@ -69,6 +69,19 @@ $fixedstats = $node->safe_psql('postgres',
"SELECT * FROM injection_points_stats_fixed();");
is($fixedstats, '0|0|0|0|0', 'fixed stats after crash');
+# On reset the stats are gone
+$node->safe_psql('postgres',
+ "SELECT injection_points_attach('stats-notice', 'notice');");
+$node->safe_psql('postgres', "SELECT injection_points_run('stats-notice');");
+$node->safe_psql('postgres', "SELECT injection_points_run('stats-notice');");
+$numcalls = $node->safe_psql('postgres',
+ "SELECT injection_points_stats_numcalls('stats-notice');");
+is($numcalls, '2', 'number of stats calls');
+$node->safe_psql('postgres', "SELECT injection_points_stats_reset();");
+$numcalls = $node->safe_psql('postgres',
+ "SELECT injection_points_stats_numcalls('stats-notice');");
+is($numcalls, '', 'number of stats after reset');
+
# Stop the server, disable the module, then restart. The server
# should be able to come up.
$node->stop;
--
2.47.1
v2-0004-Add-pg_stat_plans-contrib-extension.patchapplication/x-patch; name=v2-0004-Add-pg_stat_plans-contrib-extension.patchDownload
From a774675fca8c065e3362c1e088b3455629f1f0fe Mon Sep 17 00:00:00 2001
From: Lukas Fittl <lukas@fittl.com>
Date: Thu, 2 Jan 2025 10:47:50 -0800
Subject: [PATCH v2 4/4] Add pg_stat_plans contrib extension
This extension allows tracking per-plan call counts and execution time,
as well as capturing the plan text, aka EXPLAIN (COSTS OFF), for the
first execution of a given plan. This utilize the compute_plan_id
functionality for tracking different plans.
---
contrib/Makefile | 1 +
contrib/meson.build | 1 +
contrib/pg_stat_plans/Makefile | 29 +
contrib/pg_stat_plans/expected/cleanup.out | 1 +
contrib/pg_stat_plans/expected/privileges.out | 125 +++
contrib/pg_stat_plans/expected/select.out | 262 ++++++
contrib/pg_stat_plans/meson.build | 43 +
contrib/pg_stat_plans/pg_stat_plans--1.0.sql | 33 +
contrib/pg_stat_plans/pg_stat_plans.c | 773 ++++++++++++++++++
contrib/pg_stat_plans/pg_stat_plans.conf | 1 +
contrib/pg_stat_plans/pg_stat_plans.control | 5 +
contrib/pg_stat_plans/sql/cleanup.sql | 1 +
contrib/pg_stat_plans/sql/privileges.sql | 59 ++
contrib/pg_stat_plans/sql/select.sql | 67 ++
doc/src/sgml/contrib.sgml | 1 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/pgstatplans.sgml | 425 ++++++++++
17 files changed, 1828 insertions(+)
create mode 100644 contrib/pg_stat_plans/Makefile
create mode 100644 contrib/pg_stat_plans/expected/cleanup.out
create mode 100644 contrib/pg_stat_plans/expected/privileges.out
create mode 100644 contrib/pg_stat_plans/expected/select.out
create mode 100644 contrib/pg_stat_plans/meson.build
create mode 100644 contrib/pg_stat_plans/pg_stat_plans--1.0.sql
create mode 100644 contrib/pg_stat_plans/pg_stat_plans.c
create mode 100644 contrib/pg_stat_plans/pg_stat_plans.conf
create mode 100644 contrib/pg_stat_plans/pg_stat_plans.control
create mode 100644 contrib/pg_stat_plans/sql/cleanup.sql
create mode 100644 contrib/pg_stat_plans/sql/privileges.sql
create mode 100644 contrib/pg_stat_plans/sql/select.sql
create mode 100644 doc/src/sgml/pgstatplans.sgml
diff --git a/contrib/Makefile b/contrib/Makefile
index 952855d9b6..8de010afde 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -34,6 +34,7 @@ SUBDIRS = \
pg_freespacemap \
pg_logicalinspect \
pg_prewarm \
+ pg_stat_plans \
pg_stat_statements \
pg_surgery \
pg_trgm \
diff --git a/contrib/meson.build b/contrib/meson.build
index 1ba73ebd67..0442ec2644 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -49,6 +49,7 @@ subdir('pg_freespacemap')
subdir('pg_logicalinspect')
subdir('pg_prewarm')
subdir('pgrowlocks')
+subdir('pg_stat_plans')
subdir('pg_stat_statements')
subdir('pgstattuple')
subdir('pg_surgery')
diff --git a/contrib/pg_stat_plans/Makefile b/contrib/pg_stat_plans/Makefile
new file mode 100644
index 0000000000..e073db95ed
--- /dev/null
+++ b/contrib/pg_stat_plans/Makefile
@@ -0,0 +1,29 @@
+# contrib/pg_stat_plans/Makefile
+
+MODULE_big = pg_stat_plans
+OBJS = \
+ $(WIN32RES) \
+ pg_stat_plans.o
+
+EXTENSION = pg_stat_plans
+DATA = pg_stat_plans--1.0.sql
+PGFILEDESC = "pg_stat_plans - track per-plan call counts, execution times and EXPLAIN texts"
+
+LDFLAGS_SL += $(filter -lm, $(LIBS))
+
+REGRESS_OPTS = --temp-config $(top_srcdir)/contrib/pg_stat_plans/pg_stat_plans.conf
+REGRESS = select privileges cleanup
+# Disabled because these tests require "shared_preload_libraries=pg_stat_plans",
+# which typical installcheck users do not have (e.g. buildfarm clients).
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/pg_stat_plans
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/pg_stat_plans/expected/cleanup.out b/contrib/pg_stat_plans/expected/cleanup.out
new file mode 100644
index 0000000000..51565617ce
--- /dev/null
+++ b/contrib/pg_stat_plans/expected/cleanup.out
@@ -0,0 +1 @@
+DROP EXTENSION pg_stat_plans;
diff --git a/contrib/pg_stat_plans/expected/privileges.out b/contrib/pg_stat_plans/expected/privileges.out
new file mode 100644
index 0000000000..3e21d6d701
--- /dev/null
+++ b/contrib/pg_stat_plans/expected/privileges.out
@@ -0,0 +1,125 @@
+--
+-- Only superusers and roles with privileges of the pg_read_all_stats role
+-- are allowed to see the plan text, queryid and planid of queries executed by
+-- other users. Other users can see the statistics.
+--
+CREATE ROLE regress_stats_superuser SUPERUSER;
+CREATE ROLE regress_stats_user1;
+CREATE ROLE regress_stats_user2;
+GRANT pg_read_all_stats TO regress_stats_user2;
+SET ROLE regress_stats_superuser;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
+SELECT 1 AS "ONE";
+ ONE
+-----
+ 1
+(1 row)
+
+SET ROLE regress_stats_user1;
+SELECT 1+1 AS "TWO";
+ TWO
+-----
+ 2
+(1 row)
+
+--
+-- A superuser can read all columns of queries executed by others,
+-- including plan text, queryid and planid.
+--
+SET ROLE regress_stats_superuser;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+ rolname | queryid_bool | planid_bool | plan | calls
+-------------------------+--------------+-------------+------------------------------------------------------------------------------------+-------
+ regress_stats_superuser | t | t | Result | 1
+ regress_stats_superuser | t | t | Result | 1
+ regress_stats_superuser | t | t | Sort +| 0
+ | | | Sort Key: pg_authid.rolname, pg_stat_plans.plan COLLATE "C", pg_stat_plans.calls+|
+ | | | -> Hash Join +|
+ | | | Hash Cond: (pg_stat_plans.userid = pg_authid.oid) +|
+ | | | -> Function Scan on pg_stat_plans +|
+ | | | -> Hash +|
+ | | | -> Seq Scan on pg_authid |
+ regress_stats_user1 | t | t | Result | 1
+(4 rows)
+
+--
+-- regress_stats_user1 has no privileges to read the plan text, queryid
+-- or planid of queries executed by others but can see statistics
+-- like calls and rows.
+--
+SET ROLE regress_stats_user1;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+ rolname | queryid_bool | planid_bool | plan | calls
+-------------------------+--------------+-------------+------------------------------------------------------------------------------------+-------
+ regress_stats_superuser | | | <insufficient privilege> | 1
+ regress_stats_superuser | | | <insufficient privilege> | 1
+ regress_stats_superuser | | | <insufficient privilege> | 1
+ regress_stats_user1 | t | t | Result | 1
+ regress_stats_user1 | t | t | Sort +| 0
+ | | | Sort Key: pg_authid.rolname, pg_stat_plans.plan COLLATE "C", pg_stat_plans.calls+|
+ | | | -> Hash Join +|
+ | | | Hash Cond: (pg_stat_plans.userid = pg_authid.oid) +|
+ | | | -> Function Scan on pg_stat_plans +|
+ | | | -> Hash +|
+ | | | -> Seq Scan on pg_authid |
+(5 rows)
+
+--
+-- regress_stats_user2, with pg_read_all_stats role privileges, can
+-- read all columns, including plan text, queryid and planid, of queries
+-- executed by others.
+--
+SET ROLE regress_stats_user2;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+ rolname | queryid_bool | planid_bool | plan | calls
+-------------------------+--------------+-------------+------------------------------------------------------------------------------------+-------
+ regress_stats_superuser | t | t | Result | 1
+ regress_stats_superuser | t | t | Result | 1
+ regress_stats_superuser | t | t | Sort +| 1
+ | | | Sort Key: pg_authid.rolname, pg_stat_plans.plan COLLATE "C", pg_stat_plans.calls+|
+ | | | -> Hash Join +|
+ | | | Hash Cond: (pg_stat_plans.userid = pg_authid.oid) +|
+ | | | -> Function Scan on pg_stat_plans +|
+ | | | -> Hash +|
+ | | | -> Seq Scan on pg_authid |
+ regress_stats_user1 | t | t | Result | 1
+ regress_stats_user1 | t | t | Sort +| 1
+ | | | Sort Key: pg_authid.rolname, pg_stat_plans.plan COLLATE "C", pg_stat_plans.calls+|
+ | | | -> Hash Join +|
+ | | | Hash Cond: (pg_stat_plans.userid = pg_authid.oid) +|
+ | | | -> Function Scan on pg_stat_plans +|
+ | | | -> Hash +|
+ | | | -> Seq Scan on pg_authid |
+ regress_stats_user2 | t | t | Sort +| 0
+ | | | Sort Key: pg_authid.rolname, pg_stat_plans.plan COLLATE "C", pg_stat_plans.calls+|
+ | | | -> Hash Join +|
+ | | | Hash Cond: (pg_stat_plans.userid = pg_authid.oid) +|
+ | | | -> Function Scan on pg_stat_plans +|
+ | | | -> Hash +|
+ | | | -> Seq Scan on pg_authid |
+(6 rows)
+
+--
+-- cleanup
+--
+RESET ROLE;
+DROP ROLE regress_stats_superuser;
+DROP ROLE regress_stats_user1;
+DROP ROLE regress_stats_user2;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
diff --git a/contrib/pg_stat_plans/expected/select.out b/contrib/pg_stat_plans/expected/select.out
new file mode 100644
index 0000000000..906d8ce90d
--- /dev/null
+++ b/contrib/pg_stat_plans/expected/select.out
@@ -0,0 +1,262 @@
+--
+-- SELECT statements
+--
+CREATE EXTENSION pg_stat_plans;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
+--
+-- simple statements
+--
+SELECT 1 FROM pg_class LIMIT 1;
+ ?column?
+----------
+ 1
+(1 row)
+
+SELECT 1 FROM pg_class WHERE relname = 'pg_class';
+ ?column?
+----------
+ 1
+(1 row)
+
+SET enable_indexscan = off;
+SELECT 1 FROM pg_class WHERE relname = 'pg_class';
+ ?column?
+----------
+ 1
+(1 row)
+
+SET enable_indexscan = on;
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+ plan | calls
+--------------------------------------------------------------+-------
+ Bitmap Heap Scan on pg_class +| 1
+ Recheck Cond: (relname = 'pg_class'::name) +|
+ -> Bitmap Index Scan on pg_class_relname_nsp_index +|
+ Index Cond: (relname = 'pg_class'::name) |
+ Index Only Scan using pg_class_relname_nsp_index on pg_class+| 1
+ Index Cond: (relname = 'pg_class'::name) |
+ Limit +| 1
+ -> Seq Scan on pg_class |
+ Result | 1
+ Sort +| 0
+ Sort Key: pg_stat_plans.plan COLLATE "C" +|
+ -> Function Scan on pg_stat_plans |
+(5 rows)
+
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
+--
+-- subplans and CTEs
+--
+WITH x AS MATERIALIZED (SELECT 1)
+SELECT * FROM x;
+ ?column?
+----------
+ 1
+(1 row)
+
+SELECT a.attname,
+ (SELECT pg_catalog.pg_get_expr(d.adbin, d.adrelid)
+ FROM pg_catalog.pg_attrdef d
+ WHERE d.adrelid = a.attrelid AND d.adnum = a.attnum AND a.atthasdef)
+ FROM pg_catalog.pg_attribute a
+ WHERE a.attrelid = 'pg_class'::regclass
+ ORDER BY attnum LIMIT 1;
+ attname | pg_get_expr
+----------+-------------
+ tableoid |
+(1 row)
+
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+ plan | calls
+-------------------------------------------------------------------------------+-------
+ CTE Scan on x +| 1
+ CTE x +|
+ -> Result |
+ Limit +| 1
+ -> Index Scan using pg_attribute_relid_attnum_index on pg_attribute a +|
+ Index Cond: (attrelid = '1259'::oid) +|
+ SubPlan 1 +|
+ -> Result +|
+ One-Time Filter: a.atthasdef +|
+ -> Seq Scan on pg_attrdef d +|
+ Filter: ((adrelid = a.attrelid) AND (adnum = a.attnum)) |
+ Result | 1
+ Sort +| 0
+ Sort Key: pg_stat_plans.plan COLLATE "C" +|
+ -> Function Scan on pg_stat_plans |
+(4 rows)
+
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
+--
+-- partitoning
+--
+create table lp (a char) partition by list (a);
+create table lp_default partition of lp default;
+create table lp_ef partition of lp for values in ('e', 'f');
+create table lp_ad partition of lp for values in ('a', 'd');
+create table lp_bc partition of lp for values in ('b', 'c');
+create table lp_g partition of lp for values in ('g');
+create table lp_null partition of lp for values in (null);
+select * from lp;
+ a
+---
+(0 rows)
+
+select * from lp where a > 'a' and a < 'd';
+ a
+---
+(0 rows)
+
+select * from lp where a > 'a' and a <= 'd';
+ a
+---
+(0 rows)
+
+select * from lp where a = 'a';
+ a
+---
+(0 rows)
+
+select * from lp where 'a' = a; /* commuted */
+ a
+---
+(0 rows)
+
+select * from lp where a is not null;
+ a
+---
+(0 rows)
+
+select * from lp where a is null;
+ a
+---
+(0 rows)
+
+select * from lp where a = 'a' or a = 'c';
+ a
+---
+(0 rows)
+
+select * from lp where a is not null and (a = 'a' or a = 'c');
+ a
+---
+(0 rows)
+
+select * from lp where a <> 'g';
+ a
+---
+(0 rows)
+
+select * from lp where a <> 'a' and a <> 'd';
+ a
+---
+(0 rows)
+
+select * from lp where a not in ('a', 'd');
+ a
+---
+(0 rows)
+
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+ plan | calls
+--------------------------------------------------------------------------------+-------
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ Filter: ((a = 'a'::bpchar) OR (a = 'c'::bpchar)) +|
+ -> Seq Scan on lp_bc lp_2 +|
+ Filter: ((a = 'a'::bpchar) OR (a = 'c'::bpchar)) |
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ Filter: ((a > 'a'::bpchar) AND (a <= 'd'::bpchar)) +|
+ -> Seq Scan on lp_bc lp_2 +|
+ Filter: ((a > 'a'::bpchar) AND (a <= 'd'::bpchar)) +|
+ -> Seq Scan on lp_default lp_3 +|
+ Filter: ((a > 'a'::bpchar) AND (a <= 'd'::bpchar)) |
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ Filter: ((a IS NOT NULL) AND ((a = 'a'::bpchar) OR (a = 'c'::bpchar)))+|
+ -> Seq Scan on lp_bc lp_2 +|
+ Filter: ((a IS NOT NULL) AND ((a = 'a'::bpchar) OR (a = 'c'::bpchar))) |
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ Filter: (a <> 'g'::bpchar) +|
+ -> Seq Scan on lp_bc lp_2 +|
+ Filter: (a <> 'g'::bpchar) +|
+ -> Seq Scan on lp_ef lp_3 +|
+ Filter: (a <> 'g'::bpchar) +|
+ -> Seq Scan on lp_default lp_4 +|
+ Filter: (a <> 'g'::bpchar) |
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ Filter: (a IS NOT NULL) +|
+ -> Seq Scan on lp_bc lp_2 +|
+ Filter: (a IS NOT NULL) +|
+ -> Seq Scan on lp_ef lp_3 +|
+ Filter: (a IS NOT NULL) +|
+ -> Seq Scan on lp_g lp_4 +|
+ Filter: (a IS NOT NULL) +|
+ -> Seq Scan on lp_default lp_5 +|
+ Filter: (a IS NOT NULL) |
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ -> Seq Scan on lp_bc lp_2 +|
+ -> Seq Scan on lp_ef lp_3 +|
+ -> Seq Scan on lp_g lp_4 +|
+ -> Seq Scan on lp_null lp_5 +|
+ -> Seq Scan on lp_default lp_6 |
+ Append +| 1
+ -> Seq Scan on lp_bc lp_1 +|
+ Filter: ((a <> 'a'::bpchar) AND (a <> 'd'::bpchar)) +|
+ -> Seq Scan on lp_ef lp_2 +|
+ Filter: ((a <> 'a'::bpchar) AND (a <> 'd'::bpchar)) +|
+ -> Seq Scan on lp_g lp_3 +|
+ Filter: ((a <> 'a'::bpchar) AND (a <> 'd'::bpchar)) +|
+ -> Seq Scan on lp_default lp_4 +|
+ Filter: ((a <> 'a'::bpchar) AND (a <> 'd'::bpchar)) |
+ Append +| 1
+ -> Seq Scan on lp_bc lp_1 +|
+ Filter: ((a > 'a'::bpchar) AND (a < 'd'::bpchar)) +|
+ -> Seq Scan on lp_default lp_2 +|
+ Filter: ((a > 'a'::bpchar) AND (a < 'd'::bpchar)) |
+ Append +| 1
+ -> Seq Scan on lp_bc lp_1 +|
+ Filter: (a <> ALL ('{a,d}'::bpchar[])) +|
+ -> Seq Scan on lp_ef lp_2 +|
+ Filter: (a <> ALL ('{a,d}'::bpchar[])) +|
+ -> Seq Scan on lp_g lp_3 +|
+ Filter: (a <> ALL ('{a,d}'::bpchar[])) +|
+ -> Seq Scan on lp_default lp_4 +|
+ Filter: (a <> ALL ('{a,d}'::bpchar[])) |
+ Result | 1
+ Seq Scan on lp_ad lp +| 1
+ Filter: ('a'::bpchar = a) |
+ Seq Scan on lp_ad lp +| 1
+ Filter: (a = 'a'::bpchar) |
+ Seq Scan on lp_null lp +| 1
+ Filter: (a IS NULL) |
+ Sort +| 0
+ Sort Key: pg_stat_plans.plan COLLATE "C" +|
+ -> Function Scan on pg_stat_plans |
+(14 rows)
+
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
diff --git a/contrib/pg_stat_plans/meson.build b/contrib/pg_stat_plans/meson.build
new file mode 100644
index 0000000000..3bd884d960
--- /dev/null
+++ b/contrib/pg_stat_plans/meson.build
@@ -0,0 +1,43 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+pg_stat_plans_sources = files(
+ 'pg_stat_plans.c',
+)
+
+if host_system == 'windows'
+ pg_stat_plans_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'pg_stat_plans',
+ '--FILEDESC', 'pg_stat_plans - track per-plan call counts, execution times and EXPLAIN texts',])
+endif
+
+pg_stat_plans = shared_module('pg_stat_plans',
+ pg_stat_plans_sources,
+ kwargs: contrib_mod_args + {
+ 'dependencies': contrib_mod_args['dependencies'],
+ },
+)
+contrib_targets += pg_stat_plans
+
+install_data(
+ 'pg_stat_plans.control',
+ 'pg_stat_plans--1.0.sql',
+ kwargs: contrib_data_args,
+)
+
+tests += {
+ 'name': 'pg_stat_plans',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'regress': {
+ 'sql': [
+ 'select',
+ 'privileges',
+ 'cleanup',
+ ],
+ 'regress_args': ['--temp-config', files('pg_stat_plans.conf')],
+ # Disabled because these tests require
+ # "shared_preload_libraries=pg_stat_plans", which typical
+ # runningcheck users do not have (e.g. buildfarm clients).
+ 'runningcheck': false,
+ }
+}
diff --git a/contrib/pg_stat_plans/pg_stat_plans--1.0.sql b/contrib/pg_stat_plans/pg_stat_plans--1.0.sql
new file mode 100644
index 0000000000..f08452b274
--- /dev/null
+++ b/contrib/pg_stat_plans/pg_stat_plans--1.0.sql
@@ -0,0 +1,33 @@
+/* contrib/pg_stat_plans/pg_stat_plans--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_stat_plans" to load this file. \quit
+
+-- Register functions.
+CREATE FUNCTION pg_stat_plans_reset()
+RETURNS void
+AS 'MODULE_PATHNAME'
+LANGUAGE C PARALLEL SAFE;
+
+CREATE FUNCTION pg_stat_plans(IN showplan boolean,
+ OUT userid oid,
+ OUT dbid oid,
+ OUT toplevel bool,
+ OUT queryid bigint,
+ OUT planid bigint,
+ OUT calls int8,
+ OUT total_exec_time float8,
+ OUT plan text
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stat_plans_1_0'
+LANGUAGE C STRICT VOLATILE PARALLEL SAFE;
+
+-- Register a view on the function for ease of use.
+CREATE VIEW pg_stat_plans AS
+ SELECT * FROM pg_stat_plans(true);
+
+GRANT SELECT ON pg_stat_plans TO PUBLIC;
+
+-- Don't want this to be available to non-superusers.
+REVOKE ALL ON FUNCTION pg_stat_plans_reset() FROM PUBLIC;
diff --git a/contrib/pg_stat_plans/pg_stat_plans.c b/contrib/pg_stat_plans/pg_stat_plans.c
new file mode 100644
index 0000000000..e222b4ebb9
--- /dev/null
+++ b/contrib/pg_stat_plans/pg_stat_plans.c
@@ -0,0 +1,773 @@
+/*--------------------------------------------------------------------------
+ *
+ * pg_stat_plans.c
+ * Track per-plan call counts, execution times and EXPLAIN texts
+ * across a whole database cluster.
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * contrib/pg_stat_plans/pg_stat_plans.c
+ *
+ * -------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/parallel.h"
+#include "catalog/pg_authid.h"
+#include "commands/explain.h"
+#include "common/hashfn.h"
+#include "funcapi.h"
+#include "mb/pg_wchar.h"
+#include "nodes/queryjumble.h"
+#include "pgstat.h"
+#include "optimizer/planner.h"
+#include "utils/acl.h"
+#include "utils/builtins.h"
+#include "utils/guc.h"
+#include "utils/pgstat_internal.h"
+#include "utils/snapmgr.h"
+
+PG_MODULE_MAGIC;
+
+/* Current nesting depth of planner/ExecutorRun/ProcessUtility calls */
+static int nesting_level = 0;
+
+/* Saved hook values */
+static planner_hook_type prev_planner_hook = NULL;
+static ExecutorStart_hook_type prev_ExecutorStart = NULL;
+static ExecutorRun_hook_type prev_ExecutorRun = NULL;
+static ExecutorFinish_hook_type prev_ExecutorFinish = NULL;
+static ExecutorEnd_hook_type prev_ExecutorEnd = NULL;
+
+/*---- GUC variables ----*/
+
+typedef enum
+{
+ PGSP_TRACK_NONE, /* track no plans */
+ PGSP_TRACK_TOP, /* only plans for top level statements */
+ PGSP_TRACK_ALL, /* all plans, including for nested statements */
+} PGSPTrackLevel;
+
+static const struct config_enum_entry track_options[] =
+{
+ {"none", PGSP_TRACK_NONE, false},
+ {"top", PGSP_TRACK_TOP, false},
+ {"all", PGSP_TRACK_ALL, false},
+ {NULL, 0, false}
+};
+
+static int pgsp_max = 5000; /* max # plans to track */
+static int pgsp_max_size = 2048; /* max size of plan text to track (in
+ * bytes) */
+static int pgsp_track = PGSP_TRACK_TOP; /* tracking level */
+
+#define pgsp_enabled(level) \
+ (!IsParallelWorker() && \
+ (compute_plan_id != COMPUTE_PLAN_ID_OFF) && \
+ (pgsp_track == PGSP_TRACK_ALL || \
+ (pgsp_track == PGSP_TRACK_TOP && (level) == 0)))
+
+#define USAGE_INCREASE 0.5 /* increase by this each time we report
+ * stats */
+#define USAGE_DECREASE_FACTOR (0.99) /* decreased every
+ * pgstat_dealloc_plans */
+#define USAGE_DEALLOC_PERCENT 5 /* free this % of entries at once */
+
+/*---- Function declarations ----*/
+
+PG_FUNCTION_INFO_V1(pg_stat_plans_reset);
+PG_FUNCTION_INFO_V1(pg_stat_plans_1_0);
+
+/* Structures for statistics of plans */
+typedef struct PgStatShared_PlanInfo
+{
+ /* key elements that identify a plan (together with the dboid) */
+ uint64 planid;
+ uint64 queryid;
+ Oid userid; /* userid is tracked to allow users to see
+ * their own query plans */
+ bool toplevel; /* query executed at top level */
+
+ dsa_pointer plan_text; /* pointer to DSA memory containing plan text */
+ int plan_encoding; /* plan text encoding */
+} PgStatShared_PlanInfo;
+
+typedef struct PgStat_StatPlanEntry
+{
+ PgStat_Counter exec_count;
+ double exec_time;
+ double usage; /* Usage factor of the entry, used to
+ * prioritize which plans to age out */
+
+ /* Only used in shared structure, not in local pending stats */
+ PgStatShared_PlanInfo info;
+} PgStat_StatPlanEntry;
+
+typedef struct PgStatShared_Plan
+{
+ PgStatShared_Common header;
+ PgStat_StatPlanEntry stats;
+} PgStatShared_Plan;
+
+static bool plan_stats_flush_cb(PgStat_EntryRef *entry_ref, bool nowait);
+
+static const PgStat_KindInfo plan_stats = {
+ .name = "plan_stats",
+ .fixed_amount = false,
+
+ /*
+ * We currently don't write to a file since plan texts would get lost (and
+ * just the stats on their own aren't that useful)
+ */
+ .write_to_file = false,
+
+ /*
+ * Plan statistics are available system-wide to simplify monitoring
+ * scripts
+ */
+ .accessed_across_databases = true,
+
+ .shared_size = sizeof(PgStatShared_Plan),
+ .shared_data_off = offsetof(PgStatShared_Plan, stats),
+ .shared_data_len = sizeof(((PgStatShared_Plan *) 0)->stats),
+ .pending_size = sizeof(PgStat_StatPlanEntry),
+ .flush_pending_cb = plan_stats_flush_cb,
+};
+
+/*
+ * Compute stats entry idx from query ID and plan ID with an 8-byte hash.
+ *
+ * Whilst we could theorically just use the plan ID here, we intentionally
+ * add the query ID into the mix to ease interpreting the data in combination
+ * with pg_stat_statements.
+ */
+#define PGSTAT_PLAN_IDX(query_id, plan_id, user_id, toplevel) hash_combine64(toplevel, hash_combine64(query_id, hash_combine64(plan_id, user_id)))
+
+/*
+ * Kind ID reserved for statistics of plans.
+ */
+#define PGSTAT_KIND_PLANS PGSTAT_KIND_EXPERIMENTAL /* TODO: Assign */
+
+/*
+ * Callback for stats handling
+ */
+static bool
+plan_stats_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
+{
+ PgStat_StatPlanEntry *localent;
+ PgStatShared_Plan *shfuncent;
+
+ localent = (PgStat_StatPlanEntry *) entry_ref->pending;
+ shfuncent = (PgStatShared_Plan *) entry_ref->shared_stats;
+
+ if (!pgstat_lock_entry(entry_ref, nowait))
+ return false;
+
+ shfuncent->stats.exec_count += localent->exec_count;
+ shfuncent->stats.exec_time += localent->exec_time;
+ shfuncent->stats.usage += localent->usage;
+
+ pgstat_unlock_entry(entry_ref);
+
+ return true;
+}
+
+static char *
+pgsp_explain_plan(QueryDesc *queryDesc)
+{
+ ExplainState *es;
+ StringInfo es_str;
+
+ es = NewExplainState();
+ es_str = es->str;
+
+ /*
+ * We turn off COSTS since identical planids may have very different
+ * costs, and it could be misleading to only show the first recorded
+ * plan's costs.
+ */
+ es->costs = false;
+ es->format = EXPLAIN_FORMAT_TEXT;
+
+ ExplainBeginOutput(es);
+ ExplainPrintPlan(es, queryDesc);
+ ExplainEndOutput(es);
+
+ return es_str->data;
+}
+
+static void
+pgstat_gc_plan_memory()
+{
+ dshash_seq_status hstat;
+ PgStatShared_HashEntry *p;
+
+ /* dshash entry is not modified, take shared lock */
+ dshash_seq_init(&hstat, pgStatLocal.shared_hash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ PgStatShared_Common *header;
+ PgStat_StatPlanEntry *statent;
+
+ if (!p->dropped || p->key.kind != PGSTAT_KIND_PLANS)
+ continue;
+
+ header = dsa_get_address(pgStatLocal.dsa, p->body);
+
+ if (!LWLockConditionalAcquire(&header->lock, LW_EXCLUSIVE))
+ continue;
+
+ statent = (PgStat_StatPlanEntry *) pgstat_get_entry_data(PGSTAT_KIND_PLANS, header);
+
+ /*
+ * Clean up this entry's plan text allocation, if we haven't done so
+ * already
+ */
+ if (DsaPointerIsValid(statent->info.plan_text))
+ {
+ dsa_free(pgStatLocal.dsa, statent->info.plan_text);
+ statent->info.plan_text = InvalidDsaPointer;
+
+ /* Allow removal of the shared stats entry */
+ pg_atomic_fetch_sub_u32(&p->refcount, 1);
+ }
+
+ LWLockRelease(&header->lock);
+ }
+ dshash_seq_term(&hstat);
+
+ /* Encourage other backends to clean up dropped entry refs */
+ pgstat_request_entry_refs_gc();
+}
+
+typedef struct PlanDeallocEntry
+{
+ PgStat_HashKey key;
+ double usage;
+} PlanDeallocEntry;
+
+/*
+ * list sort comparator for sorting into decreasing usage order
+ */
+static int
+entry_cmp_lru(const union ListCell *lhs, const union ListCell *rhs)
+{
+ double l_usage = ((PlanDeallocEntry *) lfirst(lhs))->usage;
+ double r_usage = ((PlanDeallocEntry *) lfirst(rhs))->usage;
+
+ if (l_usage > r_usage)
+ return -1;
+ else if (l_usage < r_usage)
+ return +1;
+ else
+ return 0;
+}
+
+static void
+pgstat_dealloc_plans()
+{
+ dshash_seq_status hstat;
+ PgStatShared_HashEntry *p;
+ List *entries = NIL;
+ ListCell *lc;
+ int nvictims;
+
+ /* dshash entry is not modified, take shared lock */
+ dshash_seq_init(&hstat, pgStatLocal.shared_hash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ PgStatShared_Common *header;
+ PgStat_StatPlanEntry *statent;
+ PlanDeallocEntry *entry;
+
+ if (p->dropped || p->key.kind != PGSTAT_KIND_PLANS)
+ continue;
+
+ header = dsa_get_address(pgStatLocal.dsa, p->body);
+
+ if (!LWLockConditionalAcquire(&header->lock, LW_EXCLUSIVE))
+ continue;
+
+ statent = (PgStat_StatPlanEntry *) pgstat_get_entry_data(PGSTAT_KIND_PLANS, header);
+ statent->usage *= USAGE_DECREASE_FACTOR;
+
+ entry = palloc(sizeof(PlanDeallocEntry));
+ entry->key = p->key;
+ entry->usage = statent->usage;
+
+ LWLockRelease(&header->lock);
+
+ entries = lappend(entries, entry);
+ }
+ dshash_seq_term(&hstat);
+
+ /* Sort by usage ascending (lowest used entries are last) */
+ list_sort(entries, entry_cmp_lru);
+
+ /* At a minimum, deallocate 10 entries to make it worth our while */
+ nvictims = Max(10, list_length(entries) * USAGE_DEALLOC_PERCENT / 100);
+ nvictims = Min(nvictims, list_length(entries));
+
+ /* Actually drop the entries */
+ for_each_from(lc, entries, list_length(entries) - nvictims)
+ {
+ PlanDeallocEntry *entry = lfirst(lc);
+
+ pgstat_drop_entry(entry->key.kind, entry->key.dboid, entry->key.objid);
+ }
+
+ /* Clean up our working memory immediately */
+ foreach(lc, entries)
+ {
+ PlanDeallocEntry *entry = lfirst(lc);
+
+ pfree(entry);
+ }
+ pfree(entries);
+}
+
+static void
+pgstat_gc_plans()
+{
+ dshash_seq_status hstat;
+ PgStatShared_HashEntry *p;
+ bool have_dropped_entries = false;
+ size_t plan_entry_count = 0;
+
+ /* TODO: Prevent concurrent GC cycles - flag an active GC run somehow */
+
+ /*
+ * Count our active entries, and whether there are any dropped entries we
+ * may need to clean up at the end.
+ */
+ dshash_seq_init(&hstat, pgStatLocal.shared_hash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ if (p->key.kind != PGSTAT_KIND_PLANS)
+ continue;
+
+ if (p->dropped)
+ have_dropped_entries = true;
+ else
+ plan_entry_count++;
+ }
+ dshash_seq_term(&hstat);
+
+ /*
+ * If we're over the limit, delete entries with lowest usage factor.
+ */
+ if (plan_entry_count > pgsp_max)
+ {
+ pgstat_dealloc_plans();
+ have_dropped_entries = true; /* Assume we did some work */
+ }
+
+ /* If there are dropped entries, clean up their plan memory if needed */
+ if (have_dropped_entries)
+ pgstat_gc_plan_memory();
+}
+
+static void
+pgstat_report_plan_stats(QueryDesc *queryDesc,
+ PgStat_Counter exec_count,
+ double exec_time)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_Plan *shstatent;
+ PgStat_StatPlanEntry *statent;
+ bool newly_created;
+ uint64 queryId = queryDesc->plannedstmt->queryId;
+ uint64 planId = queryDesc->plannedstmt->planId;
+ Oid userid = GetUserId();
+ bool toplevel = (nesting_level == 0);
+
+ entry_ref = pgstat_prep_pending_entry(PGSTAT_KIND_PLANS, MyDatabaseId,
+ PGSTAT_PLAN_IDX(queryId, planId, userid, toplevel), &newly_created);
+
+ shstatent = (PgStatShared_Plan *) entry_ref->shared_stats;
+ statent = &shstatent->stats;
+
+ if (newly_created)
+ {
+ char *plan = pgsp_explain_plan(queryDesc);
+ size_t plan_size = Min(strlen(plan), pgsp_max_size);
+
+ (void) pgstat_lock_entry(entry_ref, false);
+
+ /*
+ * We may be over the limit, so run GC now before saving entry (we do
+ * this whilst holding the lock on the new entry so we don't remove it
+ * by accident)
+ */
+ pgstat_gc_plans();
+
+ shstatent->stats.info.planid = planId;
+ shstatent->stats.info.queryid = queryId;
+ shstatent->stats.info.userid = userid;
+ shstatent->stats.info.toplevel = toplevel;
+ shstatent->stats.info.plan_text = dsa_allocate(pgStatLocal.dsa, plan_size);
+ strlcpy(dsa_get_address(pgStatLocal.dsa, shstatent->stats.info.plan_text), plan, plan_size);
+
+ shstatent->stats.info.plan_encoding = GetDatabaseEncoding();
+
+ /*
+ * Increase refcount here so entry can't get released without us
+ * dropping the plan text
+ */
+ pg_atomic_fetch_add_u32(&entry_ref->shared_entry->refcount, 1);
+
+ pgstat_unlock_entry(entry_ref);
+
+ pfree(plan);
+ }
+
+ statent->exec_count += exec_count;
+ statent->exec_time += exec_time;
+ statent->usage += USAGE_INCREASE;
+}
+
+/*
+ * Planner hook: forward to regular planner, but increase plan count and
+ * record query plan if needed.
+ */
+static PlannedStmt *
+pgsp_planner(Query *parse,
+ const char *query_string,
+ int cursorOptions,
+ ParamListInfo boundParams)
+{
+ PlannedStmt *result;
+
+ /*
+ * Increment the nesting level, to ensure that functions evaluated during
+ * planning are not seen as top-level calls.
+ */
+ nesting_level++;
+ PG_TRY();
+ {
+ if (prev_planner_hook)
+ result = prev_planner_hook(parse, query_string, cursorOptions,
+ boundParams);
+ else
+ result = standard_planner(parse, query_string, cursorOptions,
+ boundParams);
+ }
+ PG_FINALLY();
+ {
+ nesting_level--;
+ }
+ PG_END_TRY();
+
+ return result;
+}
+
+/*
+ * ExecutorStart hook: start up tracking if needed
+ */
+static void
+pgsp_ExecutorStart(QueryDesc *queryDesc, int eflags)
+{
+ uint64 queryId = queryDesc->plannedstmt->queryId;
+ uint64 planId = queryDesc->plannedstmt->planId;
+
+ if (prev_ExecutorStart)
+ prev_ExecutorStart(queryDesc, eflags);
+ else
+ standard_ExecutorStart(queryDesc, eflags);
+
+ if (queryId != UINT64CONST(0) && planId != UINT64CONST(0) &&
+ pgsp_enabled(nesting_level))
+ {
+ /*
+ * Record initial entry now, so plan text is available for currently
+ * running queries
+ */
+ pgstat_report_plan_stats(queryDesc,
+ 0, /* executions are counted in
+ * pgsp_ExecutorEnd */
+ 0.0);
+
+ /*
+ * Set up to track total elapsed time in ExecutorRun. Make sure the
+ * space is allocated in the per-query context so it will go away at
+ * ExecutorEnd.
+ */
+ if (queryDesc->totaltime == NULL)
+ {
+ MemoryContext oldcxt;
+
+ oldcxt = MemoryContextSwitchTo(queryDesc->estate->es_query_cxt);
+ queryDesc->totaltime = InstrAlloc(1, INSTRUMENT_ALL, false);
+ MemoryContextSwitchTo(oldcxt);
+ }
+ }
+}
+
+/*
+ * ExecutorRun hook: all we need do is track nesting depth
+ */
+static void
+pgsp_ExecutorRun(QueryDesc *queryDesc, ScanDirection direction, uint64 count)
+{
+ nesting_level++;
+ PG_TRY();
+ {
+ if (prev_ExecutorRun)
+ prev_ExecutorRun(queryDesc, direction, count);
+ else
+ standard_ExecutorRun(queryDesc, direction, count);
+ }
+ PG_FINALLY();
+ {
+ nesting_level--;
+ }
+ PG_END_TRY();
+}
+
+/*
+ * ExecutorFinish hook: all we need do is track nesting depth
+ */
+static void
+pgsp_ExecutorFinish(QueryDesc *queryDesc)
+{
+ nesting_level++;
+ PG_TRY();
+ {
+ if (prev_ExecutorFinish)
+ prev_ExecutorFinish(queryDesc);
+ else
+ standard_ExecutorFinish(queryDesc);
+ }
+ PG_FINALLY();
+ {
+ nesting_level--;
+ }
+ PG_END_TRY();
+}
+
+/*
+ * ExecutorEnd hook: store results if needed
+ */
+static void
+pgsp_ExecutorEnd(QueryDesc *queryDesc)
+{
+ uint64 queryId = queryDesc->plannedstmt->queryId;
+ uint64 planId = queryDesc->plannedstmt->planId;
+
+ if (queryId != UINT64CONST(0) && planId != UINT64CONST(0) &&
+ queryDesc->totaltime && pgsp_enabled(nesting_level))
+ {
+ /*
+ * Make sure stats accumulation is done. (Note: it's okay if several
+ * levels of hook all do this.)
+ */
+ InstrEndLoop(queryDesc->totaltime);
+
+ pgstat_report_plan_stats(queryDesc,
+ 1,
+ queryDesc->totaltime->total * 1000.0 /* convert to msec */ );
+ }
+
+ if (prev_ExecutorEnd)
+ prev_ExecutorEnd(queryDesc);
+ else
+ standard_ExecutorEnd(queryDesc);
+}
+
+/*
+ * Module load callback
+ */
+void
+_PG_init(void)
+{
+ /*
+ * In order to register for shared memory stats, we have to be loaded via
+ * shared_preload_libraries. If not, fall out without hooking into any of
+ * the main system. (We don't throw error here because it seems useful to
+ * allow the pg_stat_plans functions to be created even when the module
+ * isn't active. The functions must protect themselves against being
+ * called then, however.)
+ */
+ if (!process_shared_preload_libraries_in_progress)
+ return;
+
+ /*
+ * Inform the postmaster that we want to enable query_id calculation if
+ * compute_query_id is set to auto, as well as plan_id calculation if
+ * compute_plan_id is set to auto.
+ */
+ EnableQueryId();
+ EnablePlanId();
+
+ /*
+ * Define (or redefine) custom GUC variables.
+ */
+ DefineCustomIntVariable("pg_stat_plans.max",
+ "Sets the maximum number of plans tracked by pg_stat_plans in shared memory.",
+ NULL,
+ &pgsp_max,
+ 5000,
+ 100,
+ INT_MAX / 2,
+ PGC_SIGHUP,
+ 0,
+ NULL,
+ NULL,
+ NULL);
+
+ DefineCustomIntVariable("pg_stat_plans.max_size",
+ "Sets the maximum size of plan texts tracked by pg_stat_plans in shared memory.",
+ NULL,
+ &pgsp_max_size,
+ 2048,
+ 100,
+ 1048576, /* 1MB hard limit */
+ PGC_SUSET,
+ 0,
+ NULL,
+ NULL,
+ NULL);
+
+ DefineCustomEnumVariable("pg_stat_plans.track",
+ "Selects which plans are tracked by pg_stat_plans.",
+ NULL,
+ &pgsp_track,
+ PGSP_TRACK_TOP,
+ track_options,
+ PGC_SUSET,
+ 0,
+ NULL,
+ NULL,
+ NULL);
+
+ MarkGUCPrefixReserved("pg_stat_plans");
+
+ /*
+ * Install hooks.
+ */
+ prev_planner_hook = planner_hook;
+ planner_hook = pgsp_planner;
+ prev_ExecutorStart = ExecutorStart_hook;
+ ExecutorStart_hook = pgsp_ExecutorStart;
+ prev_ExecutorRun = ExecutorRun_hook;
+ ExecutorRun_hook = pgsp_ExecutorRun;
+ prev_ExecutorFinish = ExecutorFinish_hook;
+ ExecutorFinish_hook = pgsp_ExecutorFinish;
+ prev_ExecutorEnd = ExecutorEnd_hook;
+ ExecutorEnd_hook = pgsp_ExecutorEnd;
+
+ pgstat_register_kind(PGSTAT_KIND_PLANS, &plan_stats);
+}
+
+/*
+ * Reset statement statistics.
+ */
+Datum
+pg_stat_plans_reset(PG_FUNCTION_ARGS)
+{
+ pgstat_drop_entries_of_kind(PGSTAT_KIND_PLANS);
+
+ /* Free plan text memory and allow cleanup of dropped entries */
+ pgstat_gc_plan_memory();
+
+ PG_RETURN_VOID();
+}
+
+#define PG_STAT_PLANS_COLS 8
+
+Datum
+pg_stat_plans_1_0(PG_FUNCTION_ARGS)
+{
+ bool showplan = PG_GETARG_BOOL(0);
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ Oid userid = GetUserId();
+ bool is_allowed_role = false;
+
+ dshash_seq_status hstat;
+ PgStatShared_HashEntry *p;
+
+ /*
+ * Superusers or roles with the privileges of pg_read_all_stats members
+ * are allowed
+ */
+ is_allowed_role = has_privs_of_role(userid, ROLE_PG_READ_ALL_STATS);
+
+ /* stats kind must be registered already */
+ if (!pgstat_get_kind_info(PGSTAT_KIND_PLANS))
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("pg_stat_plans must be loaded via \"shared_preload_libraries\"")));
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ /* dshash entry is not modified, take shared lock */
+ dshash_seq_init(&hstat, pgStatLocal.shared_hash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ PgStat_StatPlanEntry *statent;
+ Datum values[PG_STAT_PLANS_COLS];
+ bool nulls[PG_STAT_PLANS_COLS];
+ int i = 0;
+
+ if (p->dropped || p->key.kind != PGSTAT_KIND_PLANS)
+ continue;
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+
+ statent = pgstat_get_entry_data(p->key.kind, dsa_get_address(pgStatLocal.dsa, p->body));
+
+ values[i++] = ObjectIdGetDatum(statent->info.userid);
+ values[i++] = ObjectIdGetDatum(p->key.dboid);
+ values[i++] = BoolGetDatum(statent->info.toplevel);
+ if (is_allowed_role || statent->info.userid == userid)
+ {
+ int64 queryid = statent->info.queryid;
+ int64 planid = statent->info.planid;
+
+ values[i++] = Int64GetDatumFast(queryid);
+ values[i++] = Int64GetDatumFast(planid);
+ }
+ else
+ {
+ nulls[i++] = true;
+ nulls[i++] = true;
+ }
+ values[i++] = Int64GetDatumFast(statent->exec_count);
+ values[i++] = Float8GetDatumFast(statent->exec_time);
+
+ if (showplan && (is_allowed_role || statent->info.userid == userid))
+ {
+ char *pstr = DsaPointerIsValid(statent->info.plan_text) ? dsa_get_address(pgStatLocal.dsa, statent->info.plan_text) : NULL;
+
+ if (pstr)
+ {
+ char *enc = pg_any_to_server(pstr, strlen(pstr), statent->info.plan_encoding);
+
+ values[i++] = CStringGetTextDatum(enc);
+
+ if (enc != pstr)
+ pfree(enc);
+ }
+ else
+ {
+ nulls[i++] = true;
+ }
+ }
+ else if (showplan)
+ {
+ values[i++] = CStringGetTextDatum("<insufficient privilege>");
+ }
+ else
+ {
+ nulls[i++] = true;
+ }
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
+ }
+ dshash_seq_term(&hstat);
+
+ return (Datum) 0;
+}
diff --git a/contrib/pg_stat_plans/pg_stat_plans.conf b/contrib/pg_stat_plans/pg_stat_plans.conf
new file mode 100644
index 0000000000..6750b3e2cc
--- /dev/null
+++ b/contrib/pg_stat_plans/pg_stat_plans.conf
@@ -0,0 +1 @@
+shared_preload_libraries = 'pg_stat_plans'
diff --git a/contrib/pg_stat_plans/pg_stat_plans.control b/contrib/pg_stat_plans/pg_stat_plans.control
new file mode 100644
index 0000000000..4db3a47239
--- /dev/null
+++ b/contrib/pg_stat_plans/pg_stat_plans.control
@@ -0,0 +1,5 @@
+# pg_stat_plans extension
+comment = 'track per-plan call counts, execution times and EXPLAIN texts'
+default_version = '1.0'
+module_pathname = '$libdir/pg_stat_plans'
+relocatable = true
diff --git a/contrib/pg_stat_plans/sql/cleanup.sql b/contrib/pg_stat_plans/sql/cleanup.sql
new file mode 100644
index 0000000000..51565617ce
--- /dev/null
+++ b/contrib/pg_stat_plans/sql/cleanup.sql
@@ -0,0 +1 @@
+DROP EXTENSION pg_stat_plans;
diff --git a/contrib/pg_stat_plans/sql/privileges.sql b/contrib/pg_stat_plans/sql/privileges.sql
new file mode 100644
index 0000000000..aaad72a655
--- /dev/null
+++ b/contrib/pg_stat_plans/sql/privileges.sql
@@ -0,0 +1,59 @@
+--
+-- Only superusers and roles with privileges of the pg_read_all_stats role
+-- are allowed to see the plan text, queryid and planid of queries executed by
+-- other users. Other users can see the statistics.
+--
+
+CREATE ROLE regress_stats_superuser SUPERUSER;
+CREATE ROLE regress_stats_user1;
+CREATE ROLE regress_stats_user2;
+GRANT pg_read_all_stats TO regress_stats_user2;
+
+SET ROLE regress_stats_superuser;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+SELECT 1 AS "ONE";
+
+SET ROLE regress_stats_user1;
+SELECT 1+1 AS "TWO";
+
+--
+-- A superuser can read all columns of queries executed by others,
+-- including plan text, queryid and planid.
+--
+
+SET ROLE regress_stats_superuser;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+
+--
+-- regress_stats_user1 has no privileges to read the plan text, queryid
+-- or planid of queries executed by others but can see statistics
+-- like calls and rows.
+--
+
+SET ROLE regress_stats_user1;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+
+--
+-- regress_stats_user2, with pg_read_all_stats role privileges, can
+-- read all columns, including plan text, queryid and planid, of queries
+-- executed by others.
+--
+
+SET ROLE regress_stats_user2;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+
+--
+-- cleanup
+--
+
+RESET ROLE;
+DROP ROLE regress_stats_superuser;
+DROP ROLE regress_stats_user1;
+DROP ROLE regress_stats_user2;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
diff --git a/contrib/pg_stat_plans/sql/select.sql b/contrib/pg_stat_plans/sql/select.sql
new file mode 100644
index 0000000000..f0e803ad70
--- /dev/null
+++ b/contrib/pg_stat_plans/sql/select.sql
@@ -0,0 +1,67 @@
+--
+-- SELECT statements
+--
+
+CREATE EXTENSION pg_stat_plans;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+
+--
+-- simple statements
+--
+
+SELECT 1 FROM pg_class LIMIT 1;
+
+SELECT 1 FROM pg_class WHERE relname = 'pg_class';
+
+SET enable_indexscan = off;
+SELECT 1 FROM pg_class WHERE relname = 'pg_class';
+SET enable_indexscan = on;
+
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+
+--
+-- subplans and CTEs
+--
+
+WITH x AS MATERIALIZED (SELECT 1)
+SELECT * FROM x;
+
+SELECT a.attname,
+ (SELECT pg_catalog.pg_get_expr(d.adbin, d.adrelid)
+ FROM pg_catalog.pg_attrdef d
+ WHERE d.adrelid = a.attrelid AND d.adnum = a.attnum AND a.atthasdef)
+ FROM pg_catalog.pg_attribute a
+ WHERE a.attrelid = 'pg_class'::regclass
+ ORDER BY attnum LIMIT 1;
+
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+
+--
+-- partitoning
+--
+
+create table lp (a char) partition by list (a);
+create table lp_default partition of lp default;
+create table lp_ef partition of lp for values in ('e', 'f');
+create table lp_ad partition of lp for values in ('a', 'd');
+create table lp_bc partition of lp for values in ('b', 'c');
+create table lp_g partition of lp for values in ('g');
+create table lp_null partition of lp for values in (null);
+
+select * from lp;
+select * from lp where a > 'a' and a < 'd';
+select * from lp where a > 'a' and a <= 'd';
+select * from lp where a = 'a';
+select * from lp where 'a' = a; /* commuted */
+select * from lp where a is not null;
+select * from lp where a is null;
+select * from lp where a = 'a' or a = 'c';
+select * from lp where a is not null and (a = 'a' or a = 'c');
+select * from lp where a <> 'g';
+select * from lp where a <> 'a' and a <> 'd';
+select * from lp where a not in ('a', 'd');
+
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index 7c381949a5..4a5a02c704 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -157,6 +157,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
&pglogicalinspect;
&pgprewarm;
&pgrowlocks;
+ &pgstatplans;
&pgstatstatements;
&pgstattuple;
&pgsurgery;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 66e6dccd4c..b0afb33ce2 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -146,6 +146,7 @@
<!ENTITY pglogicalinspect SYSTEM "pglogicalinspect.sgml">
<!ENTITY pgprewarm SYSTEM "pgprewarm.sgml">
<!ENTITY pgrowlocks SYSTEM "pgrowlocks.sgml">
+<!ENTITY pgstatplans SYSTEM "pgstatplans.sgml">
<!ENTITY pgstatstatements SYSTEM "pgstatstatements.sgml">
<!ENTITY pgstattuple SYSTEM "pgstattuple.sgml">
<!ENTITY pgsurgery SYSTEM "pgsurgery.sgml">
diff --git a/doc/src/sgml/pgstatplans.sgml b/doc/src/sgml/pgstatplans.sgml
new file mode 100644
index 0000000000..4402d904f8
--- /dev/null
+++ b/doc/src/sgml/pgstatplans.sgml
@@ -0,0 +1,425 @@
+<!-- doc/src/sgml/pgstatplans.sgml -->
+
+<sect1 id="pgstatplans" xreflabel="pg_stat_plans">
+ <title>pg_stat_plans — track per-plan call counts, execution times and EXPLAIN texts</title>
+
+ <indexterm zone="pgstatplans">
+ <primary>pg_stat_plans</primary>
+ </indexterm>
+
+ <para>
+ The <filename>pg_stat_plans</filename> module provides a means for
+ tracking per-plan statistics and plan texts of all SQL statements executed by
+ a server.
+ </para>
+
+ <para>
+ The module must be loaded by adding <literal>pg_stat_plans</literal> to
+ <xref linkend="guc-shared-preload-libraries"/> in
+ <filename>postgresql.conf</filename>, because it requires additional shared memory.
+ This means that a server restart is needed to add or remove the module.
+ In addition, query and plan identifier calculation must be enabled in order for the
+ module to be active by setting both <xref linkend="guc-compute-plan-id"/> to
+ <literal>auto</literal> or <literal>on</literal> and
+ <xref linkend="guc-compute-query-id"/> to <literal>auto</literal> or <literal>on</literal>.
+ </para>
+
+ <para>
+ When <filename>pg_stat_plans</filename> is active, it tracks
+ statistics across all databases of the server. To access and manipulate
+ these statistics, the module provides the <structname>pg_stat_plans</structname>
+ view and the utility functions <function>pg_stat_plans_reset</function> and
+ <function>pg_stat_plans</function>. These are not available globally but
+ can be enabled for a specific database with
+ <command>CREATE EXTENSION pg_stat_plans</command>.
+ </para>
+
+ <sect2 id="pgstatplans-pg-stat-plans">
+ <title>The <structname>pg_stat_plans</structname> View</title>
+
+ <para>
+ The statistics gathered by the module are made available via a
+ view named <structname>pg_stat_plans</structname>. This view
+ contains one row for each distinct combination of database ID, user
+ ID, whether it's a top-level statement or not, query ID and plan ID
+ (up to the maximum number of distinct plans that the module can track).
+ The columns of the view are shown in <xref linkend="pgstatplans-columns"/>.
+ </para>
+
+ <table id="pgstatplans-columns">
+ <title><structname>pg_stat_plans</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>userid</structfield> <type>oid</type>
+ (references <link linkend="catalog-pg-authid"><structname>pg_authid</structname></link>.<structfield>oid</structfield>)
+ </para>
+ <para>
+ OID of user who executed the statement
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbid</structfield> <type>oid</type>
+ (references <link linkend="catalog-pg-database"><structname>pg_database</structname></link>.<structfield>oid</structfield>)
+ </para>
+ <para>
+ OID of database in which the statement was executed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>toplevel</structfield> <type>bool</type>
+ </para>
+ <para>
+ True if the query was executed as a top-level statement
+ (always true if <varname>pg_stat_plans.track</varname> is set to
+ <literal>top</literal>)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>queryid</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Hash code to identify identical normalized queries.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>planid</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Hash code to identify identical plan shapes.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>calls</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the plan was executed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_exec_time</structfield> <type>double precision</type>
+ </para>
+ <para>
+ Total time spent executing the plan, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>plan</structfield> <type>text</type>
+ </para>
+ <para>
+ Plan text of a representative plan. This is similar to the output of
+ <literal>EXPLAIN (COSTS OFF)</literal>. Note the plan text will contain constant
+ values of the first plan recorded, but subsequent executions of the
+ same plan hash code (<structfield>planid</structfield>) with different
+ constant values will be tracked under the same entry.
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ For security reasons, only superusers and roles with privileges of the
+ <literal>pg_read_all_stats</literal> role are allowed to see the plan text,
+ <structfield>queryid</structfield> and <structfield>planid</structfield>
+ of queries executed by other users. Other users can see the statistics,
+ however, if the view has been installed in their database.
+ </para>
+
+ <para>
+ Plannable queries (that is, <command>SELECT</command>, <command>INSERT</command>,
+ <command>UPDATE</command>, <command>DELETE</command>, and <command>MERGE</command>)
+ will have the same <structfield>planid</structfield> whenever they have identical plan
+ structures according to an internal hash calculation. Typically, two plans will be
+ considered the same for this purpose if they have the same
+ <literal>EXPLAIN (COSTS OFF)</literal> output and are semantically equivalent except
+ for the values of literal constants appearing in the query plan.
+ </para>
+
+ <para>
+ Note that queries that have not finished executing yet will show in
+ <structname>pg_stat_plans</structname> with their plan text, but without
+ the <structname>calls</structname> field being incremented. This can be
+ used to identify the query plan for a currently running statement by joining
+ <link linkend="monitoring-pg-stat-activity-view">pg_stat_activity</link>
+ with <structname>pg_stat_plans</structname>, see example usage in
+ <xref linkend="pgstatplans-sample-output"/>.
+ </para>
+
+ <para>
+ Consumers of <structname>pg_stat_plans</structname> should use
+ <structfield>planid</structfield> in combination with
+ <structfield>queryid</structfield>, <structfield>dbid</structfield>,
+ <structfield>userid</structfield> and <structfield>toplevel</structfield>
+ as a stable and reliable identifier for each entry, instead of using its
+ plan text. However, it is important to understand that there are only limited
+ guarantees around the stability of the <structfield>planid</structfield>
+ hash value. Since the identifier is derived from the plan tree, its value
+ is a function of, among other things, the internal object identifiers
+ appearing in this representation. This has some counterintuitive implications.
+ For example, <filename>pg_stat_plans</filename> will consider two
+ apparently-identical plans to be distinct, if they reference a table
+ that was dropped and recreated between the creation of the two plans.
+ The hashing process is also sensitive to differences in
+ machine architecture and other facets of the platform.
+ Furthermore, it is not safe to assume that <structfield>planid</structfield>
+ will be stable across major versions of <productname>PostgreSQL</productname>.
+ </para>
+
+ <para>
+ Two servers participating in replication based on physical WAL replay can
+ be expected to have identical <structfield>planid</structfield> values for
+ the same plan. However, logical replication schemes do not promise to
+ keep replicas identical in all relevant details, so
+ <structfield>planid</structfield> will not be a useful identifier for
+ accumulating costs across a set of logical replicas.
+ If in doubt, direct testing is recommended.
+ </para>
+
+ <para>
+ Plan texts are stored in shared memory, and limited in length. To increase
+ the maximum length of stored plan texts you can increase
+ <varname>pg_stat_plans.max_size</varname>. This value can be changed for
+ an individual connection, or set as a server-wide setting.
+ </para>
+ </sect2>
+
+ <sect2 id="pgstatplans-funcs">
+ <title>Functions</title>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ <function>pg_stat_plans_reset() returns void</function>
+ <indexterm>
+ <primary>pg_stat_plans_reset</primary>
+ </indexterm>
+ </term>
+
+ <listitem>
+ <para>
+ <function>pg_stat_plans_reset</function> discards statistics and plan texts
+ gathered so far by <filename>pg_stat_plans</filename>.
+ By default, this function can only be executed by superusers.
+ Access may be granted to others using <command>GRANT</command>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <function>pg_stat_plans(showplan boolean) returns setof record</function>
+ <indexterm>
+ <primary>pg_stat_plans</primary>
+ <secondary>function</secondary>
+ </indexterm>
+ </term>
+
+ <listitem>
+ <para>
+ The <structname>pg_stat_plans</structname> view is defined in
+ terms of a function also named <function>pg_stat_plans</function>.
+ It is possible for clients to call
+ the <function>pg_stat_plans</function> function directly, and by
+ specifying <literal>showplan := false</literal> have plan texts be
+ omitted (that is, the <literal>OUT</literal> argument that corresponds
+ to the view's <structfield>plan</structfield> column will return nulls). This
+ feature is intended to support external tools that might wish to avoid
+ the overhead of repeatedly retrieving plan texts of indeterminate
+ length. Such tools can instead cache the first plan text observed
+ for each entry themselves, since that is
+ all <filename>pg_stat_plans</filename> itself does, and then retrieve
+ plan texts only as needed.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+
+ <sect2 id="pgstatplans-config-params">
+ <title>Configuration Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ <varname>pg_stat_plans.max</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>pg_stat_plans.max</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+
+ <listitem>
+ <para>
+ <varname>pg_stat_plans.max</varname> is the maximum number of
+ plans tracked by the module (i.e., the maximum number of rows
+ in the <structname>pg_stat_plans</structname> view). If more distinct
+ plans than that are observed, information about the least-executed
+ plans is discarded. The default value is 5000.
+ Only superusers can change this setting. Changing the setting requires
+ a reload of the server.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <varname>pg_stat_plans.max_size</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>pg_stat_plans.max_size</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+
+ <listitem>
+ <para>
+ <varname>pg_stat_plans.max_size</varname> is the maximum length of
+ each plan text tracked by the module in bytes. Longer plan texts will be truncated.
+ The default value is 2048 (2kB).
+ Only superusers can change this setting.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <varname>pg_stat_plans.track</varname> (<type>enum</type>)
+ <indexterm>
+ <primary><varname>pg_stat_plans.track</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+
+ <listitem>
+ <para>
+ <varname>pg_stat_plans.track</varname> controls which plans
+ are counted by the module.
+ Specify <literal>top</literal> to track plans by top-level statements (those issued
+ directly by clients), <literal>all</literal> to also track nested statements
+ (such as statements invoked within functions), or <literal>none</literal> to
+ disable plan statistics collection.
+ The default value is <literal>top</literal>.
+ Only superusers can change this setting.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ <para>
+ The module requires additional shared memory proportional to
+ <varname>pg_stat_plans.max</varname> for statistics, as well as
+ <varname>pg_stat_plans.max</varname> multiplied by
+ <varname>pg_stat_plans.max_size</varname> for plan texts. Note that this
+ memory is only consumed when entries are created, and not if
+ <varname>pg_stat_plans.track</varname> is set to <literal>none</literal>.
+ </para>
+
+ <para>
+ These parameters must be set in <filename>postgresql.conf</filename>.
+ Typical usage might be:
+
+<programlisting>
+# postgresql.conf
+shared_preload_libraries = 'pg_stat_plans'
+
+compute_query_id = on
+compute_plan_id = on
+pg_stat_plans.max = 10000
+pg_stat_plans.max_size = 4096
+pg_stat_plans.track = all
+</programlisting>
+ </para>
+ </sect2>
+
+ <sect2 id="pgstatplans-sample-output">
+ <title>Sample Output</title>
+
+<screen>
+bench=# SELECT pg_stat_plans_reset();
+
+$ pgbench -i bench
+$ pgbench -c10 -t300 bench
+
+bench=# \x
+bench=# SELECT plan, calls, total_exec_time
+ FROM pg_stat_plans ORDER BY total_exec_time DESC LIMIT 5;
+-[ RECORD 1 ]---+-----------------------------------------------------------------
+plan | Update on pgbench_tellers +
+ | -> Seq Scan on pgbench_tellers +
+ | Filter: (tid = 5)
+calls | 3000
+total_exec_time | 642.8880919999993
+-[ RECORD 2 ]---+-----------------------------------------------------------------
+plan | Update on pgbench_branches +
+ | -> Seq Scan on pgbench_branches +
+ | Filter: (bid = 1)
+calls | 1813
+total_exec_time | 476.64152700000005
+-[ RECORD 3 ]---+-----------------------------------------------------------------
+plan | Update on pgbench_branches +
+ | -> Index Scan using pgbench_branches_pkey on pgbench_branches+
+ | Index Cond: (bid = 1)
+calls | 1187
+total_exec_time | 326.1257549999999
+-[ RECORD 4 ]---+-----------------------------------------------------------------
+plan | Update on pgbench_accounts +
+ | -> Index Scan using pgbench_accounts_pkey on pgbench_accounts+
+ | Index Cond: (aid = 48793)
+calls | 3000
+total_exec_time | 21.664690000000093
+-[ RECORD 5 ]---+-----------------------------------------------------------------
+plan | Insert on pgbench_history +
+ | -> Result
+calls | 3000
+total_exec_time | 4.365250999999957
+
+session 1:
+
+bench# SELECT pg_sleep(100), COUNT(*) FROM pgbench_accounts;
+
+session 2:
+
+bench=# SELECT query, plan FROM pg_stat_activity
+ JOIN pg_stat_plans ON (usesysid = userid AND datid = dbid AND query_id = queryid AND plan_id = planid)
+ WHERE query LIKE 'SELECT pg_sleep%';
+ query | plan
+-------------------------------------------------------+------------------------------------
+ SELECT pg_sleep(100), COUNT(*) FROM pgbench_accounts; | Aggregate +
+ | -> Seq Scan on pgbench_accounts
+(1 row)
+
+</screen>
+ </sect2>
+
+ <sect2 id="pgstatplans-authors">
+ <title>Authors</title>
+
+ <para>
+ Lukas Fittl <email>lukas@fittl.com</email>.
+ </para>
+ </sect2>
+
+</sect1>
--
2.47.1
On Fri, Jan 24, 2025 at 01:59:00AM -0800, Lukas Fittl wrote:
Overall, I also do wonder if it wouldn't be better to have a callback
mechanism in the shared memory stats, so stats plugins can do extra work
when an entry gets dropped (like freeing the DSA memory for the plan text),
vs having to add all this extra logic to do it.
Not sure about this part yet. I have looked at 0002 to begin with
something and it is really useful on its own. Stats kinds calling
this routine don't need to worry about the internals of dropping local
references or doing a seqscan on the shared hash table. However, what
you have sent lacks in flexibility to me, and the duplication with
pgstat_drop_all_entries is annoying. This had better be merged in a
single routine.
Attached is an updated version that adds an optional "do_drop"
callback in the function that does the seqscan on the dshash, to
decide if an entry should be gone or not. This follows the same model
as the "reset" part, where stats kind can push the matching function
they want to work on the individual entries. We could add a
pgstat_drop_entries_of_kind(), but I'm not feeling that this is
strongly necessary with the basic interface in place.
The changes in the module injection_points were not good. The SQL
function was named "reset" but that's a drop operation.
What do you think?
--
Michael
Attachments:
v3-0002-Add-pgstat_drop_matching_entries.patchtext/x-diff; charset=us-asciiDownload
From 48412fcfb708eb990136321686681e961626aabe Mon Sep 17 00:00:00 2001
From: Lukas Fittl <lukas@fittl.com>
Date: Thu, 2 Jan 2025 10:46:30 -0800
Subject: [PATCH v3] Add pgstat_drop_matching_entries()
This allows users of the cumulative statistics systems to drop all
entries based on the decisions of a matching function, similar to how
pgstat_reset_matching_entries() works.
---
src/include/utils/pgstat_internal.h | 2 ++
src/backend/utils/activity/pgstat_shmem.c | 31 ++++++++++++++++++-
.../injection_points--1.0.sql | 10 ++++++
.../injection_points/injection_stats.c | 19 ++++++++++++
.../modules/injection_points/t/001_stats.pl | 13 ++++++++
5 files changed, 74 insertions(+), 1 deletion(-)
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index a3d39d2b725..06dcea3f0dc 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -718,6 +718,8 @@ extern bool pgstat_lock_entry_shared(PgStat_EntryRef *entry_ref, bool nowait);
extern void pgstat_unlock_entry(PgStat_EntryRef *entry_ref);
extern bool pgstat_drop_entry(PgStat_Kind kind, Oid dboid, uint64 objid);
extern void pgstat_drop_all_entries(void);
+extern void pgstat_drop_matching_entries(bool (*do_drop) (PgStatShared_HashEntry *, Datum),
+ Datum match_data);
extern PgStat_EntryRef *pgstat_get_entry_ref_locked(PgStat_Kind kind, Oid dboid, uint64 objid,
bool nowait);
extern void pgstat_reset_entry(PgStat_Kind kind, Oid dboid, uint64 objid, TimestampTz ts);
diff --git a/src/backend/utils/activity/pgstat_shmem.c b/src/backend/utils/activity/pgstat_shmem.c
index 342586397d6..770d62425c5 100644
--- a/src/backend/utils/activity/pgstat_shmem.c
+++ b/src/backend/utils/activity/pgstat_shmem.c
@@ -993,19 +993,39 @@ pgstat_drop_entry(PgStat_Kind kind, Oid dboid, uint64 objid)
return freed;
}
+/*
+ * Scan through the shared hashtable of stats, dropping statistics if
+ * approved by the optional do_drop() function.
+ */
void
-pgstat_drop_all_entries(void)
+pgstat_drop_matching_entries(bool (*do_drop) (PgStatShared_HashEntry *, Datum),
+ Datum match_data)
{
dshash_seq_status hstat;
PgStatShared_HashEntry *ps;
uint64 not_freed_count = 0;
+ /* entries are removed, take an exclusive lock */
dshash_seq_init(&hstat, pgStatLocal.shared_hash, true);
while ((ps = dshash_seq_next(&hstat)) != NULL)
{
if (ps->dropped)
continue;
+ if (do_drop != NULL && !do_drop(ps, match_data))
+ continue;
+
+ /* delete local reference */
+ if (pgStatEntryRefHash)
+ {
+ PgStat_EntryRefHashEntry *lohashent =
+ pgstat_entry_ref_hash_lookup(pgStatEntryRefHash, ps->key);
+
+ if (lohashent)
+ pgstat_release_entry_ref(lohashent->key, lohashent->entry_ref,
+ true);
+ }
+
if (!pgstat_drop_entry_internal(ps, &hstat))
not_freed_count++;
}
@@ -1015,6 +1035,15 @@ pgstat_drop_all_entries(void)
pgstat_request_entry_refs_gc();
}
+/*
+ * Scan through the shared hashtable of stats and drop all entries.
+ */
+void
+pgstat_drop_all_entries(void)
+{
+ pgstat_drop_matching_entries(NULL, 0);
+}
+
static void
shared_stat_reset_contents(PgStat_Kind kind, PgStatShared_Common *header,
TimestampTz ts)
diff --git a/src/test/modules/injection_points/injection_points--1.0.sql b/src/test/modules/injection_points/injection_points--1.0.sql
index c445bf64e62..5d83f08811b 100644
--- a/src/test/modules/injection_points/injection_points--1.0.sql
+++ b/src/test/modules/injection_points/injection_points--1.0.sql
@@ -85,6 +85,16 @@ RETURNS bigint
AS 'MODULE_PATHNAME', 'injection_points_stats_numcalls'
LANGUAGE C STRICT;
+--
+-- injection_points_stats_drop()
+--
+-- Drop all statistics of injection points.
+--
+CREATE FUNCTION injection_points_stats_drop()
+RETURNS void
+AS 'MODULE_PATHNAME', 'injection_points_stats_drop'
+LANGUAGE C STRICT;
+
--
-- injection_points_stats_fixed()
--
diff --git a/src/test/modules/injection_points/injection_stats.c b/src/test/modules/injection_points/injection_stats.c
index 5db62bca66f..14903c629e0 100644
--- a/src/test/modules/injection_points/injection_stats.c
+++ b/src/test/modules/injection_points/injection_stats.c
@@ -197,3 +197,22 @@ injection_points_stats_numcalls(PG_FUNCTION_ARGS)
PG_RETURN_INT64(entry->numcalls);
}
+
+/* Only used by injection_points_stats_drop() */
+static bool
+match_inj_entries(PgStatShared_HashEntry *entry, Datum match_data)
+{
+ return entry->key.kind == PGSTAT_KIND_INJECTION;
+}
+
+/*
+ * SQL function that drops all injection point statistics.
+ */
+PG_FUNCTION_INFO_V1(injection_points_stats_drop);
+Datum
+injection_points_stats_drop(PG_FUNCTION_ARGS)
+{
+ pgstat_drop_matching_entries(match_inj_entries, 0);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/test/modules/injection_points/t/001_stats.pl b/src/test/modules/injection_points/t/001_stats.pl
index d4539fe8727..25de5fc46fe 100644
--- a/src/test/modules/injection_points/t/001_stats.pl
+++ b/src/test/modules/injection_points/t/001_stats.pl
@@ -69,6 +69,19 @@ $fixedstats = $node->safe_psql('postgres',
"SELECT * FROM injection_points_stats_fixed();");
is($fixedstats, '0|0|0|0|0', 'fixed stats after crash');
+# On drop all stats are gone
+$node->safe_psql('postgres',
+ "SELECT injection_points_attach('stats-notice', 'notice');");
+$node->safe_psql('postgres', "SELECT injection_points_run('stats-notice');");
+$node->safe_psql('postgres', "SELECT injection_points_run('stats-notice');");
+$numcalls = $node->safe_psql('postgres',
+ "SELECT injection_points_stats_numcalls('stats-notice');");
+is($numcalls, '2', 'number of stats calls');
+$node->safe_psql('postgres', "SELECT injection_points_stats_drop();");
+$numcalls = $node->safe_psql('postgres',
+ "SELECT injection_points_stats_numcalls('stats-notice');");
+is($numcalls, '', 'no stats after drop via SQL function');
+
# Stop the server, disable the module, then restart. The server
# should be able to come up.
$node->stop;
--
2.47.2
On Mon, Jan 27, 2025 at 12:53:36PM +0900, Michael Paquier wrote:
Not sure about this part yet. I have looked at 0002 to begin with
something and it is really useful on its own. Stats kinds calling
this routine don't need to worry about the internals of dropping local
references or doing a seqscan on the shared hash table. However, what
you have sent lacks in flexibility to me, and the duplication with
pgstat_drop_all_entries is annoying. This had better be merged in a
single routine.
After thinking more about this one, I still want this toy and hearing
nothing I have applied it, with a second commit for the addition in
injection_points to avoid multiple bullet points in a single commit.
I have noticed post-commit that I have made a mistake in the credits
of a632cd354d35 and ce5c620fb625 for your family name. Really sorry
about that! This mistake is on me..
What do you think?
Attached is a rebased version of the three remaining patches. While
looking at this stuff, I have noticed an extra cleanup that would be
good to have, as a separate change: we could reformat a bit the plan
header comments so as these do not require a rewrite when adding
node_attr to them, like d575051b9af9.
Sami's patch set posted at [1]/messages/by-id/CAA5RZ0sUPPOpkRZD=Za83op2ngcPC7dp249vcHA-X5YS7p3n8Q@mail.gmail.com -- Michael has the same problem, making the
proposals harder to parse and review, and the devil is in the details
with these pg_node_attr() properties attached to the structures. That
would be something to do on top of the proposed patch sets. Would any
of you be interested in that?
[1]: /messages/by-id/CAA5RZ0sUPPOpkRZD=Za83op2ngcPC7dp249vcHA-X5YS7p3n8Q@mail.gmail.com -- Michael
--
Michael
Attachments:
v3-0001-Allow-using-jumbling-logic-outside-of-query-jumbl.patchtext/x-diff; charset=us-asciiDownload
From a232d30792f1a9a466d032cb12ddbdf328820e9a Mon Sep 17 00:00:00 2001
From: Lukas Fittl <lukas@fittl.com>
Date: Tue, 31 Dec 2024 15:05:39 -0800
Subject: [PATCH v3 1/3] Allow using jumbling logic outside of query jumble
unit file
This can be useful either for jumbling expressions in other contexts
(e.g. to calculate a plan jumble), or to allow extensions to use
a modified jumbling logic more easily.
This intentionally supports the use case where a separate jumbling logic
does not care about recording constants, as the query jumble does.
---
src/include/nodes/queryjumble.h | 6 +++
src/backend/nodes/queryjumblefuncs.c | 59 +++++++++++++++++-----------
2 files changed, 41 insertions(+), 24 deletions(-)
diff --git a/src/include/nodes/queryjumble.h b/src/include/nodes/queryjumble.h
index 50eb9566587..5afa6f3605f 100644
--- a/src/include/nodes/queryjumble.h
+++ b/src/include/nodes/queryjumble.h
@@ -83,4 +83,10 @@ IsQueryIdEnabled(void)
return query_id_enabled;
}
+/* Functions intended for other users of jumbling (e.g. plan jumbling) */
+extern JumbleState *InitializeJumbleState(bool record_clocations);
+extern void AppendJumble(JumbleState *jstate, const unsigned char *item, Size size);
+extern void JumbleNode(JumbleState *jstate, Node *node);
+extern uint64 HashJumbleState(JumbleState *jstate);
+
#endif /* QUERYJUMBLE_H */
diff --git a/src/backend/nodes/queryjumblefuncs.c b/src/backend/nodes/queryjumblefuncs.c
index b103a281936..545d8edcae2 100644
--- a/src/backend/nodes/queryjumblefuncs.c
+++ b/src/backend/nodes/queryjumblefuncs.c
@@ -51,10 +51,7 @@ int compute_query_id = COMPUTE_QUERY_ID_AUTO;
*/
bool query_id_enabled = false;
-static void AppendJumble(JumbleState *jstate,
- const unsigned char *item, Size size);
static void RecordConstLocation(JumbleState *jstate, int location);
-static void _jumbleNode(JumbleState *jstate, Node *node);
static void _jumbleA_Const(JumbleState *jstate, Node *node);
static void _jumbleList(JumbleState *jstate, Node *node);
static void _jumbleVariableSetStmt(JumbleState *jstate, Node *node);
@@ -109,28 +106,42 @@ CleanQuerytext(const char *query, int *location, int *len)
}
JumbleState *
-JumbleQuery(Query *query)
+InitializeJumbleState(bool record_clocations)
{
- JumbleState *jstate = NULL;
-
- Assert(IsQueryIdEnabled());
-
- jstate = (JumbleState *) palloc(sizeof(JumbleState));
+ JumbleState *jstate = (JumbleState *) palloc0(sizeof(JumbleState));
/* Set up workspace for query jumbling */
jstate->jumble = (unsigned char *) palloc(JUMBLE_SIZE);
jstate->jumble_len = 0;
- jstate->clocations_buf_size = 32;
- jstate->clocations = (LocationLen *)
- palloc(jstate->clocations_buf_size * sizeof(LocationLen));
- jstate->clocations_count = 0;
- jstate->highest_extern_param_id = 0;
+
+ if (record_clocations)
+ {
+ jstate->clocations_buf_size = 32;
+ jstate->clocations = (LocationLen *)
+ palloc(jstate->clocations_buf_size * sizeof(LocationLen));
+ }
+
+ return jstate;
+}
+
+uint64
+HashJumbleState(JumbleState *jstate)
+{
+ return DatumGetUInt64(hash_any_extended(jstate->jumble,
+ jstate->jumble_len,
+ 0));
+}
+
+JumbleState *
+JumbleQuery(Query *query)
+{
+ JumbleState *jstate = InitializeJumbleState(true);
+
+ Assert(IsQueryIdEnabled());
/* Compute query ID and mark the Query node with it */
- _jumbleNode(jstate, (Node *) query);
- query->queryId = DatumGetUInt64(hash_any_extended(jstate->jumble,
- jstate->jumble_len,
- 0));
+ JumbleNode(jstate, (Node *) query);
+ query->queryId = HashJumbleState(jstate);
/*
* If we are unlucky enough to get a hash of zero, use 1 instead for
@@ -164,7 +175,7 @@ EnableQueryId(void)
* AppendJumble: Append a value that is substantive in a given query to
* the current jumble.
*/
-static void
+void
AppendJumble(JumbleState *jstate, const unsigned char *item, Size size)
{
unsigned char *jumble = jstate->jumble;
@@ -205,7 +216,7 @@ static void
RecordConstLocation(JumbleState *jstate, int location)
{
/* -1 indicates unknown or undefined location */
- if (location >= 0)
+ if (location >= 0 && jstate->clocations_buf_size > 0)
{
/* enlarge array if needed */
if (jstate->clocations_count >= jstate->clocations_buf_size)
@@ -224,7 +235,7 @@ RecordConstLocation(JumbleState *jstate, int location)
}
#define JUMBLE_NODE(item) \
- _jumbleNode(jstate, (Node *) expr->item)
+ JumbleNode(jstate, (Node *) expr->item)
#define JUMBLE_LOCATION(location) \
RecordConstLocation(jstate, expr->location)
#define JUMBLE_FIELD(item) \
@@ -239,8 +250,8 @@ do { \
#include "queryjumblefuncs.funcs.c"
-static void
-_jumbleNode(JumbleState *jstate, Node *node)
+void
+JumbleNode(JumbleState *jstate, Node *node)
{
Node *expr = node;
@@ -305,7 +316,7 @@ _jumbleList(JumbleState *jstate, Node *node)
{
case T_List:
foreach(l, expr)
- _jumbleNode(jstate, lfirst(l));
+ JumbleNode(jstate, lfirst(l));
break;
case T_IntList:
foreach(l, expr)
--
2.47.2
v3-0002-Optionally-record-a-plan_id-in-PlannedStmt-to-ide.patchtext/x-diff; charset=us-asciiDownload
From 64e3f1c52cfe393ebe6c9bfa2b355207befa2bea Mon Sep 17 00:00:00 2001
From: Michael Paquier <michael@paquier.xyz>
Date: Fri, 31 Jan 2025 13:01:59 +0900
Subject: [PATCH v3 2/3] Optionally record a plan_id in PlannedStmt to identify
plan shape
When enabled via the new compute_plan_id GUC (default off), this utilizes
the existing treewalk in setrefs.c after planning to calculate a hash
(the "plan_id", or plan identifier) that can be used to identify
which plan was chosen.
The plan_id generally intends to be the same if a given EXPLAIN (without
ANALYZE) output is the same. The plan_id includes both the top-level plan
as well as all subplans. Execution statistics are excluded.
If enabled, the plan_id is shown for currently running queries in
pg_stat_activity, as well as recorded in EXPLAIN and auto_explain output.
Other in core users or extensions can use this facility to show or
accumulate statistics about the plans used by queries, to help identify
plan regressions, or drive plan management decisions.
Note that this commit intentionally does not include a facility to map
a given plan_id to the EXPLAIN text output - it is a assumed that users
can utilize the auto_explain extension to establish this mapping as
needed, or extensions can record this via the existing planner hook.
---
src/include/catalog/pg_proc.dat | 6 +-
src/include/nodes/pathnodes.h | 3 +
src/include/nodes/plannodes.h | 399 +++++++++++-------
src/include/nodes/primnodes.h | 7 +-
src/include/nodes/queryjumble.h | 34 +-
src/include/utils/backend_status.h | 5 +
src/backend/catalog/system_views.sql | 1 +
src/backend/commands/explain.c | 16 +
src/backend/executor/execMain.c | 10 +-
src/backend/executor/execParallel.c | 1 +
src/backend/nodes/gen_node_support.pl | 50 ++-
src/backend/nodes/queryjumblefuncs.c | 78 +++-
src/backend/optimizer/plan/planner.c | 18 +
src/backend/optimizer/plan/setrefs.c | 9 +
src/backend/postmaster/launch_backend.c | 3 +
src/backend/tcop/postgres.c | 1 +
src/backend/utils/activity/backend_status.c | 70 ++-
src/backend/utils/adt/pgstatfuncs.c | 7 +-
src/backend/utils/misc/guc_tables.c | 28 ++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/test/regress/expected/explain.out | 11 +
src/test/regress/expected/rules.out | 9 +-
src/test/regress/sql/explain.sql | 4 +
doc/src/sgml/config.sgml | 34 ++
doc/src/sgml/monitoring.sgml | 16 +
25 files changed, 635 insertions(+), 186 deletions(-)
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 5b8c2ad2a54..5547db7d2f1 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5568,9 +5568,9 @@
proname => 'pg_stat_get_activity', prorows => '100', proisstrict => 'f',
proretset => 't', provolatile => 's', proparallel => 'r',
prorettype => 'record', proargtypes => 'int4',
- proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,bool,int4,int8}',
- proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,gss_delegation,leader_pid,query_id}',
+ proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,bool,int4,int8,int8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,gss_delegation,leader_pid,query_id,plan_id}',
prosrc => 'pg_stat_get_activity' },
{ oid => '6318', descr => 'describe wait events',
proname => 'pg_get_wait_events', procost => '10', prorows => '250',
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 52d44f43021..9603e0edda0 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -166,6 +166,9 @@ typedef struct PlannerGlobal
/* partition descriptors */
PartitionDirectory partition_directory pg_node_attr(read_write_ignore);
+
+ /* optional jumble state for plan identifier claculation */
+ struct JumbleState *plan_jumble_state pg_node_attr(read_write_ignore);
} PlannerGlobal;
/* macro for fetching the Plan associated with a SubPlan node */
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 8143744e89c..f5574baf379 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -53,6 +53,10 @@ typedef struct PlannedStmt
uint64 queryId; /* query identifier (copied from Query) */
+ uint64 planId; /* plan identifier (calculated if
+ * compute_plan_id is enabled, can also be set
+ * by plugins) */
+
bool hasReturning; /* is it insert|update|delete|merge RETURNING? */
bool hasModifyingCTE; /* has insert|update|delete|merge in WITH? */
@@ -121,44 +125,55 @@ typedef struct PlannedStmt
*/
typedef struct Plan
{
- pg_node_attr(abstract, no_equal, no_query_jumble)
+ pg_node_attr(abstract, no_equal)
NodeTag type;
/*
* estimated execution costs for plan (see costsize.c for more info)
*/
- int disabled_nodes; /* count of disabled nodes */
- Cost startup_cost; /* cost expended before fetching any tuples */
- Cost total_cost; /* total cost (assuming all tuples fetched) */
+ int disabled_nodes pg_node_attr(query_jumble_ignore); /* count of disabled
+ * nodes */
+ Cost startup_cost pg_node_attr(query_jumble_ignore); /* cost expended before
+ * fetching any tuples */
+ Cost total_cost pg_node_attr(query_jumble_ignore); /* total cost (assuming
+ * all tuples fetched) */
/*
* planner's estimate of result size of this plan step
*/
- Cardinality plan_rows; /* number of rows plan is expected to emit */
- int plan_width; /* average row width in bytes */
+ Cardinality plan_rows pg_node_attr(query_jumble_ignore); /* number of rows plan
+ * is expected to emit */
+ int plan_width pg_node_attr(query_jumble_ignore); /* average row width in
+ * bytes */
/*
* information needed for parallel query
*/
- bool parallel_aware; /* engage parallel-aware logic? */
- bool parallel_safe; /* OK to use as part of parallel plan? */
+ bool parallel_aware pg_node_attr(query_jumble_ignore); /* engage parallel-aware
+ * logic? */
+ bool parallel_safe pg_node_attr(query_jumble_ignore); /* OK to use as part of
+ * parallel plan? */
/*
* information needed for asynchronous execution
*/
- bool async_capable; /* engage asynchronous-capable logic? */
+ bool async_capable pg_node_attr(query_jumble_ignore); /* engage
+ * asynchronous-capable
+ * logic? */
/*
* Common structural data for all Plan types.
*/
- int plan_node_id; /* unique across entire final plan tree */
+ int plan_node_id pg_node_attr(query_jumble_ignore); /* unique across entire
+ * final plan tree */
List *targetlist; /* target list to be computed at this node */
List *qual; /* implicitly-ANDed qual conditions */
- struct Plan *lefttree; /* input plan tree(s) */
- struct Plan *righttree;
- List *initPlan; /* Init Plan nodes (un-correlated expr
- * subselects) */
+ struct Plan *lefttree pg_node_attr(query_jumble_ignore); /* input plan tree(s) */
+ struct Plan *righttree pg_node_attr(query_jumble_ignore);
+ List *initPlan pg_node_attr(query_jumble_ignore); /* Init Plan nodes
+ * (un-correlated expr
+ * subselects) */
/*
* Information for management of parameter-change-driven rescanning
@@ -171,8 +186,8 @@ typedef struct Plan
* params that affect the node (i.e., the setParams of its initplans).
* These are _all_ the PARAM_EXEC params that affect this node.
*/
- Bitmapset *extParam;
- Bitmapset *allParam;
+ Bitmapset *extParam pg_node_attr(query_jumble_ignore);
+ Bitmapset *allParam pg_node_attr(query_jumble_ignore);
} Plan;
/* ----------------
@@ -234,31 +249,47 @@ typedef struct ModifyTable
{
Plan plan;
CmdType operation; /* INSERT, UPDATE, DELETE, or MERGE */
- bool canSetTag; /* do we set the command tag/es_processed? */
- Index nominalRelation; /* Parent RT index for use of EXPLAIN */
- Index rootRelation; /* Root RT index, if partitioned/inherited */
+ bool canSetTag pg_node_attr(query_jumble_ignore); /* do we set the command
+ * tag/es_processed? */
+ Index nominalRelation pg_node_attr(query_jumble_ignore); /* Parent RT index for
+ * use of EXPLAIN */
+ Index rootRelation pg_node_attr(query_jumble_rt_index); /* Root RT index, if
+ * partitioned/inherited */
bool partColsUpdated; /* some part key in hierarchy updated? */
- List *resultRelations; /* integer list of RT indexes */
- List *updateColnosLists; /* per-target-table update_colnos lists */
- List *withCheckOptionLists; /* per-target-table WCO lists */
- char *returningOldAlias; /* alias for OLD in RETURNING lists */
- char *returningNewAlias; /* alias for NEW in RETURNING lists */
- List *returningLists; /* per-target-table RETURNING tlists */
- List *fdwPrivLists; /* per-target-table FDW private data lists */
- Bitmapset *fdwDirectModifyPlans; /* indices of FDW DM plans */
- List *rowMarks; /* PlanRowMarks (non-locking only) */
- int epqParam; /* ID of Param for EvalPlanQual re-eval */
+ List *resultRelations pg_node_attr(query_jumble_rt_index); /* integer list of RT
+ * indexes */
+ List *updateColnosLists pg_node_attr(query_jumble_ignore); /* per-target-table
+ * update_colnos lists */
+ List *withCheckOptionLists pg_node_attr(query_jumble_ignore); /* per-target-table WCO
+ * lists */
+ char *returningOldAlias pg_node_attr(query_jumble_ignore); /* alias for OLD in
+ * RETURNING lists */
+ char *returningNewAlias pg_node_attr(query_jumble_ignore); /* alias for NEW in
+ * RETURNING lists */
+ List *returningLists pg_node_attr(query_jumble_ignore); /* per-target-table
+ * RETURNING tlists */
+ List *fdwPrivLists pg_node_attr(query_jumble_ignore); /* per-target-table FDW
+ * private data lists */
+ Bitmapset *fdwDirectModifyPlans pg_node_attr(query_jumble_ignore); /* indices of FDW DM
+ * plans */
+ List *rowMarks pg_node_attr(query_jumble_ignore); /* PlanRowMarks
+ * (non-locking only) */
+ int epqParam pg_node_attr(query_jumble_ignore); /* ID of Param for
+ * EvalPlanQual re-eval */
OnConflictAction onConflictAction; /* ON CONFLICT action */
List *arbiterIndexes; /* List of ON CONFLICT arbiter index OIDs */
List *onConflictSet; /* INSERT ON CONFLICT DO UPDATE targetlist */
List *onConflictCols; /* target column numbers for onConflictSet */
Node *onConflictWhere; /* WHERE for ON CONFLICT UPDATE */
- Index exclRelRTI; /* RTI of the EXCLUDED pseudo relation */
- List *exclRelTlist; /* tlist of the EXCLUDED pseudo relation */
- List *mergeActionLists; /* per-target-table lists of actions for
- * MERGE */
- List *mergeJoinConditions; /* per-target-table join conditions
- * for MERGE */
+ Index exclRelRTI pg_node_attr(query_jumble_ignore); /* RTI of the EXCLUDED
+ * pseudo relation */
+ List *exclRelTlist pg_node_attr(query_jumble_ignore); /* tlist of the EXCLUDED
+ * pseudo relation */
+ List *mergeActionLists pg_node_attr(query_jumble_ignore); /* per-target-table
+ * lists of actions for
+ * MERGE */
+ List *mergeJoinConditions pg_node_attr(query_jumble_ignore); /* per-target-table join
+ * conditions for MERGE */
} ModifyTable;
struct PartitionPruneInfo; /* forward reference to struct below */
@@ -271,18 +302,20 @@ struct PartitionPruneInfo; /* forward reference to struct below */
typedef struct Append
{
Plan plan;
- Bitmapset *apprelids; /* RTIs of appendrel(s) formed by this node */
- List *appendplans;
- int nasyncplans; /* # of asynchronous plans */
+ /* RTIs of appendrel(s) formed by this node */
+ Bitmapset *apprelids pg_node_attr(query_jumble_ignore);
+ List *appendplans pg_node_attr(query_jumble_ignore);
+ /* # of asynchronous plans */
+ int nasyncplans pg_node_attr(query_jumble_ignore);
/*
* All 'appendplans' preceding this index are non-partial plans. All
* 'appendplans' from this index onwards are partial plans.
*/
- int first_partial_plan;
+ int first_partial_plan pg_node_attr(query_jumble_ignore);
/* Index to PlannerInfo.partPruneInfos or -1 if no run-time pruning */
- int part_prune_index;
+ int part_prune_index pg_node_attr(query_jumble_ignore);
} Append;
/* ----------------
@@ -295,29 +328,29 @@ typedef struct MergeAppend
Plan plan;
/* RTIs of appendrel(s) formed by this node */
- Bitmapset *apprelids;
+ Bitmapset *apprelids pg_node_attr(query_jumble_ignore);
- List *mergeplans;
+ List *mergeplans pg_node_attr(query_jumble_ignore);
/* these fields are just like the sort-key info in struct Sort: */
/* number of sort-key columns */
- int numCols;
+ int numCols pg_node_attr(query_jumble_ignore);
/* their indexes in the target list */
- AttrNumber *sortColIdx pg_node_attr(array_size(numCols));
+ AttrNumber *sortColIdx pg_node_attr(array_size(numCols), query_jumble_ignore);
/* OIDs of operators to sort them by */
- Oid *sortOperators pg_node_attr(array_size(numCols));
+ Oid *sortOperators pg_node_attr(array_size(numCols), query_jumble_ignore);
/* OIDs of collations */
- Oid *collations pg_node_attr(array_size(numCols));
+ Oid *collations pg_node_attr(array_size(numCols), query_jumble_ignore);
/* NULLS FIRST/LAST directions */
- bool *nullsFirst pg_node_attr(array_size(numCols));
+ bool *nullsFirst pg_node_attr(array_size(numCols), query_jumble_ignore);
/* Index to PlannerInfo.partPruneInfos or -1 if no run-time pruning */
- int part_prune_index;
+ int part_prune_index pg_node_attr(query_jumble_ignore);
} MergeAppend;
/* ----------------
@@ -333,22 +366,22 @@ typedef struct RecursiveUnion
Plan plan;
/* ID of Param representing work table */
- int wtParam;
+ int wtParam pg_node_attr(query_jumble_ignore);
/* Remaining fields are zero/null in UNION ALL case */
/* number of columns to check for duplicate-ness */
- int numCols;
+ int numCols pg_node_attr(query_jumble_ignore);
/* their indexes in the target list */
- AttrNumber *dupColIdx pg_node_attr(array_size(numCols));
+ AttrNumber *dupColIdx pg_node_attr(array_size(numCols), query_jumble_ignore);
/* equality operators to compare with */
- Oid *dupOperators pg_node_attr(array_size(numCols));
- Oid *dupCollations pg_node_attr(array_size(numCols));
+ Oid *dupOperators pg_node_attr(array_size(numCols), query_jumble_ignore);
+ Oid *dupCollations pg_node_attr(array_size(numCols), query_jumble_ignore);
/* estimated number of groups in input */
- long numGroups;
+ long numGroups pg_node_attr(query_jumble_ignore);
} RecursiveUnion;
/* ----------------
@@ -362,7 +395,7 @@ typedef struct RecursiveUnion
typedef struct BitmapAnd
{
Plan plan;
- List *bitmapplans;
+ List *bitmapplans pg_node_attr(query_jumble_ignore);
} BitmapAnd;
/* ----------------
@@ -376,8 +409,8 @@ typedef struct BitmapAnd
typedef struct BitmapOr
{
Plan plan;
- bool isshared;
- List *bitmapplans;
+ bool isshared pg_node_attr(query_jumble_ignore);
+ List *bitmapplans pg_node_attr(query_jumble_ignore);
} BitmapOr;
/*
@@ -392,7 +425,8 @@ typedef struct Scan
pg_node_attr(abstract)
Plan plan;
- Index scanrelid; /* relid is index into the range table */
+ Index scanrelid pg_node_attr(query_jumble_rt_index); /* relid is index into
+ * the range table */
} Scan;
/* ----------------
@@ -457,9 +491,11 @@ typedef struct IndexScan
Scan scan;
Oid indexid; /* OID of index to scan */
List *indexqual; /* list of index quals (usually OpExprs) */
- List *indexqualorig; /* the same in original form */
+ List *indexqualorig pg_node_attr(query_jumble_ignore); /* the same in original
+ * form */
List *indexorderby; /* list of index ORDER BY exprs */
- List *indexorderbyorig; /* the same in original form */
+ List *indexorderbyorig pg_node_attr(query_jumble_ignore); /* the same in original
+ * form */
List *indexorderbyops; /* OIDs of sort ops for ORDER BY exprs */
ScanDirection indexorderdir; /* forward or backward or don't care */
} IndexScan;
@@ -500,9 +536,12 @@ typedef struct IndexOnlyScan
Scan scan;
Oid indexid; /* OID of index to scan */
List *indexqual; /* list of index quals (usually OpExprs) */
- List *recheckqual; /* index quals in recheckable form */
+ List *recheckqual pg_node_attr(query_jumble_ignore); /* index quals in
+ * recheckable form */
List *indexorderby; /* list of index ORDER BY exprs */
- List *indextlist; /* TargetEntry list describing index's cols */
+ List *indextlist pg_node_attr(query_jumble_ignore); /* TargetEntry list
+ * describing index's
+ * cols */
ScanDirection indexorderdir; /* forward or backward or don't care */
} IndexOnlyScan;
@@ -527,9 +566,11 @@ typedef struct BitmapIndexScan
{
Scan scan;
Oid indexid; /* OID of index to scan */
- bool isshared; /* Create shared bitmap if set */
+ bool isshared pg_node_attr(query_jumble_ignore); /* Create shared bitmap
+ * if set */
List *indexqual; /* list of index quals (OpExprs) */
- List *indexqualorig; /* the same in original form */
+ List *indexqualorig pg_node_attr(query_jumble_ignore); /* the same in original
+ * form */
} BitmapIndexScan;
/* ----------------
@@ -544,7 +585,8 @@ typedef struct BitmapIndexScan
typedef struct BitmapHeapScan
{
Scan scan;
- List *bitmapqualorig; /* index quals, in standard expr form */
+ List *bitmapqualorig pg_node_attr(query_jumble_ignore); /* index quals, in
+ * standard expr form */
} BitmapHeapScan;
/* ----------------
@@ -604,8 +646,8 @@ typedef enum SubqueryScanStatus
typedef struct SubqueryScan
{
Scan scan;
- Plan *subplan;
- SubqueryScanStatus scanstatus;
+ Plan *subplan pg_node_attr(query_jumble_ignore);
+ SubqueryScanStatus scanstatus pg_node_attr(query_jumble_ignore);
} SubqueryScan;
/* ----------------
@@ -646,8 +688,11 @@ typedef struct TableFuncScan
typedef struct CteScan
{
Scan scan;
- int ctePlanId; /* ID of init SubPlan for CTE */
- int cteParam; /* ID of Param representing CTE output */
+ int ctePlanId pg_node_attr(query_jumble_ignore); /* ID of init SubPlan
+ * for CTE */
+ int cteParam pg_node_attr(query_jumble_ignore); /* ID of Param
+ * representing CTE
+ * output */
} CteScan;
/* ----------------
@@ -667,7 +712,9 @@ typedef struct NamedTuplestoreScan
typedef struct WorkTableScan
{
Scan scan;
- int wtParam; /* ID of Param representing work table */
+ int wtParam pg_node_attr(query_jumble_ignore); /* ID of Param
+ * representing work
+ * table */
} WorkTableScan;
/* ----------------
@@ -714,17 +761,26 @@ typedef struct ForeignScan
{
Scan scan;
CmdType operation; /* SELECT/INSERT/UPDATE/DELETE */
- Index resultRelation; /* direct modification target's RT index */
- Oid checkAsUser; /* user to perform the scan as; 0 means to
- * check as current user */
+ Index resultRelation pg_node_attr(query_jumble_ignore); /* direct modification
+ * target's RT index */
+ Oid checkAsUser pg_node_attr(query_jumble_ignore); /* user to perform the
+ * scan as; 0 means to
+ * check as current user */
Oid fs_server; /* OID of foreign server */
- List *fdw_exprs; /* expressions that FDW may evaluate */
- List *fdw_private; /* private data for FDW */
- List *fdw_scan_tlist; /* optional tlist describing scan tuple */
- List *fdw_recheck_quals; /* original quals not in scan.plan.qual */
- Bitmapset *fs_relids; /* base+OJ RTIs generated by this scan */
- Bitmapset *fs_base_relids; /* base RTIs generated by this scan */
- bool fsSystemCol; /* true if any "system column" is needed */
+ List *fdw_exprs pg_node_attr(query_jumble_ignore); /* expressions that FDW
+ * may evaluate */
+ List *fdw_private pg_node_attr(query_jumble_ignore); /* private data for FDW */
+ List *fdw_scan_tlist pg_node_attr(query_jumble_ignore); /* optional tlist
+ * describing scan tuple */
+ List *fdw_recheck_quals pg_node_attr(query_jumble_ignore); /* original quals not in
+ * scan.plan.qual */
+ Bitmapset *fs_relids pg_node_attr(query_jumble_ignore); /* base+OJ RTIs
+ * generated by this
+ * scan */
+ Bitmapset *fs_base_relids pg_node_attr(query_jumble_ignore); /* base RTIs generated
+ * by this scan */
+ bool fsSystemCol pg_node_attr(query_jumble_ignore); /* true if any "system
+ * column" is needed */
} ForeignScan;
/* ----------------
@@ -745,20 +801,27 @@ struct CustomScanMethods;
typedef struct CustomScan
{
Scan scan;
- uint32 flags; /* mask of CUSTOMPATH_* flags, see
- * nodes/extensible.h */
- List *custom_plans; /* list of Plan nodes, if any */
- List *custom_exprs; /* expressions that custom code may evaluate */
- List *custom_private; /* private data for custom code */
- List *custom_scan_tlist; /* optional tlist describing scan tuple */
- Bitmapset *custom_relids; /* RTIs generated by this scan */
+ uint32 flags pg_node_attr(query_jumble_ignore); /* mask of CUSTOMPATH_*
+ * flags, see
+ * nodes/extensible.h */
+ List *custom_plans pg_node_attr(query_jumble_ignore); /* list of Plan nodes,
+ * if any */
+ List *custom_exprs pg_node_attr(query_jumble_ignore); /* expressions that
+ * custom code may
+ * evaluate */
+ List *custom_private pg_node_attr(query_jumble_ignore); /* private data for
+ * custom code */
+ List *custom_scan_tlist pg_node_attr(query_jumble_ignore); /* optional tlist
+ * describing scan tuple */
+ Bitmapset *custom_relids pg_node_attr(query_jumble_ignore); /* RTIs generated by
+ * this scan */
/*
* NOTE: The method field of CustomScan is required to be a pointer to a
* static table of callback functions. So we don't copy the table itself,
* just reference the original one.
*/
- const struct CustomScanMethods *methods;
+ const struct CustomScanMethods *methods pg_node_attr(query_jumble_ignore);
} CustomScan;
/*
@@ -795,7 +858,7 @@ typedef struct Join
Plan plan;
JoinType jointype;
- bool inner_unique;
+ bool inner_unique pg_node_attr(query_jumble_ignore);
List *joinqual; /* JOIN quals (in addition to plan.qual) */
} Join;
@@ -818,7 +881,7 @@ typedef struct NestLoop
typedef struct NestLoopParam
{
- pg_node_attr(no_equal, no_query_jumble)
+ pg_node_attr(no_equal)
NodeTag type;
int paramno; /* number of the PARAM_EXEC Param to set */
@@ -841,7 +904,7 @@ typedef struct MergeJoin
Join join;
/* Can we skip mark/restore calls? */
- bool skip_mark_restore;
+ bool skip_mark_restore pg_node_attr(query_jumble_ignore);
/* mergeclauses as expression trees */
List *mergeclauses;
@@ -897,13 +960,13 @@ typedef struct Memoize
Plan plan;
/* size of the two arrays below */
- int numKeys;
+ int numKeys pg_node_attr(query_jumble_ignore);
/* hash operators for each key */
- Oid *hashOperators pg_node_attr(array_size(numKeys));
+ Oid *hashOperators pg_node_attr(array_size(numKeys), query_jumble_ignore);
/* collations for each key */
- Oid *collations pg_node_attr(array_size(numKeys));
+ Oid *collations pg_node_attr(array_size(numKeys), query_jumble_ignore);
/* cache keys in the form of exprs containing parameters */
List *param_exprs;
@@ -912,7 +975,7 @@ typedef struct Memoize
* true if the cache entry should be marked as complete after we store the
* first tuple in it.
*/
- bool singlerow;
+ bool singlerow pg_node_attr(query_jumble_ignore);
/*
* true when cache key should be compared bit by bit, false when using
@@ -924,10 +987,10 @@ typedef struct Memoize
* The maximum number of entries that the planner expects will fit in the
* cache, or 0 if unknown
*/
- uint32 est_entries;
+ uint32 est_entries pg_node_attr(query_jumble_ignore);
/* paramids from param_exprs */
- Bitmapset *keyparamids;
+ Bitmapset *keyparamids pg_node_attr(query_jumble_ignore);
} Memoize;
/* ----------------
@@ -1010,31 +1073,31 @@ typedef struct Agg
AggSplit aggsplit;
/* number of grouping columns */
- int numCols;
+ int numCols pg_node_attr(query_jumble_ignore);
/* their indexes in the target list */
- AttrNumber *grpColIdx pg_node_attr(array_size(numCols));
+ AttrNumber *grpColIdx pg_node_attr(array_size(numCols), query_jumble_ignore);
/* equality operators to compare with */
- Oid *grpOperators pg_node_attr(array_size(numCols));
- Oid *grpCollations pg_node_attr(array_size(numCols));
+ Oid *grpOperators pg_node_attr(array_size(numCols), query_jumble_ignore);
+ Oid *grpCollations pg_node_attr(array_size(numCols), query_jumble_ignore);
/* estimated number of groups in input */
- long numGroups;
+ long numGroups pg_node_attr(query_jumble_ignore);
/* for pass-by-ref transition data */
- uint64 transitionSpace;
+ uint64 transitionSpace pg_node_attr(query_jumble_ignore);
/* IDs of Params used in Aggref inputs */
- Bitmapset *aggParams;
+ Bitmapset *aggParams pg_node_attr(query_jumble_ignore);
/* Note: planner provides numGroups & aggParams only in HASHED/MIXED case */
/* grouping sets to use */
- List *groupingSets;
+ List *groupingSets pg_node_attr(query_jumble_ignore);
/* chained Agg/Sort nodes */
- List *chain;
+ List *chain pg_node_attr(query_jumble_ignore);
} Agg;
/* ----------------
@@ -1046,43 +1109,43 @@ typedef struct WindowAgg
Plan plan;
/* ID referenced by window functions */
- Index winref;
+ Index winref pg_node_attr(query_jumble_ignore);
/* number of columns in partition clause */
- int partNumCols;
+ int partNumCols pg_node_attr(query_jumble_ignore);
/* their indexes in the target list */
- AttrNumber *partColIdx pg_node_attr(array_size(partNumCols));
+ AttrNumber *partColIdx pg_node_attr(array_size(partNumCols), query_jumble_ignore);
/* equality operators for partition columns */
- Oid *partOperators pg_node_attr(array_size(partNumCols));
+ Oid *partOperators pg_node_attr(array_size(partNumCols), query_jumble_ignore);
/* collations for partition columns */
- Oid *partCollations pg_node_attr(array_size(partNumCols));
+ Oid *partCollations pg_node_attr(array_size(partNumCols), query_jumble_ignore);
/* number of columns in ordering clause */
- int ordNumCols;
+ int ordNumCols pg_node_attr(query_jumble_ignore);
/* their indexes in the target list */
- AttrNumber *ordColIdx pg_node_attr(array_size(ordNumCols));
+ AttrNumber *ordColIdx pg_node_attr(array_size(ordNumCols), query_jumble_ignore);
/* equality operators for ordering columns */
- Oid *ordOperators pg_node_attr(array_size(ordNumCols));
+ Oid *ordOperators pg_node_attr(array_size(ordNumCols), query_jumble_ignore);
/* collations for ordering columns */
- Oid *ordCollations pg_node_attr(array_size(ordNumCols));
+ Oid *ordCollations pg_node_attr(array_size(ordNumCols), query_jumble_ignore);
/* frame_clause options, see WindowDef */
- int frameOptions;
+ int frameOptions pg_node_attr(query_jumble_ignore);
/* expression for starting bound, if any */
- Node *startOffset;
+ Node *startOffset pg_node_attr(query_jumble_ignore);
/* expression for ending bound, if any */
- Node *endOffset;
+ Node *endOffset pg_node_attr(query_jumble_ignore);
/* qual to help short-circuit execution */
- List *runCondition;
+ List *runCondition pg_node_attr(query_jumble_ignore);
/* runCondition for display in EXPLAIN */
List *runConditionOrig;
@@ -1090,25 +1153,25 @@ typedef struct WindowAgg
/* these fields are used with RANGE offset PRECEDING/FOLLOWING: */
/* in_range function for startOffset */
- Oid startInRangeFunc;
+ Oid startInRangeFunc pg_node_attr(query_jumble_ignore);
/* in_range function for endOffset */
- Oid endInRangeFunc;
+ Oid endInRangeFunc pg_node_attr(query_jumble_ignore);
/* collation for in_range tests */
- Oid inRangeColl;
+ Oid inRangeColl pg_node_attr(query_jumble_ignore);
/* use ASC sort order for in_range tests? */
- bool inRangeAsc;
+ bool inRangeAsc pg_node_attr(query_jumble_ignore);
/* nulls sort first for in_range tests? */
- bool inRangeNullsFirst;
+ bool inRangeNullsFirst pg_node_attr(query_jumble_ignore);
/*
* false for all apart from the WindowAgg that's closest to the root of
* the plan
*/
- bool topWindow;
+ bool topWindow pg_node_attr(query_jumble_ignore);
} WindowAgg;
/* ----------------
@@ -1120,16 +1183,16 @@ typedef struct Unique
Plan plan;
/* number of columns to check for uniqueness */
- int numCols;
+ int numCols pg_node_attr(query_jumble_ignore);
/* their indexes in the target list */
- AttrNumber *uniqColIdx pg_node_attr(array_size(numCols));
+ AttrNumber *uniqColIdx pg_node_attr(array_size(numCols), query_jumble_ignore);
/* equality operators to compare with */
- Oid *uniqOperators pg_node_attr(array_size(numCols));
+ Oid *uniqOperators pg_node_attr(array_size(numCols), query_jumble_ignore);
/* collations for equality comparisons */
- Oid *uniqCollations pg_node_attr(array_size(numCols));
+ Oid *uniqCollations pg_node_attr(array_size(numCols), query_jumble_ignore);
} Unique;
/* ------------
@@ -1146,12 +1209,19 @@ typedef struct Unique
typedef struct Gather
{
Plan plan;
- int num_workers; /* planned number of worker processes */
- int rescan_param; /* ID of Param that signals a rescan, or -1 */
- bool single_copy; /* don't execute plan more than once */
- bool invisible; /* suppress EXPLAIN display (for testing)? */
- Bitmapset *initParam; /* param id's of initplans which are referred
- * at gather or one of its child nodes */
+ /* planned number of worker processes */
+ int num_workers;
+ /* ID of Param that signals a rescan, or -1 */
+ int rescan_param pg_node_attr(query_jumble_ignore);
+ /* don't execute plan more than once */
+ bool single_copy pg_node_attr(query_jumble_ignore);
+ /* suppress EXPLAIN display (for testing)? */
+ bool invisible pg_node_attr(query_jumble_ignore);
+ /*
+ * param id's of initplans which are referred at gather or one of its
+ * child nodes
+ */
+ Bitmapset *initParam pg_node_attr(query_jumble_ignore);
} Gather;
/* ------------
@@ -1166,30 +1236,30 @@ typedef struct GatherMerge
int num_workers;
/* ID of Param that signals a rescan, or -1 */
- int rescan_param;
+ int rescan_param pg_node_attr(query_jumble_ignore);
/* remaining fields are just like the sort-key info in struct Sort */
/* number of sort-key columns */
- int numCols;
+ int numCols pg_node_attr(query_jumble_ignore);
/* their indexes in the target list */
- AttrNumber *sortColIdx pg_node_attr(array_size(numCols));
+ AttrNumber *sortColIdx pg_node_attr(array_size(numCols), query_jumble_ignore);
/* OIDs of operators to sort them by */
- Oid *sortOperators pg_node_attr(array_size(numCols));
+ Oid *sortOperators pg_node_attr(array_size(numCols), query_jumble_ignore);
/* OIDs of collations */
- Oid *collations pg_node_attr(array_size(numCols));
+ Oid *collations pg_node_attr(array_size(numCols), query_jumble_ignore);
/* NULLS FIRST/LAST directions */
- bool *nullsFirst pg_node_attr(array_size(numCols));
+ bool *nullsFirst pg_node_attr(array_size(numCols), query_jumble_ignore);
/*
* param id's of initplans which are referred at gather merge or one of
* its child nodes
*/
- Bitmapset *initParam;
+ Bitmapset *initParam pg_node_attr(query_jumble_ignore);
} GatherMerge;
/* ----------------
@@ -1209,11 +1279,16 @@ typedef struct Hash
* needed to put them into the hashtable.
*/
List *hashkeys; /* hash keys for the hashjoin condition */
- Oid skewTable; /* outer join key's table OID, or InvalidOid */
- AttrNumber skewColumn; /* outer join key's column #, or zero */
- bool skewInherit; /* is outer join rel an inheritance tree? */
+ Oid skewTable pg_node_attr(query_jumble_ignore); /* outer join key's
+ * table OID, or
+ * InvalidOid */
+ AttrNumber skewColumn pg_node_attr(query_jumble_ignore); /* outer join key's
+ * column #, or zero */
+ bool skewInherit pg_node_attr(query_jumble_ignore); /* is outer join rel an
+ * inheritance tree? */
/* all other info is in the parent HashJoin node */
- Cardinality rows_total; /* estimate total rows if parallel_aware */
+ Cardinality rows_total pg_node_attr(query_jumble_ignore); /* estimate total rows
+ * if parallel_aware */
} Hash;
/* ----------------
@@ -1231,20 +1306,20 @@ typedef struct SetOp
SetOpStrategy strategy;
/* number of columns to compare */
- int numCols;
+ int numCols pg_node_attr(query_jumble_ignore);
/* their indexes in the target list */
- AttrNumber *cmpColIdx pg_node_attr(array_size(numCols));
+ AttrNumber *cmpColIdx pg_node_attr(array_size(numCols), query_jumble_ignore);
/* comparison operators (either equality operators or sort operators) */
- Oid *cmpOperators pg_node_attr(array_size(numCols));
- Oid *cmpCollations pg_node_attr(array_size(numCols));
+ Oid *cmpOperators pg_node_attr(array_size(numCols), query_jumble_ignore);
+ Oid *cmpCollations pg_node_attr(array_size(numCols), query_jumble_ignore);
/* nulls-first flags if sorting, otherwise not interesting */
- bool *cmpNullsFirst pg_node_attr(array_size(numCols));
+ bool *cmpNullsFirst pg_node_attr(array_size(numCols), query_jumble_ignore);
/* estimated number of groups in left input */
- long numGroups;
+ long numGroups pg_node_attr(query_jumble_ignore);
} SetOp;
/* ----------------
@@ -1259,8 +1334,10 @@ typedef struct SetOp
typedef struct LockRows
{
Plan plan;
- List *rowMarks; /* a list of PlanRowMark's */
- int epqParam; /* ID of Param for EvalPlanQual re-eval */
+ List *rowMarks pg_node_attr(query_jumble_ignore); /* a list of
+ * PlanRowMark's */
+ int epqParam pg_node_attr(query_jumble_ignore); /* ID of Param for
+ * EvalPlanQual re-eval */
} LockRows;
/* ----------------
@@ -1275,25 +1352,25 @@ typedef struct Limit
Plan plan;
/* OFFSET parameter, or NULL if none */
- Node *limitOffset;
+ Node *limitOffset pg_node_attr(query_jumble_ignore);
/* COUNT parameter, or NULL if none */
- Node *limitCount;
+ Node *limitCount pg_node_attr(query_jumble_ignore);
/* limit type */
- LimitOption limitOption;
+ LimitOption limitOption pg_node_attr(query_jumble_ignore);
/* number of columns to check for similarity */
- int uniqNumCols;
+ int uniqNumCols pg_node_attr(query_jumble_ignore);
/* their indexes in the target list */
- AttrNumber *uniqColIdx pg_node_attr(array_size(uniqNumCols));
+ AttrNumber *uniqColIdx pg_node_attr(array_size(uniqNumCols), query_jumble_ignore);
/* equality operators to compare with */
- Oid *uniqOperators pg_node_attr(array_size(uniqNumCols));
+ Oid *uniqOperators pg_node_attr(array_size(uniqNumCols), query_jumble_ignore);
/* collations for equality comparisons */
- Oid *uniqCollations pg_node_attr(array_size(uniqNumCols));
+ Oid *uniqCollations pg_node_attr(array_size(uniqNumCols), query_jumble_ignore);
} Limit;
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
index 59e7bb26bbd..c09785a0728 100644
--- a/src/include/nodes/primnodes.h
+++ b/src/include/nodes/primnodes.h
@@ -1074,8 +1074,6 @@ typedef struct SubLink
*/
typedef struct SubPlan
{
- pg_node_attr(no_query_jumble)
-
Expr xpr;
/* Fields copied from original SubLink: */
SubLinkType subLinkType; /* see above */
@@ -1106,8 +1104,9 @@ typedef struct SubPlan
List *parParam; /* indices of input Params from parent plan */
List *args; /* exprs to pass as parParam values */
/* Estimated execution costs: */
- Cost startup_cost; /* one-time setup cost */
- Cost per_call_cost; /* cost for each subplan evaluation */
+ Cost startup_cost pg_node_attr(query_jumble_ignore); /* one-time setup cost */
+ Cost per_call_cost pg_node_attr(query_jumble_ignore); /* cost for each subplan
+ * evaluation */
} SubPlan;
/*
diff --git a/src/include/nodes/queryjumble.h b/src/include/nodes/queryjumble.h
index 5afa6f3605f..6c3b787b46f 100644
--- a/src/include/nodes/queryjumble.h
+++ b/src/include/nodes/queryjumble.h
@@ -15,6 +15,7 @@
#define QUERYJUMBLE_H
#include "nodes/parsenodes.h"
+#include "nodes/pathnodes.h"
/*
* Struct for tracking locations/lengths of constants during normalization
@@ -48,6 +49,9 @@ typedef struct JumbleState
/* highest Param id we've seen, in order to start normalization correctly */
int highest_extern_param_id;
+
+ /* planner global info for resolving RT indexes when plan jumbling */
+ PlannerGlobal *glob;
} JumbleState;
/* Values for the compute_query_id GUC */
@@ -59,15 +63,27 @@ enum ComputeQueryIdType
COMPUTE_QUERY_ID_REGRESS,
};
+/* Values for the compute_plan_id GUC */
+enum ComputePlanIdType
+{
+ COMPUTE_PLAN_ID_OFF,
+ COMPUTE_PLAN_ID_ON,
+ COMPUTE_PLAN_ID_AUTO,
+ COMPUTE_PLAN_ID_REGRESS,
+};
+
/* GUC parameters */
extern PGDLLIMPORT int compute_query_id;
+extern PGDLLIMPORT int compute_plan_id;
extern const char *CleanQuerytext(const char *query, int *location, int *len);
extern JumbleState *JumbleQuery(Query *query);
extern void EnableQueryId(void);
+extern void EnablePlanId(void);
extern PGDLLIMPORT bool query_id_enabled;
+extern PGDLLIMPORT bool plan_id_enabled;
/*
* Returns whether query identifier computation has been enabled, either
@@ -83,8 +99,22 @@ IsQueryIdEnabled(void)
return query_id_enabled;
}
-/* Functions intended for other users of jumbling (e.g. plan jumbling) */
-extern JumbleState *InitializeJumbleState(bool record_clocations);
+/*
+ * Returns whether plan identifier computation has been enabled, either
+ * directly in the GUC or by a module when the setting is 'auto'.
+ */
+static inline bool
+IsPlanIdEnabled(void)
+{
+ if (compute_plan_id == COMPUTE_PLAN_ID_OFF)
+ return false;
+ if (compute_plan_id == COMPUTE_PLAN_ID_ON)
+ return true;
+ return plan_id_enabled;
+}
+
+/* Functions called for plan jumbling or extensions doing their own jumbling */
+extern JumbleState *InitializeJumbleState(bool record_clocations, PlannerGlobal *glob);
extern void AppendJumble(JumbleState *jstate, const unsigned char *item, Size size);
extern void JumbleNode(JumbleState *jstate, Node *node);
extern uint64 HashJumbleState(JumbleState *jstate);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index d3d4ff6c5c9..437a4cec5bf 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -170,6 +170,9 @@ typedef struct PgBackendStatus
/* query identifier, optionally computed using post_parse_analyze_hook */
uint64 st_query_id;
+
+ /* plan identifier, optionally computed after planning */
+ uint64 st_plan_id;
} PgBackendStatus;
@@ -316,6 +319,7 @@ extern void pgstat_clear_backend_activity_snapshot(void);
/* Activity reporting functions */
extern void pgstat_report_activity(BackendState state, const char *cmd_str);
extern void pgstat_report_query_id(uint64 query_id, bool force);
+extern void pgstat_report_plan_id(uint64 query_id, uint64 plan_id, bool force);
extern void pgstat_report_tempfile(size_t filesize);
extern void pgstat_report_appname(const char *appname);
extern void pgstat_report_xact_timestamp(TimestampTz tstamp);
@@ -323,6 +327,7 @@ extern const char *pgstat_get_backend_current_activity(int pid, bool checkUser);
extern const char *pgstat_get_crashed_backend_activity(int pid, char *buffer,
int buflen);
extern uint64 pgstat_get_my_query_id(void);
+extern uint64 pgstat_get_my_plan_id(void);
extern BackendType pgstat_get_backend_type_by_proc_number(ProcNumber procNumber);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index cddc3ea9b53..d3d5578d2c8 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -889,6 +889,7 @@ CREATE VIEW pg_stat_activity AS
S.backend_xid,
s.backend_xmin,
S.query_id,
+ S.plan_id,
S.query,
S.backend_type
FROM pg_stat_get_activity(NULL) AS S
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index c24e66f82e1..31a1761f539 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -966,6 +966,22 @@ ExplainPrintPlan(ExplainState *es, QueryDesc *queryDesc)
ExplainPropertyInteger("Query Identifier", NULL, (int64)
queryDesc->plannedstmt->queryId, es);
}
+
+ /*
+ * COMPUTE_PLAN_ID_REGRESS means COMPUTE_PLAN_ID_YES, but we don't show
+ * the queryid in any of the EXPLAIN plans to keep stable the results
+ * generated by regression test suites.
+ */
+ if (es->verbose && queryDesc->plannedstmt->planId != UINT64CONST(0) &&
+ compute_plan_id != COMPUTE_PLAN_ID_REGRESS)
+ {
+ /*
+ * Output the queryid as an int64 rather than a uint64 so we match
+ * what would be seen in the BIGINT pg_stat_activity.plan_id column.
+ */
+ ExplainPropertyInteger("Plan Identifier", NULL, (int64)
+ queryDesc->plannedstmt->planId, es);
+ }
}
/*
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 1d27b840ca9..7f16f38f240 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -120,13 +120,15 @@ ExecutorStart(QueryDesc *queryDesc, int eflags)
{
/*
* In some cases (e.g. an EXECUTE statement or an execute message with the
- * extended query protocol) the query_id won't be reported, so do it now.
+ * extended query protocol) the query_id and plan_id won't be reported, so
+ * do it now.
*
- * Note that it's harmless to report the query_id multiple times, as the
- * call will be ignored if the top level query_id has already been
- * reported.
+ * Note that it's harmless to report the identifiers multiple times, as
+ * the call will be ignored if the top level query_id / plan_id has
+ * already been reported.
*/
pgstat_report_query_id(queryDesc->plannedstmt->queryId, false);
+ pgstat_report_plan_id(queryDesc->plannedstmt->planId, queryDesc->plannedstmt->queryId, false);
if (ExecutorStart_hook)
(*ExecutorStart_hook) (queryDesc, eflags);
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 9c313d81315..da2279579a8 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -174,6 +174,7 @@ ExecSerializePlan(Plan *plan, EState *estate)
pstmt = makeNode(PlannedStmt);
pstmt->commandType = CMD_SELECT;
pstmt->queryId = pgstat_get_my_query_id();
+ pstmt->planId = pgstat_get_my_plan_id();
pstmt->hasReturning = false;
pstmt->hasModifyingCTE = false;
pstmt->canSetTag = true;
diff --git a/src/backend/nodes/gen_node_support.pl b/src/backend/nodes/gen_node_support.pl
index 7c012c27f88..e7b04678e06 100644
--- a/src/backend/nodes/gen_node_support.pl
+++ b/src/backend/nodes/gen_node_support.pl
@@ -475,6 +475,7 @@ foreach my $infile (@ARGV)
equal_ignore_if_zero
query_jumble_ignore
query_jumble_location
+ query_jumble_rt_index
read_write_ignore
write_only_relids
write_only_nondefault_pathtarget
@@ -1280,13 +1281,19 @@ _jumble${n}(JumbleState *jstate, Node *node)
{
my $t = $node_type_info{$n}->{field_types}{$f};
my @a = @{ $node_type_info{$n}->{field_attrs}{$f} };
+ my $array_size_field;
my $query_jumble_ignore = $struct_no_query_jumble;
my $query_jumble_location = 0;
+ my $query_jumble_rt_index = 0;
# extract per-field attributes
foreach my $a (@a)
{
- if ($a eq 'query_jumble_ignore')
+ if ($a =~ /^array_size\(([\w.]+)\)$/)
+ {
+ $array_size_field = $1;
+ }
+ elsif ($a eq 'query_jumble_ignore')
{
$query_jumble_ignore = 1;
}
@@ -1294,10 +1301,29 @@ _jumble${n}(JumbleState *jstate, Node *node)
{
$query_jumble_location = 1;
}
+ elsif ($a eq 'query_jumble_rt_index')
+ {
+ $query_jumble_rt_index = 1;
+ }
}
+ next if $query_jumble_ignore;
+
+ if ($query_jumble_rt_index)
+ {
+ if ($t eq 'List*')
+ {
+ print $jff "\tJUMBLE_RT_INDEX_LIST($f);\n"
+ unless $query_jumble_ignore;
+ }
+ else
+ {
+ print $jff "\tJUMBLE_RT_INDEX($f);\n"
+ unless $query_jumble_ignore;
+ }
+ }
# node type
- if (($t =~ /^(\w+)\*$/ or $t =~ /^struct\s+(\w+)\*$/)
+ elsif (($t =~ /^(\w+)\*$/ or $t =~ /^struct\s+(\w+)\*$/)
and elem $1, @node_types)
{
print $jff "\tJUMBLE_NODE($f);\n"
@@ -1317,6 +1343,26 @@ _jumble${n}(JumbleState *jstate, Node *node)
print $jff "\tJUMBLE_STRING($f);\n"
unless $query_jumble_ignore;
}
+ elsif ($t =~ /^(\w+)(\*|\[\w+\])$/ and elem $1, @scalar_types)
+ {
+ if (!defined $array_size_field)
+ {
+ die "no array size defined for $n.$f of type $t\n";
+ }
+ if ($node_type_info{$n}->{field_types}{$array_size_field} eq
+ 'List*')
+ {
+ print $jff
+ "\tJUMBLE_ARRAY($f, list_length(expr->$array_size_field));\n"
+ unless $query_jumble_ignore;
+ }
+ else
+ {
+ print $jff
+ "\tJUMBLE_ARRAY($f, expr->$array_size_field);\n"
+ unless $query_jumble_ignore;
+ }
+ }
else
{
print $jff "\tJUMBLE_FIELD($f);\n"
diff --git a/src/backend/nodes/queryjumblefuncs.c b/src/backend/nodes/queryjumblefuncs.c
index 545d8edcae2..482d1ea8283 100644
--- a/src/backend/nodes/queryjumblefuncs.c
+++ b/src/backend/nodes/queryjumblefuncs.c
@@ -35,12 +35,14 @@
#include "common/hashfn.h"
#include "miscadmin.h"
#include "nodes/queryjumble.h"
+#include "parser/parsetree.h"
#include "parser/scansup.h"
#define JUMBLE_SIZE 1024 /* query serialization buffer size */
/* GUC parameters */
int compute_query_id = COMPUTE_QUERY_ID_AUTO;
+int compute_plan_id = COMPUTE_PLAN_ID_AUTO;
/*
* True when compute_query_id is ON or AUTO, and a module requests them.
@@ -51,7 +53,18 @@ int compute_query_id = COMPUTE_QUERY_ID_AUTO;
*/
bool query_id_enabled = false;
+/*
+ * True when compute_plan_id is ON or AUTO, and a module requests them.
+ *
+ * Note that IsPlanIdEnabled() should be used instead of checking
+ * plan_id_enabled or plan_query_id directly when we want to know
+ * whether plan identifiers are computed in the core or not.
+ */
+bool plan_id_enabled = false;
+
static void RecordConstLocation(JumbleState *jstate, int location);
+static void JumbleRangeTableIndex(JumbleState *jstate, Index rti);
+static void JumbleRangeTableIndexList(JumbleState *jstate, List *l);
static void _jumbleA_Const(JumbleState *jstate, Node *node);
static void _jumbleList(JumbleState *jstate, Node *node);
static void _jumbleVariableSetStmt(JumbleState *jstate, Node *node);
@@ -106,7 +119,7 @@ CleanQuerytext(const char *query, int *location, int *len)
}
JumbleState *
-InitializeJumbleState(bool record_clocations)
+InitializeJumbleState(bool record_clocations, PlannerGlobal *glob)
{
JumbleState *jstate = (JumbleState *) palloc0(sizeof(JumbleState));
@@ -121,6 +134,8 @@ InitializeJumbleState(bool record_clocations)
palloc(jstate->clocations_buf_size * sizeof(LocationLen));
}
+ jstate->glob = glob;
+
return jstate;
}
@@ -135,7 +150,7 @@ HashJumbleState(JumbleState *jstate)
JumbleState *
JumbleQuery(Query *query)
{
- JumbleState *jstate = InitializeJumbleState(true);
+ JumbleState *jstate = InitializeJumbleState(true, NULL);
Assert(IsQueryIdEnabled());
@@ -171,6 +186,19 @@ EnableQueryId(void)
query_id_enabled = true;
}
+/*
+ * Enables plan identifier computation.
+ *
+ * Third-party plugins can use this function to inform core that they require
+ * a query identifier to be computed.
+ */
+void
+EnablePlanId(void)
+{
+ if (compute_plan_id != COMPUTE_PLAN_ID_OFF)
+ plan_id_enabled = true;
+}
+
/*
* AppendJumble: Append a value that is substantive in a given query to
* the current jumble.
@@ -238,8 +266,17 @@ RecordConstLocation(JumbleState *jstate, int location)
JumbleNode(jstate, (Node *) expr->item)
#define JUMBLE_LOCATION(location) \
RecordConstLocation(jstate, expr->location)
+#define JUMBLE_RT_INDEX(item) \
+do { \
+ if (expr->item) \
+ JumbleRangeTableIndex(jstate, expr->item); \
+} while(0)
+#define JUMBLE_RT_INDEX_LIST(item) \
+ JumbleRangeTableIndexList(jstate, expr->item)
#define JUMBLE_FIELD(item) \
AppendJumble(jstate, (const unsigned char *) &(expr->item), sizeof(expr->item))
+#define JUMBLE_ARRAY(item, len) \
+ AppendJumble(jstate, (const unsigned char *) &(expr->item), sizeof(*(expr->item)) * len)
#define JUMBLE_FIELD_SINGLE(item) \
AppendJumble(jstate, (const unsigned char *) &(item), sizeof(item))
#define JUMBLE_STRING(str) \
@@ -388,3 +425,40 @@ _jumbleVariableSetStmt(JumbleState *jstate, Node *node)
JUMBLE_FIELD(is_local);
JUMBLE_LOCATION(location);
}
+
+/*
+ * Jumble the target of a rangle table index, e.g. in a Scan or Modify node
+ */
+static void
+JumbleRangeTableIndex(JumbleState *jstate, Index rti)
+{
+ RangeTblEntry *expr = rt_fetch(rti, jstate->glob->finalrtable);
+
+ switch (expr->rtekind)
+ {
+ case RTE_RELATION:
+ JUMBLE_FIELD(relid);
+ break;
+ case RTE_CTE:
+ JUMBLE_STRING(ctename);
+ break;
+ default:
+
+ /*
+ * Ignore other targets, the jumble includes something identifying
+ * about them already
+ */
+ break;
+ }
+}
+
+static void
+JumbleRangeTableIndexList(JumbleState *jstate, List *l)
+{
+ ListCell *lc;
+
+ Assert(l->type == T_IntList);
+
+ foreach(lc, l)
+ JumbleRangeTableIndex(jstate, lfirst_int(lc));
+}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 8a474a50be7..d14c3218e17 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -37,6 +37,7 @@
#ifdef OPTIMIZER_DEBUG
#include "nodes/print.h"
#endif
+#include "nodes/queryjumble.h"
#include "nodes/supportnodes.h"
#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
@@ -532,6 +533,16 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions,
Assert(glob->finalrowmarks == NIL);
Assert(glob->resultRelations == NIL);
Assert(glob->appendRelations == NIL);
+
+ /*
+ * Initialize plan identifier jumble if needed
+ *
+ * Note the actual jumbling is done in the tree walk in
+ * set_plan_references
+ */
+ if (IsPlanIdEnabled())
+ glob->plan_jumble_state = InitializeJumbleState(false, glob);
+
top_plan = set_plan_references(root, top_plan);
/* ... and the subplans (both regular subplans and initplans) */
Assert(list_length(glob->subplans) == list_length(glob->subroots));
@@ -571,6 +582,13 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions,
result->stmt_location = parse->stmt_location;
result->stmt_len = parse->stmt_len;
+ if (IsPlanIdEnabled())
+ {
+ result->planId = HashJumbleState(glob->plan_jumble_state);
+ pfree(glob->plan_jumble_state->jumble);
+ pfree(glob->plan_jumble_state);
+ }
+
result->jitFlags = PGJIT_NONE;
if (jit_enabled && jit_above_cost >= 0 &&
top_plan->total_cost > jit_above_cost)
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 0868249be94..fc48337f1df 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -19,6 +19,7 @@
#include "catalog/pg_type.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
+#include "nodes/queryjumble.h"
#include "optimizer/optimizer.h"
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
@@ -1295,6 +1296,14 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset)
break;
}
+ /*
+ * If enabled, append significant information to the plan identifier
+ * jumble (we do this here since we're already walking the tree in a
+ * near-final state)
+ */
+ if (IsPlanIdEnabled())
+ JumbleNode(root->glob->plan_jumble_state, (Node *) plan);
+
/*
* Now recurse into child plans, if any
*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index a97a1eda6da..654acf5bf04 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -115,6 +115,7 @@ typedef struct
bool redirection_done;
bool IsBinaryUpgrade;
bool query_id_enabled;
+ bool plan_id_enabled;
int max_safe_fds;
int MaxBackends;
int num_pmchild_slots;
@@ -744,6 +745,7 @@ save_backend_variables(BackendParameters *param,
param->redirection_done = redirection_done;
param->IsBinaryUpgrade = IsBinaryUpgrade;
param->query_id_enabled = query_id_enabled;
+ param->plan_id_enabled = plan_id_enabled;
param->max_safe_fds = max_safe_fds;
param->MaxBackends = MaxBackends;
@@ -1004,6 +1006,7 @@ restore_backend_variables(BackendParameters *param)
redirection_done = param->redirection_done;
IsBinaryUpgrade = param->IsBinaryUpgrade;
query_id_enabled = param->query_id_enabled;
+ plan_id_enabled = param->plan_id_enabled;
max_safe_fds = param->max_safe_fds;
MaxBackends = param->MaxBackends;
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 5655348a2e2..6d8947bae9b 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -1106,6 +1106,7 @@ exec_simple_query(const char *query_string)
size_t cmdtaglen;
pgstat_report_query_id(0, true);
+ pgstat_report_plan_id(0, 0, true);
/*
* Get the command name for use in status display (it also becomes the
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 731342799a6..1dfb7a58f87 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -379,6 +379,7 @@ pgstat_bestart(void)
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = UINT64CONST(0);
+ lbeentry.st_plan_id = UINT64CONST(0);
/*
* we don't zero st_progress_param here to save cycles; nobody should
@@ -533,6 +534,7 @@ pgstat_report_activity(BackendState state, const char *cmd_str)
/* st_xact_start_timestamp and wait_event_info are also disabled */
beentry->st_xact_start_timestamp = 0;
beentry->st_query_id = UINT64CONST(0);
+ beentry->st_plan_id = UINT64CONST(0);
proc->wait_event_info = 0;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -588,12 +590,15 @@ pgstat_report_activity(BackendState state, const char *cmd_str)
beentry->st_state_start_timestamp = current_timestamp;
/*
- * If a new query is started, we reset the query identifier as it'll only
- * be known after parse analysis, to avoid reporting last query's
- * identifier.
+ * If a new query is started, we reset the query and plan identifier as
+ * it'll only be known after parse analysis / planning, to avoid reporting
+ * last query's identifiers.
*/
if (state == STATE_RUNNING)
+ {
beentry->st_query_id = UINT64CONST(0);
+ beentry->st_plan_id = UINT64CONST(0);
+ }
if (cmd_str != NULL)
{
@@ -644,6 +649,45 @@ pgstat_report_query_id(uint64 query_id, bool force)
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
+/* --------
+ * pgstat_report_plan_id() -
+ *
+ * Called to update top-level plan identifier.
+ * --------
+ */
+void
+pgstat_report_plan_id(uint64 plan_id, uint64 query_id, bool force)
+{
+ volatile PgBackendStatus *beentry = MyBEEntry;
+
+ /*
+ * if track_activities is disabled, st_plan_id should already have been
+ * reset
+ */
+ if (!beentry || !pgstat_track_activities)
+ return;
+
+ /*
+ * We only report the top-level plan identifiers. The stored plan_id is
+ * reset when a backend calls pgstat_report_activity(STATE_RUNNING), or
+ * with an explicit call to this function using the force flag. If the
+ * saved plan identifier is not zero or the query identifier is 0, it
+ * means that it's not a top-level command, so ignore the one provided
+ * unless it's an explicit call to reset the identifier.
+ */
+ if ((beentry->st_plan_id != 0 || query_id == 0) && !force)
+ return;
+
+ /*
+ * Update my status entry, following the protocol of bumping
+ * st_changecount before and after. We use a volatile pointer here to
+ * ensure the compiler doesn't try to get cute.
+ */
+ PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
+ beentry->st_plan_id = plan_id;
+ PGSTAT_END_WRITE_ACTIVITY(beentry);
+}
+
/* ----------
* pgstat_report_appname() -
@@ -1040,6 +1084,26 @@ pgstat_get_my_query_id(void)
return MyBEEntry->st_query_id;
}
+/* ----------
+ * pgstat_get_my_plan_id() -
+ *
+ * Return current backend's plan identifier.
+ */
+uint64
+pgstat_get_my_plan_id(void)
+{
+ if (!MyBEEntry)
+ return 0;
+
+ /*
+ * There's no need for a lock around pgstat_begin_read_activity /
+ * pgstat_end_read_activity here as it's only called from
+ * pg_stat_get_activity which is already protected, or from the same
+ * backend which means that there won't be concurrent writes.
+ */
+ return MyBEEntry->st_plan_id;
+}
+
/* ----------
* pgstat_get_backend_type_by_proc_number() -
*
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index e9096a88492..a3d5592beb6 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -330,7 +330,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum
pg_stat_get_activity(PG_FUNCTION_ARGS)
{
-#define PG_STAT_GET_ACTIVITY_COLS 31
+#define PG_STAT_GET_ACTIVITY_COLS 32
int num_backends = pgstat_fetch_stat_numbackends();
int curr_backend;
int pid = PG_ARGISNULL(0) ? -1 : PG_GETARG_INT32(0);
@@ -641,6 +641,10 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
nulls[30] = true;
else
values[30] = UInt64GetDatum(beentry->st_query_id);
+ if (beentry->st_plan_id == 0)
+ nulls[31] = true;
+ else
+ values[31] = UInt64GetDatum(beentry->st_plan_id);
}
else
{
@@ -670,6 +674,7 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
nulls[28] = true;
nulls[29] = true;
nulls[30] = true;
+ nulls[31] = true;
}
tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 38cb9e970d5..9ddb8e97310 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -309,6 +309,24 @@ static const struct config_enum_entry compute_query_id_options[] = {
{NULL, 0, false}
};
+/*
+ * Although only "on" and "off" are documented, we accept
+ * all the likely variants of "on" and "off".
+ */
+static const struct config_enum_entry compute_plan_id_options[] = {
+ {"auto", COMPUTE_PLAN_ID_AUTO, false},
+ {"regress", COMPUTE_PLAN_ID_REGRESS, false},
+ {"on", COMPUTE_PLAN_ID_ON, false},
+ {"off", COMPUTE_PLAN_ID_OFF, false},
+ {"true", COMPUTE_PLAN_ID_ON, true},
+ {"false", COMPUTE_PLAN_ID_OFF, true},
+ {"yes", COMPUTE_PLAN_ID_ON, true},
+ {"no", COMPUTE_PLAN_ID_OFF, true},
+ {"1", COMPUTE_PLAN_ID_ON, true},
+ {"0", COMPUTE_PLAN_ID_OFF, true},
+ {NULL, 0, false}
+};
+
/*
* Although only "on", "off", and "partition" are documented, we
* accept all the likely variants of "on" and "off".
@@ -4873,6 +4891,16 @@ struct config_enum ConfigureNamesEnum[] =
NULL, NULL, NULL
},
+ {
+ {"compute_plan_id", PGC_SUSET, STATS_MONITORING,
+ gettext_noop("Enables in-core computation of plan identifiers."),
+ NULL
+ },
+ &compute_plan_id,
+ COMPUTE_PLAN_ID_AUTO, compute_plan_id_options,
+ NULL, NULL, NULL
+ },
+
{
{"constraint_exclusion", PGC_USERSET, QUERY_TUNING_OTHER,
gettext_noop("Enables the planner to use constraints to optimize queries."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 079efa1baa7..0634ae90dd3 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -641,6 +641,7 @@
# - Monitoring -
#compute_query_id = auto
+#compute_plan_id = auto
#log_statement_stats = off
#log_parser_stats = off
#log_planner_stats = off
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index ee31e41d508..8bfa3c1a5f9 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -699,6 +699,17 @@ select explain_filter('explain (verbose) create table test_ctas as select 1');
Query Identifier: N
(3 rows)
+-- Test compute_plan_id
+set compute_plan_id = on;
+select explain_filter('explain (verbose) select * from int8_tbl i8');
+ explain_filter
+----------------------------------------------------------------
+ Seq Scan on public.int8_tbl i8 (cost=N.N..N.N rows=N width=N)
+ Output: q1, q2
+ Query Identifier: N
+ Plan Identifier: N
+(4 rows)
+
-- Test SERIALIZE option
select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
explain_filter
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 3361f6a69c9..6778b79e658 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1760,9 +1760,10 @@ pg_stat_activity| SELECT s.datid,
s.backend_xid,
s.backend_xmin,
s.query_id,
+ s.plan_id,
s.query,
s.backend_type
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
LEFT JOIN pg_database d ON ((s.datid = d.oid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_all_indexes| SELECT c.oid AS relid,
@@ -1890,7 +1891,7 @@ pg_stat_gssapi| SELECT pid,
gss_princ AS principal,
gss_enc AS encrypted,
gss_delegation AS credentials_delegated
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
WHERE (client_port IS NOT NULL);
pg_stat_io| SELECT backend_type,
object,
@@ -2096,7 +2097,7 @@ pg_stat_replication| SELECT s.pid,
w.sync_priority,
w.sync_state,
w.reply_time
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
JOIN pg_stat_get_wal_senders() w(pid, state, sent_lsn, write_lsn, flush_lsn, replay_lsn, write_lag, flush_lag, replay_lag, sync_priority, sync_state, reply_time) ON ((s.pid = w.pid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_replication_slots| SELECT s.slot_name,
@@ -2130,7 +2131,7 @@ pg_stat_ssl| SELECT pid,
ssl_client_dn AS client_dn,
ssl_client_serial AS client_serial,
ssl_issuer_dn AS issuer_dn
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
WHERE (client_port IS NOT NULL);
pg_stat_subscription| SELECT su.oid AS subid,
su.subname,
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index 0bafa870496..d787ad2cda3 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -167,6 +167,10 @@ select explain_filter('explain (verbose) select * from int8_tbl i8');
select explain_filter('explain (verbose) declare test_cur cursor for select * from int8_tbl');
select explain_filter('explain (verbose) create table test_ctas as select 1');
+-- Test compute_plan_id
+set compute_plan_id = on;
+select explain_filter('explain (verbose) select * from int8_tbl i8');
+
-- Test SERIALIZE option
select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
select explain_filter('explain (analyze,serialize text,buffers,timing off) select * from int8_tbl i8');
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index a782f109982..85a1e0d8d4b 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -8406,6 +8406,40 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-compute-plan-id" xreflabel="compute_plan_id">
+ <term><varname>compute_plan_id</varname> (<type>enum</type>)
+ <indexterm>
+ <primary><varname>compute_plan_id</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Enables in-core computation of a plan identifier.
+ Plan identifiers can be displayed in the <link
+ linkend="monitoring-pg-stat-activity-view"><structname>pg_stat_activity</structname></link>
+ view or using <command>EXPLAIN</command>.
+ Note that an external module can alternatively be used if the
+ in-core plan identifier computation method is not acceptable.
+ In this case, in-core computation must be always disabled.
+ Valid values are <literal>off</literal> (always disabled),
+ <literal>on</literal> (always enabled), <literal>auto</literal>,
+ which lets modules that utilize plan identifiers enable
+ it automatically, and <literal>regress</literal> which
+ has the same effect as <literal>on</literal>, except that the
+ query identifier is not shown in the <literal>EXPLAIN</literal> output
+ in order to facilitate automated regression testing.
+ The default is <literal>auto</literal>.
+ </para>
+ <note>
+ <para>
+ To ensure that only one plan identifier is calculated and
+ displayed, extensions that calculate plan identifiers should
+ throw an error if a plan identifier has already been computed.
+ </para>
+ </note>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-log-statement-stats">
<term><varname>log_statement_stats</varname> (<type>boolean</type>)
<indexterm>
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 4e917f159aa..c4ecf20cb2c 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -972,6 +972,22 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>plan_id</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Identifier of this backend's most recent query plan. If
+ <structfield>state</structfield> is <literal>active</literal> this
+ field shows the identifier of the currently executing query plan. In
+ all other states, it shows the identifier of last query plan that
+ was executed. Plan identifiers are not computed by default so this
+ field will be null unless <xref linkend="guc-compute-plan-id"/>
+ parameter is enabled or a third-party module that computes plan
+ identifiers is configured.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>query</structfield> <type>text</type>
--
2.47.2
v3-0003-Add-pg_stat_plans-contrib-extension.patchtext/x-diff; charset=us-asciiDownload
From 4ffc49cadac2519a7e40dabcf7df5cc888bd3144 Mon Sep 17 00:00:00 2001
From: Michael Paquier <michael@paquier.xyz>
Date: Fri, 31 Jan 2025 13:08:19 +0900
Subject: [PATCH v3 3/3] Add pg_stat_plans contrib extension
This extension allows tracking per-plan call counts and execution time,
as well as capturing the plan text, aka EXPLAIN (COSTS OFF), for the
first execution of a given plan. This utilize the compute_plan_id
functionality for tracking different plans.
---
doc/src/sgml/contrib.sgml | 1 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/pgstatplans.sgml | 425 ++++++++++
contrib/Makefile | 1 +
contrib/meson.build | 1 +
contrib/pg_stat_plans/Makefile | 29 +
contrib/pg_stat_plans/expected/cleanup.out | 1 +
contrib/pg_stat_plans/expected/privileges.out | 125 +++
contrib/pg_stat_plans/expected/select.out | 262 ++++++
contrib/pg_stat_plans/meson.build | 43 +
contrib/pg_stat_plans/pg_stat_plans--1.0.sql | 33 +
contrib/pg_stat_plans/pg_stat_plans.c | 779 ++++++++++++++++++
contrib/pg_stat_plans/pg_stat_plans.conf | 1 +
contrib/pg_stat_plans/pg_stat_plans.control | 5 +
contrib/pg_stat_plans/sql/cleanup.sql | 1 +
contrib/pg_stat_plans/sql/privileges.sql | 59 ++
contrib/pg_stat_plans/sql/select.sql | 67 ++
17 files changed, 1834 insertions(+)
create mode 100644 doc/src/sgml/pgstatplans.sgml
create mode 100644 contrib/pg_stat_plans/Makefile
create mode 100644 contrib/pg_stat_plans/expected/cleanup.out
create mode 100644 contrib/pg_stat_plans/expected/privileges.out
create mode 100644 contrib/pg_stat_plans/expected/select.out
create mode 100644 contrib/pg_stat_plans/meson.build
create mode 100644 contrib/pg_stat_plans/pg_stat_plans--1.0.sql
create mode 100644 contrib/pg_stat_plans/pg_stat_plans.c
create mode 100644 contrib/pg_stat_plans/pg_stat_plans.conf
create mode 100644 contrib/pg_stat_plans/pg_stat_plans.control
create mode 100644 contrib/pg_stat_plans/sql/cleanup.sql
create mode 100644 contrib/pg_stat_plans/sql/privileges.sql
create mode 100644 contrib/pg_stat_plans/sql/select.sql
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index 7c381949a53..4a5a02c7049 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -157,6 +157,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
&pglogicalinspect;
&pgprewarm;
&pgrowlocks;
+ &pgstatplans;
&pgstatstatements;
&pgstattuple;
&pgsurgery;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 66e6dccd4c9..b0afb33ce22 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -146,6 +146,7 @@
<!ENTITY pglogicalinspect SYSTEM "pglogicalinspect.sgml">
<!ENTITY pgprewarm SYSTEM "pgprewarm.sgml">
<!ENTITY pgrowlocks SYSTEM "pgrowlocks.sgml">
+<!ENTITY pgstatplans SYSTEM "pgstatplans.sgml">
<!ENTITY pgstatstatements SYSTEM "pgstatstatements.sgml">
<!ENTITY pgstattuple SYSTEM "pgstattuple.sgml">
<!ENTITY pgsurgery SYSTEM "pgsurgery.sgml">
diff --git a/doc/src/sgml/pgstatplans.sgml b/doc/src/sgml/pgstatplans.sgml
new file mode 100644
index 00000000000..2d5faff15a9
--- /dev/null
+++ b/doc/src/sgml/pgstatplans.sgml
@@ -0,0 +1,425 @@
+<!-- doc/src/sgml/pgstatplans.sgml -->
+
+<sect1 id="pgstatplans" xreflabel="pg_stat_plans">
+ <title>pg_stat_plans — track per-plan call counts, execution times and EXPLAIN texts</title>
+
+ <indexterm zone="pgstatplans">
+ <primary>pg_stat_plans</primary>
+ </indexterm>
+
+ <para>
+ The <filename>pg_stat_plans</filename> module provides a means for
+ tracking per-plan statistics and plan texts of all SQL statements executed by
+ a server.
+ </para>
+
+ <para>
+ The module must be loaded by adding <literal>pg_stat_plans</literal> to
+ <xref linkend="guc-shared-preload-libraries"/> in
+ <filename>postgresql.conf</filename>, because it requires additional shared memory.
+ This means that a server restart is needed to add or remove the module.
+ In addition, query and plan identifier calculation must be enabled in order for the
+ module to be active by setting both <xref linkend="guc-compute-plan-id"/> to
+ <literal>auto</literal> or <literal>on</literal> and
+ <xref linkend="guc-compute-query-id"/> to <literal>auto</literal> or <literal>on</literal>.
+ </para>
+
+ <para>
+ When <filename>pg_stat_plans</filename> is active, it tracks
+ statistics across all databases of the server. To access and manipulate
+ these statistics, the module provides the <structname>pg_stat_plans</structname>
+ view and the utility functions <function>pg_stat_plans_reset</function> and
+ <function>pg_stat_plans</function>. These are not available globally but
+ can be enabled for a specific database with
+ <command>CREATE EXTENSION pg_stat_plans</command>.
+ </para>
+
+ <sect2 id="pgstatplans-pg-stat-plans">
+ <title>The <structname>pg_stat_plans</structname> View</title>
+
+ <para>
+ The statistics gathered by the module are made available via a
+ view named <structname>pg_stat_plans</structname>. This view
+ contains one row for each distinct combination of database ID, user
+ ID, whether it's a top-level statement or not, query ID and plan ID
+ (up to the maximum number of distinct plans that the module can track).
+ The columns of the view are shown in <xref linkend="pgstatplans-columns"/>.
+ </para>
+
+ <table id="pgstatplans-columns">
+ <title><structname>pg_stat_plans</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>userid</structfield> <type>oid</type>
+ (references <link linkend="catalog-pg-authid"><structname>pg_authid</structname></link>.<structfield>oid</structfield>)
+ </para>
+ <para>
+ OID of user who executed the statement
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbid</structfield> <type>oid</type>
+ (references <link linkend="catalog-pg-database"><structname>pg_database</structname></link>.<structfield>oid</structfield>)
+ </para>
+ <para>
+ OID of database in which the statement was executed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>toplevel</structfield> <type>bool</type>
+ </para>
+ <para>
+ True if the query was executed as a top-level statement
+ (always true if <varname>pg_stat_plans.track</varname> is set to
+ <literal>top</literal>)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>queryid</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Hash code to identify identical normalized queries.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>planid</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Hash code to identify identical plan shapes.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>calls</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the plan was executed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_exec_time</structfield> <type>double precision</type>
+ </para>
+ <para>
+ Total time spent executing the plan, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>plan</structfield> <type>text</type>
+ </para>
+ <para>
+ Plan text of a representative plan. This is similar to the output of
+ <literal>EXPLAIN (COSTS OFF)</literal>. Note the plan text will contain constant
+ values of the first plan recorded, but subsequent executions of the
+ same plan hash code (<structfield>planid</structfield>) with different
+ constant values will be tracked under the same entry.
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ For security reasons, only superusers and roles with privileges of the
+ <literal>pg_read_all_stats</literal> role are allowed to see the plan text,
+ <structfield>queryid</structfield> and <structfield>planid</structfield>
+ of queries executed by other users. Other users can see the statistics,
+ however, if the view has been installed in their database.
+ </para>
+
+ <para>
+ Plannable queries (that is, <command>SELECT</command>, <command>INSERT</command>,
+ <command>UPDATE</command>, <command>DELETE</command>, and <command>MERGE</command>)
+ will have the same <structfield>planid</structfield> whenever they have identical plan
+ structures according to an internal hash calculation. Typically, two plans will be
+ considered the same for this purpose if they have the same
+ <literal>EXPLAIN (COSTS OFF)</literal> output and are semantically equivalent except
+ for the values of literal constants appearing in the query plan.
+ </para>
+
+ <para>
+ Note that queries that have not finished executing yet will show in
+ <structname>pg_stat_plans</structname> with their plan text, but without
+ the <structname>calls</structname> field being incremented. This can be
+ used to identify the query plan for a currently running statement by joining
+ <link linkend="monitoring-pg-stat-activity-view">pg_stat_activity</link>
+ with <structname>pg_stat_plans</structname>, see example usage in
+ <xref linkend="pgstatplans-sample-output"/>.
+ </para>
+
+ <para>
+ Consumers of <structname>pg_stat_plans</structname> should use
+ <structfield>planid</structfield> in combination with
+ <structfield>queryid</structfield>, <structfield>dbid</structfield>,
+ <structfield>userid</structfield> and <structfield>toplevel</structfield>
+ as a stable and reliable identifier for each entry, instead of using its
+ plan text. However, it is important to understand that there are only limited
+ guarantees around the stability of the <structfield>planid</structfield>
+ hash value. Since the identifier is derived from the plan tree, its value
+ is a function of, among other things, the internal object identifiers
+ appearing in this representation. This has some counterintuitive implications.
+ For example, <filename>pg_stat_plans</filename> will consider two
+ apparently-identical plans to be distinct, if they reference a table
+ that was dropped and recreated between the creation of the two plans.
+ The hashing process is also sensitive to differences in
+ machine architecture and other facets of the platform.
+ Furthermore, it is not safe to assume that <structfield>planid</structfield>
+ will be stable across major versions of <productname>PostgreSQL</productname>.
+ </para>
+
+ <para>
+ Two servers participating in replication based on physical WAL replay can
+ be expected to have identical <structfield>planid</structfield> values for
+ the same plan. However, logical replication schemes do not promise to
+ keep replicas identical in all relevant details, so
+ <structfield>planid</structfield> will not be a useful identifier for
+ accumulating costs across a set of logical replicas.
+ If in doubt, direct testing is recommended.
+ </para>
+
+ <para>
+ Plan texts are stored in shared memory, and limited in length. To increase
+ the maximum length of stored plan texts you can increase
+ <varname>pg_stat_plans.max_size</varname>. This value can be changed for
+ an individual connection, or set as a server-wide setting.
+ </para>
+ </sect2>
+
+ <sect2 id="pgstatplans-funcs">
+ <title>Functions</title>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ <function>pg_stat_plans_reset() returns void</function>
+ <indexterm>
+ <primary>pg_stat_plans_reset</primary>
+ </indexterm>
+ </term>
+
+ <listitem>
+ <para>
+ <function>pg_stat_plans_reset</function> discards statistics and plan texts
+ gathered so far by <filename>pg_stat_plans</filename>.
+ By default, this function can only be executed by superusers.
+ Access may be granted to others using <command>GRANT</command>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <function>pg_stat_plans(showplan boolean) returns setof record</function>
+ <indexterm>
+ <primary>pg_stat_plans</primary>
+ <secondary>function</secondary>
+ </indexterm>
+ </term>
+
+ <listitem>
+ <para>
+ The <structname>pg_stat_plans</structname> view is defined in
+ terms of a function also named <function>pg_stat_plans</function>.
+ It is possible for clients to call
+ the <function>pg_stat_plans</function> function directly, and by
+ specifying <literal>showplan := false</literal> have plan texts be
+ omitted (that is, the <literal>OUT</literal> argument that corresponds
+ to the view's <structfield>plan</structfield> column will return nulls). This
+ feature is intended to support external tools that might wish to avoid
+ the overhead of repeatedly retrieving plan texts of indeterminate
+ length. Such tools can instead cache the first plan text observed
+ for each entry themselves, since that is
+ all <filename>pg_stat_plans</filename> itself does, and then retrieve
+ plan texts only as needed.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+
+ <sect2 id="pgstatplans-config-params">
+ <title>Configuration Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ <varname>pg_stat_plans.max</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>pg_stat_plans.max</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+
+ <listitem>
+ <para>
+ <varname>pg_stat_plans.max</varname> is the maximum number of
+ plans tracked by the module (i.e., the maximum number of rows
+ in the <structname>pg_stat_plans</structname> view). If more distinct
+ plans than that are observed, information about the least-executed
+ plans is discarded. The default value is 5000.
+ Only superusers can change this setting. Changing the setting requires
+ a reload of the server.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <varname>pg_stat_plans.max_size</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>pg_stat_plans.max_size</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+
+ <listitem>
+ <para>
+ <varname>pg_stat_plans.max_size</varname> is the maximum length of
+ each plan text tracked by the module in bytes. Longer plan texts will be truncated.
+ The default value is 2048 (2kB).
+ Only superusers can change this setting.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <varname>pg_stat_plans.track</varname> (<type>enum</type>)
+ <indexterm>
+ <primary><varname>pg_stat_plans.track</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+
+ <listitem>
+ <para>
+ <varname>pg_stat_plans.track</varname> controls which plans
+ are counted by the module.
+ Specify <literal>top</literal> to track plans by top-level statements (those issued
+ directly by clients), <literal>all</literal> to also track nested statements
+ (such as statements invoked within functions), or <literal>none</literal> to
+ disable plan statistics collection.
+ The default value is <literal>top</literal>.
+ Only superusers can change this setting.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ <para>
+ The module requires additional shared memory proportional to
+ <varname>pg_stat_plans.max</varname> for statistics, as well as
+ <varname>pg_stat_plans.max</varname> multiplied by
+ <varname>pg_stat_plans.max_size</varname> for plan texts. Note that this
+ memory is only consumed when entries are created, and not if
+ <varname>pg_stat_plans.track</varname> is set to <literal>none</literal>.
+ </para>
+
+ <para>
+ These parameters must be set in <filename>postgresql.conf</filename>.
+ Typical usage might be:
+
+<programlisting>
+# postgresql.conf
+shared_preload_libraries = 'pg_stat_plans'
+
+compute_query_id = on
+compute_plan_id = on
+pg_stat_plans.max = 10000
+pg_stat_plans.max_size = 4096
+pg_stat_plans.track = all
+</programlisting>
+ </para>
+ </sect2>
+
+ <sect2 id="pgstatplans-sample-output">
+ <title>Sample Output</title>
+
+<screen>
+bench=# SELECT pg_stat_plans_reset();
+
+$ pgbench -i bench
+$ pgbench -c10 -t300 bench
+
+bench=# \x
+bench=# SELECT plan, calls, total_exec_time
+ FROM pg_stat_plans ORDER BY total_exec_time DESC LIMIT 5;
+-[ RECORD 1 ]---+-----------------------------------------------------------------
+plan | Update on pgbench_tellers +
+ | -> Seq Scan on pgbench_tellers +
+ | Filter: (tid = 5)
+calls | 3000
+total_exec_time | 642.8880919999993
+-[ RECORD 2 ]---+-----------------------------------------------------------------
+plan | Update on pgbench_branches +
+ | -> Seq Scan on pgbench_branches +
+ | Filter: (bid = 1)
+calls | 1813
+total_exec_time | 476.64152700000005
+-[ RECORD 3 ]---+-----------------------------------------------------------------
+plan | Update on pgbench_branches +
+ | -> Index Scan using pgbench_branches_pkey on pgbench_branches+
+ | Index Cond: (bid = 1)
+calls | 1187
+total_exec_time | 326.1257549999999
+-[ RECORD 4 ]---+-----------------------------------------------------------------
+plan | Update on pgbench_accounts +
+ | -> Index Scan using pgbench_accounts_pkey on pgbench_accounts+
+ | Index Cond: (aid = 48793)
+calls | 3000
+total_exec_time | 21.664690000000093
+-[ RECORD 5 ]---+-----------------------------------------------------------------
+plan | Insert on pgbench_history +
+ | -> Result
+calls | 3000
+total_exec_time | 4.365250999999957
+
+session 1:
+
+bench# SELECT pg_sleep(100), COUNT(*) FROM pgbench_accounts;
+
+session 2:
+
+bench=# SELECT query, plan FROM pg_stat_activity
+ JOIN pg_stat_plans ON (usesysid = userid AND datid = dbid AND query_id = queryid AND plan_id = planid)
+ WHERE query LIKE 'SELECT pg_sleep%';
+ query | plan
+-------------------------------------------------------+------------------------------------
+ SELECT pg_sleep(100), COUNT(*) FROM pgbench_accounts; | Aggregate +
+ | -> Seq Scan on pgbench_accounts
+(1 row)
+
+</screen>
+ </sect2>
+
+ <sect2 id="pgstatplans-authors">
+ <title>Authors</title>
+
+ <para>
+ Lukas Fittl <email>lukas@fittl.com</email>.
+ </para>
+ </sect2>
+
+</sect1>
diff --git a/contrib/Makefile b/contrib/Makefile
index 952855d9b61..8de010afdef 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -34,6 +34,7 @@ SUBDIRS = \
pg_freespacemap \
pg_logicalinspect \
pg_prewarm \
+ pg_stat_plans \
pg_stat_statements \
pg_surgery \
pg_trgm \
diff --git a/contrib/meson.build b/contrib/meson.build
index 1ba73ebd67a..0442ec2644a 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -49,6 +49,7 @@ subdir('pg_freespacemap')
subdir('pg_logicalinspect')
subdir('pg_prewarm')
subdir('pgrowlocks')
+subdir('pg_stat_plans')
subdir('pg_stat_statements')
subdir('pgstattuple')
subdir('pg_surgery')
diff --git a/contrib/pg_stat_plans/Makefile b/contrib/pg_stat_plans/Makefile
new file mode 100644
index 00000000000..e073db95edd
--- /dev/null
+++ b/contrib/pg_stat_plans/Makefile
@@ -0,0 +1,29 @@
+# contrib/pg_stat_plans/Makefile
+
+MODULE_big = pg_stat_plans
+OBJS = \
+ $(WIN32RES) \
+ pg_stat_plans.o
+
+EXTENSION = pg_stat_plans
+DATA = pg_stat_plans--1.0.sql
+PGFILEDESC = "pg_stat_plans - track per-plan call counts, execution times and EXPLAIN texts"
+
+LDFLAGS_SL += $(filter -lm, $(LIBS))
+
+REGRESS_OPTS = --temp-config $(top_srcdir)/contrib/pg_stat_plans/pg_stat_plans.conf
+REGRESS = select privileges cleanup
+# Disabled because these tests require "shared_preload_libraries=pg_stat_plans",
+# which typical installcheck users do not have (e.g. buildfarm clients).
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/pg_stat_plans
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/pg_stat_plans/expected/cleanup.out b/contrib/pg_stat_plans/expected/cleanup.out
new file mode 100644
index 00000000000..51565617cef
--- /dev/null
+++ b/contrib/pg_stat_plans/expected/cleanup.out
@@ -0,0 +1 @@
+DROP EXTENSION pg_stat_plans;
diff --git a/contrib/pg_stat_plans/expected/privileges.out b/contrib/pg_stat_plans/expected/privileges.out
new file mode 100644
index 00000000000..3e21d6d7019
--- /dev/null
+++ b/contrib/pg_stat_plans/expected/privileges.out
@@ -0,0 +1,125 @@
+--
+-- Only superusers and roles with privileges of the pg_read_all_stats role
+-- are allowed to see the plan text, queryid and planid of queries executed by
+-- other users. Other users can see the statistics.
+--
+CREATE ROLE regress_stats_superuser SUPERUSER;
+CREATE ROLE regress_stats_user1;
+CREATE ROLE regress_stats_user2;
+GRANT pg_read_all_stats TO regress_stats_user2;
+SET ROLE regress_stats_superuser;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
+SELECT 1 AS "ONE";
+ ONE
+-----
+ 1
+(1 row)
+
+SET ROLE regress_stats_user1;
+SELECT 1+1 AS "TWO";
+ TWO
+-----
+ 2
+(1 row)
+
+--
+-- A superuser can read all columns of queries executed by others,
+-- including plan text, queryid and planid.
+--
+SET ROLE regress_stats_superuser;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+ rolname | queryid_bool | planid_bool | plan | calls
+-------------------------+--------------+-------------+------------------------------------------------------------------------------------+-------
+ regress_stats_superuser | t | t | Result | 1
+ regress_stats_superuser | t | t | Result | 1
+ regress_stats_superuser | t | t | Sort +| 0
+ | | | Sort Key: pg_authid.rolname, pg_stat_plans.plan COLLATE "C", pg_stat_plans.calls+|
+ | | | -> Hash Join +|
+ | | | Hash Cond: (pg_stat_plans.userid = pg_authid.oid) +|
+ | | | -> Function Scan on pg_stat_plans +|
+ | | | -> Hash +|
+ | | | -> Seq Scan on pg_authid |
+ regress_stats_user1 | t | t | Result | 1
+(4 rows)
+
+--
+-- regress_stats_user1 has no privileges to read the plan text, queryid
+-- or planid of queries executed by others but can see statistics
+-- like calls and rows.
+--
+SET ROLE regress_stats_user1;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+ rolname | queryid_bool | planid_bool | plan | calls
+-------------------------+--------------+-------------+------------------------------------------------------------------------------------+-------
+ regress_stats_superuser | | | <insufficient privilege> | 1
+ regress_stats_superuser | | | <insufficient privilege> | 1
+ regress_stats_superuser | | | <insufficient privilege> | 1
+ regress_stats_user1 | t | t | Result | 1
+ regress_stats_user1 | t | t | Sort +| 0
+ | | | Sort Key: pg_authid.rolname, pg_stat_plans.plan COLLATE "C", pg_stat_plans.calls+|
+ | | | -> Hash Join +|
+ | | | Hash Cond: (pg_stat_plans.userid = pg_authid.oid) +|
+ | | | -> Function Scan on pg_stat_plans +|
+ | | | -> Hash +|
+ | | | -> Seq Scan on pg_authid |
+(5 rows)
+
+--
+-- regress_stats_user2, with pg_read_all_stats role privileges, can
+-- read all columns, including plan text, queryid and planid, of queries
+-- executed by others.
+--
+SET ROLE regress_stats_user2;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+ rolname | queryid_bool | planid_bool | plan | calls
+-------------------------+--------------+-------------+------------------------------------------------------------------------------------+-------
+ regress_stats_superuser | t | t | Result | 1
+ regress_stats_superuser | t | t | Result | 1
+ regress_stats_superuser | t | t | Sort +| 1
+ | | | Sort Key: pg_authid.rolname, pg_stat_plans.plan COLLATE "C", pg_stat_plans.calls+|
+ | | | -> Hash Join +|
+ | | | Hash Cond: (pg_stat_plans.userid = pg_authid.oid) +|
+ | | | -> Function Scan on pg_stat_plans +|
+ | | | -> Hash +|
+ | | | -> Seq Scan on pg_authid |
+ regress_stats_user1 | t | t | Result | 1
+ regress_stats_user1 | t | t | Sort +| 1
+ | | | Sort Key: pg_authid.rolname, pg_stat_plans.plan COLLATE "C", pg_stat_plans.calls+|
+ | | | -> Hash Join +|
+ | | | Hash Cond: (pg_stat_plans.userid = pg_authid.oid) +|
+ | | | -> Function Scan on pg_stat_plans +|
+ | | | -> Hash +|
+ | | | -> Seq Scan on pg_authid |
+ regress_stats_user2 | t | t | Sort +| 0
+ | | | Sort Key: pg_authid.rolname, pg_stat_plans.plan COLLATE "C", pg_stat_plans.calls+|
+ | | | -> Hash Join +|
+ | | | Hash Cond: (pg_stat_plans.userid = pg_authid.oid) +|
+ | | | -> Function Scan on pg_stat_plans +|
+ | | | -> Hash +|
+ | | | -> Seq Scan on pg_authid |
+(6 rows)
+
+--
+-- cleanup
+--
+RESET ROLE;
+DROP ROLE regress_stats_superuser;
+DROP ROLE regress_stats_user1;
+DROP ROLE regress_stats_user2;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
diff --git a/contrib/pg_stat_plans/expected/select.out b/contrib/pg_stat_plans/expected/select.out
new file mode 100644
index 00000000000..906d8ce90d6
--- /dev/null
+++ b/contrib/pg_stat_plans/expected/select.out
@@ -0,0 +1,262 @@
+--
+-- SELECT statements
+--
+CREATE EXTENSION pg_stat_plans;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
+--
+-- simple statements
+--
+SELECT 1 FROM pg_class LIMIT 1;
+ ?column?
+----------
+ 1
+(1 row)
+
+SELECT 1 FROM pg_class WHERE relname = 'pg_class';
+ ?column?
+----------
+ 1
+(1 row)
+
+SET enable_indexscan = off;
+SELECT 1 FROM pg_class WHERE relname = 'pg_class';
+ ?column?
+----------
+ 1
+(1 row)
+
+SET enable_indexscan = on;
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+ plan | calls
+--------------------------------------------------------------+-------
+ Bitmap Heap Scan on pg_class +| 1
+ Recheck Cond: (relname = 'pg_class'::name) +|
+ -> Bitmap Index Scan on pg_class_relname_nsp_index +|
+ Index Cond: (relname = 'pg_class'::name) |
+ Index Only Scan using pg_class_relname_nsp_index on pg_class+| 1
+ Index Cond: (relname = 'pg_class'::name) |
+ Limit +| 1
+ -> Seq Scan on pg_class |
+ Result | 1
+ Sort +| 0
+ Sort Key: pg_stat_plans.plan COLLATE "C" +|
+ -> Function Scan on pg_stat_plans |
+(5 rows)
+
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
+--
+-- subplans and CTEs
+--
+WITH x AS MATERIALIZED (SELECT 1)
+SELECT * FROM x;
+ ?column?
+----------
+ 1
+(1 row)
+
+SELECT a.attname,
+ (SELECT pg_catalog.pg_get_expr(d.adbin, d.adrelid)
+ FROM pg_catalog.pg_attrdef d
+ WHERE d.adrelid = a.attrelid AND d.adnum = a.attnum AND a.atthasdef)
+ FROM pg_catalog.pg_attribute a
+ WHERE a.attrelid = 'pg_class'::regclass
+ ORDER BY attnum LIMIT 1;
+ attname | pg_get_expr
+----------+-------------
+ tableoid |
+(1 row)
+
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+ plan | calls
+-------------------------------------------------------------------------------+-------
+ CTE Scan on x +| 1
+ CTE x +|
+ -> Result |
+ Limit +| 1
+ -> Index Scan using pg_attribute_relid_attnum_index on pg_attribute a +|
+ Index Cond: (attrelid = '1259'::oid) +|
+ SubPlan 1 +|
+ -> Result +|
+ One-Time Filter: a.atthasdef +|
+ -> Seq Scan on pg_attrdef d +|
+ Filter: ((adrelid = a.attrelid) AND (adnum = a.attnum)) |
+ Result | 1
+ Sort +| 0
+ Sort Key: pg_stat_plans.plan COLLATE "C" +|
+ -> Function Scan on pg_stat_plans |
+(4 rows)
+
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
+--
+-- partitoning
+--
+create table lp (a char) partition by list (a);
+create table lp_default partition of lp default;
+create table lp_ef partition of lp for values in ('e', 'f');
+create table lp_ad partition of lp for values in ('a', 'd');
+create table lp_bc partition of lp for values in ('b', 'c');
+create table lp_g partition of lp for values in ('g');
+create table lp_null partition of lp for values in (null);
+select * from lp;
+ a
+---
+(0 rows)
+
+select * from lp where a > 'a' and a < 'd';
+ a
+---
+(0 rows)
+
+select * from lp where a > 'a' and a <= 'd';
+ a
+---
+(0 rows)
+
+select * from lp where a = 'a';
+ a
+---
+(0 rows)
+
+select * from lp where 'a' = a; /* commuted */
+ a
+---
+(0 rows)
+
+select * from lp where a is not null;
+ a
+---
+(0 rows)
+
+select * from lp where a is null;
+ a
+---
+(0 rows)
+
+select * from lp where a = 'a' or a = 'c';
+ a
+---
+(0 rows)
+
+select * from lp where a is not null and (a = 'a' or a = 'c');
+ a
+---
+(0 rows)
+
+select * from lp where a <> 'g';
+ a
+---
+(0 rows)
+
+select * from lp where a <> 'a' and a <> 'd';
+ a
+---
+(0 rows)
+
+select * from lp where a not in ('a', 'd');
+ a
+---
+(0 rows)
+
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+ plan | calls
+--------------------------------------------------------------------------------+-------
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ Filter: ((a = 'a'::bpchar) OR (a = 'c'::bpchar)) +|
+ -> Seq Scan on lp_bc lp_2 +|
+ Filter: ((a = 'a'::bpchar) OR (a = 'c'::bpchar)) |
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ Filter: ((a > 'a'::bpchar) AND (a <= 'd'::bpchar)) +|
+ -> Seq Scan on lp_bc lp_2 +|
+ Filter: ((a > 'a'::bpchar) AND (a <= 'd'::bpchar)) +|
+ -> Seq Scan on lp_default lp_3 +|
+ Filter: ((a > 'a'::bpchar) AND (a <= 'd'::bpchar)) |
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ Filter: ((a IS NOT NULL) AND ((a = 'a'::bpchar) OR (a = 'c'::bpchar)))+|
+ -> Seq Scan on lp_bc lp_2 +|
+ Filter: ((a IS NOT NULL) AND ((a = 'a'::bpchar) OR (a = 'c'::bpchar))) |
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ Filter: (a <> 'g'::bpchar) +|
+ -> Seq Scan on lp_bc lp_2 +|
+ Filter: (a <> 'g'::bpchar) +|
+ -> Seq Scan on lp_ef lp_3 +|
+ Filter: (a <> 'g'::bpchar) +|
+ -> Seq Scan on lp_default lp_4 +|
+ Filter: (a <> 'g'::bpchar) |
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ Filter: (a IS NOT NULL) +|
+ -> Seq Scan on lp_bc lp_2 +|
+ Filter: (a IS NOT NULL) +|
+ -> Seq Scan on lp_ef lp_3 +|
+ Filter: (a IS NOT NULL) +|
+ -> Seq Scan on lp_g lp_4 +|
+ Filter: (a IS NOT NULL) +|
+ -> Seq Scan on lp_default lp_5 +|
+ Filter: (a IS NOT NULL) |
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ -> Seq Scan on lp_bc lp_2 +|
+ -> Seq Scan on lp_ef lp_3 +|
+ -> Seq Scan on lp_g lp_4 +|
+ -> Seq Scan on lp_null lp_5 +|
+ -> Seq Scan on lp_default lp_6 |
+ Append +| 1
+ -> Seq Scan on lp_bc lp_1 +|
+ Filter: ((a <> 'a'::bpchar) AND (a <> 'd'::bpchar)) +|
+ -> Seq Scan on lp_ef lp_2 +|
+ Filter: ((a <> 'a'::bpchar) AND (a <> 'd'::bpchar)) +|
+ -> Seq Scan on lp_g lp_3 +|
+ Filter: ((a <> 'a'::bpchar) AND (a <> 'd'::bpchar)) +|
+ -> Seq Scan on lp_default lp_4 +|
+ Filter: ((a <> 'a'::bpchar) AND (a <> 'd'::bpchar)) |
+ Append +| 1
+ -> Seq Scan on lp_bc lp_1 +|
+ Filter: ((a > 'a'::bpchar) AND (a < 'd'::bpchar)) +|
+ -> Seq Scan on lp_default lp_2 +|
+ Filter: ((a > 'a'::bpchar) AND (a < 'd'::bpchar)) |
+ Append +| 1
+ -> Seq Scan on lp_bc lp_1 +|
+ Filter: (a <> ALL ('{a,d}'::bpchar[])) +|
+ -> Seq Scan on lp_ef lp_2 +|
+ Filter: (a <> ALL ('{a,d}'::bpchar[])) +|
+ -> Seq Scan on lp_g lp_3 +|
+ Filter: (a <> ALL ('{a,d}'::bpchar[])) +|
+ -> Seq Scan on lp_default lp_4 +|
+ Filter: (a <> ALL ('{a,d}'::bpchar[])) |
+ Result | 1
+ Seq Scan on lp_ad lp +| 1
+ Filter: ('a'::bpchar = a) |
+ Seq Scan on lp_ad lp +| 1
+ Filter: (a = 'a'::bpchar) |
+ Seq Scan on lp_null lp +| 1
+ Filter: (a IS NULL) |
+ Sort +| 0
+ Sort Key: pg_stat_plans.plan COLLATE "C" +|
+ -> Function Scan on pg_stat_plans |
+(14 rows)
+
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
diff --git a/contrib/pg_stat_plans/meson.build b/contrib/pg_stat_plans/meson.build
new file mode 100644
index 00000000000..3bd884d9601
--- /dev/null
+++ b/contrib/pg_stat_plans/meson.build
@@ -0,0 +1,43 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+pg_stat_plans_sources = files(
+ 'pg_stat_plans.c',
+)
+
+if host_system == 'windows'
+ pg_stat_plans_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'pg_stat_plans',
+ '--FILEDESC', 'pg_stat_plans - track per-plan call counts, execution times and EXPLAIN texts',])
+endif
+
+pg_stat_plans = shared_module('pg_stat_plans',
+ pg_stat_plans_sources,
+ kwargs: contrib_mod_args + {
+ 'dependencies': contrib_mod_args['dependencies'],
+ },
+)
+contrib_targets += pg_stat_plans
+
+install_data(
+ 'pg_stat_plans.control',
+ 'pg_stat_plans--1.0.sql',
+ kwargs: contrib_data_args,
+)
+
+tests += {
+ 'name': 'pg_stat_plans',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'regress': {
+ 'sql': [
+ 'select',
+ 'privileges',
+ 'cleanup',
+ ],
+ 'regress_args': ['--temp-config', files('pg_stat_plans.conf')],
+ # Disabled because these tests require
+ # "shared_preload_libraries=pg_stat_plans", which typical
+ # runningcheck users do not have (e.g. buildfarm clients).
+ 'runningcheck': false,
+ }
+}
diff --git a/contrib/pg_stat_plans/pg_stat_plans--1.0.sql b/contrib/pg_stat_plans/pg_stat_plans--1.0.sql
new file mode 100644
index 00000000000..f08452b274b
--- /dev/null
+++ b/contrib/pg_stat_plans/pg_stat_plans--1.0.sql
@@ -0,0 +1,33 @@
+/* contrib/pg_stat_plans/pg_stat_plans--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_stat_plans" to load this file. \quit
+
+-- Register functions.
+CREATE FUNCTION pg_stat_plans_reset()
+RETURNS void
+AS 'MODULE_PATHNAME'
+LANGUAGE C PARALLEL SAFE;
+
+CREATE FUNCTION pg_stat_plans(IN showplan boolean,
+ OUT userid oid,
+ OUT dbid oid,
+ OUT toplevel bool,
+ OUT queryid bigint,
+ OUT planid bigint,
+ OUT calls int8,
+ OUT total_exec_time float8,
+ OUT plan text
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stat_plans_1_0'
+LANGUAGE C STRICT VOLATILE PARALLEL SAFE;
+
+-- Register a view on the function for ease of use.
+CREATE VIEW pg_stat_plans AS
+ SELECT * FROM pg_stat_plans(true);
+
+GRANT SELECT ON pg_stat_plans TO PUBLIC;
+
+-- Don't want this to be available to non-superusers.
+REVOKE ALL ON FUNCTION pg_stat_plans_reset() FROM PUBLIC;
diff --git a/contrib/pg_stat_plans/pg_stat_plans.c b/contrib/pg_stat_plans/pg_stat_plans.c
new file mode 100644
index 00000000000..5fa31fcbf3e
--- /dev/null
+++ b/contrib/pg_stat_plans/pg_stat_plans.c
@@ -0,0 +1,779 @@
+/*--------------------------------------------------------------------------
+ *
+ * pg_stat_plans.c
+ * Track per-plan call counts, execution times and EXPLAIN texts
+ * across a whole database cluster.
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * contrib/pg_stat_plans/pg_stat_plans.c
+ *
+ * -------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/parallel.h"
+#include "catalog/pg_authid.h"
+#include "commands/explain.h"
+#include "common/hashfn.h"
+#include "funcapi.h"
+#include "mb/pg_wchar.h"
+#include "nodes/queryjumble.h"
+#include "pgstat.h"
+#include "optimizer/planner.h"
+#include "utils/acl.h"
+#include "utils/builtins.h"
+#include "utils/guc.h"
+#include "utils/pgstat_internal.h"
+#include "utils/snapmgr.h"
+
+PG_MODULE_MAGIC;
+
+/* Current nesting depth of planner/ExecutorRun/ProcessUtility calls */
+static int nesting_level = 0;
+
+/* Saved hook values */
+static planner_hook_type prev_planner_hook = NULL;
+static ExecutorStart_hook_type prev_ExecutorStart = NULL;
+static ExecutorRun_hook_type prev_ExecutorRun = NULL;
+static ExecutorFinish_hook_type prev_ExecutorFinish = NULL;
+static ExecutorEnd_hook_type prev_ExecutorEnd = NULL;
+
+/*---- GUC variables ----*/
+
+typedef enum
+{
+ PGSP_TRACK_NONE, /* track no plans */
+ PGSP_TRACK_TOP, /* only plans for top level statements */
+ PGSP_TRACK_ALL, /* all plans, including for nested statements */
+} PGSPTrackLevel;
+
+static const struct config_enum_entry track_options[] =
+{
+ {"none", PGSP_TRACK_NONE, false},
+ {"top", PGSP_TRACK_TOP, false},
+ {"all", PGSP_TRACK_ALL, false},
+ {NULL, 0, false}
+};
+
+static int pgsp_max = 5000; /* max # plans to track */
+static int pgsp_max_size = 2048; /* max size of plan text to track (in
+ * bytes) */
+static int pgsp_track = PGSP_TRACK_TOP; /* tracking level */
+
+#define pgsp_enabled(level) \
+ (!IsParallelWorker() && \
+ (compute_plan_id != COMPUTE_PLAN_ID_OFF) && \
+ (pgsp_track == PGSP_TRACK_ALL || \
+ (pgsp_track == PGSP_TRACK_TOP && (level) == 0)))
+
+#define USAGE_INCREASE 0.5 /* increase by this each time we report
+ * stats */
+#define USAGE_DECREASE_FACTOR (0.99) /* decreased every
+ * pgstat_dealloc_plans */
+#define USAGE_DEALLOC_PERCENT 5 /* free this % of entries at once */
+
+/*---- Function declarations ----*/
+
+PG_FUNCTION_INFO_V1(pg_stat_plans_reset);
+PG_FUNCTION_INFO_V1(pg_stat_plans_1_0);
+
+/* Structures for statistics of plans */
+typedef struct PgStatShared_PlanInfo
+{
+ /* key elements that identify a plan (together with the dboid) */
+ uint64 planid;
+ uint64 queryid;
+ Oid userid; /* userid is tracked to allow users to see
+ * their own query plans */
+ bool toplevel; /* query executed at top level */
+
+ dsa_pointer plan_text; /* pointer to DSA memory containing plan text */
+ int plan_encoding; /* plan text encoding */
+} PgStatShared_PlanInfo;
+
+typedef struct PgStat_StatPlanEntry
+{
+ PgStat_Counter exec_count;
+ double exec_time;
+ double usage; /* Usage factor of the entry, used to
+ * prioritize which plans to age out */
+
+ /* Only used in shared structure, not in local pending stats */
+ PgStatShared_PlanInfo info;
+} PgStat_StatPlanEntry;
+
+typedef struct PgStatShared_Plan
+{
+ PgStatShared_Common header;
+ PgStat_StatPlanEntry stats;
+} PgStatShared_Plan;
+
+static bool plan_stats_flush_cb(PgStat_EntryRef *entry_ref, bool nowait);
+
+static const PgStat_KindInfo plan_stats = {
+ .name = "plan_stats",
+ .fixed_amount = false,
+
+ /*
+ * We currently don't write to a file since plan texts would get lost (and
+ * just the stats on their own aren't that useful)
+ */
+ .write_to_file = false,
+
+ /*
+ * Plan statistics are available system-wide to simplify monitoring
+ * scripts
+ */
+ .accessed_across_databases = true,
+
+ .shared_size = sizeof(PgStatShared_Plan),
+ .shared_data_off = offsetof(PgStatShared_Plan, stats),
+ .shared_data_len = sizeof(((PgStatShared_Plan *) 0)->stats),
+ .pending_size = sizeof(PgStat_StatPlanEntry),
+ .flush_pending_cb = plan_stats_flush_cb,
+};
+
+/*
+ * Compute stats entry idx from query ID and plan ID with an 8-byte hash.
+ *
+ * Whilst we could theorically just use the plan ID here, we intentionally
+ * add the query ID into the mix to ease interpreting the data in combination
+ * with pg_stat_statements.
+ */
+#define PGSTAT_PLAN_IDX(query_id, plan_id, user_id, toplevel) hash_combine64(toplevel, hash_combine64(query_id, hash_combine64(plan_id, user_id)))
+
+/*
+ * Kind ID reserved for statistics of plans.
+ */
+#define PGSTAT_KIND_PLANS PGSTAT_KIND_EXPERIMENTAL /* TODO: Assign */
+
+/*
+ * Callback for stats handling
+ */
+static bool
+plan_stats_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
+{
+ PgStat_StatPlanEntry *localent;
+ PgStatShared_Plan *shfuncent;
+
+ localent = (PgStat_StatPlanEntry *) entry_ref->pending;
+ shfuncent = (PgStatShared_Plan *) entry_ref->shared_stats;
+
+ if (!pgstat_lock_entry(entry_ref, nowait))
+ return false;
+
+ shfuncent->stats.exec_count += localent->exec_count;
+ shfuncent->stats.exec_time += localent->exec_time;
+ shfuncent->stats.usage += localent->usage;
+
+ pgstat_unlock_entry(entry_ref);
+
+ return true;
+}
+
+static char *
+pgsp_explain_plan(QueryDesc *queryDesc)
+{
+ ExplainState *es;
+ StringInfo es_str;
+
+ es = NewExplainState();
+ es_str = es->str;
+
+ /*
+ * We turn off COSTS since identical planids may have very different
+ * costs, and it could be misleading to only show the first recorded
+ * plan's costs.
+ */
+ es->costs = false;
+ es->format = EXPLAIN_FORMAT_TEXT;
+
+ ExplainBeginOutput(es);
+ ExplainPrintPlan(es, queryDesc);
+ ExplainEndOutput(es);
+
+ return es_str->data;
+}
+
+static void
+pgstat_gc_plan_memory()
+{
+ dshash_seq_status hstat;
+ PgStatShared_HashEntry *p;
+
+ /* dshash entry is not modified, take shared lock */
+ dshash_seq_init(&hstat, pgStatLocal.shared_hash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ PgStatShared_Common *header;
+ PgStat_StatPlanEntry *statent;
+
+ if (!p->dropped || p->key.kind != PGSTAT_KIND_PLANS)
+ continue;
+
+ header = dsa_get_address(pgStatLocal.dsa, p->body);
+
+ if (!LWLockConditionalAcquire(&header->lock, LW_EXCLUSIVE))
+ continue;
+
+ statent = (PgStat_StatPlanEntry *) pgstat_get_entry_data(PGSTAT_KIND_PLANS, header);
+
+ /*
+ * Clean up this entry's plan text allocation, if we haven't done so
+ * already
+ */
+ if (DsaPointerIsValid(statent->info.plan_text))
+ {
+ dsa_free(pgStatLocal.dsa, statent->info.plan_text);
+ statent->info.plan_text = InvalidDsaPointer;
+
+ /* Allow removal of the shared stats entry */
+ pg_atomic_fetch_sub_u32(&p->refcount, 1);
+ }
+
+ LWLockRelease(&header->lock);
+ }
+ dshash_seq_term(&hstat);
+
+ /* Encourage other backends to clean up dropped entry refs */
+ pgstat_request_entry_refs_gc();
+}
+
+typedef struct PlanDeallocEntry
+{
+ PgStat_HashKey key;
+ double usage;
+} PlanDeallocEntry;
+
+/*
+ * list sort comparator for sorting into decreasing usage order
+ */
+static int
+entry_cmp_lru(const union ListCell *lhs, const union ListCell *rhs)
+{
+ double l_usage = ((PlanDeallocEntry *) lfirst(lhs))->usage;
+ double r_usage = ((PlanDeallocEntry *) lfirst(rhs))->usage;
+
+ if (l_usage > r_usage)
+ return -1;
+ else if (l_usage < r_usage)
+ return +1;
+ else
+ return 0;
+}
+
+static void
+pgstat_dealloc_plans()
+{
+ dshash_seq_status hstat;
+ PgStatShared_HashEntry *p;
+ List *entries = NIL;
+ ListCell *lc;
+ int nvictims;
+
+ /* dshash entry is not modified, take shared lock */
+ dshash_seq_init(&hstat, pgStatLocal.shared_hash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ PgStatShared_Common *header;
+ PgStat_StatPlanEntry *statent;
+ PlanDeallocEntry *entry;
+
+ if (p->dropped || p->key.kind != PGSTAT_KIND_PLANS)
+ continue;
+
+ header = dsa_get_address(pgStatLocal.dsa, p->body);
+
+ if (!LWLockConditionalAcquire(&header->lock, LW_EXCLUSIVE))
+ continue;
+
+ statent = (PgStat_StatPlanEntry *) pgstat_get_entry_data(PGSTAT_KIND_PLANS, header);
+ statent->usage *= USAGE_DECREASE_FACTOR;
+
+ entry = palloc(sizeof(PlanDeallocEntry));
+ entry->key = p->key;
+ entry->usage = statent->usage;
+
+ LWLockRelease(&header->lock);
+
+ entries = lappend(entries, entry);
+ }
+ dshash_seq_term(&hstat);
+
+ /* Sort by usage ascending (lowest used entries are last) */
+ list_sort(entries, entry_cmp_lru);
+
+ /* At a minimum, deallocate 10 entries to make it worth our while */
+ nvictims = Max(10, list_length(entries) * USAGE_DEALLOC_PERCENT / 100);
+ nvictims = Min(nvictims, list_length(entries));
+
+ /* Actually drop the entries */
+ for_each_from(lc, entries, list_length(entries) - nvictims)
+ {
+ PlanDeallocEntry *entry = lfirst(lc);
+
+ pgstat_drop_entry(entry->key.kind, entry->key.dboid, entry->key.objid);
+ }
+
+ /* Clean up our working memory immediately */
+ foreach(lc, entries)
+ {
+ PlanDeallocEntry *entry = lfirst(lc);
+
+ pfree(entry);
+ }
+ pfree(entries);
+}
+
+static void
+pgstat_gc_plans()
+{
+ dshash_seq_status hstat;
+ PgStatShared_HashEntry *p;
+ bool have_dropped_entries = false;
+ size_t plan_entry_count = 0;
+
+ /* TODO: Prevent concurrent GC cycles - flag an active GC run somehow */
+
+ /*
+ * Count our active entries, and whether there are any dropped entries we
+ * may need to clean up at the end.
+ */
+ dshash_seq_init(&hstat, pgStatLocal.shared_hash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ if (p->key.kind != PGSTAT_KIND_PLANS)
+ continue;
+
+ if (p->dropped)
+ have_dropped_entries = true;
+ else
+ plan_entry_count++;
+ }
+ dshash_seq_term(&hstat);
+
+ /*
+ * If we're over the limit, delete entries with lowest usage factor.
+ */
+ if (plan_entry_count > pgsp_max)
+ {
+ pgstat_dealloc_plans();
+ have_dropped_entries = true; /* Assume we did some work */
+ }
+
+ /* If there are dropped entries, clean up their plan memory if needed */
+ if (have_dropped_entries)
+ pgstat_gc_plan_memory();
+}
+
+static void
+pgstat_report_plan_stats(QueryDesc *queryDesc,
+ PgStat_Counter exec_count,
+ double exec_time)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_Plan *shstatent;
+ PgStat_StatPlanEntry *statent;
+ bool newly_created;
+ uint64 queryId = queryDesc->plannedstmt->queryId;
+ uint64 planId = queryDesc->plannedstmt->planId;
+ Oid userid = GetUserId();
+ bool toplevel = (nesting_level == 0);
+
+ entry_ref = pgstat_prep_pending_entry(PGSTAT_KIND_PLANS, MyDatabaseId,
+ PGSTAT_PLAN_IDX(queryId, planId, userid, toplevel), &newly_created);
+
+ shstatent = (PgStatShared_Plan *) entry_ref->shared_stats;
+ statent = &shstatent->stats;
+
+ if (newly_created)
+ {
+ char *plan = pgsp_explain_plan(queryDesc);
+ size_t plan_size = Min(strlen(plan), pgsp_max_size);
+
+ (void) pgstat_lock_entry(entry_ref, false);
+
+ /*
+ * We may be over the limit, so run GC now before saving entry (we do
+ * this whilst holding the lock on the new entry so we don't remove it
+ * by accident)
+ */
+ pgstat_gc_plans();
+
+ shstatent->stats.info.planid = planId;
+ shstatent->stats.info.queryid = queryId;
+ shstatent->stats.info.userid = userid;
+ shstatent->stats.info.toplevel = toplevel;
+ shstatent->stats.info.plan_text = dsa_allocate(pgStatLocal.dsa, plan_size);
+ strlcpy(dsa_get_address(pgStatLocal.dsa, shstatent->stats.info.plan_text), plan, plan_size);
+
+ shstatent->stats.info.plan_encoding = GetDatabaseEncoding();
+
+ /*
+ * Increase refcount here so entry can't get released without us
+ * dropping the plan text
+ */
+ pg_atomic_fetch_add_u32(&entry_ref->shared_entry->refcount, 1);
+
+ pgstat_unlock_entry(entry_ref);
+
+ pfree(plan);
+ }
+
+ statent->exec_count += exec_count;
+ statent->exec_time += exec_time;
+ statent->usage += USAGE_INCREASE;
+}
+
+/*
+ * Planner hook: forward to regular planner, but increase plan count and
+ * record query plan if needed.
+ */
+static PlannedStmt *
+pgsp_planner(Query *parse,
+ const char *query_string,
+ int cursorOptions,
+ ParamListInfo boundParams)
+{
+ PlannedStmt *result;
+
+ /*
+ * Increment the nesting level, to ensure that functions evaluated during
+ * planning are not seen as top-level calls.
+ */
+ nesting_level++;
+ PG_TRY();
+ {
+ if (prev_planner_hook)
+ result = prev_planner_hook(parse, query_string, cursorOptions,
+ boundParams);
+ else
+ result = standard_planner(parse, query_string, cursorOptions,
+ boundParams);
+ }
+ PG_FINALLY();
+ {
+ nesting_level--;
+ }
+ PG_END_TRY();
+
+ return result;
+}
+
+/*
+ * ExecutorStart hook: start up tracking if needed
+ */
+static void
+pgsp_ExecutorStart(QueryDesc *queryDesc, int eflags)
+{
+ uint64 queryId = queryDesc->plannedstmt->queryId;
+ uint64 planId = queryDesc->plannedstmt->planId;
+
+ if (prev_ExecutorStart)
+ prev_ExecutorStart(queryDesc, eflags);
+ else
+ standard_ExecutorStart(queryDesc, eflags);
+
+ if (queryId != UINT64CONST(0) && planId != UINT64CONST(0) &&
+ pgsp_enabled(nesting_level))
+ {
+ /*
+ * Record initial entry now, so plan text is available for currently
+ * running queries
+ */
+ pgstat_report_plan_stats(queryDesc,
+ 0, /* executions are counted in
+ * pgsp_ExecutorEnd */
+ 0.0);
+
+ /*
+ * Set up to track total elapsed time in ExecutorRun. Make sure the
+ * space is allocated in the per-query context so it will go away at
+ * ExecutorEnd.
+ */
+ if (queryDesc->totaltime == NULL)
+ {
+ MemoryContext oldcxt;
+
+ oldcxt = MemoryContextSwitchTo(queryDesc->estate->es_query_cxt);
+ queryDesc->totaltime = InstrAlloc(1, INSTRUMENT_ALL, false);
+ MemoryContextSwitchTo(oldcxt);
+ }
+ }
+}
+
+/*
+ * ExecutorRun hook: all we need do is track nesting depth
+ */
+static void
+pgsp_ExecutorRun(QueryDesc *queryDesc, ScanDirection direction, uint64 count)
+{
+ nesting_level++;
+ PG_TRY();
+ {
+ if (prev_ExecutorRun)
+ prev_ExecutorRun(queryDesc, direction, count);
+ else
+ standard_ExecutorRun(queryDesc, direction, count);
+ }
+ PG_FINALLY();
+ {
+ nesting_level--;
+ }
+ PG_END_TRY();
+}
+
+/*
+ * ExecutorFinish hook: all we need do is track nesting depth
+ */
+static void
+pgsp_ExecutorFinish(QueryDesc *queryDesc)
+{
+ nesting_level++;
+ PG_TRY();
+ {
+ if (prev_ExecutorFinish)
+ prev_ExecutorFinish(queryDesc);
+ else
+ standard_ExecutorFinish(queryDesc);
+ }
+ PG_FINALLY();
+ {
+ nesting_level--;
+ }
+ PG_END_TRY();
+}
+
+/*
+ * ExecutorEnd hook: store results if needed
+ */
+static void
+pgsp_ExecutorEnd(QueryDesc *queryDesc)
+{
+ uint64 queryId = queryDesc->plannedstmt->queryId;
+ uint64 planId = queryDesc->plannedstmt->planId;
+
+ if (queryId != UINT64CONST(0) && planId != UINT64CONST(0) &&
+ queryDesc->totaltime && pgsp_enabled(nesting_level))
+ {
+ /*
+ * Make sure stats accumulation is done. (Note: it's okay if several
+ * levels of hook all do this.)
+ */
+ InstrEndLoop(queryDesc->totaltime);
+
+ pgstat_report_plan_stats(queryDesc,
+ 1,
+ queryDesc->totaltime->total * 1000.0 /* convert to msec */ );
+ }
+
+ if (prev_ExecutorEnd)
+ prev_ExecutorEnd(queryDesc);
+ else
+ standard_ExecutorEnd(queryDesc);
+}
+
+/*
+ * Module load callback
+ */
+void
+_PG_init(void)
+{
+ /*
+ * In order to register for shared memory stats, we have to be loaded via
+ * shared_preload_libraries. If not, fall out without hooking into any of
+ * the main system. (We don't throw error here because it seems useful to
+ * allow the pg_stat_plans functions to be created even when the module
+ * isn't active. The functions must protect themselves against being
+ * called then, however.)
+ */
+ if (!process_shared_preload_libraries_in_progress)
+ return;
+
+ /*
+ * Inform the postmaster that we want to enable query_id calculation if
+ * compute_query_id is set to auto, as well as plan_id calculation if
+ * compute_plan_id is set to auto.
+ */
+ EnableQueryId();
+ EnablePlanId();
+
+ /*
+ * Define (or redefine) custom GUC variables.
+ */
+ DefineCustomIntVariable("pg_stat_plans.max",
+ "Sets the maximum number of plans tracked by pg_stat_plans in shared memory.",
+ NULL,
+ &pgsp_max,
+ 5000,
+ 100,
+ INT_MAX / 2,
+ PGC_SIGHUP,
+ 0,
+ NULL,
+ NULL,
+ NULL);
+
+ DefineCustomIntVariable("pg_stat_plans.max_size",
+ "Sets the maximum size of plan texts tracked by pg_stat_plans in shared memory.",
+ NULL,
+ &pgsp_max_size,
+ 2048,
+ 100,
+ 1048576, /* 1MB hard limit */
+ PGC_SUSET,
+ 0,
+ NULL,
+ NULL,
+ NULL);
+
+ DefineCustomEnumVariable("pg_stat_plans.track",
+ "Selects which plans are tracked by pg_stat_plans.",
+ NULL,
+ &pgsp_track,
+ PGSP_TRACK_TOP,
+ track_options,
+ PGC_SUSET,
+ 0,
+ NULL,
+ NULL,
+ NULL);
+
+ MarkGUCPrefixReserved("pg_stat_plans");
+
+ /*
+ * Install hooks.
+ */
+ prev_planner_hook = planner_hook;
+ planner_hook = pgsp_planner;
+ prev_ExecutorStart = ExecutorStart_hook;
+ ExecutorStart_hook = pgsp_ExecutorStart;
+ prev_ExecutorRun = ExecutorRun_hook;
+ ExecutorRun_hook = pgsp_ExecutorRun;
+ prev_ExecutorFinish = ExecutorFinish_hook;
+ ExecutorFinish_hook = pgsp_ExecutorFinish;
+ prev_ExecutorEnd = ExecutorEnd_hook;
+ ExecutorEnd_hook = pgsp_ExecutorEnd;
+
+ pgstat_register_kind(PGSTAT_KIND_PLANS, &plan_stats);
+}
+
+static bool
+match_plans_entries(PgStatShared_HashEntry *entry, Datum match_data)
+{
+ return entry->key.kind == PGSTAT_KIND_PLANS;
+}
+
+/*
+ * Reset statement statistics.
+ */
+Datum
+pg_stat_plans_reset(PG_FUNCTION_ARGS)
+{
+ pgstat_drop_matching_entries(match_plans_entries, 0);
+
+ /* Free plan text memory and allow cleanup of dropped entries */
+ pgstat_gc_plan_memory();
+
+ PG_RETURN_VOID();
+}
+
+#define PG_STAT_PLANS_COLS 8
+
+Datum
+pg_stat_plans_1_0(PG_FUNCTION_ARGS)
+{
+ bool showplan = PG_GETARG_BOOL(0);
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ Oid userid = GetUserId();
+ bool is_allowed_role = false;
+
+ dshash_seq_status hstat;
+ PgStatShared_HashEntry *p;
+
+ /*
+ * Superusers or roles with the privileges of pg_read_all_stats members
+ * are allowed
+ */
+ is_allowed_role = has_privs_of_role(userid, ROLE_PG_READ_ALL_STATS);
+
+ /* stats kind must be registered already */
+ if (!pgstat_get_kind_info(PGSTAT_KIND_PLANS))
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("pg_stat_plans must be loaded via \"shared_preload_libraries\"")));
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ /* dshash entry is not modified, take shared lock */
+ dshash_seq_init(&hstat, pgStatLocal.shared_hash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ PgStat_StatPlanEntry *statent;
+ Datum values[PG_STAT_PLANS_COLS];
+ bool nulls[PG_STAT_PLANS_COLS];
+ int i = 0;
+
+ if (p->dropped || p->key.kind != PGSTAT_KIND_PLANS)
+ continue;
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+
+ statent = pgstat_get_entry_data(p->key.kind, dsa_get_address(pgStatLocal.dsa, p->body));
+
+ values[i++] = ObjectIdGetDatum(statent->info.userid);
+ values[i++] = ObjectIdGetDatum(p->key.dboid);
+ values[i++] = BoolGetDatum(statent->info.toplevel);
+ if (is_allowed_role || statent->info.userid == userid)
+ {
+ int64 queryid = statent->info.queryid;
+ int64 planid = statent->info.planid;
+
+ values[i++] = Int64GetDatumFast(queryid);
+ values[i++] = Int64GetDatumFast(planid);
+ }
+ else
+ {
+ nulls[i++] = true;
+ nulls[i++] = true;
+ }
+ values[i++] = Int64GetDatumFast(statent->exec_count);
+ values[i++] = Float8GetDatumFast(statent->exec_time);
+
+ if (showplan && (is_allowed_role || statent->info.userid == userid))
+ {
+ char *pstr = DsaPointerIsValid(statent->info.plan_text) ? dsa_get_address(pgStatLocal.dsa, statent->info.plan_text) : NULL;
+
+ if (pstr)
+ {
+ char *enc = pg_any_to_server(pstr, strlen(pstr), statent->info.plan_encoding);
+
+ values[i++] = CStringGetTextDatum(enc);
+
+ if (enc != pstr)
+ pfree(enc);
+ }
+ else
+ {
+ nulls[i++] = true;
+ }
+ }
+ else if (showplan)
+ {
+ values[i++] = CStringGetTextDatum("<insufficient privilege>");
+ }
+ else
+ {
+ nulls[i++] = true;
+ }
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
+ }
+ dshash_seq_term(&hstat);
+
+ return (Datum) 0;
+}
diff --git a/contrib/pg_stat_plans/pg_stat_plans.conf b/contrib/pg_stat_plans/pg_stat_plans.conf
new file mode 100644
index 00000000000..6750b3e2cc0
--- /dev/null
+++ b/contrib/pg_stat_plans/pg_stat_plans.conf
@@ -0,0 +1 @@
+shared_preload_libraries = 'pg_stat_plans'
diff --git a/contrib/pg_stat_plans/pg_stat_plans.control b/contrib/pg_stat_plans/pg_stat_plans.control
new file mode 100644
index 00000000000..4db3a47239b
--- /dev/null
+++ b/contrib/pg_stat_plans/pg_stat_plans.control
@@ -0,0 +1,5 @@
+# pg_stat_plans extension
+comment = 'track per-plan call counts, execution times and EXPLAIN texts'
+default_version = '1.0'
+module_pathname = '$libdir/pg_stat_plans'
+relocatable = true
diff --git a/contrib/pg_stat_plans/sql/cleanup.sql b/contrib/pg_stat_plans/sql/cleanup.sql
new file mode 100644
index 00000000000..51565617cef
--- /dev/null
+++ b/contrib/pg_stat_plans/sql/cleanup.sql
@@ -0,0 +1 @@
+DROP EXTENSION pg_stat_plans;
diff --git a/contrib/pg_stat_plans/sql/privileges.sql b/contrib/pg_stat_plans/sql/privileges.sql
new file mode 100644
index 00000000000..aaad72a6553
--- /dev/null
+++ b/contrib/pg_stat_plans/sql/privileges.sql
@@ -0,0 +1,59 @@
+--
+-- Only superusers and roles with privileges of the pg_read_all_stats role
+-- are allowed to see the plan text, queryid and planid of queries executed by
+-- other users. Other users can see the statistics.
+--
+
+CREATE ROLE regress_stats_superuser SUPERUSER;
+CREATE ROLE regress_stats_user1;
+CREATE ROLE regress_stats_user2;
+GRANT pg_read_all_stats TO regress_stats_user2;
+
+SET ROLE regress_stats_superuser;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+SELECT 1 AS "ONE";
+
+SET ROLE regress_stats_user1;
+SELECT 1+1 AS "TWO";
+
+--
+-- A superuser can read all columns of queries executed by others,
+-- including plan text, queryid and planid.
+--
+
+SET ROLE regress_stats_superuser;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+
+--
+-- regress_stats_user1 has no privileges to read the plan text, queryid
+-- or planid of queries executed by others but can see statistics
+-- like calls and rows.
+--
+
+SET ROLE regress_stats_user1;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+
+--
+-- regress_stats_user2, with pg_read_all_stats role privileges, can
+-- read all columns, including plan text, queryid and planid, of queries
+-- executed by others.
+--
+
+SET ROLE regress_stats_user2;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+
+--
+-- cleanup
+--
+
+RESET ROLE;
+DROP ROLE regress_stats_superuser;
+DROP ROLE regress_stats_user1;
+DROP ROLE regress_stats_user2;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
diff --git a/contrib/pg_stat_plans/sql/select.sql b/contrib/pg_stat_plans/sql/select.sql
new file mode 100644
index 00000000000..f0e803ad70c
--- /dev/null
+++ b/contrib/pg_stat_plans/sql/select.sql
@@ -0,0 +1,67 @@
+--
+-- SELECT statements
+--
+
+CREATE EXTENSION pg_stat_plans;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+
+--
+-- simple statements
+--
+
+SELECT 1 FROM pg_class LIMIT 1;
+
+SELECT 1 FROM pg_class WHERE relname = 'pg_class';
+
+SET enable_indexscan = off;
+SELECT 1 FROM pg_class WHERE relname = 'pg_class';
+SET enable_indexscan = on;
+
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+
+--
+-- subplans and CTEs
+--
+
+WITH x AS MATERIALIZED (SELECT 1)
+SELECT * FROM x;
+
+SELECT a.attname,
+ (SELECT pg_catalog.pg_get_expr(d.adbin, d.adrelid)
+ FROM pg_catalog.pg_attrdef d
+ WHERE d.adrelid = a.attrelid AND d.adnum = a.attnum AND a.atthasdef)
+ FROM pg_catalog.pg_attribute a
+ WHERE a.attrelid = 'pg_class'::regclass
+ ORDER BY attnum LIMIT 1;
+
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+
+--
+-- partitoning
+--
+
+create table lp (a char) partition by list (a);
+create table lp_default partition of lp default;
+create table lp_ef partition of lp for values in ('e', 'f');
+create table lp_ad partition of lp for values in ('a', 'd');
+create table lp_bc partition of lp for values in ('b', 'c');
+create table lp_g partition of lp for values in ('g');
+create table lp_null partition of lp for values in (null);
+
+select * from lp;
+select * from lp where a > 'a' and a < 'd';
+select * from lp where a > 'a' and a <= 'd';
+select * from lp where a = 'a';
+select * from lp where 'a' = a; /* commuted */
+select * from lp where a is not null;
+select * from lp where a is null;
+select * from lp where a = 'a' or a = 'c';
+select * from lp where a is not null and (a = 'a' or a = 'c');
+select * from lp where a <> 'g';
+select * from lp where a <> 'a' and a <> 'd';
+select * from lp where a not in ('a', 'd');
+
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
--
2.47.2
On Thu, Jan 30, 2025 at 8:37 PM Michael Paquier <michael@paquier.xyz> wrote:
After thinking more about this one, I still want this toy and hearing
nothing I have applied it, with a second commit for the addition in
injection_points to avoid multiple bullet points in a single commit.
Thanks for committing! I had intended to review/test your patch, but the
earlier parts of the week got way too busy.
I think the API with do_drop makes sense, and whilst I'd think there is
some extra overhead to calling the function vs having an inline check for
kind, it seems unlikely this would be used in a performance critical
context, and the flexibility seems useful.
I have noticed post-commit that I have made a mistake in the credits
of a632cd354d35 and ce5c620fb625 for your family name. Really sorry
about that! This mistake is on me..
No worries regarding the name, happens to me all the time :)
What do you think?
Attached is a rebased version of the three remaining patches. While
looking at this stuff, I have noticed an extra cleanup that would be
good to have, as a separate change: we could reformat a bit the plan
header comments so as these do not require a rewrite when adding
node_attr to them, like d575051b9af9.
Yeah, I think that'd be helpful to move the comments before the fields - it
definitely gets hard to read.
Sami's patch set posted at [1] has the same problem, making the
proposals harder to parse and review, and the devil is in the details
with these pg_node_attr() properties attached to the structures. That
would be something to do on top of the proposed patch sets. Would any
of you be interested in that?
I'd be happy to tackle that - were you thinking to simply move any comments
before the field, in each case where we're adding an annotation?
Separately I've been thinking how we could best have a discussion/review on
whether the jumbling of specific plan struct fields is correct. I was
thinking maybe a quick wiki page could be helpful, noting why to jumble/not
jumble certain fields?
Thanks,
Lukas
--
Lukas Fittl
On Thu, Jan 30, 2025 at 09:19:49PM -0800, Lukas Fittl wrote:
I'd be happy to tackle that - were you thinking to simply move any comments
before the field, in each case where we're adding an annotation?
Yes.
Separately I've been thinking how we could best have a discussion/review on
whether the jumbling of specific plan struct fields is correct. I was
thinking maybe a quick wiki page could be helpful, noting why to jumble/not
jumble certain fields?
Makes sense. This is a complicated topic.
--
Michael
Separately I've been thinking how we could best have a discussion/review on
whether the jumbling of specific plan struct fields is correct. I was
thinking maybe a quick wiki page could be helpful, noting why to jumble/not
jumble certain fields?
Makes sense. This is a complicated topic.
+1 for the Wiki page
I started looking at the set of patches and started with v3-0001.
For that one, I think we need to refactor a bit more for
maintainability/readability.
queryjumblefuncs.c now has dual purposes which is the generic node jumbling
code and now it also has the specific query jumbling code. That seems wrong
from a readability/maintainability perspective.
Here are my high-level thoughts on this:
1. rename queryjumblefuncs.c to jumblefuncs.c
2. move the query jumbling related code to parser/analyze.c,
since query jumbling occurs there during parsing.
3. Rewrite the comments in the new jumblefuncs.c to
make it clear the intention of this infrastructure; that
it is used to jumble nodes for query or plan trees.
I can work on this if you agree.
Regards,
Sami
On Tue, Feb 04, 2025 at 05:14:48PM -0600, Sami Imseih wrote:
Here are my high-level thoughts on this:
1. rename queryjumblefuncs.c to jumblefuncs.c
If these APIs are used for somethings else than Query structure, yes,
the renaming makes sense.
2. move the query jumbling related code to parser/analyze.c,
since query jumbling occurs there during parsing.
Not sure about this one. It depends on how much is changed. As long
as everything related to the nodes stays in src/backend/nodes/,
perhaps that's OK.
3. Rewrite the comments in the new jumblefuncs.c to
make it clear the intention of this infrastructure; that
it is used to jumble nodes for query or plan trees.
Seems to me that this could be done before 2, as well.
I can work on this if you agree.
I'd welcome an extra patch to rework a bit the format of the comments
for the Plan nodes, to ease the addition of pg_node_attr(), making any
proposed patches more readable.
--
Michael
I can work on this if you agree.
I'd welcome an extra patch to rework a bit the format of the comments
for the Plan nodes, to ease the addition of pg_node_attr(), making any
proposed patches more readable.
I'll take care of this also.
Regards,
Sami
Looks like some emails were sent before I could send my draft email, but
hopefully this should make the follow up work easier :)
Attached a v4 patch set with a few minor changes to plan ID jumbling:
* Range table jumbling is now done in a separate JumbleRangeTable function
after setrefs.c walked the tree - this way we avoid having custom logic for
RT Indexes in the node jumbling, and keeping a reference to PlannerGlobal
in the jumble struct
* Moved the JumbleNode call to the bottom of the set_plan_references
function for clarity - previously it was before descending into inner/outer
plan, but after some other recursive calls to set_plan_references, which
didn't really make sense
* Fixed a bug with JUMBLE_ARRAY incorrectly taking the reference of the
array (which caused planid to change incorrectly between runs)
* Added JUMBLE_BITMAPSET
Further, I've significantly reduced the number of fields ignored for plan
jumbling:
Basically the approach taken in this version is that only things that would
negatively affect the planid (i.e. make it unique when it shouldn't be) are
ignored, vs ignoring duplicate fields and fields that are only used by the
executor (which is what v1-v3 did). I'm not 100% sure that's the right
approach (but it does keep the diff a good amount smaller), I think the
tradeoff here is basically jumbling performance vs maintenance overhead
when fields are added/changed.
This does not yet move field-specific comments to their own line in nodes
where we're adding node attributes, I'll leave that for Sami to work on.
On Tue, Feb 4, 2025 at 3:15 PM Sami Imseih <samimseih@gmail.com> wrote:
Separately I've been thinking how we could best have a
discussion/review on
whether the jumbling of specific plan struct fields is correct. I was
thinking maybe a quick wiki page could be helpful, noting why tojumble/not
jumble certain fields?
Makes sense. This is a complicated topic.
+1 for the Wiki page
Started here: https://wiki.postgresql.org/wiki/Plan_ID_Jumbling
Thanks,
Lukas
--
Lukas Fittl
Attachments:
v4-0002-Optionally-record-a-plan_id-in-PlannedStmt-to-ide.patchapplication/octet-stream; name=v4-0002-Optionally-record-a-plan_id-in-PlannedStmt-to-ide.patchDownload
From 59227e046cf2848b34ac77741d040b6e489be1bd Mon Sep 17 00:00:00 2001
From: Lukas Fittl <lukas@fittl.com>
Date: Sun, 2 Feb 2025 12:18:49 -0800
Subject: [PATCH v4 2/3] Optionally record a plan_id in PlannedStmt to identify
plan shape
When enabled via the new compute_plan_id GUC (default off), this utilizes
the existing treewalk in setrefs.c after planning to calculate a hash
(the "plan_id", or plan identifier) that can be used to identify
which plan was chosen.
The plan_id generally intends to be the same if a given EXPLAIN (without
ANALYZE) output is the same. The plan_id includes both the top-level plan
as well as all subplans. Execution statistics are excluded.
If enabled, the plan_id is shown for currently running queries in
pg_stat_activity, as well as recorded in EXPLAIN and auto_explain output.
Other in core users or extensions can use this facility to show or
accumulate statistics about the plans used by queries, to help identify
plan regressions, or drive plan management decisions.
Note that this commit intentionally does not include a facility to map
a given plan_id to the EXPLAIN text output - it is a assumed that users
can utilize the auto_explain extension to establish this mapping as
needed, or extensions can record this via the existing planner hook.
---
doc/src/sgml/config.sgml | 34 +++
doc/src/sgml/monitoring.sgml | 16 ++
src/backend/catalog/system_views.sql | 1 +
src/backend/commands/explain.c | 16 ++
src/backend/executor/execMain.c | 10 +-
src/backend/executor/execParallel.c | 1 +
src/backend/nodes/gen_node_support.pl | 36 ++-
src/backend/nodes/queryjumblefuncs.c | 65 +++++
src/backend/optimizer/plan/planner.c | 19 ++
src/backend/optimizer/plan/setrefs.c | 9 +
src/backend/postmaster/launch_backend.c | 3 +
src/backend/tcop/postgres.c | 1 +
src/backend/utils/activity/backend_status.c | 70 ++++-
src/backend/utils/adt/pgstatfuncs.c | 7 +-
src/backend/utils/misc/guc_tables.c | 28 ++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/catalog/pg_proc.dat | 6 +-
src/include/nodes/pathnodes.h | 3 +
src/include/nodes/plannodes.h | 243 ++++++++++++------
src/include/nodes/primnodes.h | 7 +-
src/include/nodes/queryjumble.h | 30 ++-
src/include/utils/backend_status.h | 5 +
src/test/regress/expected/explain.out | 11 +
src/test/regress/expected/rules.out | 9 +-
src/test/regress/sql/explain.sql | 4 +
25 files changed, 529 insertions(+), 106 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index a782f109982..85a1e0d8d4b 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -8406,6 +8406,40 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-compute-plan-id" xreflabel="compute_plan_id">
+ <term><varname>compute_plan_id</varname> (<type>enum</type>)
+ <indexterm>
+ <primary><varname>compute_plan_id</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Enables in-core computation of a plan identifier.
+ Plan identifiers can be displayed in the <link
+ linkend="monitoring-pg-stat-activity-view"><structname>pg_stat_activity</structname></link>
+ view or using <command>EXPLAIN</command>.
+ Note that an external module can alternatively be used if the
+ in-core plan identifier computation method is not acceptable.
+ In this case, in-core computation must be always disabled.
+ Valid values are <literal>off</literal> (always disabled),
+ <literal>on</literal> (always enabled), <literal>auto</literal>,
+ which lets modules that utilize plan identifiers enable
+ it automatically, and <literal>regress</literal> which
+ has the same effect as <literal>on</literal>, except that the
+ query identifier is not shown in the <literal>EXPLAIN</literal> output
+ in order to facilitate automated regression testing.
+ The default is <literal>auto</literal>.
+ </para>
+ <note>
+ <para>
+ To ensure that only one plan identifier is calculated and
+ displayed, extensions that calculate plan identifiers should
+ throw an error if a plan identifier has already been computed.
+ </para>
+ </note>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-log-statement-stats">
<term><varname>log_statement_stats</varname> (<type>boolean</type>)
<indexterm>
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index edc2470bcf9..64e62a09059 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -972,6 +972,22 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>plan_id</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Identifier of this backend's most recent query plan. If
+ <structfield>state</structfield> is <literal>active</literal> this
+ field shows the identifier of the currently executing query plan. In
+ all other states, it shows the identifier of last query plan that
+ was executed. Plan identifiers are not computed by default so this
+ field will be null unless <xref linkend="guc-compute-plan-id"/>
+ parameter is enabled or a third-party module that computes plan
+ identifiers is configured.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>query</structfield> <type>text</type>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index cddc3ea9b53..d3d5578d2c8 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -889,6 +889,7 @@ CREATE VIEW pg_stat_activity AS
S.backend_xid,
s.backend_xmin,
S.query_id,
+ S.plan_id,
S.query,
S.backend_type
FROM pg_stat_get_activity(NULL) AS S
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index c24e66f82e1..31a1761f539 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -966,6 +966,22 @@ ExplainPrintPlan(ExplainState *es, QueryDesc *queryDesc)
ExplainPropertyInteger("Query Identifier", NULL, (int64)
queryDesc->plannedstmt->queryId, es);
}
+
+ /*
+ * COMPUTE_PLAN_ID_REGRESS means COMPUTE_PLAN_ID_YES, but we don't show
+ * the queryid in any of the EXPLAIN plans to keep stable the results
+ * generated by regression test suites.
+ */
+ if (es->verbose && queryDesc->plannedstmt->planId != UINT64CONST(0) &&
+ compute_plan_id != COMPUTE_PLAN_ID_REGRESS)
+ {
+ /*
+ * Output the queryid as an int64 rather than a uint64 so we match
+ * what would be seen in the BIGINT pg_stat_activity.plan_id column.
+ */
+ ExplainPropertyInteger("Plan Identifier", NULL, (int64)
+ queryDesc->plannedstmt->planId, es);
+ }
}
/*
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 604cb0625b8..6890082d745 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -121,13 +121,15 @@ ExecutorStart(QueryDesc *queryDesc, int eflags)
{
/*
* In some cases (e.g. an EXECUTE statement or an execute message with the
- * extended query protocol) the query_id won't be reported, so do it now.
+ * extended query protocol) the query_id and plan_id won't be reported, so
+ * do it now.
*
- * Note that it's harmless to report the query_id multiple times, as the
- * call will be ignored if the top level query_id has already been
- * reported.
+ * Note that it's harmless to report the identifiers multiple times, as
+ * the call will be ignored if the top level query_id / plan_id has
+ * already been reported.
*/
pgstat_report_query_id(queryDesc->plannedstmt->queryId, false);
+ pgstat_report_plan_id(queryDesc->plannedstmt->planId, queryDesc->plannedstmt->queryId, false);
if (ExecutorStart_hook)
(*ExecutorStart_hook) (queryDesc, eflags);
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 9c313d81315..da2279579a8 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -174,6 +174,7 @@ ExecSerializePlan(Plan *plan, EState *estate)
pstmt = makeNode(PlannedStmt);
pstmt->commandType = CMD_SELECT;
pstmt->queryId = pgstat_get_my_query_id();
+ pstmt->planId = pgstat_get_my_plan_id();
pstmt->hasReturning = false;
pstmt->hasModifyingCTE = false;
pstmt->canSetTag = true;
diff --git a/src/backend/nodes/gen_node_support.pl b/src/backend/nodes/gen_node_support.pl
index 1a657f7e0ae..6c20bfc5986 100644
--- a/src/backend/nodes/gen_node_support.pl
+++ b/src/backend/nodes/gen_node_support.pl
@@ -1281,13 +1281,18 @@ _jumble${n}(JumbleState *jstate, Node *node)
{
my $t = $node_type_info{$n}->{field_types}{$f};
my @a = @{ $node_type_info{$n}->{field_attrs}{$f} };
+ my $array_size_field;
my $query_jumble_ignore = $struct_no_query_jumble;
my $query_jumble_location = 0;
# extract per-field attributes
foreach my $a (@a)
{
- if ($a eq 'query_jumble_ignore')
+ if ($a =~ /^array_size\(([\w.]+)\)$/)
+ {
+ $array_size_field = $1;
+ }
+ elsif ($a eq 'query_jumble_ignore')
{
$query_jumble_ignore = 1;
}
@@ -1297,8 +1302,15 @@ _jumble${n}(JumbleState *jstate, Node *node)
}
}
+ next if $query_jumble_ignore;
+
+ if ($t eq 'Bitmapset*')
+ {
+ print $jff "\tJUMBLE_BITMAPSET($f);\n"
+ unless $query_jumble_ignore;
+ }
# node type
- if (($t =~ /^(\w+)\*$/ or $t =~ /^struct\s+(\w+)\*$/)
+ elsif (($t =~ /^(\w+)\*$/ or $t =~ /^struct\s+(\w+)\*$/)
and elem $1, @node_types)
{
print $jff "\tJUMBLE_NODE($f);\n"
@@ -1318,6 +1330,26 @@ _jumble${n}(JumbleState *jstate, Node *node)
print $jff "\tJUMBLE_STRING($f);\n"
unless $query_jumble_ignore;
}
+ elsif ($t =~ /^(\w+)(\*|\[\w+\])$/ and elem $1, @scalar_types)
+ {
+ if (!defined $array_size_field)
+ {
+ die "no array size defined for $n.$f of type $t\n";
+ }
+ if ($node_type_info{$n}->{field_types}{$array_size_field} eq
+ 'List*')
+ {
+ print $jff
+ "\tJUMBLE_ARRAY($f, list_length(expr->$array_size_field));\n"
+ unless $query_jumble_ignore;
+ }
+ else
+ {
+ print $jff
+ "\tJUMBLE_ARRAY($f, expr->$array_size_field);\n"
+ unless $query_jumble_ignore;
+ }
+ }
else
{
print $jff "\tJUMBLE_FIELD($f);\n"
diff --git a/src/backend/nodes/queryjumblefuncs.c b/src/backend/nodes/queryjumblefuncs.c
index 545d8edcae2..64bec54a211 100644
--- a/src/backend/nodes/queryjumblefuncs.c
+++ b/src/backend/nodes/queryjumblefuncs.c
@@ -35,12 +35,14 @@
#include "common/hashfn.h"
#include "miscadmin.h"
#include "nodes/queryjumble.h"
+#include "parser/parsetree.h"
#include "parser/scansup.h"
#define JUMBLE_SIZE 1024 /* query serialization buffer size */
/* GUC parameters */
int compute_query_id = COMPUTE_QUERY_ID_AUTO;
+int compute_plan_id = COMPUTE_PLAN_ID_AUTO;
/*
* True when compute_query_id is ON or AUTO, and a module requests them.
@@ -51,6 +53,15 @@ int compute_query_id = COMPUTE_QUERY_ID_AUTO;
*/
bool query_id_enabled = false;
+/*
+ * True when compute_plan_id is ON or AUTO, and a module requests them.
+ *
+ * Note that IsPlanIdEnabled() should be used instead of checking
+ * plan_id_enabled or plan_query_id directly when we want to know
+ * whether plan identifiers are computed in the core or not.
+ */
+bool plan_id_enabled = false;
+
static void RecordConstLocation(JumbleState *jstate, int location);
static void _jumbleA_Const(JumbleState *jstate, Node *node);
static void _jumbleList(JumbleState *jstate, Node *node);
@@ -171,6 +182,19 @@ EnableQueryId(void)
query_id_enabled = true;
}
+/*
+ * Enables plan identifier computation.
+ *
+ * Third-party plugins can use this function to inform core that they require
+ * a query identifier to be computed.
+ */
+void
+EnablePlanId(void)
+{
+ if (compute_plan_id != COMPUTE_PLAN_ID_OFF)
+ plan_id_enabled = true;
+}
+
/*
* AppendJumble: Append a value that is substantive in a given query to
* the current jumble.
@@ -240,6 +264,13 @@ RecordConstLocation(JumbleState *jstate, int location)
RecordConstLocation(jstate, expr->location)
#define JUMBLE_FIELD(item) \
AppendJumble(jstate, (const unsigned char *) &(expr->item), sizeof(expr->item))
+#define JUMBLE_BITMAPSET(item) \
+do { \
+ if (expr->item) \
+ AppendJumble(jstate, (const unsigned char *) expr->item->words, sizeof(bitmapword) * expr->item->nwords); \
+} while(0)
+#define JUMBLE_ARRAY(item, len) \
+ AppendJumble(jstate, (const unsigned char *) expr->item, sizeof(*(expr->item)) * len)
#define JUMBLE_FIELD_SINGLE(item) \
AppendJumble(jstate, (const unsigned char *) &(item), sizeof(item))
#define JUMBLE_STRING(str) \
@@ -388,3 +419,37 @@ _jumbleVariableSetStmt(JumbleState *jstate, Node *node)
JUMBLE_FIELD(is_local);
JUMBLE_LOCATION(location);
}
+
+/*
+ * Jumble the entries in the rangle table to map RT indexes to relations
+ *
+ * This ensures jumbled RT indexes (e.g. in a Scan or Modify node), are
+ * distinguished by the target of the RT entry, even if the index is the same.
+ */
+void
+JumbleRangeTable(JumbleState *jstate, List *rtable)
+{
+ ListCell *lc;
+
+ foreach(lc, rtable)
+ {
+ RangeTblEntry *expr = lfirst_node(RangeTblEntry, lc);
+
+ switch (expr->rtekind)
+ {
+ case RTE_RELATION:
+ JUMBLE_FIELD(relid);
+ break;
+ case RTE_CTE:
+ JUMBLE_STRING(ctename);
+ break;
+ default:
+
+ /*
+ * Ignore other targets, the jumble includes something identifying
+ * about them already
+ */
+ break;
+ }
+ }
+}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index ffd7517ea97..9d43a174b9e 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -37,6 +37,7 @@
#ifdef OPTIMIZER_DEBUG
#include "nodes/print.h"
#endif
+#include "nodes/queryjumble.h"
#include "nodes/supportnodes.h"
#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
@@ -532,6 +533,16 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions,
Assert(glob->finalrowmarks == NIL);
Assert(glob->resultRelations == NIL);
Assert(glob->appendRelations == NIL);
+
+ /*
+ * Initialize plan identifier jumble if needed
+ *
+ * Note the actual jumbling is done in the tree walk in
+ * set_plan_references
+ */
+ if (IsPlanIdEnabled())
+ glob->plan_jumble_state = InitializeJumbleState(false);
+
top_plan = set_plan_references(root, top_plan);
/* ... and the subplans (both regular subplans and initplans) */
Assert(list_length(glob->subplans) == list_length(glob->subroots));
@@ -596,6 +607,14 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions,
result->jitFlags |= PGJIT_DEFORM;
}
+ if (IsPlanIdEnabled())
+ {
+ JumbleRangeTable(glob->plan_jumble_state, glob->finalrtable);
+ result->planId = HashJumbleState(glob->plan_jumble_state);
+ pfree(glob->plan_jumble_state->jumble);
+ pfree(glob->plan_jumble_state);
+ }
+
if (glob->partition_directory != NULL)
DestroyPartitionDirectory(glob->partition_directory);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 0868249be94..5e890f576b0 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -19,6 +19,7 @@
#include "catalog/pg_type.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
+#include "nodes/queryjumble.h"
#include "optimizer/optimizer.h"
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
@@ -1306,6 +1307,14 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset)
plan->lefttree = set_plan_refs(root, plan->lefttree, rtoffset);
plan->righttree = set_plan_refs(root, plan->righttree, rtoffset);
+ /*
+ * If enabled, append significant information to the plan identifier
+ * jumble (we do this here since we're already walking the tree in a
+ * near-final state)
+ */
+ if (IsPlanIdEnabled())
+ JumbleNode(root->glob->plan_jumble_state, (Node *) plan);
+
return plan;
}
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index a97a1eda6da..654acf5bf04 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -115,6 +115,7 @@ typedef struct
bool redirection_done;
bool IsBinaryUpgrade;
bool query_id_enabled;
+ bool plan_id_enabled;
int max_safe_fds;
int MaxBackends;
int num_pmchild_slots;
@@ -744,6 +745,7 @@ save_backend_variables(BackendParameters *param,
param->redirection_done = redirection_done;
param->IsBinaryUpgrade = IsBinaryUpgrade;
param->query_id_enabled = query_id_enabled;
+ param->plan_id_enabled = plan_id_enabled;
param->max_safe_fds = max_safe_fds;
param->MaxBackends = MaxBackends;
@@ -1004,6 +1006,7 @@ restore_backend_variables(BackendParameters *param)
redirection_done = param->redirection_done;
IsBinaryUpgrade = param->IsBinaryUpgrade;
query_id_enabled = param->query_id_enabled;
+ plan_id_enabled = param->plan_id_enabled;
max_safe_fds = param->max_safe_fds;
MaxBackends = param->MaxBackends;
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 5655348a2e2..6d8947bae9b 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -1106,6 +1106,7 @@ exec_simple_query(const char *query_string)
size_t cmdtaglen;
pgstat_report_query_id(0, true);
+ pgstat_report_plan_id(0, 0, true);
/*
* Get the command name for use in status display (it also becomes the
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 731342799a6..1dfb7a58f87 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -379,6 +379,7 @@ pgstat_bestart(void)
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = UINT64CONST(0);
+ lbeentry.st_plan_id = UINT64CONST(0);
/*
* we don't zero st_progress_param here to save cycles; nobody should
@@ -533,6 +534,7 @@ pgstat_report_activity(BackendState state, const char *cmd_str)
/* st_xact_start_timestamp and wait_event_info are also disabled */
beentry->st_xact_start_timestamp = 0;
beentry->st_query_id = UINT64CONST(0);
+ beentry->st_plan_id = UINT64CONST(0);
proc->wait_event_info = 0;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -588,12 +590,15 @@ pgstat_report_activity(BackendState state, const char *cmd_str)
beentry->st_state_start_timestamp = current_timestamp;
/*
- * If a new query is started, we reset the query identifier as it'll only
- * be known after parse analysis, to avoid reporting last query's
- * identifier.
+ * If a new query is started, we reset the query and plan identifier as
+ * it'll only be known after parse analysis / planning, to avoid reporting
+ * last query's identifiers.
*/
if (state == STATE_RUNNING)
+ {
beentry->st_query_id = UINT64CONST(0);
+ beentry->st_plan_id = UINT64CONST(0);
+ }
if (cmd_str != NULL)
{
@@ -644,6 +649,45 @@ pgstat_report_query_id(uint64 query_id, bool force)
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
+/* --------
+ * pgstat_report_plan_id() -
+ *
+ * Called to update top-level plan identifier.
+ * --------
+ */
+void
+pgstat_report_plan_id(uint64 plan_id, uint64 query_id, bool force)
+{
+ volatile PgBackendStatus *beentry = MyBEEntry;
+
+ /*
+ * if track_activities is disabled, st_plan_id should already have been
+ * reset
+ */
+ if (!beentry || !pgstat_track_activities)
+ return;
+
+ /*
+ * We only report the top-level plan identifiers. The stored plan_id is
+ * reset when a backend calls pgstat_report_activity(STATE_RUNNING), or
+ * with an explicit call to this function using the force flag. If the
+ * saved plan identifier is not zero or the query identifier is 0, it
+ * means that it's not a top-level command, so ignore the one provided
+ * unless it's an explicit call to reset the identifier.
+ */
+ if ((beentry->st_plan_id != 0 || query_id == 0) && !force)
+ return;
+
+ /*
+ * Update my status entry, following the protocol of bumping
+ * st_changecount before and after. We use a volatile pointer here to
+ * ensure the compiler doesn't try to get cute.
+ */
+ PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
+ beentry->st_plan_id = plan_id;
+ PGSTAT_END_WRITE_ACTIVITY(beentry);
+}
+
/* ----------
* pgstat_report_appname() -
@@ -1040,6 +1084,26 @@ pgstat_get_my_query_id(void)
return MyBEEntry->st_query_id;
}
+/* ----------
+ * pgstat_get_my_plan_id() -
+ *
+ * Return current backend's plan identifier.
+ */
+uint64
+pgstat_get_my_plan_id(void)
+{
+ if (!MyBEEntry)
+ return 0;
+
+ /*
+ * There's no need for a lock around pgstat_begin_read_activity /
+ * pgstat_end_read_activity here as it's only called from
+ * pg_stat_get_activity which is already protected, or from the same
+ * backend which means that there won't be concurrent writes.
+ */
+ return MyBEEntry->st_plan_id;
+}
+
/* ----------
* pgstat_get_backend_type_by_proc_number() -
*
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index e9096a88492..a3d5592beb6 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -330,7 +330,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum
pg_stat_get_activity(PG_FUNCTION_ARGS)
{
-#define PG_STAT_GET_ACTIVITY_COLS 31
+#define PG_STAT_GET_ACTIVITY_COLS 32
int num_backends = pgstat_fetch_stat_numbackends();
int curr_backend;
int pid = PG_ARGISNULL(0) ? -1 : PG_GETARG_INT32(0);
@@ -641,6 +641,10 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
nulls[30] = true;
else
values[30] = UInt64GetDatum(beentry->st_query_id);
+ if (beentry->st_plan_id == 0)
+ nulls[31] = true;
+ else
+ values[31] = UInt64GetDatum(beentry->st_plan_id);
}
else
{
@@ -670,6 +674,7 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
nulls[28] = true;
nulls[29] = true;
nulls[30] = true;
+ nulls[31] = true;
}
tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 71448bb4fdd..22fdfb60ccc 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -309,6 +309,24 @@ static const struct config_enum_entry compute_query_id_options[] = {
{NULL, 0, false}
};
+/*
+ * Although only "on" and "off" are documented, we accept
+ * all the likely variants of "on" and "off".
+ */
+static const struct config_enum_entry compute_plan_id_options[] = {
+ {"auto", COMPUTE_PLAN_ID_AUTO, false},
+ {"regress", COMPUTE_PLAN_ID_REGRESS, false},
+ {"on", COMPUTE_PLAN_ID_ON, false},
+ {"off", COMPUTE_PLAN_ID_OFF, false},
+ {"true", COMPUTE_PLAN_ID_ON, true},
+ {"false", COMPUTE_PLAN_ID_OFF, true},
+ {"yes", COMPUTE_PLAN_ID_ON, true},
+ {"no", COMPUTE_PLAN_ID_OFF, true},
+ {"1", COMPUTE_PLAN_ID_ON, true},
+ {"0", COMPUTE_PLAN_ID_OFF, true},
+ {NULL, 0, false}
+};
+
/*
* Although only "on", "off", and "partition" are documented, we
* accept all the likely variants of "on" and "off".
@@ -4873,6 +4891,16 @@ struct config_enum ConfigureNamesEnum[] =
NULL, NULL, NULL
},
+ {
+ {"compute_plan_id", PGC_SUSET, STATS_MONITORING,
+ gettext_noop("Enables in-core computation of plan identifiers."),
+ NULL
+ },
+ &compute_plan_id,
+ COMPUTE_PLAN_ID_AUTO, compute_plan_id_options,
+ NULL, NULL, NULL
+ },
+
{
{"constraint_exclusion", PGC_USERSET, QUERY_TUNING_OTHER,
gettext_noop("Enables the planner to use constraints to optimize queries."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 079efa1baa7..0634ae90dd3 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -641,6 +641,7 @@
# - Monitoring -
#compute_query_id = auto
+#compute_plan_id = auto
#log_statement_stats = off
#log_parser_stats = off
#log_planner_stats = off
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 5b8c2ad2a54..5547db7d2f1 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5568,9 +5568,9 @@
proname => 'pg_stat_get_activity', prorows => '100', proisstrict => 'f',
proretset => 't', provolatile => 's', proparallel => 'r',
prorettype => 'record', proargtypes => 'int4',
- proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,bool,int4,int8}',
- proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,gss_delegation,leader_pid,query_id}',
+ proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,bool,int4,int8,int8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,gss_delegation,leader_pid,query_id,plan_id}',
prosrc => 'pg_stat_get_activity' },
{ oid => '6318', descr => 'describe wait events',
proname => 'pg_get_wait_events', procost => '10', prorows => '250',
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 52d44f43021..4baf829f644 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -166,6 +166,9 @@ typedef struct PlannerGlobal
/* partition descriptors */
PartitionDirectory partition_directory pg_node_attr(read_write_ignore);
+
+ /* optional jumble state for plan identifier calculation */
+ struct JumbleState *plan_jumble_state pg_node_attr(read_write_ignore);
} PlannerGlobal;
/* macro for fetching the Plan associated with a SubPlan node */
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 06d9559ebb9..2ccf772d87e 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -53,6 +53,10 @@ typedef struct PlannedStmt
uint64 queryId; /* query identifier (copied from Query) */
+ uint64 planId; /* plan identifier (calculated if
+ * compute_plan_id is enabled, can also be set
+ * by plugins) */
+
bool hasReturning; /* is it insert|update|delete|merge RETURNING? */
bool hasModifyingCTE; /* has insert|update|delete|merge in WITH? */
@@ -121,44 +125,55 @@ typedef struct PlannedStmt
*/
typedef struct Plan
{
- pg_node_attr(abstract, no_equal, no_query_jumble)
+ pg_node_attr(abstract, no_equal)
NodeTag type;
/*
* estimated execution costs for plan (see costsize.c for more info)
*/
- int disabled_nodes; /* count of disabled nodes */
- Cost startup_cost; /* cost expended before fetching any tuples */
- Cost total_cost; /* total cost (assuming all tuples fetched) */
+ int disabled_nodes pg_node_attr(query_jumble_ignore); /* count of disabled
+ * nodes */
+ Cost startup_cost pg_node_attr(query_jumble_ignore); /* cost expended before
+ * fetching any tuples */
+ Cost total_cost pg_node_attr(query_jumble_ignore); /* total cost (assuming
+ * all tuples fetched) */
/*
* planner's estimate of result size of this plan step
*/
- Cardinality plan_rows; /* number of rows plan is expected to emit */
- int plan_width; /* average row width in bytes */
+ Cardinality plan_rows pg_node_attr(query_jumble_ignore); /* number of rows plan
+ * is expected to emit */
+ int plan_width pg_node_attr(query_jumble_ignore); /* average row width in
+ * bytes */
/*
* information needed for parallel query
*/
- bool parallel_aware; /* engage parallel-aware logic? */
- bool parallel_safe; /* OK to use as part of parallel plan? */
+ bool parallel_aware; /* engage parallel-aware
+ * logic? */
+ bool parallel_safe; /* OK to use as part of
+ * parallel plan? */
/*
* information needed for asynchronous execution
*/
- bool async_capable; /* engage asynchronous-capable logic? */
+ bool async_capable; /* engage
+ * asynchronous-capable
+ * logic? */
/*
* Common structural data for all Plan types.
*/
- int plan_node_id; /* unique across entire final plan tree */
+ int plan_node_id; /* unique across entire
+ * final plan tree */
List *targetlist; /* target list to be computed at this node */
List *qual; /* implicitly-ANDed qual conditions */
- struct Plan *lefttree; /* input plan tree(s) */
- struct Plan *righttree;
- List *initPlan; /* Init Plan nodes (un-correlated expr
- * subselects) */
+ struct Plan *lefttree pg_node_attr(query_jumble_ignore); /* input plan tree(s) */
+ struct Plan *righttree pg_node_attr(query_jumble_ignore);
+ List *initPlan pg_node_attr(query_jumble_ignore); /* Init Plan nodes
+ * (un-correlated expr
+ * subselects) */
/*
* Information for management of parameter-change-driven rescanning
@@ -234,31 +249,47 @@ typedef struct ModifyTable
{
Plan plan;
CmdType operation; /* INSERT, UPDATE, DELETE, or MERGE */
- bool canSetTag; /* do we set the command tag/es_processed? */
- Index nominalRelation; /* Parent RT index for use of EXPLAIN */
- Index rootRelation; /* Root RT index, if partitioned/inherited */
+ bool canSetTag; /* do we set the command
+ * tag/es_processed? */
+ Index nominalRelation; /* Parent RT index for
+ * use of EXPLAIN */
+ Index rootRelation; /* Root RT index, if
+ * partitioned/inherited */
bool partColsUpdated; /* some part key in hierarchy updated? */
- List *resultRelations; /* integer list of RT indexes */
- List *updateColnosLists; /* per-target-table update_colnos lists */
- List *withCheckOptionLists; /* per-target-table WCO lists */
- char *returningOldAlias; /* alias for OLD in RETURNING lists */
- char *returningNewAlias; /* alias for NEW in RETURNING lists */
- List *returningLists; /* per-target-table RETURNING tlists */
- List *fdwPrivLists; /* per-target-table FDW private data lists */
- Bitmapset *fdwDirectModifyPlans; /* indices of FDW DM plans */
- List *rowMarks; /* PlanRowMarks (non-locking only) */
- int epqParam; /* ID of Param for EvalPlanQual re-eval */
+ List *resultRelations; /* integer list of RT
+ * indexes */
+ List *updateColnosLists; /* per-target-table
+ * update_colnos lists */
+ List *withCheckOptionLists; /* per-target-table WCO
+ * lists */
+ char *returningOldAlias; /* alias for OLD in
+ * RETURNING lists */
+ char *returningNewAlias; /* alias for NEW in
+ * RETURNING lists */
+ List *returningLists; /* per-target-table
+ * RETURNING tlists */
+ List *fdwPrivLists pg_node_attr(query_jumble_ignore); /* per-target-table FDW
+ * private data lists */
+ Bitmapset *fdwDirectModifyPlans; /* indices of FDW DM
+ * plans */
+ List *rowMarks; /* PlanRowMarks
+ * (non-locking only) */
+ int epqParam; /* ID of Param for
+ * EvalPlanQual re-eval */
OnConflictAction onConflictAction; /* ON CONFLICT action */
List *arbiterIndexes; /* List of ON CONFLICT arbiter index OIDs */
List *onConflictSet; /* INSERT ON CONFLICT DO UPDATE targetlist */
List *onConflictCols; /* target column numbers for onConflictSet */
Node *onConflictWhere; /* WHERE for ON CONFLICT UPDATE */
- Index exclRelRTI; /* RTI of the EXCLUDED pseudo relation */
- List *exclRelTlist; /* tlist of the EXCLUDED pseudo relation */
- List *mergeActionLists; /* per-target-table lists of actions for
- * MERGE */
- List *mergeJoinConditions; /* per-target-table join conditions
- * for MERGE */
+ Index exclRelRTI; /* RTI of the EXCLUDED
+ * pseudo relation */
+ List *exclRelTlist; /* tlist of the EXCLUDED
+ * pseudo relation */
+ List *mergeActionLists; /* per-target-table
+ * lists of actions for
+ * MERGE */
+ List *mergeJoinConditions; /* per-target-table join
+ * conditions for MERGE */
} ModifyTable;
struct PartitionPruneInfo; /* forward reference to struct below */
@@ -272,7 +303,7 @@ typedef struct Append
{
Plan plan;
Bitmapset *apprelids; /* RTIs of appendrel(s) formed by this node */
- List *appendplans;
+ List *appendplans pg_node_attr(query_jumble_ignore);
int nasyncplans; /* # of asynchronous plans */
/*
@@ -301,7 +332,7 @@ typedef struct MergeAppend
/* RTIs of appendrel(s) formed by this node */
Bitmapset *apprelids;
- List *mergeplans;
+ List *mergeplans pg_node_attr(query_jumble_ignore);
/* these fields are just like the sort-key info in struct Sort: */
@@ -356,7 +387,7 @@ typedef struct RecursiveUnion
Oid *dupCollations pg_node_attr(array_size(numCols));
/* estimated number of groups in input */
- long numGroups;
+ long numGroups pg_node_attr(query_jumble_ignore);
} RecursiveUnion;
/* ----------------
@@ -370,7 +401,7 @@ typedef struct RecursiveUnion
typedef struct BitmapAnd
{
Plan plan;
- List *bitmapplans;
+ List *bitmapplans pg_node_attr(query_jumble_ignore);
} BitmapAnd;
/* ----------------
@@ -385,7 +416,7 @@ typedef struct BitmapOr
{
Plan plan;
bool isshared;
- List *bitmapplans;
+ List *bitmapplans pg_node_attr(query_jumble_ignore);
} BitmapOr;
/*
@@ -400,7 +431,8 @@ typedef struct Scan
pg_node_attr(abstract)
Plan plan;
- Index scanrelid; /* relid is index into the range table */
+ Index scanrelid; /* relid is index into
+ * the range table */
} Scan;
/* ----------------
@@ -465,9 +497,11 @@ typedef struct IndexScan
Scan scan;
Oid indexid; /* OID of index to scan */
List *indexqual; /* list of index quals (usually OpExprs) */
- List *indexqualorig; /* the same in original form */
+ List *indexqualorig; /* the same in original
+ * form */
List *indexorderby; /* list of index ORDER BY exprs */
- List *indexorderbyorig; /* the same in original form */
+ List *indexorderbyorig; /* the same in original
+ * form */
List *indexorderbyops; /* OIDs of sort ops for ORDER BY exprs */
ScanDirection indexorderdir; /* forward or backward or don't care */
} IndexScan;
@@ -508,9 +542,12 @@ typedef struct IndexOnlyScan
Scan scan;
Oid indexid; /* OID of index to scan */
List *indexqual; /* list of index quals (usually OpExprs) */
- List *recheckqual; /* index quals in recheckable form */
+ List *recheckqual; /* index quals in
+ * recheckable form */
List *indexorderby; /* list of index ORDER BY exprs */
- List *indextlist; /* TargetEntry list describing index's cols */
+ List *indextlist; /* TargetEntry list
+ * describing index's
+ * cols */
ScanDirection indexorderdir; /* forward or backward or don't care */
} IndexOnlyScan;
@@ -535,9 +572,11 @@ typedef struct BitmapIndexScan
{
Scan scan;
Oid indexid; /* OID of index to scan */
- bool isshared; /* Create shared bitmap if set */
+ bool isshared; /* Create shared bitmap
+ * if set */
List *indexqual; /* list of index quals (OpExprs) */
- List *indexqualorig; /* the same in original form */
+ List *indexqualorig; /* the same in original
+ * form */
} BitmapIndexScan;
/* ----------------
@@ -552,7 +591,8 @@ typedef struct BitmapIndexScan
typedef struct BitmapHeapScan
{
Scan scan;
- List *bitmapqualorig; /* index quals, in standard expr form */
+ List *bitmapqualorig; /* index quals, in
+ * standard expr form */
} BitmapHeapScan;
/* ----------------
@@ -612,7 +652,7 @@ typedef enum SubqueryScanStatus
typedef struct SubqueryScan
{
Scan scan;
- Plan *subplan;
+ Plan *subplan pg_node_attr(query_jumble_ignore);
SubqueryScanStatus scanstatus;
} SubqueryScan;
@@ -654,8 +694,11 @@ typedef struct TableFuncScan
typedef struct CteScan
{
Scan scan;
- int ctePlanId; /* ID of init SubPlan for CTE */
- int cteParam; /* ID of Param representing CTE output */
+ int ctePlanId; /* ID of init SubPlan
+ * for CTE */
+ int cteParam; /* ID of Param
+ * representing CTE
+ * output */
} CteScan;
/* ----------------
@@ -675,7 +718,9 @@ typedef struct NamedTuplestoreScan
typedef struct WorkTableScan
{
Scan scan;
- int wtParam; /* ID of Param representing work table */
+ int wtParam; /* ID of Param
+ * representing work
+ * table */
} WorkTableScan;
/* ----------------
@@ -722,17 +767,26 @@ typedef struct ForeignScan
{
Scan scan;
CmdType operation; /* SELECT/INSERT/UPDATE/DELETE */
- Index resultRelation; /* direct modification target's RT index */
- Oid checkAsUser; /* user to perform the scan as; 0 means to
- * check as current user */
+ Index resultRelation; /* direct modification
+ * target's RT index */
+ Oid checkAsUser; /* user to perform the
+ * scan as; 0 means to
+ * check as current user */
Oid fs_server; /* OID of foreign server */
- List *fdw_exprs; /* expressions that FDW may evaluate */
- List *fdw_private; /* private data for FDW */
- List *fdw_scan_tlist; /* optional tlist describing scan tuple */
- List *fdw_recheck_quals; /* original quals not in scan.plan.qual */
- Bitmapset *fs_relids; /* base+OJ RTIs generated by this scan */
- Bitmapset *fs_base_relids; /* base RTIs generated by this scan */
- bool fsSystemCol; /* true if any "system column" is needed */
+ List *fdw_exprs; /* expressions that FDW
+ * may evaluate */
+ List *fdw_private pg_node_attr(query_jumble_ignore); /* private data for FDW */
+ List *fdw_scan_tlist; /* optional tlist
+ * describing scan tuple */
+ List *fdw_recheck_quals; /* original quals not in
+ * scan.plan.qual */
+ Bitmapset *fs_relids; /* base+OJ RTIs
+ * generated by this
+ * scan */
+ Bitmapset *fs_base_relids; /* base RTIs generated
+ * by this scan */
+ bool fsSystemCol; /* true if any "system
+ * column" is needed */
} ForeignScan;
/* ----------------
@@ -753,20 +807,27 @@ struct CustomScanMethods;
typedef struct CustomScan
{
Scan scan;
- uint32 flags; /* mask of CUSTOMPATH_* flags, see
- * nodes/extensible.h */
- List *custom_plans; /* list of Plan nodes, if any */
- List *custom_exprs; /* expressions that custom code may evaluate */
- List *custom_private; /* private data for custom code */
- List *custom_scan_tlist; /* optional tlist describing scan tuple */
- Bitmapset *custom_relids; /* RTIs generated by this scan */
+ uint32 flags; /* mask of CUSTOMPATH_*
+ * flags, see
+ * nodes/extensible.h */
+ List *custom_plans; /* list of Plan nodes,
+ * if any */
+ List *custom_exprs; /* expressions that
+ * custom code may
+ * evaluate */
+ List *custom_private pg_node_attr(query_jumble_ignore); /* private data for
+ * custom code */
+ List *custom_scan_tlist; /* optional tlist
+ * describing scan tuple */
+ Bitmapset *custom_relids; /* RTIs generated by
+ * this scan */
/*
* NOTE: The method field of CustomScan is required to be a pointer to a
* static table of callback functions. So we don't copy the table itself,
* just reference the original one.
*/
- const struct CustomScanMethods *methods;
+ const struct CustomScanMethods *methods pg_node_attr(query_jumble_ignore);
} CustomScan;
/*
@@ -826,7 +887,7 @@ typedef struct NestLoop
typedef struct NestLoopParam
{
- pg_node_attr(no_equal, no_query_jumble)
+ pg_node_attr(no_equal)
NodeTag type;
int paramno; /* number of the PARAM_EXEC Param to set */
@@ -932,7 +993,7 @@ typedef struct Memoize
* The maximum number of entries that the planner expects will fit in the
* cache, or 0 if unknown
*/
- uint32 est_entries;
+ uint32 est_entries pg_node_attr(query_jumble_ignore);
/* paramids from param_exprs */
Bitmapset *keyparamids;
@@ -1028,7 +1089,7 @@ typedef struct Agg
Oid *grpCollations pg_node_attr(array_size(numCols));
/* estimated number of groups in input */
- long numGroups;
+ long numGroups pg_node_attr(query_jumble_ignore);
/* for pass-by-ref transition data */
uint64 transitionSpace;
@@ -1154,12 +1215,19 @@ typedef struct Unique
typedef struct Gather
{
Plan plan;
- int num_workers; /* planned number of worker processes */
- int rescan_param; /* ID of Param that signals a rescan, or -1 */
- bool single_copy; /* don't execute plan more than once */
- bool invisible; /* suppress EXPLAIN display (for testing)? */
- Bitmapset *initParam; /* param id's of initplans which are referred
- * at gather or one of its child nodes */
+ /* planned number of worker processes */
+ int num_workers;
+ /* ID of Param that signals a rescan, or -1 */
+ int rescan_param;
+ /* don't execute plan more than once */
+ bool single_copy;
+ /* suppress EXPLAIN display (for testing)? */
+ bool invisible;
+ /*
+ * param id's of initplans which are referred at gather or one of its
+ * child nodes
+ */
+ Bitmapset *initParam;
} Gather;
/* ------------
@@ -1217,11 +1285,16 @@ typedef struct Hash
* needed to put them into the hashtable.
*/
List *hashkeys; /* hash keys for the hashjoin condition */
- Oid skewTable; /* outer join key's table OID, or InvalidOid */
- AttrNumber skewColumn; /* outer join key's column #, or zero */
- bool skewInherit; /* is outer join rel an inheritance tree? */
+ Oid skewTable; /* outer join key's
+ * table OID, or
+ * InvalidOid */
+ AttrNumber skewColumn; /* outer join key's
+ * column #, or zero */
+ bool skewInherit; /* is outer join rel an
+ * inheritance tree? */
/* all other info is in the parent HashJoin node */
- Cardinality rows_total; /* estimate total rows if parallel_aware */
+ Cardinality rows_total pg_node_attr(query_jumble_ignore); /* estimate total rows
+ * if parallel_aware */
} Hash;
/* ----------------
@@ -1252,7 +1325,7 @@ typedef struct SetOp
bool *cmpNullsFirst pg_node_attr(array_size(numCols));
/* estimated number of groups in left input */
- long numGroups;
+ long numGroups pg_node_attr(query_jumble_ignore);
} SetOp;
/* ----------------
@@ -1267,8 +1340,10 @@ typedef struct SetOp
typedef struct LockRows
{
Plan plan;
- List *rowMarks; /* a list of PlanRowMark's */
- int epqParam; /* ID of Param for EvalPlanQual re-eval */
+ List *rowMarks; /* a list of
+ * PlanRowMark's */
+ int epqParam; /* ID of Param for
+ * EvalPlanQual re-eval */
} LockRows;
/* ----------------
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
index 839e71d52f4..85702f75070 100644
--- a/src/include/nodes/primnodes.h
+++ b/src/include/nodes/primnodes.h
@@ -1075,8 +1075,6 @@ typedef struct SubLink
*/
typedef struct SubPlan
{
- pg_node_attr(no_query_jumble)
-
Expr xpr;
/* Fields copied from original SubLink: */
SubLinkType subLinkType; /* see above */
@@ -1107,8 +1105,9 @@ typedef struct SubPlan
List *parParam; /* indices of input Params from parent plan */
List *args; /* exprs to pass as parParam values */
/* Estimated execution costs: */
- Cost startup_cost; /* one-time setup cost */
- Cost per_call_cost; /* cost for each subplan evaluation */
+ Cost startup_cost pg_node_attr(query_jumble_ignore); /* one-time setup cost */
+ Cost per_call_cost pg_node_attr(query_jumble_ignore); /* cost for each subplan
+ * evaluation */
} SubPlan;
/*
diff --git a/src/include/nodes/queryjumble.h b/src/include/nodes/queryjumble.h
index 5afa6f3605f..6356dd73fbc 100644
--- a/src/include/nodes/queryjumble.h
+++ b/src/include/nodes/queryjumble.h
@@ -15,6 +15,7 @@
#define QUERYJUMBLE_H
#include "nodes/parsenodes.h"
+#include "nodes/pathnodes.h"
/*
* Struct for tracking locations/lengths of constants during normalization
@@ -59,15 +60,27 @@ enum ComputeQueryIdType
COMPUTE_QUERY_ID_REGRESS,
};
+/* Values for the compute_plan_id GUC */
+enum ComputePlanIdType
+{
+ COMPUTE_PLAN_ID_OFF,
+ COMPUTE_PLAN_ID_ON,
+ COMPUTE_PLAN_ID_AUTO,
+ COMPUTE_PLAN_ID_REGRESS,
+};
+
/* GUC parameters */
extern PGDLLIMPORT int compute_query_id;
+extern PGDLLIMPORT int compute_plan_id;
extern const char *CleanQuerytext(const char *query, int *location, int *len);
extern JumbleState *JumbleQuery(Query *query);
extern void EnableQueryId(void);
+extern void EnablePlanId(void);
extern PGDLLIMPORT bool query_id_enabled;
+extern PGDLLIMPORT bool plan_id_enabled;
/*
* Returns whether query identifier computation has been enabled, either
@@ -83,10 +96,25 @@ IsQueryIdEnabled(void)
return query_id_enabled;
}
-/* Functions intended for other users of jumbling (e.g. plan jumbling) */
+/*
+ * Returns whether plan identifier computation has been enabled, either
+ * directly in the GUC or by a module when the setting is 'auto'.
+ */
+static inline bool
+IsPlanIdEnabled(void)
+{
+ if (compute_plan_id == COMPUTE_PLAN_ID_OFF)
+ return false;
+ if (compute_plan_id == COMPUTE_PLAN_ID_ON)
+ return true;
+ return plan_id_enabled;
+}
+
+/* Functions called for plan jumbling or extensions doing their own jumbling */
extern JumbleState *InitializeJumbleState(bool record_clocations);
extern void AppendJumble(JumbleState *jstate, const unsigned char *item, Size size);
extern void JumbleNode(JumbleState *jstate, Node *node);
+extern void JumbleRangeTable(JumbleState *jstate, List *rtable);
extern uint64 HashJumbleState(JumbleState *jstate);
#endif /* QUERYJUMBLE_H */
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index d3d4ff6c5c9..437a4cec5bf 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -170,6 +170,9 @@ typedef struct PgBackendStatus
/* query identifier, optionally computed using post_parse_analyze_hook */
uint64 st_query_id;
+
+ /* plan identifier, optionally computed after planning */
+ uint64 st_plan_id;
} PgBackendStatus;
@@ -316,6 +319,7 @@ extern void pgstat_clear_backend_activity_snapshot(void);
/* Activity reporting functions */
extern void pgstat_report_activity(BackendState state, const char *cmd_str);
extern void pgstat_report_query_id(uint64 query_id, bool force);
+extern void pgstat_report_plan_id(uint64 query_id, uint64 plan_id, bool force);
extern void pgstat_report_tempfile(size_t filesize);
extern void pgstat_report_appname(const char *appname);
extern void pgstat_report_xact_timestamp(TimestampTz tstamp);
@@ -323,6 +327,7 @@ extern const char *pgstat_get_backend_current_activity(int pid, bool checkUser);
extern const char *pgstat_get_crashed_backend_activity(int pid, char *buffer,
int buflen);
extern uint64 pgstat_get_my_query_id(void);
+extern uint64 pgstat_get_my_plan_id(void);
extern BackendType pgstat_get_backend_type_by_proc_number(ProcNumber procNumber);
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index ee31e41d508..8bfa3c1a5f9 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -699,6 +699,17 @@ select explain_filter('explain (verbose) create table test_ctas as select 1');
Query Identifier: N
(3 rows)
+-- Test compute_plan_id
+set compute_plan_id = on;
+select explain_filter('explain (verbose) select * from int8_tbl i8');
+ explain_filter
+----------------------------------------------------------------
+ Seq Scan on public.int8_tbl i8 (cost=N.N..N.N rows=N width=N)
+ Output: q1, q2
+ Query Identifier: N
+ Plan Identifier: N
+(4 rows)
+
-- Test SERIALIZE option
select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
explain_filter
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 3361f6a69c9..6778b79e658 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1760,9 +1760,10 @@ pg_stat_activity| SELECT s.datid,
s.backend_xid,
s.backend_xmin,
s.query_id,
+ s.plan_id,
s.query,
s.backend_type
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
LEFT JOIN pg_database d ON ((s.datid = d.oid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_all_indexes| SELECT c.oid AS relid,
@@ -1890,7 +1891,7 @@ pg_stat_gssapi| SELECT pid,
gss_princ AS principal,
gss_enc AS encrypted,
gss_delegation AS credentials_delegated
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
WHERE (client_port IS NOT NULL);
pg_stat_io| SELECT backend_type,
object,
@@ -2096,7 +2097,7 @@ pg_stat_replication| SELECT s.pid,
w.sync_priority,
w.sync_state,
w.reply_time
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
JOIN pg_stat_get_wal_senders() w(pid, state, sent_lsn, write_lsn, flush_lsn, replay_lsn, write_lag, flush_lag, replay_lag, sync_priority, sync_state, reply_time) ON ((s.pid = w.pid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_replication_slots| SELECT s.slot_name,
@@ -2130,7 +2131,7 @@ pg_stat_ssl| SELECT pid,
ssl_client_dn AS client_dn,
ssl_client_serial AS client_serial,
ssl_issuer_dn AS issuer_dn
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
WHERE (client_port IS NOT NULL);
pg_stat_subscription| SELECT su.oid AS subid,
su.subname,
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index 0bafa870496..d787ad2cda3 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -167,6 +167,10 @@ select explain_filter('explain (verbose) select * from int8_tbl i8');
select explain_filter('explain (verbose) declare test_cur cursor for select * from int8_tbl');
select explain_filter('explain (verbose) create table test_ctas as select 1');
+-- Test compute_plan_id
+set compute_plan_id = on;
+select explain_filter('explain (verbose) select * from int8_tbl i8');
+
-- Test SERIALIZE option
select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
select explain_filter('explain (analyze,serialize text,buffers,timing off) select * from int8_tbl i8');
--
2.47.1
v4-0003-Add-pg_stat_plans-contrib-extension.patchapplication/octet-stream; name=v4-0003-Add-pg_stat_plans-contrib-extension.patchDownload
From 2c39ebd625e20784daa0e15541d5a417142139e7 Mon Sep 17 00:00:00 2001
From: Michael Paquier <michael@paquier.xyz>
Date: Fri, 31 Jan 2025 13:08:19 +0900
Subject: [PATCH v4 3/3] Add pg_stat_plans contrib extension
This extension allows tracking per-plan call counts and execution time,
as well as capturing the plan text, aka EXPLAIN (COSTS OFF), for the
first execution of a given plan. This utilize the compute_plan_id
functionality for tracking different plans.
---
contrib/Makefile | 1 +
contrib/meson.build | 1 +
contrib/pg_stat_plans/Makefile | 29 +
contrib/pg_stat_plans/expected/cleanup.out | 1 +
contrib/pg_stat_plans/expected/privileges.out | 125 +++
contrib/pg_stat_plans/expected/select.out | 262 ++++++
contrib/pg_stat_plans/meson.build | 43 +
contrib/pg_stat_plans/pg_stat_plans--1.0.sql | 33 +
contrib/pg_stat_plans/pg_stat_plans.c | 779 ++++++++++++++++++
contrib/pg_stat_plans/pg_stat_plans.conf | 1 +
contrib/pg_stat_plans/pg_stat_plans.control | 5 +
contrib/pg_stat_plans/sql/cleanup.sql | 1 +
contrib/pg_stat_plans/sql/privileges.sql | 59 ++
contrib/pg_stat_plans/sql/select.sql | 67 ++
doc/src/sgml/contrib.sgml | 1 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/pgstatplans.sgml | 425 ++++++++++
17 files changed, 1834 insertions(+)
create mode 100644 contrib/pg_stat_plans/Makefile
create mode 100644 contrib/pg_stat_plans/expected/cleanup.out
create mode 100644 contrib/pg_stat_plans/expected/privileges.out
create mode 100644 contrib/pg_stat_plans/expected/select.out
create mode 100644 contrib/pg_stat_plans/meson.build
create mode 100644 contrib/pg_stat_plans/pg_stat_plans--1.0.sql
create mode 100644 contrib/pg_stat_plans/pg_stat_plans.c
create mode 100644 contrib/pg_stat_plans/pg_stat_plans.conf
create mode 100644 contrib/pg_stat_plans/pg_stat_plans.control
create mode 100644 contrib/pg_stat_plans/sql/cleanup.sql
create mode 100644 contrib/pg_stat_plans/sql/privileges.sql
create mode 100644 contrib/pg_stat_plans/sql/select.sql
create mode 100644 doc/src/sgml/pgstatplans.sgml
diff --git a/contrib/Makefile b/contrib/Makefile
index 952855d9b61..8de010afdef 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -34,6 +34,7 @@ SUBDIRS = \
pg_freespacemap \
pg_logicalinspect \
pg_prewarm \
+ pg_stat_plans \
pg_stat_statements \
pg_surgery \
pg_trgm \
diff --git a/contrib/meson.build b/contrib/meson.build
index 1ba73ebd67a..0442ec2644a 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -49,6 +49,7 @@ subdir('pg_freespacemap')
subdir('pg_logicalinspect')
subdir('pg_prewarm')
subdir('pgrowlocks')
+subdir('pg_stat_plans')
subdir('pg_stat_statements')
subdir('pgstattuple')
subdir('pg_surgery')
diff --git a/contrib/pg_stat_plans/Makefile b/contrib/pg_stat_plans/Makefile
new file mode 100644
index 00000000000..e073db95edd
--- /dev/null
+++ b/contrib/pg_stat_plans/Makefile
@@ -0,0 +1,29 @@
+# contrib/pg_stat_plans/Makefile
+
+MODULE_big = pg_stat_plans
+OBJS = \
+ $(WIN32RES) \
+ pg_stat_plans.o
+
+EXTENSION = pg_stat_plans
+DATA = pg_stat_plans--1.0.sql
+PGFILEDESC = "pg_stat_plans - track per-plan call counts, execution times and EXPLAIN texts"
+
+LDFLAGS_SL += $(filter -lm, $(LIBS))
+
+REGRESS_OPTS = --temp-config $(top_srcdir)/contrib/pg_stat_plans/pg_stat_plans.conf
+REGRESS = select privileges cleanup
+# Disabled because these tests require "shared_preload_libraries=pg_stat_plans",
+# which typical installcheck users do not have (e.g. buildfarm clients).
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/pg_stat_plans
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/pg_stat_plans/expected/cleanup.out b/contrib/pg_stat_plans/expected/cleanup.out
new file mode 100644
index 00000000000..51565617cef
--- /dev/null
+++ b/contrib/pg_stat_plans/expected/cleanup.out
@@ -0,0 +1 @@
+DROP EXTENSION pg_stat_plans;
diff --git a/contrib/pg_stat_plans/expected/privileges.out b/contrib/pg_stat_plans/expected/privileges.out
new file mode 100644
index 00000000000..3e21d6d7019
--- /dev/null
+++ b/contrib/pg_stat_plans/expected/privileges.out
@@ -0,0 +1,125 @@
+--
+-- Only superusers and roles with privileges of the pg_read_all_stats role
+-- are allowed to see the plan text, queryid and planid of queries executed by
+-- other users. Other users can see the statistics.
+--
+CREATE ROLE regress_stats_superuser SUPERUSER;
+CREATE ROLE regress_stats_user1;
+CREATE ROLE regress_stats_user2;
+GRANT pg_read_all_stats TO regress_stats_user2;
+SET ROLE regress_stats_superuser;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
+SELECT 1 AS "ONE";
+ ONE
+-----
+ 1
+(1 row)
+
+SET ROLE regress_stats_user1;
+SELECT 1+1 AS "TWO";
+ TWO
+-----
+ 2
+(1 row)
+
+--
+-- A superuser can read all columns of queries executed by others,
+-- including plan text, queryid and planid.
+--
+SET ROLE regress_stats_superuser;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+ rolname | queryid_bool | planid_bool | plan | calls
+-------------------------+--------------+-------------+------------------------------------------------------------------------------------+-------
+ regress_stats_superuser | t | t | Result | 1
+ regress_stats_superuser | t | t | Result | 1
+ regress_stats_superuser | t | t | Sort +| 0
+ | | | Sort Key: pg_authid.rolname, pg_stat_plans.plan COLLATE "C", pg_stat_plans.calls+|
+ | | | -> Hash Join +|
+ | | | Hash Cond: (pg_stat_plans.userid = pg_authid.oid) +|
+ | | | -> Function Scan on pg_stat_plans +|
+ | | | -> Hash +|
+ | | | -> Seq Scan on pg_authid |
+ regress_stats_user1 | t | t | Result | 1
+(4 rows)
+
+--
+-- regress_stats_user1 has no privileges to read the plan text, queryid
+-- or planid of queries executed by others but can see statistics
+-- like calls and rows.
+--
+SET ROLE regress_stats_user1;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+ rolname | queryid_bool | planid_bool | plan | calls
+-------------------------+--------------+-------------+------------------------------------------------------------------------------------+-------
+ regress_stats_superuser | | | <insufficient privilege> | 1
+ regress_stats_superuser | | | <insufficient privilege> | 1
+ regress_stats_superuser | | | <insufficient privilege> | 1
+ regress_stats_user1 | t | t | Result | 1
+ regress_stats_user1 | t | t | Sort +| 0
+ | | | Sort Key: pg_authid.rolname, pg_stat_plans.plan COLLATE "C", pg_stat_plans.calls+|
+ | | | -> Hash Join +|
+ | | | Hash Cond: (pg_stat_plans.userid = pg_authid.oid) +|
+ | | | -> Function Scan on pg_stat_plans +|
+ | | | -> Hash +|
+ | | | -> Seq Scan on pg_authid |
+(5 rows)
+
+--
+-- regress_stats_user2, with pg_read_all_stats role privileges, can
+-- read all columns, including plan text, queryid and planid, of queries
+-- executed by others.
+--
+SET ROLE regress_stats_user2;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+ rolname | queryid_bool | planid_bool | plan | calls
+-------------------------+--------------+-------------+------------------------------------------------------------------------------------+-------
+ regress_stats_superuser | t | t | Result | 1
+ regress_stats_superuser | t | t | Result | 1
+ regress_stats_superuser | t | t | Sort +| 1
+ | | | Sort Key: pg_authid.rolname, pg_stat_plans.plan COLLATE "C", pg_stat_plans.calls+|
+ | | | -> Hash Join +|
+ | | | Hash Cond: (pg_stat_plans.userid = pg_authid.oid) +|
+ | | | -> Function Scan on pg_stat_plans +|
+ | | | -> Hash +|
+ | | | -> Seq Scan on pg_authid |
+ regress_stats_user1 | t | t | Result | 1
+ regress_stats_user1 | t | t | Sort +| 1
+ | | | Sort Key: pg_authid.rolname, pg_stat_plans.plan COLLATE "C", pg_stat_plans.calls+|
+ | | | -> Hash Join +|
+ | | | Hash Cond: (pg_stat_plans.userid = pg_authid.oid) +|
+ | | | -> Function Scan on pg_stat_plans +|
+ | | | -> Hash +|
+ | | | -> Seq Scan on pg_authid |
+ regress_stats_user2 | t | t | Sort +| 0
+ | | | Sort Key: pg_authid.rolname, pg_stat_plans.plan COLLATE "C", pg_stat_plans.calls+|
+ | | | -> Hash Join +|
+ | | | Hash Cond: (pg_stat_plans.userid = pg_authid.oid) +|
+ | | | -> Function Scan on pg_stat_plans +|
+ | | | -> Hash +|
+ | | | -> Seq Scan on pg_authid |
+(6 rows)
+
+--
+-- cleanup
+--
+RESET ROLE;
+DROP ROLE regress_stats_superuser;
+DROP ROLE regress_stats_user1;
+DROP ROLE regress_stats_user2;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
diff --git a/contrib/pg_stat_plans/expected/select.out b/contrib/pg_stat_plans/expected/select.out
new file mode 100644
index 00000000000..906d8ce90d6
--- /dev/null
+++ b/contrib/pg_stat_plans/expected/select.out
@@ -0,0 +1,262 @@
+--
+-- SELECT statements
+--
+CREATE EXTENSION pg_stat_plans;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
+--
+-- simple statements
+--
+SELECT 1 FROM pg_class LIMIT 1;
+ ?column?
+----------
+ 1
+(1 row)
+
+SELECT 1 FROM pg_class WHERE relname = 'pg_class';
+ ?column?
+----------
+ 1
+(1 row)
+
+SET enable_indexscan = off;
+SELECT 1 FROM pg_class WHERE relname = 'pg_class';
+ ?column?
+----------
+ 1
+(1 row)
+
+SET enable_indexscan = on;
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+ plan | calls
+--------------------------------------------------------------+-------
+ Bitmap Heap Scan on pg_class +| 1
+ Recheck Cond: (relname = 'pg_class'::name) +|
+ -> Bitmap Index Scan on pg_class_relname_nsp_index +|
+ Index Cond: (relname = 'pg_class'::name) |
+ Index Only Scan using pg_class_relname_nsp_index on pg_class+| 1
+ Index Cond: (relname = 'pg_class'::name) |
+ Limit +| 1
+ -> Seq Scan on pg_class |
+ Result | 1
+ Sort +| 0
+ Sort Key: pg_stat_plans.plan COLLATE "C" +|
+ -> Function Scan on pg_stat_plans |
+(5 rows)
+
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
+--
+-- subplans and CTEs
+--
+WITH x AS MATERIALIZED (SELECT 1)
+SELECT * FROM x;
+ ?column?
+----------
+ 1
+(1 row)
+
+SELECT a.attname,
+ (SELECT pg_catalog.pg_get_expr(d.adbin, d.adrelid)
+ FROM pg_catalog.pg_attrdef d
+ WHERE d.adrelid = a.attrelid AND d.adnum = a.attnum AND a.atthasdef)
+ FROM pg_catalog.pg_attribute a
+ WHERE a.attrelid = 'pg_class'::regclass
+ ORDER BY attnum LIMIT 1;
+ attname | pg_get_expr
+----------+-------------
+ tableoid |
+(1 row)
+
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+ plan | calls
+-------------------------------------------------------------------------------+-------
+ CTE Scan on x +| 1
+ CTE x +|
+ -> Result |
+ Limit +| 1
+ -> Index Scan using pg_attribute_relid_attnum_index on pg_attribute a +|
+ Index Cond: (attrelid = '1259'::oid) +|
+ SubPlan 1 +|
+ -> Result +|
+ One-Time Filter: a.atthasdef +|
+ -> Seq Scan on pg_attrdef d +|
+ Filter: ((adrelid = a.attrelid) AND (adnum = a.attnum)) |
+ Result | 1
+ Sort +| 0
+ Sort Key: pg_stat_plans.plan COLLATE "C" +|
+ -> Function Scan on pg_stat_plans |
+(4 rows)
+
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
+--
+-- partitoning
+--
+create table lp (a char) partition by list (a);
+create table lp_default partition of lp default;
+create table lp_ef partition of lp for values in ('e', 'f');
+create table lp_ad partition of lp for values in ('a', 'd');
+create table lp_bc partition of lp for values in ('b', 'c');
+create table lp_g partition of lp for values in ('g');
+create table lp_null partition of lp for values in (null);
+select * from lp;
+ a
+---
+(0 rows)
+
+select * from lp where a > 'a' and a < 'd';
+ a
+---
+(0 rows)
+
+select * from lp where a > 'a' and a <= 'd';
+ a
+---
+(0 rows)
+
+select * from lp where a = 'a';
+ a
+---
+(0 rows)
+
+select * from lp where 'a' = a; /* commuted */
+ a
+---
+(0 rows)
+
+select * from lp where a is not null;
+ a
+---
+(0 rows)
+
+select * from lp where a is null;
+ a
+---
+(0 rows)
+
+select * from lp where a = 'a' or a = 'c';
+ a
+---
+(0 rows)
+
+select * from lp where a is not null and (a = 'a' or a = 'c');
+ a
+---
+(0 rows)
+
+select * from lp where a <> 'g';
+ a
+---
+(0 rows)
+
+select * from lp where a <> 'a' and a <> 'd';
+ a
+---
+(0 rows)
+
+select * from lp where a not in ('a', 'd');
+ a
+---
+(0 rows)
+
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+ plan | calls
+--------------------------------------------------------------------------------+-------
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ Filter: ((a = 'a'::bpchar) OR (a = 'c'::bpchar)) +|
+ -> Seq Scan on lp_bc lp_2 +|
+ Filter: ((a = 'a'::bpchar) OR (a = 'c'::bpchar)) |
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ Filter: ((a > 'a'::bpchar) AND (a <= 'd'::bpchar)) +|
+ -> Seq Scan on lp_bc lp_2 +|
+ Filter: ((a > 'a'::bpchar) AND (a <= 'd'::bpchar)) +|
+ -> Seq Scan on lp_default lp_3 +|
+ Filter: ((a > 'a'::bpchar) AND (a <= 'd'::bpchar)) |
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ Filter: ((a IS NOT NULL) AND ((a = 'a'::bpchar) OR (a = 'c'::bpchar)))+|
+ -> Seq Scan on lp_bc lp_2 +|
+ Filter: ((a IS NOT NULL) AND ((a = 'a'::bpchar) OR (a = 'c'::bpchar))) |
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ Filter: (a <> 'g'::bpchar) +|
+ -> Seq Scan on lp_bc lp_2 +|
+ Filter: (a <> 'g'::bpchar) +|
+ -> Seq Scan on lp_ef lp_3 +|
+ Filter: (a <> 'g'::bpchar) +|
+ -> Seq Scan on lp_default lp_4 +|
+ Filter: (a <> 'g'::bpchar) |
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ Filter: (a IS NOT NULL) +|
+ -> Seq Scan on lp_bc lp_2 +|
+ Filter: (a IS NOT NULL) +|
+ -> Seq Scan on lp_ef lp_3 +|
+ Filter: (a IS NOT NULL) +|
+ -> Seq Scan on lp_g lp_4 +|
+ Filter: (a IS NOT NULL) +|
+ -> Seq Scan on lp_default lp_5 +|
+ Filter: (a IS NOT NULL) |
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ -> Seq Scan on lp_bc lp_2 +|
+ -> Seq Scan on lp_ef lp_3 +|
+ -> Seq Scan on lp_g lp_4 +|
+ -> Seq Scan on lp_null lp_5 +|
+ -> Seq Scan on lp_default lp_6 |
+ Append +| 1
+ -> Seq Scan on lp_bc lp_1 +|
+ Filter: ((a <> 'a'::bpchar) AND (a <> 'd'::bpchar)) +|
+ -> Seq Scan on lp_ef lp_2 +|
+ Filter: ((a <> 'a'::bpchar) AND (a <> 'd'::bpchar)) +|
+ -> Seq Scan on lp_g lp_3 +|
+ Filter: ((a <> 'a'::bpchar) AND (a <> 'd'::bpchar)) +|
+ -> Seq Scan on lp_default lp_4 +|
+ Filter: ((a <> 'a'::bpchar) AND (a <> 'd'::bpchar)) |
+ Append +| 1
+ -> Seq Scan on lp_bc lp_1 +|
+ Filter: ((a > 'a'::bpchar) AND (a < 'd'::bpchar)) +|
+ -> Seq Scan on lp_default lp_2 +|
+ Filter: ((a > 'a'::bpchar) AND (a < 'd'::bpchar)) |
+ Append +| 1
+ -> Seq Scan on lp_bc lp_1 +|
+ Filter: (a <> ALL ('{a,d}'::bpchar[])) +|
+ -> Seq Scan on lp_ef lp_2 +|
+ Filter: (a <> ALL ('{a,d}'::bpchar[])) +|
+ -> Seq Scan on lp_g lp_3 +|
+ Filter: (a <> ALL ('{a,d}'::bpchar[])) +|
+ -> Seq Scan on lp_default lp_4 +|
+ Filter: (a <> ALL ('{a,d}'::bpchar[])) |
+ Result | 1
+ Seq Scan on lp_ad lp +| 1
+ Filter: ('a'::bpchar = a) |
+ Seq Scan on lp_ad lp +| 1
+ Filter: (a = 'a'::bpchar) |
+ Seq Scan on lp_null lp +| 1
+ Filter: (a IS NULL) |
+ Sort +| 0
+ Sort Key: pg_stat_plans.plan COLLATE "C" +|
+ -> Function Scan on pg_stat_plans |
+(14 rows)
+
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
diff --git a/contrib/pg_stat_plans/meson.build b/contrib/pg_stat_plans/meson.build
new file mode 100644
index 00000000000..3bd884d9601
--- /dev/null
+++ b/contrib/pg_stat_plans/meson.build
@@ -0,0 +1,43 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+pg_stat_plans_sources = files(
+ 'pg_stat_plans.c',
+)
+
+if host_system == 'windows'
+ pg_stat_plans_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'pg_stat_plans',
+ '--FILEDESC', 'pg_stat_plans - track per-plan call counts, execution times and EXPLAIN texts',])
+endif
+
+pg_stat_plans = shared_module('pg_stat_plans',
+ pg_stat_plans_sources,
+ kwargs: contrib_mod_args + {
+ 'dependencies': contrib_mod_args['dependencies'],
+ },
+)
+contrib_targets += pg_stat_plans
+
+install_data(
+ 'pg_stat_plans.control',
+ 'pg_stat_plans--1.0.sql',
+ kwargs: contrib_data_args,
+)
+
+tests += {
+ 'name': 'pg_stat_plans',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'regress': {
+ 'sql': [
+ 'select',
+ 'privileges',
+ 'cleanup',
+ ],
+ 'regress_args': ['--temp-config', files('pg_stat_plans.conf')],
+ # Disabled because these tests require
+ # "shared_preload_libraries=pg_stat_plans", which typical
+ # runningcheck users do not have (e.g. buildfarm clients).
+ 'runningcheck': false,
+ }
+}
diff --git a/contrib/pg_stat_plans/pg_stat_plans--1.0.sql b/contrib/pg_stat_plans/pg_stat_plans--1.0.sql
new file mode 100644
index 00000000000..f08452b274b
--- /dev/null
+++ b/contrib/pg_stat_plans/pg_stat_plans--1.0.sql
@@ -0,0 +1,33 @@
+/* contrib/pg_stat_plans/pg_stat_plans--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_stat_plans" to load this file. \quit
+
+-- Register functions.
+CREATE FUNCTION pg_stat_plans_reset()
+RETURNS void
+AS 'MODULE_PATHNAME'
+LANGUAGE C PARALLEL SAFE;
+
+CREATE FUNCTION pg_stat_plans(IN showplan boolean,
+ OUT userid oid,
+ OUT dbid oid,
+ OUT toplevel bool,
+ OUT queryid bigint,
+ OUT planid bigint,
+ OUT calls int8,
+ OUT total_exec_time float8,
+ OUT plan text
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stat_plans_1_0'
+LANGUAGE C STRICT VOLATILE PARALLEL SAFE;
+
+-- Register a view on the function for ease of use.
+CREATE VIEW pg_stat_plans AS
+ SELECT * FROM pg_stat_plans(true);
+
+GRANT SELECT ON pg_stat_plans TO PUBLIC;
+
+-- Don't want this to be available to non-superusers.
+REVOKE ALL ON FUNCTION pg_stat_plans_reset() FROM PUBLIC;
diff --git a/contrib/pg_stat_plans/pg_stat_plans.c b/contrib/pg_stat_plans/pg_stat_plans.c
new file mode 100644
index 00000000000..5fa31fcbf3e
--- /dev/null
+++ b/contrib/pg_stat_plans/pg_stat_plans.c
@@ -0,0 +1,779 @@
+/*--------------------------------------------------------------------------
+ *
+ * pg_stat_plans.c
+ * Track per-plan call counts, execution times and EXPLAIN texts
+ * across a whole database cluster.
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * contrib/pg_stat_plans/pg_stat_plans.c
+ *
+ * -------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/parallel.h"
+#include "catalog/pg_authid.h"
+#include "commands/explain.h"
+#include "common/hashfn.h"
+#include "funcapi.h"
+#include "mb/pg_wchar.h"
+#include "nodes/queryjumble.h"
+#include "pgstat.h"
+#include "optimizer/planner.h"
+#include "utils/acl.h"
+#include "utils/builtins.h"
+#include "utils/guc.h"
+#include "utils/pgstat_internal.h"
+#include "utils/snapmgr.h"
+
+PG_MODULE_MAGIC;
+
+/* Current nesting depth of planner/ExecutorRun/ProcessUtility calls */
+static int nesting_level = 0;
+
+/* Saved hook values */
+static planner_hook_type prev_planner_hook = NULL;
+static ExecutorStart_hook_type prev_ExecutorStart = NULL;
+static ExecutorRun_hook_type prev_ExecutorRun = NULL;
+static ExecutorFinish_hook_type prev_ExecutorFinish = NULL;
+static ExecutorEnd_hook_type prev_ExecutorEnd = NULL;
+
+/*---- GUC variables ----*/
+
+typedef enum
+{
+ PGSP_TRACK_NONE, /* track no plans */
+ PGSP_TRACK_TOP, /* only plans for top level statements */
+ PGSP_TRACK_ALL, /* all plans, including for nested statements */
+} PGSPTrackLevel;
+
+static const struct config_enum_entry track_options[] =
+{
+ {"none", PGSP_TRACK_NONE, false},
+ {"top", PGSP_TRACK_TOP, false},
+ {"all", PGSP_TRACK_ALL, false},
+ {NULL, 0, false}
+};
+
+static int pgsp_max = 5000; /* max # plans to track */
+static int pgsp_max_size = 2048; /* max size of plan text to track (in
+ * bytes) */
+static int pgsp_track = PGSP_TRACK_TOP; /* tracking level */
+
+#define pgsp_enabled(level) \
+ (!IsParallelWorker() && \
+ (compute_plan_id != COMPUTE_PLAN_ID_OFF) && \
+ (pgsp_track == PGSP_TRACK_ALL || \
+ (pgsp_track == PGSP_TRACK_TOP && (level) == 0)))
+
+#define USAGE_INCREASE 0.5 /* increase by this each time we report
+ * stats */
+#define USAGE_DECREASE_FACTOR (0.99) /* decreased every
+ * pgstat_dealloc_plans */
+#define USAGE_DEALLOC_PERCENT 5 /* free this % of entries at once */
+
+/*---- Function declarations ----*/
+
+PG_FUNCTION_INFO_V1(pg_stat_plans_reset);
+PG_FUNCTION_INFO_V1(pg_stat_plans_1_0);
+
+/* Structures for statistics of plans */
+typedef struct PgStatShared_PlanInfo
+{
+ /* key elements that identify a plan (together with the dboid) */
+ uint64 planid;
+ uint64 queryid;
+ Oid userid; /* userid is tracked to allow users to see
+ * their own query plans */
+ bool toplevel; /* query executed at top level */
+
+ dsa_pointer plan_text; /* pointer to DSA memory containing plan text */
+ int plan_encoding; /* plan text encoding */
+} PgStatShared_PlanInfo;
+
+typedef struct PgStat_StatPlanEntry
+{
+ PgStat_Counter exec_count;
+ double exec_time;
+ double usage; /* Usage factor of the entry, used to
+ * prioritize which plans to age out */
+
+ /* Only used in shared structure, not in local pending stats */
+ PgStatShared_PlanInfo info;
+} PgStat_StatPlanEntry;
+
+typedef struct PgStatShared_Plan
+{
+ PgStatShared_Common header;
+ PgStat_StatPlanEntry stats;
+} PgStatShared_Plan;
+
+static bool plan_stats_flush_cb(PgStat_EntryRef *entry_ref, bool nowait);
+
+static const PgStat_KindInfo plan_stats = {
+ .name = "plan_stats",
+ .fixed_amount = false,
+
+ /*
+ * We currently don't write to a file since plan texts would get lost (and
+ * just the stats on their own aren't that useful)
+ */
+ .write_to_file = false,
+
+ /*
+ * Plan statistics are available system-wide to simplify monitoring
+ * scripts
+ */
+ .accessed_across_databases = true,
+
+ .shared_size = sizeof(PgStatShared_Plan),
+ .shared_data_off = offsetof(PgStatShared_Plan, stats),
+ .shared_data_len = sizeof(((PgStatShared_Plan *) 0)->stats),
+ .pending_size = sizeof(PgStat_StatPlanEntry),
+ .flush_pending_cb = plan_stats_flush_cb,
+};
+
+/*
+ * Compute stats entry idx from query ID and plan ID with an 8-byte hash.
+ *
+ * Whilst we could theorically just use the plan ID here, we intentionally
+ * add the query ID into the mix to ease interpreting the data in combination
+ * with pg_stat_statements.
+ */
+#define PGSTAT_PLAN_IDX(query_id, plan_id, user_id, toplevel) hash_combine64(toplevel, hash_combine64(query_id, hash_combine64(plan_id, user_id)))
+
+/*
+ * Kind ID reserved for statistics of plans.
+ */
+#define PGSTAT_KIND_PLANS PGSTAT_KIND_EXPERIMENTAL /* TODO: Assign */
+
+/*
+ * Callback for stats handling
+ */
+static bool
+plan_stats_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
+{
+ PgStat_StatPlanEntry *localent;
+ PgStatShared_Plan *shfuncent;
+
+ localent = (PgStat_StatPlanEntry *) entry_ref->pending;
+ shfuncent = (PgStatShared_Plan *) entry_ref->shared_stats;
+
+ if (!pgstat_lock_entry(entry_ref, nowait))
+ return false;
+
+ shfuncent->stats.exec_count += localent->exec_count;
+ shfuncent->stats.exec_time += localent->exec_time;
+ shfuncent->stats.usage += localent->usage;
+
+ pgstat_unlock_entry(entry_ref);
+
+ return true;
+}
+
+static char *
+pgsp_explain_plan(QueryDesc *queryDesc)
+{
+ ExplainState *es;
+ StringInfo es_str;
+
+ es = NewExplainState();
+ es_str = es->str;
+
+ /*
+ * We turn off COSTS since identical planids may have very different
+ * costs, and it could be misleading to only show the first recorded
+ * plan's costs.
+ */
+ es->costs = false;
+ es->format = EXPLAIN_FORMAT_TEXT;
+
+ ExplainBeginOutput(es);
+ ExplainPrintPlan(es, queryDesc);
+ ExplainEndOutput(es);
+
+ return es_str->data;
+}
+
+static void
+pgstat_gc_plan_memory()
+{
+ dshash_seq_status hstat;
+ PgStatShared_HashEntry *p;
+
+ /* dshash entry is not modified, take shared lock */
+ dshash_seq_init(&hstat, pgStatLocal.shared_hash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ PgStatShared_Common *header;
+ PgStat_StatPlanEntry *statent;
+
+ if (!p->dropped || p->key.kind != PGSTAT_KIND_PLANS)
+ continue;
+
+ header = dsa_get_address(pgStatLocal.dsa, p->body);
+
+ if (!LWLockConditionalAcquire(&header->lock, LW_EXCLUSIVE))
+ continue;
+
+ statent = (PgStat_StatPlanEntry *) pgstat_get_entry_data(PGSTAT_KIND_PLANS, header);
+
+ /*
+ * Clean up this entry's plan text allocation, if we haven't done so
+ * already
+ */
+ if (DsaPointerIsValid(statent->info.plan_text))
+ {
+ dsa_free(pgStatLocal.dsa, statent->info.plan_text);
+ statent->info.plan_text = InvalidDsaPointer;
+
+ /* Allow removal of the shared stats entry */
+ pg_atomic_fetch_sub_u32(&p->refcount, 1);
+ }
+
+ LWLockRelease(&header->lock);
+ }
+ dshash_seq_term(&hstat);
+
+ /* Encourage other backends to clean up dropped entry refs */
+ pgstat_request_entry_refs_gc();
+}
+
+typedef struct PlanDeallocEntry
+{
+ PgStat_HashKey key;
+ double usage;
+} PlanDeallocEntry;
+
+/*
+ * list sort comparator for sorting into decreasing usage order
+ */
+static int
+entry_cmp_lru(const union ListCell *lhs, const union ListCell *rhs)
+{
+ double l_usage = ((PlanDeallocEntry *) lfirst(lhs))->usage;
+ double r_usage = ((PlanDeallocEntry *) lfirst(rhs))->usage;
+
+ if (l_usage > r_usage)
+ return -1;
+ else if (l_usage < r_usage)
+ return +1;
+ else
+ return 0;
+}
+
+static void
+pgstat_dealloc_plans()
+{
+ dshash_seq_status hstat;
+ PgStatShared_HashEntry *p;
+ List *entries = NIL;
+ ListCell *lc;
+ int nvictims;
+
+ /* dshash entry is not modified, take shared lock */
+ dshash_seq_init(&hstat, pgStatLocal.shared_hash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ PgStatShared_Common *header;
+ PgStat_StatPlanEntry *statent;
+ PlanDeallocEntry *entry;
+
+ if (p->dropped || p->key.kind != PGSTAT_KIND_PLANS)
+ continue;
+
+ header = dsa_get_address(pgStatLocal.dsa, p->body);
+
+ if (!LWLockConditionalAcquire(&header->lock, LW_EXCLUSIVE))
+ continue;
+
+ statent = (PgStat_StatPlanEntry *) pgstat_get_entry_data(PGSTAT_KIND_PLANS, header);
+ statent->usage *= USAGE_DECREASE_FACTOR;
+
+ entry = palloc(sizeof(PlanDeallocEntry));
+ entry->key = p->key;
+ entry->usage = statent->usage;
+
+ LWLockRelease(&header->lock);
+
+ entries = lappend(entries, entry);
+ }
+ dshash_seq_term(&hstat);
+
+ /* Sort by usage ascending (lowest used entries are last) */
+ list_sort(entries, entry_cmp_lru);
+
+ /* At a minimum, deallocate 10 entries to make it worth our while */
+ nvictims = Max(10, list_length(entries) * USAGE_DEALLOC_PERCENT / 100);
+ nvictims = Min(nvictims, list_length(entries));
+
+ /* Actually drop the entries */
+ for_each_from(lc, entries, list_length(entries) - nvictims)
+ {
+ PlanDeallocEntry *entry = lfirst(lc);
+
+ pgstat_drop_entry(entry->key.kind, entry->key.dboid, entry->key.objid);
+ }
+
+ /* Clean up our working memory immediately */
+ foreach(lc, entries)
+ {
+ PlanDeallocEntry *entry = lfirst(lc);
+
+ pfree(entry);
+ }
+ pfree(entries);
+}
+
+static void
+pgstat_gc_plans()
+{
+ dshash_seq_status hstat;
+ PgStatShared_HashEntry *p;
+ bool have_dropped_entries = false;
+ size_t plan_entry_count = 0;
+
+ /* TODO: Prevent concurrent GC cycles - flag an active GC run somehow */
+
+ /*
+ * Count our active entries, and whether there are any dropped entries we
+ * may need to clean up at the end.
+ */
+ dshash_seq_init(&hstat, pgStatLocal.shared_hash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ if (p->key.kind != PGSTAT_KIND_PLANS)
+ continue;
+
+ if (p->dropped)
+ have_dropped_entries = true;
+ else
+ plan_entry_count++;
+ }
+ dshash_seq_term(&hstat);
+
+ /*
+ * If we're over the limit, delete entries with lowest usage factor.
+ */
+ if (plan_entry_count > pgsp_max)
+ {
+ pgstat_dealloc_plans();
+ have_dropped_entries = true; /* Assume we did some work */
+ }
+
+ /* If there are dropped entries, clean up their plan memory if needed */
+ if (have_dropped_entries)
+ pgstat_gc_plan_memory();
+}
+
+static void
+pgstat_report_plan_stats(QueryDesc *queryDesc,
+ PgStat_Counter exec_count,
+ double exec_time)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_Plan *shstatent;
+ PgStat_StatPlanEntry *statent;
+ bool newly_created;
+ uint64 queryId = queryDesc->plannedstmt->queryId;
+ uint64 planId = queryDesc->plannedstmt->planId;
+ Oid userid = GetUserId();
+ bool toplevel = (nesting_level == 0);
+
+ entry_ref = pgstat_prep_pending_entry(PGSTAT_KIND_PLANS, MyDatabaseId,
+ PGSTAT_PLAN_IDX(queryId, planId, userid, toplevel), &newly_created);
+
+ shstatent = (PgStatShared_Plan *) entry_ref->shared_stats;
+ statent = &shstatent->stats;
+
+ if (newly_created)
+ {
+ char *plan = pgsp_explain_plan(queryDesc);
+ size_t plan_size = Min(strlen(plan), pgsp_max_size);
+
+ (void) pgstat_lock_entry(entry_ref, false);
+
+ /*
+ * We may be over the limit, so run GC now before saving entry (we do
+ * this whilst holding the lock on the new entry so we don't remove it
+ * by accident)
+ */
+ pgstat_gc_plans();
+
+ shstatent->stats.info.planid = planId;
+ shstatent->stats.info.queryid = queryId;
+ shstatent->stats.info.userid = userid;
+ shstatent->stats.info.toplevel = toplevel;
+ shstatent->stats.info.plan_text = dsa_allocate(pgStatLocal.dsa, plan_size);
+ strlcpy(dsa_get_address(pgStatLocal.dsa, shstatent->stats.info.plan_text), plan, plan_size);
+
+ shstatent->stats.info.plan_encoding = GetDatabaseEncoding();
+
+ /*
+ * Increase refcount here so entry can't get released without us
+ * dropping the plan text
+ */
+ pg_atomic_fetch_add_u32(&entry_ref->shared_entry->refcount, 1);
+
+ pgstat_unlock_entry(entry_ref);
+
+ pfree(plan);
+ }
+
+ statent->exec_count += exec_count;
+ statent->exec_time += exec_time;
+ statent->usage += USAGE_INCREASE;
+}
+
+/*
+ * Planner hook: forward to regular planner, but increase plan count and
+ * record query plan if needed.
+ */
+static PlannedStmt *
+pgsp_planner(Query *parse,
+ const char *query_string,
+ int cursorOptions,
+ ParamListInfo boundParams)
+{
+ PlannedStmt *result;
+
+ /*
+ * Increment the nesting level, to ensure that functions evaluated during
+ * planning are not seen as top-level calls.
+ */
+ nesting_level++;
+ PG_TRY();
+ {
+ if (prev_planner_hook)
+ result = prev_planner_hook(parse, query_string, cursorOptions,
+ boundParams);
+ else
+ result = standard_planner(parse, query_string, cursorOptions,
+ boundParams);
+ }
+ PG_FINALLY();
+ {
+ nesting_level--;
+ }
+ PG_END_TRY();
+
+ return result;
+}
+
+/*
+ * ExecutorStart hook: start up tracking if needed
+ */
+static void
+pgsp_ExecutorStart(QueryDesc *queryDesc, int eflags)
+{
+ uint64 queryId = queryDesc->plannedstmt->queryId;
+ uint64 planId = queryDesc->plannedstmt->planId;
+
+ if (prev_ExecutorStart)
+ prev_ExecutorStart(queryDesc, eflags);
+ else
+ standard_ExecutorStart(queryDesc, eflags);
+
+ if (queryId != UINT64CONST(0) && planId != UINT64CONST(0) &&
+ pgsp_enabled(nesting_level))
+ {
+ /*
+ * Record initial entry now, so plan text is available for currently
+ * running queries
+ */
+ pgstat_report_plan_stats(queryDesc,
+ 0, /* executions are counted in
+ * pgsp_ExecutorEnd */
+ 0.0);
+
+ /*
+ * Set up to track total elapsed time in ExecutorRun. Make sure the
+ * space is allocated in the per-query context so it will go away at
+ * ExecutorEnd.
+ */
+ if (queryDesc->totaltime == NULL)
+ {
+ MemoryContext oldcxt;
+
+ oldcxt = MemoryContextSwitchTo(queryDesc->estate->es_query_cxt);
+ queryDesc->totaltime = InstrAlloc(1, INSTRUMENT_ALL, false);
+ MemoryContextSwitchTo(oldcxt);
+ }
+ }
+}
+
+/*
+ * ExecutorRun hook: all we need do is track nesting depth
+ */
+static void
+pgsp_ExecutorRun(QueryDesc *queryDesc, ScanDirection direction, uint64 count)
+{
+ nesting_level++;
+ PG_TRY();
+ {
+ if (prev_ExecutorRun)
+ prev_ExecutorRun(queryDesc, direction, count);
+ else
+ standard_ExecutorRun(queryDesc, direction, count);
+ }
+ PG_FINALLY();
+ {
+ nesting_level--;
+ }
+ PG_END_TRY();
+}
+
+/*
+ * ExecutorFinish hook: all we need do is track nesting depth
+ */
+static void
+pgsp_ExecutorFinish(QueryDesc *queryDesc)
+{
+ nesting_level++;
+ PG_TRY();
+ {
+ if (prev_ExecutorFinish)
+ prev_ExecutorFinish(queryDesc);
+ else
+ standard_ExecutorFinish(queryDesc);
+ }
+ PG_FINALLY();
+ {
+ nesting_level--;
+ }
+ PG_END_TRY();
+}
+
+/*
+ * ExecutorEnd hook: store results if needed
+ */
+static void
+pgsp_ExecutorEnd(QueryDesc *queryDesc)
+{
+ uint64 queryId = queryDesc->plannedstmt->queryId;
+ uint64 planId = queryDesc->plannedstmt->planId;
+
+ if (queryId != UINT64CONST(0) && planId != UINT64CONST(0) &&
+ queryDesc->totaltime && pgsp_enabled(nesting_level))
+ {
+ /*
+ * Make sure stats accumulation is done. (Note: it's okay if several
+ * levels of hook all do this.)
+ */
+ InstrEndLoop(queryDesc->totaltime);
+
+ pgstat_report_plan_stats(queryDesc,
+ 1,
+ queryDesc->totaltime->total * 1000.0 /* convert to msec */ );
+ }
+
+ if (prev_ExecutorEnd)
+ prev_ExecutorEnd(queryDesc);
+ else
+ standard_ExecutorEnd(queryDesc);
+}
+
+/*
+ * Module load callback
+ */
+void
+_PG_init(void)
+{
+ /*
+ * In order to register for shared memory stats, we have to be loaded via
+ * shared_preload_libraries. If not, fall out without hooking into any of
+ * the main system. (We don't throw error here because it seems useful to
+ * allow the pg_stat_plans functions to be created even when the module
+ * isn't active. The functions must protect themselves against being
+ * called then, however.)
+ */
+ if (!process_shared_preload_libraries_in_progress)
+ return;
+
+ /*
+ * Inform the postmaster that we want to enable query_id calculation if
+ * compute_query_id is set to auto, as well as plan_id calculation if
+ * compute_plan_id is set to auto.
+ */
+ EnableQueryId();
+ EnablePlanId();
+
+ /*
+ * Define (or redefine) custom GUC variables.
+ */
+ DefineCustomIntVariable("pg_stat_plans.max",
+ "Sets the maximum number of plans tracked by pg_stat_plans in shared memory.",
+ NULL,
+ &pgsp_max,
+ 5000,
+ 100,
+ INT_MAX / 2,
+ PGC_SIGHUP,
+ 0,
+ NULL,
+ NULL,
+ NULL);
+
+ DefineCustomIntVariable("pg_stat_plans.max_size",
+ "Sets the maximum size of plan texts tracked by pg_stat_plans in shared memory.",
+ NULL,
+ &pgsp_max_size,
+ 2048,
+ 100,
+ 1048576, /* 1MB hard limit */
+ PGC_SUSET,
+ 0,
+ NULL,
+ NULL,
+ NULL);
+
+ DefineCustomEnumVariable("pg_stat_plans.track",
+ "Selects which plans are tracked by pg_stat_plans.",
+ NULL,
+ &pgsp_track,
+ PGSP_TRACK_TOP,
+ track_options,
+ PGC_SUSET,
+ 0,
+ NULL,
+ NULL,
+ NULL);
+
+ MarkGUCPrefixReserved("pg_stat_plans");
+
+ /*
+ * Install hooks.
+ */
+ prev_planner_hook = planner_hook;
+ planner_hook = pgsp_planner;
+ prev_ExecutorStart = ExecutorStart_hook;
+ ExecutorStart_hook = pgsp_ExecutorStart;
+ prev_ExecutorRun = ExecutorRun_hook;
+ ExecutorRun_hook = pgsp_ExecutorRun;
+ prev_ExecutorFinish = ExecutorFinish_hook;
+ ExecutorFinish_hook = pgsp_ExecutorFinish;
+ prev_ExecutorEnd = ExecutorEnd_hook;
+ ExecutorEnd_hook = pgsp_ExecutorEnd;
+
+ pgstat_register_kind(PGSTAT_KIND_PLANS, &plan_stats);
+}
+
+static bool
+match_plans_entries(PgStatShared_HashEntry *entry, Datum match_data)
+{
+ return entry->key.kind == PGSTAT_KIND_PLANS;
+}
+
+/*
+ * Reset statement statistics.
+ */
+Datum
+pg_stat_plans_reset(PG_FUNCTION_ARGS)
+{
+ pgstat_drop_matching_entries(match_plans_entries, 0);
+
+ /* Free plan text memory and allow cleanup of dropped entries */
+ pgstat_gc_plan_memory();
+
+ PG_RETURN_VOID();
+}
+
+#define PG_STAT_PLANS_COLS 8
+
+Datum
+pg_stat_plans_1_0(PG_FUNCTION_ARGS)
+{
+ bool showplan = PG_GETARG_BOOL(0);
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ Oid userid = GetUserId();
+ bool is_allowed_role = false;
+
+ dshash_seq_status hstat;
+ PgStatShared_HashEntry *p;
+
+ /*
+ * Superusers or roles with the privileges of pg_read_all_stats members
+ * are allowed
+ */
+ is_allowed_role = has_privs_of_role(userid, ROLE_PG_READ_ALL_STATS);
+
+ /* stats kind must be registered already */
+ if (!pgstat_get_kind_info(PGSTAT_KIND_PLANS))
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("pg_stat_plans must be loaded via \"shared_preload_libraries\"")));
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ /* dshash entry is not modified, take shared lock */
+ dshash_seq_init(&hstat, pgStatLocal.shared_hash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ PgStat_StatPlanEntry *statent;
+ Datum values[PG_STAT_PLANS_COLS];
+ bool nulls[PG_STAT_PLANS_COLS];
+ int i = 0;
+
+ if (p->dropped || p->key.kind != PGSTAT_KIND_PLANS)
+ continue;
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+
+ statent = pgstat_get_entry_data(p->key.kind, dsa_get_address(pgStatLocal.dsa, p->body));
+
+ values[i++] = ObjectIdGetDatum(statent->info.userid);
+ values[i++] = ObjectIdGetDatum(p->key.dboid);
+ values[i++] = BoolGetDatum(statent->info.toplevel);
+ if (is_allowed_role || statent->info.userid == userid)
+ {
+ int64 queryid = statent->info.queryid;
+ int64 planid = statent->info.planid;
+
+ values[i++] = Int64GetDatumFast(queryid);
+ values[i++] = Int64GetDatumFast(planid);
+ }
+ else
+ {
+ nulls[i++] = true;
+ nulls[i++] = true;
+ }
+ values[i++] = Int64GetDatumFast(statent->exec_count);
+ values[i++] = Float8GetDatumFast(statent->exec_time);
+
+ if (showplan && (is_allowed_role || statent->info.userid == userid))
+ {
+ char *pstr = DsaPointerIsValid(statent->info.plan_text) ? dsa_get_address(pgStatLocal.dsa, statent->info.plan_text) : NULL;
+
+ if (pstr)
+ {
+ char *enc = pg_any_to_server(pstr, strlen(pstr), statent->info.plan_encoding);
+
+ values[i++] = CStringGetTextDatum(enc);
+
+ if (enc != pstr)
+ pfree(enc);
+ }
+ else
+ {
+ nulls[i++] = true;
+ }
+ }
+ else if (showplan)
+ {
+ values[i++] = CStringGetTextDatum("<insufficient privilege>");
+ }
+ else
+ {
+ nulls[i++] = true;
+ }
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
+ }
+ dshash_seq_term(&hstat);
+
+ return (Datum) 0;
+}
diff --git a/contrib/pg_stat_plans/pg_stat_plans.conf b/contrib/pg_stat_plans/pg_stat_plans.conf
new file mode 100644
index 00000000000..6750b3e2cc0
--- /dev/null
+++ b/contrib/pg_stat_plans/pg_stat_plans.conf
@@ -0,0 +1 @@
+shared_preload_libraries = 'pg_stat_plans'
diff --git a/contrib/pg_stat_plans/pg_stat_plans.control b/contrib/pg_stat_plans/pg_stat_plans.control
new file mode 100644
index 00000000000..4db3a47239b
--- /dev/null
+++ b/contrib/pg_stat_plans/pg_stat_plans.control
@@ -0,0 +1,5 @@
+# pg_stat_plans extension
+comment = 'track per-plan call counts, execution times and EXPLAIN texts'
+default_version = '1.0'
+module_pathname = '$libdir/pg_stat_plans'
+relocatable = true
diff --git a/contrib/pg_stat_plans/sql/cleanup.sql b/contrib/pg_stat_plans/sql/cleanup.sql
new file mode 100644
index 00000000000..51565617cef
--- /dev/null
+++ b/contrib/pg_stat_plans/sql/cleanup.sql
@@ -0,0 +1 @@
+DROP EXTENSION pg_stat_plans;
diff --git a/contrib/pg_stat_plans/sql/privileges.sql b/contrib/pg_stat_plans/sql/privileges.sql
new file mode 100644
index 00000000000..aaad72a6553
--- /dev/null
+++ b/contrib/pg_stat_plans/sql/privileges.sql
@@ -0,0 +1,59 @@
+--
+-- Only superusers and roles with privileges of the pg_read_all_stats role
+-- are allowed to see the plan text, queryid and planid of queries executed by
+-- other users. Other users can see the statistics.
+--
+
+CREATE ROLE regress_stats_superuser SUPERUSER;
+CREATE ROLE regress_stats_user1;
+CREATE ROLE regress_stats_user2;
+GRANT pg_read_all_stats TO regress_stats_user2;
+
+SET ROLE regress_stats_superuser;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+SELECT 1 AS "ONE";
+
+SET ROLE regress_stats_user1;
+SELECT 1+1 AS "TWO";
+
+--
+-- A superuser can read all columns of queries executed by others,
+-- including plan text, queryid and planid.
+--
+
+SET ROLE regress_stats_superuser;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+
+--
+-- regress_stats_user1 has no privileges to read the plan text, queryid
+-- or planid of queries executed by others but can see statistics
+-- like calls and rows.
+--
+
+SET ROLE regress_stats_user1;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+
+--
+-- regress_stats_user2, with pg_read_all_stats role privileges, can
+-- read all columns, including plan text, queryid and planid, of queries
+-- executed by others.
+--
+
+SET ROLE regress_stats_user2;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+
+--
+-- cleanup
+--
+
+RESET ROLE;
+DROP ROLE regress_stats_superuser;
+DROP ROLE regress_stats_user1;
+DROP ROLE regress_stats_user2;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
diff --git a/contrib/pg_stat_plans/sql/select.sql b/contrib/pg_stat_plans/sql/select.sql
new file mode 100644
index 00000000000..f0e803ad70c
--- /dev/null
+++ b/contrib/pg_stat_plans/sql/select.sql
@@ -0,0 +1,67 @@
+--
+-- SELECT statements
+--
+
+CREATE EXTENSION pg_stat_plans;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+
+--
+-- simple statements
+--
+
+SELECT 1 FROM pg_class LIMIT 1;
+
+SELECT 1 FROM pg_class WHERE relname = 'pg_class';
+
+SET enable_indexscan = off;
+SELECT 1 FROM pg_class WHERE relname = 'pg_class';
+SET enable_indexscan = on;
+
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+
+--
+-- subplans and CTEs
+--
+
+WITH x AS MATERIALIZED (SELECT 1)
+SELECT * FROM x;
+
+SELECT a.attname,
+ (SELECT pg_catalog.pg_get_expr(d.adbin, d.adrelid)
+ FROM pg_catalog.pg_attrdef d
+ WHERE d.adrelid = a.attrelid AND d.adnum = a.attnum AND a.atthasdef)
+ FROM pg_catalog.pg_attribute a
+ WHERE a.attrelid = 'pg_class'::regclass
+ ORDER BY attnum LIMIT 1;
+
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+
+--
+-- partitoning
+--
+
+create table lp (a char) partition by list (a);
+create table lp_default partition of lp default;
+create table lp_ef partition of lp for values in ('e', 'f');
+create table lp_ad partition of lp for values in ('a', 'd');
+create table lp_bc partition of lp for values in ('b', 'c');
+create table lp_g partition of lp for values in ('g');
+create table lp_null partition of lp for values in (null);
+
+select * from lp;
+select * from lp where a > 'a' and a < 'd';
+select * from lp where a > 'a' and a <= 'd';
+select * from lp where a = 'a';
+select * from lp where 'a' = a; /* commuted */
+select * from lp where a is not null;
+select * from lp where a is null;
+select * from lp where a = 'a' or a = 'c';
+select * from lp where a is not null and (a = 'a' or a = 'c');
+select * from lp where a <> 'g';
+select * from lp where a <> 'a' and a <> 'd';
+select * from lp where a not in ('a', 'd');
+
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index 7c381949a53..4a5a02c7049 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -157,6 +157,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
&pglogicalinspect;
&pgprewarm;
&pgrowlocks;
+ &pgstatplans;
&pgstatstatements;
&pgstattuple;
&pgsurgery;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 66e6dccd4c9..b0afb33ce22 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -146,6 +146,7 @@
<!ENTITY pglogicalinspect SYSTEM "pglogicalinspect.sgml">
<!ENTITY pgprewarm SYSTEM "pgprewarm.sgml">
<!ENTITY pgrowlocks SYSTEM "pgrowlocks.sgml">
+<!ENTITY pgstatplans SYSTEM "pgstatplans.sgml">
<!ENTITY pgstatstatements SYSTEM "pgstatstatements.sgml">
<!ENTITY pgstattuple SYSTEM "pgstattuple.sgml">
<!ENTITY pgsurgery SYSTEM "pgsurgery.sgml">
diff --git a/doc/src/sgml/pgstatplans.sgml b/doc/src/sgml/pgstatplans.sgml
new file mode 100644
index 00000000000..2d5faff15a9
--- /dev/null
+++ b/doc/src/sgml/pgstatplans.sgml
@@ -0,0 +1,425 @@
+<!-- doc/src/sgml/pgstatplans.sgml -->
+
+<sect1 id="pgstatplans" xreflabel="pg_stat_plans">
+ <title>pg_stat_plans — track per-plan call counts, execution times and EXPLAIN texts</title>
+
+ <indexterm zone="pgstatplans">
+ <primary>pg_stat_plans</primary>
+ </indexterm>
+
+ <para>
+ The <filename>pg_stat_plans</filename> module provides a means for
+ tracking per-plan statistics and plan texts of all SQL statements executed by
+ a server.
+ </para>
+
+ <para>
+ The module must be loaded by adding <literal>pg_stat_plans</literal> to
+ <xref linkend="guc-shared-preload-libraries"/> in
+ <filename>postgresql.conf</filename>, because it requires additional shared memory.
+ This means that a server restart is needed to add or remove the module.
+ In addition, query and plan identifier calculation must be enabled in order for the
+ module to be active by setting both <xref linkend="guc-compute-plan-id"/> to
+ <literal>auto</literal> or <literal>on</literal> and
+ <xref linkend="guc-compute-query-id"/> to <literal>auto</literal> or <literal>on</literal>.
+ </para>
+
+ <para>
+ When <filename>pg_stat_plans</filename> is active, it tracks
+ statistics across all databases of the server. To access and manipulate
+ these statistics, the module provides the <structname>pg_stat_plans</structname>
+ view and the utility functions <function>pg_stat_plans_reset</function> and
+ <function>pg_stat_plans</function>. These are not available globally but
+ can be enabled for a specific database with
+ <command>CREATE EXTENSION pg_stat_plans</command>.
+ </para>
+
+ <sect2 id="pgstatplans-pg-stat-plans">
+ <title>The <structname>pg_stat_plans</structname> View</title>
+
+ <para>
+ The statistics gathered by the module are made available via a
+ view named <structname>pg_stat_plans</structname>. This view
+ contains one row for each distinct combination of database ID, user
+ ID, whether it's a top-level statement or not, query ID and plan ID
+ (up to the maximum number of distinct plans that the module can track).
+ The columns of the view are shown in <xref linkend="pgstatplans-columns"/>.
+ </para>
+
+ <table id="pgstatplans-columns">
+ <title><structname>pg_stat_plans</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>userid</structfield> <type>oid</type>
+ (references <link linkend="catalog-pg-authid"><structname>pg_authid</structname></link>.<structfield>oid</structfield>)
+ </para>
+ <para>
+ OID of user who executed the statement
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbid</structfield> <type>oid</type>
+ (references <link linkend="catalog-pg-database"><structname>pg_database</structname></link>.<structfield>oid</structfield>)
+ </para>
+ <para>
+ OID of database in which the statement was executed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>toplevel</structfield> <type>bool</type>
+ </para>
+ <para>
+ True if the query was executed as a top-level statement
+ (always true if <varname>pg_stat_plans.track</varname> is set to
+ <literal>top</literal>)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>queryid</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Hash code to identify identical normalized queries.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>planid</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Hash code to identify identical plan shapes.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>calls</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the plan was executed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_exec_time</structfield> <type>double precision</type>
+ </para>
+ <para>
+ Total time spent executing the plan, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>plan</structfield> <type>text</type>
+ </para>
+ <para>
+ Plan text of a representative plan. This is similar to the output of
+ <literal>EXPLAIN (COSTS OFF)</literal>. Note the plan text will contain constant
+ values of the first plan recorded, but subsequent executions of the
+ same plan hash code (<structfield>planid</structfield>) with different
+ constant values will be tracked under the same entry.
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ For security reasons, only superusers and roles with privileges of the
+ <literal>pg_read_all_stats</literal> role are allowed to see the plan text,
+ <structfield>queryid</structfield> and <structfield>planid</structfield>
+ of queries executed by other users. Other users can see the statistics,
+ however, if the view has been installed in their database.
+ </para>
+
+ <para>
+ Plannable queries (that is, <command>SELECT</command>, <command>INSERT</command>,
+ <command>UPDATE</command>, <command>DELETE</command>, and <command>MERGE</command>)
+ will have the same <structfield>planid</structfield> whenever they have identical plan
+ structures according to an internal hash calculation. Typically, two plans will be
+ considered the same for this purpose if they have the same
+ <literal>EXPLAIN (COSTS OFF)</literal> output and are semantically equivalent except
+ for the values of literal constants appearing in the query plan.
+ </para>
+
+ <para>
+ Note that queries that have not finished executing yet will show in
+ <structname>pg_stat_plans</structname> with their plan text, but without
+ the <structname>calls</structname> field being incremented. This can be
+ used to identify the query plan for a currently running statement by joining
+ <link linkend="monitoring-pg-stat-activity-view">pg_stat_activity</link>
+ with <structname>pg_stat_plans</structname>, see example usage in
+ <xref linkend="pgstatplans-sample-output"/>.
+ </para>
+
+ <para>
+ Consumers of <structname>pg_stat_plans</structname> should use
+ <structfield>planid</structfield> in combination with
+ <structfield>queryid</structfield>, <structfield>dbid</structfield>,
+ <structfield>userid</structfield> and <structfield>toplevel</structfield>
+ as a stable and reliable identifier for each entry, instead of using its
+ plan text. However, it is important to understand that there are only limited
+ guarantees around the stability of the <structfield>planid</structfield>
+ hash value. Since the identifier is derived from the plan tree, its value
+ is a function of, among other things, the internal object identifiers
+ appearing in this representation. This has some counterintuitive implications.
+ For example, <filename>pg_stat_plans</filename> will consider two
+ apparently-identical plans to be distinct, if they reference a table
+ that was dropped and recreated between the creation of the two plans.
+ The hashing process is also sensitive to differences in
+ machine architecture and other facets of the platform.
+ Furthermore, it is not safe to assume that <structfield>planid</structfield>
+ will be stable across major versions of <productname>PostgreSQL</productname>.
+ </para>
+
+ <para>
+ Two servers participating in replication based on physical WAL replay can
+ be expected to have identical <structfield>planid</structfield> values for
+ the same plan. However, logical replication schemes do not promise to
+ keep replicas identical in all relevant details, so
+ <structfield>planid</structfield> will not be a useful identifier for
+ accumulating costs across a set of logical replicas.
+ If in doubt, direct testing is recommended.
+ </para>
+
+ <para>
+ Plan texts are stored in shared memory, and limited in length. To increase
+ the maximum length of stored plan texts you can increase
+ <varname>pg_stat_plans.max_size</varname>. This value can be changed for
+ an individual connection, or set as a server-wide setting.
+ </para>
+ </sect2>
+
+ <sect2 id="pgstatplans-funcs">
+ <title>Functions</title>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ <function>pg_stat_plans_reset() returns void</function>
+ <indexterm>
+ <primary>pg_stat_plans_reset</primary>
+ </indexterm>
+ </term>
+
+ <listitem>
+ <para>
+ <function>pg_stat_plans_reset</function> discards statistics and plan texts
+ gathered so far by <filename>pg_stat_plans</filename>.
+ By default, this function can only be executed by superusers.
+ Access may be granted to others using <command>GRANT</command>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <function>pg_stat_plans(showplan boolean) returns setof record</function>
+ <indexterm>
+ <primary>pg_stat_plans</primary>
+ <secondary>function</secondary>
+ </indexterm>
+ </term>
+
+ <listitem>
+ <para>
+ The <structname>pg_stat_plans</structname> view is defined in
+ terms of a function also named <function>pg_stat_plans</function>.
+ It is possible for clients to call
+ the <function>pg_stat_plans</function> function directly, and by
+ specifying <literal>showplan := false</literal> have plan texts be
+ omitted (that is, the <literal>OUT</literal> argument that corresponds
+ to the view's <structfield>plan</structfield> column will return nulls). This
+ feature is intended to support external tools that might wish to avoid
+ the overhead of repeatedly retrieving plan texts of indeterminate
+ length. Such tools can instead cache the first plan text observed
+ for each entry themselves, since that is
+ all <filename>pg_stat_plans</filename> itself does, and then retrieve
+ plan texts only as needed.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+
+ <sect2 id="pgstatplans-config-params">
+ <title>Configuration Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ <varname>pg_stat_plans.max</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>pg_stat_plans.max</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+
+ <listitem>
+ <para>
+ <varname>pg_stat_plans.max</varname> is the maximum number of
+ plans tracked by the module (i.e., the maximum number of rows
+ in the <structname>pg_stat_plans</structname> view). If more distinct
+ plans than that are observed, information about the least-executed
+ plans is discarded. The default value is 5000.
+ Only superusers can change this setting. Changing the setting requires
+ a reload of the server.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <varname>pg_stat_plans.max_size</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>pg_stat_plans.max_size</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+
+ <listitem>
+ <para>
+ <varname>pg_stat_plans.max_size</varname> is the maximum length of
+ each plan text tracked by the module in bytes. Longer plan texts will be truncated.
+ The default value is 2048 (2kB).
+ Only superusers can change this setting.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <varname>pg_stat_plans.track</varname> (<type>enum</type>)
+ <indexterm>
+ <primary><varname>pg_stat_plans.track</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+
+ <listitem>
+ <para>
+ <varname>pg_stat_plans.track</varname> controls which plans
+ are counted by the module.
+ Specify <literal>top</literal> to track plans by top-level statements (those issued
+ directly by clients), <literal>all</literal> to also track nested statements
+ (such as statements invoked within functions), or <literal>none</literal> to
+ disable plan statistics collection.
+ The default value is <literal>top</literal>.
+ Only superusers can change this setting.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ <para>
+ The module requires additional shared memory proportional to
+ <varname>pg_stat_plans.max</varname> for statistics, as well as
+ <varname>pg_stat_plans.max</varname> multiplied by
+ <varname>pg_stat_plans.max_size</varname> for plan texts. Note that this
+ memory is only consumed when entries are created, and not if
+ <varname>pg_stat_plans.track</varname> is set to <literal>none</literal>.
+ </para>
+
+ <para>
+ These parameters must be set in <filename>postgresql.conf</filename>.
+ Typical usage might be:
+
+<programlisting>
+# postgresql.conf
+shared_preload_libraries = 'pg_stat_plans'
+
+compute_query_id = on
+compute_plan_id = on
+pg_stat_plans.max = 10000
+pg_stat_plans.max_size = 4096
+pg_stat_plans.track = all
+</programlisting>
+ </para>
+ </sect2>
+
+ <sect2 id="pgstatplans-sample-output">
+ <title>Sample Output</title>
+
+<screen>
+bench=# SELECT pg_stat_plans_reset();
+
+$ pgbench -i bench
+$ pgbench -c10 -t300 bench
+
+bench=# \x
+bench=# SELECT plan, calls, total_exec_time
+ FROM pg_stat_plans ORDER BY total_exec_time DESC LIMIT 5;
+-[ RECORD 1 ]---+-----------------------------------------------------------------
+plan | Update on pgbench_tellers +
+ | -> Seq Scan on pgbench_tellers +
+ | Filter: (tid = 5)
+calls | 3000
+total_exec_time | 642.8880919999993
+-[ RECORD 2 ]---+-----------------------------------------------------------------
+plan | Update on pgbench_branches +
+ | -> Seq Scan on pgbench_branches +
+ | Filter: (bid = 1)
+calls | 1813
+total_exec_time | 476.64152700000005
+-[ RECORD 3 ]---+-----------------------------------------------------------------
+plan | Update on pgbench_branches +
+ | -> Index Scan using pgbench_branches_pkey on pgbench_branches+
+ | Index Cond: (bid = 1)
+calls | 1187
+total_exec_time | 326.1257549999999
+-[ RECORD 4 ]---+-----------------------------------------------------------------
+plan | Update on pgbench_accounts +
+ | -> Index Scan using pgbench_accounts_pkey on pgbench_accounts+
+ | Index Cond: (aid = 48793)
+calls | 3000
+total_exec_time | 21.664690000000093
+-[ RECORD 5 ]---+-----------------------------------------------------------------
+plan | Insert on pgbench_history +
+ | -> Result
+calls | 3000
+total_exec_time | 4.365250999999957
+
+session 1:
+
+bench# SELECT pg_sleep(100), COUNT(*) FROM pgbench_accounts;
+
+session 2:
+
+bench=# SELECT query, plan FROM pg_stat_activity
+ JOIN pg_stat_plans ON (usesysid = userid AND datid = dbid AND query_id = queryid AND plan_id = planid)
+ WHERE query LIKE 'SELECT pg_sleep%';
+ query | plan
+-------------------------------------------------------+------------------------------------
+ SELECT pg_sleep(100), COUNT(*) FROM pgbench_accounts; | Aggregate +
+ | -> Seq Scan on pgbench_accounts
+(1 row)
+
+</screen>
+ </sect2>
+
+ <sect2 id="pgstatplans-authors">
+ <title>Authors</title>
+
+ <para>
+ Lukas Fittl <email>lukas@fittl.com</email>.
+ </para>
+ </sect2>
+
+</sect1>
--
2.47.1
v4-0001-Allow-using-jumbling-logic-outside-of-query-jumbl.patchapplication/octet-stream; name=v4-0001-Allow-using-jumbling-logic-outside-of-query-jumbl.patchDownload
From e590cd2be54923aa608d0e575f272fde84b7a193 Mon Sep 17 00:00:00 2001
From: Lukas Fittl <lukas@fittl.com>
Date: Tue, 31 Dec 2024 15:05:39 -0800
Subject: [PATCH v4 1/3] Allow using jumbling logic outside of query jumble
unit file
This can be useful either for jumbling expressions in other contexts
(e.g. to calculate a plan jumble), or to allow extensions to use
a modified jumbling logic more easily.
This intentionally supports the use case where a separate jumbling logic
does not care about recording constants, as the query jumble does.
---
src/backend/nodes/queryjumblefuncs.c | 59 +++++++++++++++++-----------
src/include/nodes/queryjumble.h | 6 +++
2 files changed, 41 insertions(+), 24 deletions(-)
diff --git a/src/backend/nodes/queryjumblefuncs.c b/src/backend/nodes/queryjumblefuncs.c
index b103a281936..545d8edcae2 100644
--- a/src/backend/nodes/queryjumblefuncs.c
+++ b/src/backend/nodes/queryjumblefuncs.c
@@ -51,10 +51,7 @@ int compute_query_id = COMPUTE_QUERY_ID_AUTO;
*/
bool query_id_enabled = false;
-static void AppendJumble(JumbleState *jstate,
- const unsigned char *item, Size size);
static void RecordConstLocation(JumbleState *jstate, int location);
-static void _jumbleNode(JumbleState *jstate, Node *node);
static void _jumbleA_Const(JumbleState *jstate, Node *node);
static void _jumbleList(JumbleState *jstate, Node *node);
static void _jumbleVariableSetStmt(JumbleState *jstate, Node *node);
@@ -109,28 +106,42 @@ CleanQuerytext(const char *query, int *location, int *len)
}
JumbleState *
-JumbleQuery(Query *query)
+InitializeJumbleState(bool record_clocations)
{
- JumbleState *jstate = NULL;
-
- Assert(IsQueryIdEnabled());
-
- jstate = (JumbleState *) palloc(sizeof(JumbleState));
+ JumbleState *jstate = (JumbleState *) palloc0(sizeof(JumbleState));
/* Set up workspace for query jumbling */
jstate->jumble = (unsigned char *) palloc(JUMBLE_SIZE);
jstate->jumble_len = 0;
- jstate->clocations_buf_size = 32;
- jstate->clocations = (LocationLen *)
- palloc(jstate->clocations_buf_size * sizeof(LocationLen));
- jstate->clocations_count = 0;
- jstate->highest_extern_param_id = 0;
+
+ if (record_clocations)
+ {
+ jstate->clocations_buf_size = 32;
+ jstate->clocations = (LocationLen *)
+ palloc(jstate->clocations_buf_size * sizeof(LocationLen));
+ }
+
+ return jstate;
+}
+
+uint64
+HashJumbleState(JumbleState *jstate)
+{
+ return DatumGetUInt64(hash_any_extended(jstate->jumble,
+ jstate->jumble_len,
+ 0));
+}
+
+JumbleState *
+JumbleQuery(Query *query)
+{
+ JumbleState *jstate = InitializeJumbleState(true);
+
+ Assert(IsQueryIdEnabled());
/* Compute query ID and mark the Query node with it */
- _jumbleNode(jstate, (Node *) query);
- query->queryId = DatumGetUInt64(hash_any_extended(jstate->jumble,
- jstate->jumble_len,
- 0));
+ JumbleNode(jstate, (Node *) query);
+ query->queryId = HashJumbleState(jstate);
/*
* If we are unlucky enough to get a hash of zero, use 1 instead for
@@ -164,7 +175,7 @@ EnableQueryId(void)
* AppendJumble: Append a value that is substantive in a given query to
* the current jumble.
*/
-static void
+void
AppendJumble(JumbleState *jstate, const unsigned char *item, Size size)
{
unsigned char *jumble = jstate->jumble;
@@ -205,7 +216,7 @@ static void
RecordConstLocation(JumbleState *jstate, int location)
{
/* -1 indicates unknown or undefined location */
- if (location >= 0)
+ if (location >= 0 && jstate->clocations_buf_size > 0)
{
/* enlarge array if needed */
if (jstate->clocations_count >= jstate->clocations_buf_size)
@@ -224,7 +235,7 @@ RecordConstLocation(JumbleState *jstate, int location)
}
#define JUMBLE_NODE(item) \
- _jumbleNode(jstate, (Node *) expr->item)
+ JumbleNode(jstate, (Node *) expr->item)
#define JUMBLE_LOCATION(location) \
RecordConstLocation(jstate, expr->location)
#define JUMBLE_FIELD(item) \
@@ -239,8 +250,8 @@ do { \
#include "queryjumblefuncs.funcs.c"
-static void
-_jumbleNode(JumbleState *jstate, Node *node)
+void
+JumbleNode(JumbleState *jstate, Node *node)
{
Node *expr = node;
@@ -305,7 +316,7 @@ _jumbleList(JumbleState *jstate, Node *node)
{
case T_List:
foreach(l, expr)
- _jumbleNode(jstate, lfirst(l));
+ JumbleNode(jstate, lfirst(l));
break;
case T_IntList:
foreach(l, expr)
diff --git a/src/include/nodes/queryjumble.h b/src/include/nodes/queryjumble.h
index 50eb9566587..5afa6f3605f 100644
--- a/src/include/nodes/queryjumble.h
+++ b/src/include/nodes/queryjumble.h
@@ -83,4 +83,10 @@ IsQueryIdEnabled(void)
return query_id_enabled;
}
+/* Functions intended for other users of jumbling (e.g. plan jumbling) */
+extern JumbleState *InitializeJumbleState(bool record_clocations);
+extern void AppendJumble(JumbleState *jstate, const unsigned char *item, Size size);
+extern void JumbleNode(JumbleState *jstate, Node *node);
+extern uint64 HashJumbleState(JumbleState *jstate);
+
#endif /* QUERYJUMBLE_H */
--
2.47.1
Hi Andrei,
On Fri, Jan 24, 2025 at 1:23 AM Andrei Lepikhov <lepihov@gmail.com> wrote:
I may not be close to the task monitoring area, but I utilise queryId
and other tools to differ plan nodes inside extensions. Initially, like
queryId serves as a class identifier for queries, plan_id identifies a
class of nodes, not a single node. In the implementation provided here,
nodes with the same hash can represent different subtrees. For example,
JOIN(A, JOIN(B,C)) and JOIN(JOIN(B,C),A) may have the same ID.
Moreover, I wonder if this version of plan_id reacts to the join level
change. It appears that only a change of the join clause alters the
plan_id hash value, which means you would end up with a single hash for
very different plan nodes. Is that acceptable? To address this, we
should consider the hashes of the left and right subtrees and the hashes
of each subplan (especially in the case of Append).
I looked back at this again just to confirm we're not missing anything:
I don't think any of the posted patch versions (including the just shared
v4) have a problem with distinguishing two plans that are very similar but
only differ in JOIN order. Since we descend into the inner/outer plans via
the setrefs.c treewalk, the placement of JOIN nodes vs other nodes should
cause a different plan jumble (and we include both the node tag for the
join/scan nodes, as well as the RT index the scans point to in the jumble).
Do you have a reproducer that shows these two generate the same plan ID?
Thanks,
Lukas
--
Lukas Fittl
On 2/5/25 09:16, Lukas Fittl wrote:
Hi Andrei,
On Fri, Jan 24, 2025 at 1:23 AM Andrei Lepikhov <lepihov@gmail.com
<mailto:lepihov@gmail.com>> wrote:I may not be close to the task monitoring area, but I utilise queryId
and other tools to differ plan nodes inside extensions. Initially, like
queryId serves as a class identifier for queries, plan_id identifies a
class of nodes, not a single node. In the implementation provided here,
nodes with the same hash can represent different subtrees. For example,
JOIN(A, JOIN(B,C)) and JOIN(JOIN(B,C),A) may have the same ID.Moreover, I wonder if this version of plan_id reacts to the join level
change. It appears that only a change of the join clause alters the
plan_id hash value, which means you would end up with a single hash for
very different plan nodes. Is that acceptable? To address this, we
should consider the hashes of the left and right subtrees and the
hashes
of each subplan (especially in the case of Append).I looked back at this again just to confirm we're not missing anything:
I don't think any of the posted patch versions (including the just
shared v4) have a problem with distinguishing two plans that are very
similar but only differ in JOIN order. Since we descend into the inner/
outer plans via the setrefs.c treewalk, the placement of JOIN nodes vs
other nodes should cause a different plan jumble (and we include both
the node tag for the join/scan nodes, as well as the RT index the scans
point to in the jumble).
Maybe. I haven't dive into that stuff deeply yet. It is not difficult to
check.
The main point was that different extensions want different plan_ids.
For example, planner extensions want to guarantee the distinctness and
sort of stability of this field inside a query plan. Does the hash value
guarantee that?
We have discussed how queryId should be generated more than once. That's
why I think the plan_id generation logic should be implemented inside an
extension, not in the core.
--
regards, Andrei Lepikhov
This does not yet move field-specific comments to their own line in nodes where we're adding node attributes, I'll leave that for Sami to work on.
Hi,
Attached is a new set of patches for fixing the long comments
in plannodes.h and to refactor queryjumblefuncs.c
v5-0001
-----------
This fixes the long comments in plannodes.h to make it easier to add the
attribute annotation. It made the most sense to make this the first patch
in the set.
v5-0002
-----------
Here are my high-level thoughts on this:
1. rename queryjumblefuncs.c to jumblefuncs.c
If these APIs are used for somethings else than Query structure, yes,
the renaming makes sense.
Done. Also rewrote the header comment in jumblefuncs.c to describe
a more generic node jumbling mechanism that this file now offers.
2. move the query jumbling related code to parser/analyze.c,
since query jumbling occurs there during parsing.
Not sure about this one. It depends on how much is changed. As long
as everything related to the nodes stays in src/backend/nodes/,
perhaps that's OK.
Yes, after getting my hands on this, I agree with you. It made more sense
to keep all the jumbling work in jumblefuncs.c
v5-0003 and v5-0004 introduce the planId in core and pg_stat_plans. These
needed rebasing only; but I have not yet looked at this thoroughly.
We should aim to get 0001 and 0002 committed next.
Regards,
Sami
Attachments:
v5-0002-Allow-using-jumbling-logic-outside-of-query-jumbl.patchapplication/octet-stream; name=v5-0002-Allow-using-jumbling-logic-outside-of-query-jumbl.patchDownload
From 560481ec3408bf98ccf696ffb99b6c8ed1eeb6be Mon Sep 17 00:00:00 2001
From: Lukas Fittl <lukas@fittl.com>
Date: Tue, 31 Dec 2024 15:05:39 -0800
Subject: [PATCH v5 2/4] Allow using jumbling logic outside of query jumble
unit file
This can be useful either for jumbling expressions in other contexts
(e.g. to calculate a plan jumble), or to allow extensions to use
a modified jumbling logic more easily.
This intentionally supports the use case where a separate jumbling logic
does not care about recording constants, as the query jumble does.
---
.../pg_stat_statements/pg_stat_statements.c | 2 +-
src/backend/commands/createas.c | 2 +-
src/backend/commands/extension.c | 2 +-
src/backend/commands/portalcmds.c | 2 +-
src/backend/executor/execMain.c | 2 +-
src/backend/nodes/Makefile | 6 +-
src/backend/nodes/README | 4 +-
src/backend/nodes/gen_node_support.pl | 14 +--
.../{queryjumblefuncs.c => jumblefuncs.c} | 109 ++++++++++--------
src/backend/nodes/meson.build | 2 +-
src/backend/parser/analyze.c | 2 +-
src/backend/postmaster/launch_backend.c | 2 +-
src/backend/utils/misc/guc_tables.c | 2 +-
src/include/nodes/{queryjumble.h => jumble.h} | 18 ++-
src/include/nodes/meson.build | 2 +-
src/include/nodes/nodes.h | 2 +-
src/include/parser/analyze.h | 2 +-
17 files changed, 98 insertions(+), 77 deletions(-)
rename src/backend/nodes/{queryjumblefuncs.c => jumblefuncs.c} (78%)
rename src/include/nodes/{queryjumble.h => jumble.h} (80%)
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index bebf8134eb..26ef7f3e03 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -55,7 +55,7 @@
#include "jit/jit.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
-#include "nodes/queryjumble.h"
+#include "nodes/jumble.h"
#include "optimizer/planner.h"
#include "parser/analyze.h"
#include "parser/scanner.h"
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 23cecd99c9..a8498e370c 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -37,7 +37,7 @@
#include "commands/view.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
-#include "nodes/queryjumble.h"
+#include "nodes/jumble.h"
#include "parser/analyze.h"
#include "rewrite/rewriteHandler.h"
#include "tcop/tcopprot.h"
diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c
index ba540e3de5..3a462d708b 100644
--- a/src/backend/commands/extension.c
+++ b/src/backend/commands/extension.c
@@ -54,7 +54,7 @@
#include "funcapi.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
-#include "nodes/queryjumble.h"
+#include "nodes/jumble.h"
#include "storage/fd.h"
#include "tcop/utility.h"
#include "utils/acl.h"
diff --git a/src/backend/commands/portalcmds.c b/src/backend/commands/portalcmds.c
index e7c8171c10..926ec2af36 100644
--- a/src/backend/commands/portalcmds.c
+++ b/src/backend/commands/portalcmds.c
@@ -28,7 +28,7 @@
#include "executor/executor.h"
#include "executor/tstoreReceiver.h"
#include "miscadmin.h"
-#include "nodes/queryjumble.h"
+#include "nodes/jumble.h"
#include "parser/analyze.h"
#include "rewrite/rewriteHandler.h"
#include "tcop/pquery.h"
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 604cb0625b..fa6002d5f9 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -51,7 +51,7 @@
#include "foreign/fdwapi.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
-#include "nodes/queryjumble.h"
+#include "nodes/jumble.h"
#include "parser/parse_relation.h"
#include "pgstat.h"
#include "rewrite/rewriteHandler.h"
diff --git a/src/backend/nodes/Makefile b/src/backend/nodes/Makefile
index 77ddb9ca53..4545649e2d 100644
--- a/src/backend/nodes/Makefile
+++ b/src/backend/nodes/Makefile
@@ -26,7 +26,7 @@ OBJS = \
outfuncs.o \
params.o \
print.o \
- queryjumblefuncs.o \
+ jumblefuncs.o \
read.o \
readfuncs.o \
tidbitmap.o \
@@ -91,8 +91,8 @@ $(top_builddir)/src/include/nodes/header-stamp: node-support-stamp
copyfuncs.o: copyfuncs.c copyfuncs.funcs.c copyfuncs.switch.c | node-support-stamp
equalfuncs.o: equalfuncs.c equalfuncs.funcs.c equalfuncs.switch.c | node-support-stamp
outfuncs.o: outfuncs.c outfuncs.funcs.c outfuncs.switch.c | node-support-stamp
-queryjumblefuncs.o: queryjumblefuncs.c queryjumblefuncs.funcs.c queryjumblefuncs.switch.c | node-support-stamp
+jumblefuncs.o: jumblefuncs.c jumblefuncs.funcs.c jumblefuncs.switch.c | node-support-stamp
readfuncs.o: readfuncs.c readfuncs.funcs.c readfuncs.switch.c | node-support-stamp
clean:
- rm -f node-support-stamp $(addsuffix funcs.funcs.c,copy equal out queryjumble read) $(addsuffix funcs.switch.c,copy equal out queryjumble read) nodetags.h
+ rm -f node-support-stamp $(addsuffix funcs.funcs.c,copy equal out jumble read) $(addsuffix funcs.switch.c,copy equal out jumble read) nodetags.h
diff --git a/src/backend/nodes/README b/src/backend/nodes/README
index f8bbd60538..a43290fbab 100644
--- a/src/backend/nodes/README
+++ b/src/backend/nodes/README
@@ -51,7 +51,7 @@ FILES IN THIS DIRECTORY (src/backend/nodes/)
readfuncs.c - convert text representation back to a node tree (*)
makefuncs.c - creator functions for some common node types
nodeFuncs.c - some other general-purpose manipulation functions
- queryjumblefuncs.c - compute a node tree for query jumbling (*)
+ jumblefuncs.c - compute a node tree for jumbling (*)
(*) - Most functions in these files are generated by
gen_node_support.pl and #include'd there.
@@ -88,7 +88,7 @@ Suppose you want to define a node Foo:
If you intend to inherit from, say a Plan node, put Plan as the first field
of your struct definition. (The T_Foo tag is created automatically.)
2. Check that the generated support functions in copyfuncs.funcs.c,
- equalfuncs.funcs.c, outfuncs.funcs.c, queryjumblefuncs.funcs.c and
+ equalfuncs.funcs.c, outfuncs.funcs.c, jumblefuncs.funcs.c and
readfuncs.funcs.c look correct. Add attributes as necessary to control the
outcome. (For some classes of node types, you don't need all the support
functions. Use node attributes similar to those of related node types.)
diff --git a/src/backend/nodes/gen_node_support.pl b/src/backend/nodes/gen_node_support.pl
index 1a657f7e0a..26ec7e0d59 100644
--- a/src/backend/nodes/gen_node_support.pl
+++ b/src/backend/nodes/gen_node_support.pl
@@ -1244,15 +1244,15 @@ close $ofs;
close $rfs;
-# queryjumblefuncs.c
+# jumblefuncs.c
-push @output_files, 'queryjumblefuncs.funcs.c';
-open my $jff, '>', "$output_path/queryjumblefuncs.funcs.c$tmpext" or die $!;
-push @output_files, 'queryjumblefuncs.switch.c';
-open my $jfs, '>', "$output_path/queryjumblefuncs.switch.c$tmpext" or die $!;
+push @output_files, 'jumblefuncs.funcs.c';
+open my $jff, '>', "$output_path/jumblefuncs.funcs.c$tmpext" or die $!;
+push @output_files, 'jumblefuncs.switch.c';
+open my $jfs, '>', "$output_path/jumblefuncs.switch.c$tmpext" or die $!;
-printf $jff $header_comment, 'queryjumblefuncs.funcs.c';
-printf $jfs $header_comment, 'queryjumblefuncs.switch.c';
+printf $jff $header_comment, 'jumblefuncs.funcs.c';
+printf $jfs $header_comment, 'jumblefuncs.switch.c';
print $jff $node_includes;
diff --git a/src/backend/nodes/queryjumblefuncs.c b/src/backend/nodes/jumblefuncs.c
similarity index 78%
rename from src/backend/nodes/queryjumblefuncs.c
rename to src/backend/nodes/jumblefuncs.c
index b103a28193..fcf34ae479 100644
--- a/src/backend/nodes/queryjumblefuncs.c
+++ b/src/backend/nodes/jumblefuncs.c
@@ -1,32 +1,36 @@
/*-------------------------------------------------------------------------
*
- * queryjumblefuncs.c
- * Query normalization and fingerprinting.
+ * jumblefuncs.c
+ * Fingerprinting and jumbling.
*
- * Normalization is a process whereby similar queries, typically differing only
- * in their constants (though the exact rules are somewhat more subtle than
- * that) are recognized as equivalent, and are tracked as a single entry. This
- * is particularly useful for non-prepared queries.
+ * Fingerprinting selectively serializes key fields within a tree structure,
+ * such as a Query or Plan tree, to create a unique identifier while ignoring
+ * extraneous details. These essential fields are concatenated into a jumble,
+ * from which a 64-bit hash is computed. Unlike regular serialization, this
+ * approach excludes irrelevant information.
*
- * Normalization is implemented by fingerprinting queries, selectively
- * serializing those fields of each query tree's nodes that are judged to be
- * essential to the query. This is referred to as a query jumble. This is
- * distinct from a regular serialization in that various extraneous
- * information is ignored as irrelevant or not essential to the query, such
- * as the collations of Vars and, most notably, the values of constants.
+ * Use Cases:
*
- * This jumble is acquired at the end of parse analysis of each query, and
- * a 64-bit hash of it is stored into the query's Query.queryId field.
- * The server then copies this value around, making it available in plan
- * tree(s) generated from the query. The executor can then use this value
- * to blame query costs on the proper queryId.
+ * 1. In-Core Query Normalization & Identification
*
- * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
- * Portions Copyright (c) 1994, Regents of the University of California
+ * Fingerprinting is used to normalize query trees by generating a hash stored
+ * in the Query.queryId field. This ID is propagated to plan tree(s), allowing
+ * the executor to attribute query costs on the proper queryId. The process
+ * excludes information like typmod, collation, and most notably, the values
+ * of constants.
*
+ * Example: The following queries produce the same queryId:
+ *
+ * SELECT t.* FROM s1.t WHERE c1 = 1;
+ * SELECT t.* FROM s1.t WHERE c1 = 2;
+ *
+ * 2. Modified jumbling logic for extensions
+ *
+ * Extensions can modify the fingerprinting logic for queryId, or fingerprint
+ * other types of trees, such as a plan tree, to compute a plan identifier.
*
* IDENTIFICATION
- * src/backend/nodes/queryjumblefuncs.c
+ * src/backend/nodes/jumblefuncs.c
*
*-------------------------------------------------------------------------
*/
@@ -34,7 +38,7 @@
#include "common/hashfn.h"
#include "miscadmin.h"
-#include "nodes/queryjumble.h"
+#include "nodes/jumble.h"
#include "parser/scansup.h"
#define JUMBLE_SIZE 1024 /* query serialization buffer size */
@@ -51,10 +55,7 @@ int compute_query_id = COMPUTE_QUERY_ID_AUTO;
*/
bool query_id_enabled = false;
-static void AppendJumble(JumbleState *jstate,
- const unsigned char *item, Size size);
static void RecordConstLocation(JumbleState *jstate, int location);
-static void _jumbleNode(JumbleState *jstate, Node *node);
static void _jumbleA_Const(JumbleState *jstate, Node *node);
static void _jumbleList(JumbleState *jstate, Node *node);
static void _jumbleVariableSetStmt(JumbleState *jstate, Node *node);
@@ -109,28 +110,42 @@ CleanQuerytext(const char *query, int *location, int *len)
}
JumbleState *
-JumbleQuery(Query *query)
+InitializeJumbleState(bool record_clocations)
{
- JumbleState *jstate = NULL;
-
- Assert(IsQueryIdEnabled());
-
- jstate = (JumbleState *) palloc(sizeof(JumbleState));
+ JumbleState *jstate = (JumbleState *) palloc0(sizeof(JumbleState));
/* Set up workspace for query jumbling */
jstate->jumble = (unsigned char *) palloc(JUMBLE_SIZE);
jstate->jumble_len = 0;
- jstate->clocations_buf_size = 32;
- jstate->clocations = (LocationLen *)
- palloc(jstate->clocations_buf_size * sizeof(LocationLen));
- jstate->clocations_count = 0;
- jstate->highest_extern_param_id = 0;
+
+ if (record_clocations)
+ {
+ jstate->clocations_buf_size = 32;
+ jstate->clocations = (LocationLen *)
+ palloc(jstate->clocations_buf_size * sizeof(LocationLen));
+ }
+
+ return jstate;
+}
+
+uint64
+HashJumbleState(JumbleState *jstate)
+{
+ return DatumGetUInt64(hash_any_extended(jstate->jumble,
+ jstate->jumble_len,
+ 0));
+}
+
+JumbleState *
+JumbleQuery(Query *query)
+{
+ JumbleState *jstate = InitializeJumbleState(true);
+
+ Assert(IsQueryIdEnabled());
/* Compute query ID and mark the Query node with it */
- _jumbleNode(jstate, (Node *) query);
- query->queryId = DatumGetUInt64(hash_any_extended(jstate->jumble,
- jstate->jumble_len,
- 0));
+ JumbleNode(jstate, (Node *) query);
+ query->queryId = HashJumbleState(jstate);
/*
* If we are unlucky enough to get a hash of zero, use 1 instead for
@@ -164,7 +179,7 @@ EnableQueryId(void)
* AppendJumble: Append a value that is substantive in a given query to
* the current jumble.
*/
-static void
+void
AppendJumble(JumbleState *jstate, const unsigned char *item, Size size)
{
unsigned char *jumble = jstate->jumble;
@@ -205,7 +220,7 @@ static void
RecordConstLocation(JumbleState *jstate, int location)
{
/* -1 indicates unknown or undefined location */
- if (location >= 0)
+ if (location >= 0 && jstate->clocations_buf_size > 0)
{
/* enlarge array if needed */
if (jstate->clocations_count >= jstate->clocations_buf_size)
@@ -224,7 +239,7 @@ RecordConstLocation(JumbleState *jstate, int location)
}
#define JUMBLE_NODE(item) \
- _jumbleNode(jstate, (Node *) expr->item)
+ JumbleNode(jstate, (Node *) expr->item)
#define JUMBLE_LOCATION(location) \
RecordConstLocation(jstate, expr->location)
#define JUMBLE_FIELD(item) \
@@ -237,10 +252,10 @@ do { \
AppendJumble(jstate, (const unsigned char *) (expr->str), strlen(expr->str) + 1); \
} while(0)
-#include "queryjumblefuncs.funcs.c"
+#include "jumblefuncs.funcs.c"
-static void
-_jumbleNode(JumbleState *jstate, Node *node)
+void
+JumbleNode(JumbleState *jstate, Node *node)
{
Node *expr = node;
@@ -258,7 +273,7 @@ _jumbleNode(JumbleState *jstate, Node *node)
switch (nodeTag(expr))
{
-#include "queryjumblefuncs.switch.c"
+#include "jumblefuncs.switch.c"
case T_List:
case T_IntList:
@@ -305,7 +320,7 @@ _jumbleList(JumbleState *jstate, Node *node)
{
case T_List:
foreach(l, expr)
- _jumbleNode(jstate, lfirst(l));
+ JumbleNode(jstate, lfirst(l));
break;
case T_IntList:
foreach(l, expr)
diff --git a/src/backend/nodes/meson.build b/src/backend/nodes/meson.build
index 9a1c1b7b98..b7ebb86bab 100644
--- a/src/backend/nodes/meson.build
+++ b/src/backend/nodes/meson.build
@@ -19,7 +19,7 @@ backend_sources += files(
nodefunc_sources = files(
'copyfuncs.c',
'equalfuncs.c',
- 'queryjumblefuncs.c',
+ 'jumblefuncs.c',
'outfuncs.c',
'readfuncs.c',
)
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 76f58b3aca..cd235921b4 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -31,7 +31,7 @@
#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
-#include "nodes/queryjumble.h"
+#include "nodes/jumble.h"
#include "optimizer/optimizer.h"
#include "parser/analyze.h"
#include "parser/parse_agg.h"
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index a97a1eda6d..b127b1bef3 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -53,7 +53,7 @@
#include "utils/memutils.h"
#ifdef EXEC_BACKEND
-#include "nodes/queryjumble.h"
+#include "nodes/jumble.h"
#include "storage/pg_shmem.h"
#include "storage/spin.h"
#endif
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index ce7534d4d2..9a22512fef 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -50,7 +50,7 @@
#include "libpq/auth.h"
#include "libpq/libpq.h"
#include "libpq/scram.h"
-#include "nodes/queryjumble.h"
+#include "nodes/jumble.h"
#include "optimizer/cost.h"
#include "optimizer/geqo.h"
#include "optimizer/optimizer.h"
diff --git a/src/include/nodes/queryjumble.h b/src/include/nodes/jumble.h
similarity index 80%
rename from src/include/nodes/queryjumble.h
rename to src/include/nodes/jumble.h
index 50eb956658..7587c9f708 100644
--- a/src/include/nodes/queryjumble.h
+++ b/src/include/nodes/jumble.h
@@ -1,18 +1,18 @@
/*-------------------------------------------------------------------------
*
- * queryjumble.h
- * Query normalization and fingerprinting.
+ * jumble.h
+ * Fingerprinting and jumbling.
*
* Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* IDENTIFICATION
- * src/include/nodes/queryjumble.h
+ * src/include/nodes/jumble.h
*
*-------------------------------------------------------------------------
*/
-#ifndef QUERYJUMBLE_H
-#define QUERYJUMBLE_H
+#ifndef JUMBLE_H
+#define JUMBLE_H
#include "nodes/parsenodes.h"
@@ -83,4 +83,10 @@ IsQueryIdEnabled(void)
return query_id_enabled;
}
-#endif /* QUERYJUMBLE_H */
+/* Functions called for plan jumbling or extensions doing their own jumbling */
+extern JumbleState *InitializeJumbleState(bool record_clocations);
+extern void AppendJumble(JumbleState *jstate, const unsigned char *item, Size size);
+extern void JumbleNode(JumbleState *jstate, Node *node);
+extern uint64 HashJumbleState(JumbleState *jstate);
+
+#endif /* JUMBLE_H */
diff --git a/src/include/nodes/meson.build b/src/include/nodes/meson.build
index d1ca24dd32..5c0ee5b0f6 100644
--- a/src/include/nodes/meson.build
+++ b/src/include/nodes/meson.build
@@ -37,7 +37,7 @@ node_support_output = [
'readfuncs.funcs.c', 'readfuncs.switch.c',
'copyfuncs.funcs.c', 'copyfuncs.switch.c',
'equalfuncs.funcs.c', 'equalfuncs.switch.c',
- 'queryjumblefuncs.funcs.c', 'queryjumblefuncs.switch.c',
+ 'jumblefuncs.funcs.c', 'jumblefuncs.switch.c',
]
node_support_install = [
dir_include_server / 'nodes',
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 580238bfab..7330d5ffc4 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -53,7 +53,7 @@ typedef enum NodeTag
* - custom_read_write: Has custom implementations in outfuncs.c and
* readfuncs.c.
*
- * - custom_query_jumble: Has custom implementation in queryjumblefuncs.c.
+ * - custom_query_jumble: Has custom implementation in jumblefuncs.c.
*
* - no_copy: Does not support copyObject() at all.
*
diff --git a/src/include/parser/analyze.h b/src/include/parser/analyze.h
index f1bd18c49f..e414b630bc 100644
--- a/src/include/parser/analyze.h
+++ b/src/include/parser/analyze.h
@@ -15,7 +15,7 @@
#define ANALYZE_H
#include "nodes/params.h"
-#include "nodes/queryjumble.h"
+#include "nodes/jumble.h"
#include "parser/parse_node.h"
/* Hook for plugins to get control at end of parse analysis */
--
2.47.1
v5-0001-reformat-comments-in-plannode.h.patchapplication/octet-stream; name=v5-0001-reformat-comments-in-plannode.h.patchDownload
From 6f3d0250ca13c6368d2b8ee10a8b14768508902e Mon Sep 17 00:00:00 2001
From: "Sami Imseih (AWS)"
<simseih@dev-dsk-simseih-1d-3940b79e.us-east-1.amazon.com>
Date: Fri, 7 Feb 2025 00:27:52 +0000
Subject: [PATCH v5 1/4] reformat comments in plannode.h
Similar to d575051b9af9, reformat comments in plannodes.h to avoid
long lines.
This makes room for per-field annotations in a future patch to
generate node support functions automatically.
Discussion: https://www.postgresql.org/message-id/Z5xTb5iBHVGns35R%40paquier.xyz
---
src/include/nodes/plannodes.h | 426 ++++++++++++++++++++++------------
1 file changed, 279 insertions(+), 147 deletions(-)
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 06d9559ebb..1e082bfdcd 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -49,57 +49,78 @@ typedef struct PlannedStmt
NodeTag type;
- CmdType commandType; /* select|insert|update|delete|merge|utility */
+ /* select|insert|update|delete|merge|utility */
+ CmdType commandType;
- uint64 queryId; /* query identifier (copied from Query) */
+ /* query identifier (copied from Query) */
+ uint64 queryId;
- bool hasReturning; /* is it insert|update|delete|merge RETURNING? */
+ /* is it insert|update|delete|merge RETURNING? */
+ bool hasReturning;
- bool hasModifyingCTE; /* has insert|update|delete|merge in WITH? */
+ /* has insert|update|delete|merge in WITH? */
+ bool hasModifyingCTE;
- bool canSetTag; /* do I set the command result tag? */
+ /* do I set the command result tag? */
+ bool canSetTag;
- bool transientPlan; /* redo plan when TransactionXmin changes? */
+ /* redo plan when TransactionXmin changes? */
+ bool transientPlan;
- bool dependsOnRole; /* is plan specific to current role? */
+ /* is plan specific to current role? */
+ bool dependsOnRole;
- bool parallelModeNeeded; /* parallel mode required to execute? */
+ /* parallel mode required to execute? */
+ bool parallelModeNeeded;
- int jitFlags; /* which forms of JIT should be performed */
+ /* which forms of JIT should be performed */
+ int jitFlags;
- struct Plan *planTree; /* tree of Plan nodes */
+ /* tree of Plan nodes */
+ struct Plan *planTree;
- List *partPruneInfos; /* List of PartitionPruneInfo contained in the
- * plan */
+ /* List of PartitionPruneInfo contained in the plan */
+ List *partPruneInfos;
- List *rtable; /* list of RangeTblEntry nodes */
+ /* list of RangeTblEntry nodes */
+ List *rtable;
- List *permInfos; /* list of RTEPermissionInfo nodes for rtable
- * entries needing one */
+ /* list of RTEPermissionInfo nodes for rtable entries needing one */
+ List *permInfos;
/* rtable indexes of target relations for INSERT/UPDATE/DELETE/MERGE */
- List *resultRelations; /* integer list of RT indexes, or NIL */
+ /* integer list of RT indexes, or NIL */
+ List *resultRelations;
- List *appendRelations; /* list of AppendRelInfo nodes */
+ /* list of AppendRelInfo nodes */
+ List *appendRelations;
- List *subplans; /* Plan trees for SubPlan expressions; note
- * that some could be NULL */
+ /* Plan trees for SubPlan expressions; note that some could be NULL */
+ List *subplans;
- Bitmapset *rewindPlanIDs; /* indices of subplans that require REWIND */
+ /* indices of subplans that require REWIND */
+ Bitmapset *rewindPlanIDs;
- List *rowMarks; /* a list of PlanRowMark's */
+ /* a list of PlanRowMark's */
+ List *rowMarks;
- List *relationOids; /* OIDs of relations the plan depends on */
+ /* OIDs of relations the plan depends on */
+ List *relationOids;
- List *invalItems; /* other dependencies, as PlanInvalItems */
+ /* other dependencies, as PlanInvalItems */
+ List *invalItems;
- List *paramExecTypes; /* type OIDs for PARAM_EXEC Params */
+ /* type OIDs for PARAM_EXEC Params */
+ List *paramExecTypes;
- Node *utilityStmt; /* non-null if this is utility stmt */
+ /* non-null if this is utility stmt */
+ Node *utilityStmt;
/* statement location in source string (copied from Query) */
- ParseLoc stmt_location; /* start location, or -1 if unknown */
- ParseLoc stmt_len; /* length in bytes; 0 means "rest of string" */
+ /* start location, or -1 if unknown */
+ ParseLoc stmt_location;
+ /* length in bytes; 0 means "rest of string" */
+ ParseLoc stmt_len;
} PlannedStmt;
/* macro for fetching the Plan associated with a SubPlan node */
@@ -128,37 +149,49 @@ typedef struct Plan
/*
* estimated execution costs for plan (see costsize.c for more info)
*/
- int disabled_nodes; /* count of disabled nodes */
- Cost startup_cost; /* cost expended before fetching any tuples */
- Cost total_cost; /* total cost (assuming all tuples fetched) */
+ /* count of disabled nodes */
+ int disabled_nodes;
+ /* cost expended before fetching any tuples */
+ Cost startup_cost;
+ /* total cost (assuming all tuples fetched) */
+ Cost total_cost;
/*
* planner's estimate of result size of this plan step
*/
- Cardinality plan_rows; /* number of rows plan is expected to emit */
- int plan_width; /* average row width in bytes */
+ /* number of rows plan is expected to emit */
+ Cardinality plan_rows;
+ /* average row width in bytes */
+ int plan_width;
/*
* information needed for parallel query
*/
- bool parallel_aware; /* engage parallel-aware logic? */
- bool parallel_safe; /* OK to use as part of parallel plan? */
+ /* engage parallel-aware logic? */
+ bool parallel_aware;
+ /* OK to use as part of parallel plan? */
+ bool parallel_safe;
/*
* information needed for asynchronous execution
*/
- bool async_capable; /* engage asynchronous-capable logic? */
+ /* engage asynchronous-capable logic? */
+ bool async_capable;
/*
* Common structural data for all Plan types.
*/
- int plan_node_id; /* unique across entire final plan tree */
- List *targetlist; /* target list to be computed at this node */
- List *qual; /* implicitly-ANDed qual conditions */
- struct Plan *lefttree; /* input plan tree(s) */
+ /* unique across entire final plan tree */
+ int plan_node_id;
+ /* target list to be computed at this node */
+ List *targetlist;
+ /* implicitly-ANDed qual conditions */
+ List *qual;
+ /* input plan tree(s) */
+ struct Plan *lefttree;
struct Plan *righttree;
- List *initPlan; /* Init Plan nodes (un-correlated expr
- * subselects) */
+ /* Init Plan nodes (un-correlated expr subselects) */
+ List *initPlan;
/*
* Information for management of parameter-change-driven rescanning
@@ -233,32 +266,54 @@ typedef struct ProjectSet
typedef struct ModifyTable
{
Plan plan;
- CmdType operation; /* INSERT, UPDATE, DELETE, or MERGE */
- bool canSetTag; /* do we set the command tag/es_processed? */
- Index nominalRelation; /* Parent RT index for use of EXPLAIN */
- Index rootRelation; /* Root RT index, if partitioned/inherited */
- bool partColsUpdated; /* some part key in hierarchy updated? */
- List *resultRelations; /* integer list of RT indexes */
- List *updateColnosLists; /* per-target-table update_colnos lists */
- List *withCheckOptionLists; /* per-target-table WCO lists */
- char *returningOldAlias; /* alias for OLD in RETURNING lists */
- char *returningNewAlias; /* alias for NEW in RETURNING lists */
- List *returningLists; /* per-target-table RETURNING tlists */
- List *fdwPrivLists; /* per-target-table FDW private data lists */
- Bitmapset *fdwDirectModifyPlans; /* indices of FDW DM plans */
- List *rowMarks; /* PlanRowMarks (non-locking only) */
- int epqParam; /* ID of Param for EvalPlanQual re-eval */
- OnConflictAction onConflictAction; /* ON CONFLICT action */
- List *arbiterIndexes; /* List of ON CONFLICT arbiter index OIDs */
- List *onConflictSet; /* INSERT ON CONFLICT DO UPDATE targetlist */
- List *onConflictCols; /* target column numbers for onConflictSet */
- Node *onConflictWhere; /* WHERE for ON CONFLICT UPDATE */
- Index exclRelRTI; /* RTI of the EXCLUDED pseudo relation */
- List *exclRelTlist; /* tlist of the EXCLUDED pseudo relation */
- List *mergeActionLists; /* per-target-table lists of actions for
- * MERGE */
- List *mergeJoinConditions; /* per-target-table join conditions
- * for MERGE */
+ /* INSERT, UPDATE, DELETE, or MERGE */
+ CmdType operation;
+ /* do we set the command tag/es_processed? */
+ bool canSetTag;
+ /* Parent RT index for use of EXPLAIN */
+ Index nominalRelation;
+ /* Root RT index, if partitioned/inherited */
+ Index rootRelation;
+ /* some part key in hierarchy updated? */
+ bool partColsUpdated;
+ /* integer list of RT indexes */
+ List *resultRelations;
+ /* per-target-table update_colnos lists */
+ List *updateColnosLists;
+ /* per-target-table WCO lists */
+ List *withCheckOptionLists;
+ /* alias for OLD in RETURNING lists */
+ char *returningOldAlias;
+ /* alias for NEW in RETURNING lists */
+ char *returningNewAlias;
+ /* per-target-table RETURNING tlists */
+ List *returningLists;
+ /* per-target-table FDW private data lists */
+ List *fdwPrivLists;
+ /* indices of FDW DM plans */
+ Bitmapset *fdwDirectModifyPlans;
+ /* PlanRowMarks (non-locking only) */
+ List *rowMarks;
+ /* ID of Param for EvalPlanQual re-eval */
+ int epqParam;
+ /* ON CONFLICT action */
+ OnConflictAction onConflictAction;
+ /* List of ON CONFLICT arbiter index OIDs */
+ List *arbiterIndexes;
+ /* INSERT ON CONFLICT DO UPDATE targetlist */
+ List *onConflictSet;
+ /* target column numbers for onConflictSet */
+ List *onConflictCols;
+ /* WHERE for ON CONFLICT UPDATE */
+ Node *onConflictWhere;
+ /* RTI of the EXCLUDED pseudo relation */
+ Index exclRelRTI;
+ /* tlist of the EXCLUDED pseudo relation */
+ List *exclRelTlist;
+ /* per-target-table lists of actions for MERGE */
+ List *mergeActionLists;
+ /* per-target-table join conditions for MERGE */
+ List *mergeJoinConditions;
} ModifyTable;
struct PartitionPruneInfo; /* forward reference to struct below */
@@ -271,9 +326,11 @@ struct PartitionPruneInfo; /* forward reference to struct below */
typedef struct Append
{
Plan plan;
- Bitmapset *apprelids; /* RTIs of appendrel(s) formed by this node */
+ /* RTIs of appendrel(s) formed by this node */
+ Bitmapset *apprelids;
List *appendplans;
- int nasyncplans; /* # of asynchronous plans */
+ /* # of asynchronous plans */
+ int nasyncplans;
/*
* All 'appendplans' preceding this index are non-partial plans. All
@@ -400,7 +457,8 @@ typedef struct Scan
pg_node_attr(abstract)
Plan plan;
- Index scanrelid; /* relid is index into the range table */
+ /* relid is index into the range table */
+ Index scanrelid;
} Scan;
/* ----------------
@@ -463,13 +521,20 @@ typedef struct SampleScan
typedef struct IndexScan
{
Scan scan;
- Oid indexid; /* OID of index to scan */
- List *indexqual; /* list of index quals (usually OpExprs) */
- List *indexqualorig; /* the same in original form */
- List *indexorderby; /* list of index ORDER BY exprs */
- List *indexorderbyorig; /* the same in original form */
- List *indexorderbyops; /* OIDs of sort ops for ORDER BY exprs */
- ScanDirection indexorderdir; /* forward or backward or don't care */
+ /* OID of index to scan */
+ Oid indexid;
+ /* list of index quals (usually OpExprs) */
+ List *indexqual;
+ /* the same in original form */
+ List *indexqualorig;
+ /* list of index ORDER BY exprs */
+ List *indexorderby;
+ /* the same in original form */
+ List *indexorderbyorig;
+ /* OIDs of sort ops for ORDER BY exprs */
+ List *indexorderbyops;
+ /* forward or backward or don't care */
+ ScanDirection indexorderdir;
} IndexScan;
/* ----------------
@@ -506,12 +571,18 @@ typedef struct IndexScan
typedef struct IndexOnlyScan
{
Scan scan;
- Oid indexid; /* OID of index to scan */
- List *indexqual; /* list of index quals (usually OpExprs) */
- List *recheckqual; /* index quals in recheckable form */
- List *indexorderby; /* list of index ORDER BY exprs */
- List *indextlist; /* TargetEntry list describing index's cols */
- ScanDirection indexorderdir; /* forward or backward or don't care */
+ /* OID of index to scan */
+ Oid indexid;
+ /* list of index quals (usually OpExprs) */
+ List *indexqual;
+ /* index quals in recheckable form */
+ List *recheckqual;
+ /* list of index ORDER BY exprs */
+ List *indexorderby;
+ /* TargetEntry list describing index's cols */
+ List *indextlist;
+ /* forward or backward or don't care */
+ ScanDirection indexorderdir;
} IndexOnlyScan;
/* ----------------
@@ -534,10 +605,14 @@ typedef struct IndexOnlyScan
typedef struct BitmapIndexScan
{
Scan scan;
- Oid indexid; /* OID of index to scan */
- bool isshared; /* Create shared bitmap if set */
- List *indexqual; /* list of index quals (OpExprs) */
- List *indexqualorig; /* the same in original form */
+ /* OID of index to scan */
+ Oid indexid;
+ /* Create shared bitmap if set */
+ bool isshared;
+ /* list of index quals (OpExprs) */
+ List *indexqual;
+ /* the same in original form */
+ List *indexqualorig;
} BitmapIndexScan;
/* ----------------
@@ -552,7 +627,8 @@ typedef struct BitmapIndexScan
typedef struct BitmapHeapScan
{
Scan scan;
- List *bitmapqualorig; /* index quals, in standard expr form */
+ /* index quals, in standard expr form */
+ List *bitmapqualorig;
} BitmapHeapScan;
/* ----------------
@@ -566,7 +642,8 @@ typedef struct BitmapHeapScan
typedef struct TidScan
{
Scan scan;
- List *tidquals; /* qual(s) involving CTID = something */
+ /* qual(s) involving CTID = something */
+ List *tidquals;
} TidScan;
/* ----------------
@@ -579,7 +656,8 @@ typedef struct TidScan
typedef struct TidRangeScan
{
Scan scan;
- List *tidrangequals; /* qual(s) involving CTID op something */
+ /* qual(s) involving CTID op something */
+ List *tidrangequals;
} TidRangeScan;
/* ----------------
@@ -623,8 +701,10 @@ typedef struct SubqueryScan
typedef struct FunctionScan
{
Scan scan;
- List *functions; /* list of RangeTblFunction nodes */
- bool funcordinality; /* WITH ORDINALITY */
+ /* list of RangeTblFunction nodes */
+ List *functions;
+ /* WITH ORDINALITY */
+ bool funcordinality;
} FunctionScan;
/* ----------------
@@ -634,7 +714,8 @@ typedef struct FunctionScan
typedef struct ValuesScan
{
Scan scan;
- List *values_lists; /* list of expression lists */
+ /* list of expression lists */
+ List *values_lists;
} ValuesScan;
/* ----------------
@@ -644,7 +725,8 @@ typedef struct ValuesScan
typedef struct TableFuncScan
{
Scan scan;
- TableFunc *tablefunc; /* table function node */
+ /* table function node */
+ TableFunc *tablefunc;
} TableFuncScan;
/* ----------------
@@ -654,8 +736,10 @@ typedef struct TableFuncScan
typedef struct CteScan
{
Scan scan;
- int ctePlanId; /* ID of init SubPlan for CTE */
- int cteParam; /* ID of Param representing CTE output */
+ /* ID of init SubPlan for CTE */
+ int ctePlanId;
+ /* ID of Param representing CTE output */
+ int cteParam;
} CteScan;
/* ----------------
@@ -665,7 +749,8 @@ typedef struct CteScan
typedef struct NamedTuplestoreScan
{
Scan scan;
- char *enrname; /* Name given to Ephemeral Named Relation */
+ /* Name given to Ephemeral Named Relation */
+ char *enrname;
} NamedTuplestoreScan;
/* ----------------
@@ -675,7 +760,8 @@ typedef struct NamedTuplestoreScan
typedef struct WorkTableScan
{
Scan scan;
- int wtParam; /* ID of Param representing work table */
+ /* ID of Param representing work table */
+ int wtParam;
} WorkTableScan;
/* ----------------
@@ -721,18 +807,28 @@ typedef struct WorkTableScan
typedef struct ForeignScan
{
Scan scan;
- CmdType operation; /* SELECT/INSERT/UPDATE/DELETE */
- Index resultRelation; /* direct modification target's RT index */
- Oid checkAsUser; /* user to perform the scan as; 0 means to
- * check as current user */
- Oid fs_server; /* OID of foreign server */
- List *fdw_exprs; /* expressions that FDW may evaluate */
- List *fdw_private; /* private data for FDW */
- List *fdw_scan_tlist; /* optional tlist describing scan tuple */
- List *fdw_recheck_quals; /* original quals not in scan.plan.qual */
- Bitmapset *fs_relids; /* base+OJ RTIs generated by this scan */
- Bitmapset *fs_base_relids; /* base RTIs generated by this scan */
- bool fsSystemCol; /* true if any "system column" is needed */
+ /* SELECT/INSERT/UPDATE/DELETE */
+ CmdType operation;
+ /* direct modification target's RT index */
+ Index resultRelation;
+ /* user to perform the scan as; 0 means to check as current user */
+ Oid checkAsUser;
+ /* OID of foreign server */
+ Oid fs_server;
+ /* expressions that FDW may evaluate */
+ List *fdw_exprs;
+ /* private data for FDW */
+ List *fdw_private;
+ /* optional tlist describing scan tuple */
+ List *fdw_scan_tlist;
+ /* original quals not in scan.plan.qual */
+ List *fdw_recheck_quals;
+ /* base+OJ RTIs generated by this scan */
+ Bitmapset *fs_relids;
+ /* base RTIs generated by this scan */
+ Bitmapset *fs_base_relids;
+ /* true if any "system column" is needed */
+ bool fsSystemCol;
} ForeignScan;
/* ----------------
@@ -753,13 +849,18 @@ struct CustomScanMethods;
typedef struct CustomScan
{
Scan scan;
- uint32 flags; /* mask of CUSTOMPATH_* flags, see
- * nodes/extensible.h */
- List *custom_plans; /* list of Plan nodes, if any */
- List *custom_exprs; /* expressions that custom code may evaluate */
- List *custom_private; /* private data for custom code */
- List *custom_scan_tlist; /* optional tlist describing scan tuple */
- Bitmapset *custom_relids; /* RTIs generated by this scan */
+ /* mask of CUSTOMPATH_* flags, see nodes/extensible.h */
+ uint32 flags;
+ /* list of Plan nodes, if any */
+ List *custom_plans;
+ /* expressions that custom code may evaluate */
+ List *custom_exprs;
+ /* private data for custom code */
+ List *custom_private;
+ /* optional tlist describing scan tuple */
+ List *custom_scan_tlist;
+ /* RTIs generated by this scan */
+ Bitmapset *custom_relids;
/*
* NOTE: The method field of CustomScan is required to be a pointer to a
@@ -804,7 +905,8 @@ typedef struct Join
Plan plan;
JoinType jointype;
bool inner_unique;
- List *joinqual; /* JOIN quals (in addition to plan.qual) */
+ /* JOIN quals (in addition to plan.qual) */
+ List *joinqual;
} Join;
/* ----------------
@@ -821,7 +923,8 @@ typedef struct Join
typedef struct NestLoop
{
Join join;
- List *nestParams; /* list of NestLoopParam nodes */
+ /* list of NestLoopParam nodes */
+ List *nestParams;
} NestLoop;
typedef struct NestLoopParam
@@ -829,8 +932,10 @@ typedef struct NestLoopParam
pg_node_attr(no_equal, no_query_jumble)
NodeTag type;
- int paramno; /* number of the PARAM_EXEC Param to set */
- Var *paramval; /* outer-relation Var to assign to Param */
+ /* number of the PARAM_EXEC Param to set */
+ int paramno;
+ /* outer-relation Var to assign to Param */
+ Var *paramval;
} NestLoopParam;
/* ----------------
@@ -969,7 +1074,8 @@ typedef struct Sort
typedef struct IncrementalSort
{
Sort sort;
- int nPresortedCols; /* number of presorted columns */
+ /* number of presorted columns */
+ int nPresortedCols;
} IncrementalSort;
/* ---------------
@@ -1154,12 +1260,19 @@ typedef struct Unique
typedef struct Gather
{
Plan plan;
- int num_workers; /* planned number of worker processes */
- int rescan_param; /* ID of Param that signals a rescan, or -1 */
- bool single_copy; /* don't execute plan more than once */
- bool invisible; /* suppress EXPLAIN display (for testing)? */
- Bitmapset *initParam; /* param id's of initplans which are referred
- * at gather or one of its child nodes */
+ /* planned number of worker processes */
+ int num_workers;
+ /* ID of Param that signals a rescan, or -1 */
+ int rescan_param;
+ /* don't execute plan more than once */
+ bool single_copy;
+ /* suppress EXPLAIN display (for testing)? */
+ bool invisible;
+ /*
+ * param id's of initplans which are referred at gather
+ * or one of its child nodes
+ */
+ Bitmapset *initParam;
} Gather;
/* ------------
@@ -1216,12 +1329,17 @@ typedef struct Hash
* List of expressions to be hashed for tuples from Hash's outer plan,
* needed to put them into the hashtable.
*/
- List *hashkeys; /* hash keys for the hashjoin condition */
- Oid skewTable; /* outer join key's table OID, or InvalidOid */
- AttrNumber skewColumn; /* outer join key's column #, or zero */
- bool skewInherit; /* is outer join rel an inheritance tree? */
+ /* hash keys for the hashjoin condition */
+ List *hashkeys;
+ /* outer join key's table OID, or InvalidOid */
+ Oid skewTable;
+ /* outer join key's column #, or zero */
+ AttrNumber skewColumn;
+ /* is outer join rel an inheritance tree? */
+ bool skewInherit;
/* all other info is in the parent HashJoin node */
- Cardinality rows_total; /* estimate total rows if parallel_aware */
+ /* estimate total rows if parallel_aware */
+ Cardinality rows_total;
} Hash;
/* ----------------
@@ -1267,8 +1385,10 @@ typedef struct SetOp
typedef struct LockRows
{
Plan plan;
- List *rowMarks; /* a list of PlanRowMark's */
- int epqParam; /* ID of Param for EvalPlanQual re-eval */
+ /* a list of PlanRowMark's */
+ List *rowMarks;
+ /* ID of Param for EvalPlanQual re-eval */
+ int epqParam;
} LockRows;
/* ----------------
@@ -1390,14 +1510,22 @@ typedef struct PlanRowMark
pg_node_attr(no_equal, no_query_jumble)
NodeTag type;
- Index rti; /* range table index of markable relation */
- Index prti; /* range table index of parent relation */
- Index rowmarkId; /* unique identifier for resjunk columns */
- RowMarkType markType; /* see enum above */
- int allMarkTypes; /* OR of (1<<markType) for all children */
- LockClauseStrength strength; /* LockingClause's strength, or LCS_NONE */
- LockWaitPolicy waitPolicy; /* NOWAIT and SKIP LOCKED options */
- bool isParent; /* true if this is a "dummy" parent entry */
+ /* range table index of markable relation */
+ Index rti;
+ /* range table index of parent relation */
+ Index prti;
+ /* unique identifier for resjunk columns */
+ Index rowmarkId;
+ /* see enum above */
+ RowMarkType markType;
+ /* OR of (1<<markType) for all children */
+ int allMarkTypes;
+ /* LockingClause's strength, or LCS_NONE */
+ LockClauseStrength strength;
+ /* NOWAIT and SKIP LOCKED options */
+ LockWaitPolicy waitPolicy;
+ /* true if this is a "dummy" parent entry */
+ bool isParent;
} PlanRowMark;
@@ -1492,8 +1620,10 @@ typedef struct PartitionedRelPruneInfo
* is required. exec_pruning_steps shows how to prune with PARAM_EXEC
* Params; it is NIL if no per-scan pruning is required.
*/
- List *initial_pruning_steps; /* List of PartitionPruneStep */
- List *exec_pruning_steps; /* List of PartitionPruneStep */
+ /* List of PartitionPruneStep */
+ List *initial_pruning_steps;
+ /* List of PartitionPruneStep */
+ List *exec_pruning_steps;
/* All PARAM_EXEC Param IDs in exec_pruning_steps */
Bitmapset *execparamids;
@@ -1585,8 +1715,10 @@ typedef struct PlanInvalItem
pg_node_attr(no_equal, no_query_jumble)
NodeTag type;
- int cacheId; /* a syscache ID, see utils/syscache.h */
- uint32 hashValue; /* hash value of object's cache lookup key */
+ /* a syscache ID, see utils/syscache.h */
+ int cacheId;
+ /* hash value of object's cache lookup key */
+ uint32 hashValue;
} PlanInvalItem;
/*
--
2.47.1
v5-0004-Add-pg_stat_plans-contrib-extension.patchapplication/octet-stream; name=v5-0004-Add-pg_stat_plans-contrib-extension.patchDownload
From 9a41c7fa17018080d50b1bc1a4be760e4371a608 Mon Sep 17 00:00:00 2001
From: Michael Paquier <michael@paquier.xyz>
Date: Fri, 31 Jan 2025 13:08:19 +0900
Subject: [PATCH v5 4/4] Add pg_stat_plans contrib extension
This extension allows tracking per-plan call counts and execution time,
as well as capturing the plan text, aka EXPLAIN (COSTS OFF), for the
first execution of a given plan. This utilize the compute_plan_id
functionality for tracking different plans.
---
contrib/Makefile | 1 +
contrib/meson.build | 1 +
contrib/pg_stat_plans/Makefile | 29 +
contrib/pg_stat_plans/expected/cleanup.out | 1 +
contrib/pg_stat_plans/expected/privileges.out | 125 +++
contrib/pg_stat_plans/expected/select.out | 262 ++++++
contrib/pg_stat_plans/meson.build | 43 +
contrib/pg_stat_plans/pg_stat_plans--1.0.sql | 33 +
contrib/pg_stat_plans/pg_stat_plans.c | 779 ++++++++++++++++++
contrib/pg_stat_plans/pg_stat_plans.conf | 1 +
contrib/pg_stat_plans/pg_stat_plans.control | 5 +
contrib/pg_stat_plans/sql/cleanup.sql | 1 +
contrib/pg_stat_plans/sql/privileges.sql | 59 ++
contrib/pg_stat_plans/sql/select.sql | 67 ++
doc/src/sgml/contrib.sgml | 1 +
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/pgstatplans.sgml | 425 ++++++++++
17 files changed, 1834 insertions(+)
create mode 100644 contrib/pg_stat_plans/Makefile
create mode 100644 contrib/pg_stat_plans/expected/cleanup.out
create mode 100644 contrib/pg_stat_plans/expected/privileges.out
create mode 100644 contrib/pg_stat_plans/expected/select.out
create mode 100644 contrib/pg_stat_plans/meson.build
create mode 100644 contrib/pg_stat_plans/pg_stat_plans--1.0.sql
create mode 100644 contrib/pg_stat_plans/pg_stat_plans.c
create mode 100644 contrib/pg_stat_plans/pg_stat_plans.conf
create mode 100644 contrib/pg_stat_plans/pg_stat_plans.control
create mode 100644 contrib/pg_stat_plans/sql/cleanup.sql
create mode 100644 contrib/pg_stat_plans/sql/privileges.sql
create mode 100644 contrib/pg_stat_plans/sql/select.sql
create mode 100644 doc/src/sgml/pgstatplans.sgml
diff --git a/contrib/Makefile b/contrib/Makefile
index 952855d9b6..8de010afde 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -34,6 +34,7 @@ SUBDIRS = \
pg_freespacemap \
pg_logicalinspect \
pg_prewarm \
+ pg_stat_plans \
pg_stat_statements \
pg_surgery \
pg_trgm \
diff --git a/contrib/meson.build b/contrib/meson.build
index 1ba73ebd67..0442ec2644 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -49,6 +49,7 @@ subdir('pg_freespacemap')
subdir('pg_logicalinspect')
subdir('pg_prewarm')
subdir('pgrowlocks')
+subdir('pg_stat_plans')
subdir('pg_stat_statements')
subdir('pgstattuple')
subdir('pg_surgery')
diff --git a/contrib/pg_stat_plans/Makefile b/contrib/pg_stat_plans/Makefile
new file mode 100644
index 0000000000..e073db95ed
--- /dev/null
+++ b/contrib/pg_stat_plans/Makefile
@@ -0,0 +1,29 @@
+# contrib/pg_stat_plans/Makefile
+
+MODULE_big = pg_stat_plans
+OBJS = \
+ $(WIN32RES) \
+ pg_stat_plans.o
+
+EXTENSION = pg_stat_plans
+DATA = pg_stat_plans--1.0.sql
+PGFILEDESC = "pg_stat_plans - track per-plan call counts, execution times and EXPLAIN texts"
+
+LDFLAGS_SL += $(filter -lm, $(LIBS))
+
+REGRESS_OPTS = --temp-config $(top_srcdir)/contrib/pg_stat_plans/pg_stat_plans.conf
+REGRESS = select privileges cleanup
+# Disabled because these tests require "shared_preload_libraries=pg_stat_plans",
+# which typical installcheck users do not have (e.g. buildfarm clients).
+NO_INSTALLCHECK = 1
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/pg_stat_plans
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/pg_stat_plans/expected/cleanup.out b/contrib/pg_stat_plans/expected/cleanup.out
new file mode 100644
index 0000000000..51565617ce
--- /dev/null
+++ b/contrib/pg_stat_plans/expected/cleanup.out
@@ -0,0 +1 @@
+DROP EXTENSION pg_stat_plans;
diff --git a/contrib/pg_stat_plans/expected/privileges.out b/contrib/pg_stat_plans/expected/privileges.out
new file mode 100644
index 0000000000..3e21d6d701
--- /dev/null
+++ b/contrib/pg_stat_plans/expected/privileges.out
@@ -0,0 +1,125 @@
+--
+-- Only superusers and roles with privileges of the pg_read_all_stats role
+-- are allowed to see the plan text, queryid and planid of queries executed by
+-- other users. Other users can see the statistics.
+--
+CREATE ROLE regress_stats_superuser SUPERUSER;
+CREATE ROLE regress_stats_user1;
+CREATE ROLE regress_stats_user2;
+GRANT pg_read_all_stats TO regress_stats_user2;
+SET ROLE regress_stats_superuser;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
+SELECT 1 AS "ONE";
+ ONE
+-----
+ 1
+(1 row)
+
+SET ROLE regress_stats_user1;
+SELECT 1+1 AS "TWO";
+ TWO
+-----
+ 2
+(1 row)
+
+--
+-- A superuser can read all columns of queries executed by others,
+-- including plan text, queryid and planid.
+--
+SET ROLE regress_stats_superuser;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+ rolname | queryid_bool | planid_bool | plan | calls
+-------------------------+--------------+-------------+------------------------------------------------------------------------------------+-------
+ regress_stats_superuser | t | t | Result | 1
+ regress_stats_superuser | t | t | Result | 1
+ regress_stats_superuser | t | t | Sort +| 0
+ | | | Sort Key: pg_authid.rolname, pg_stat_plans.plan COLLATE "C", pg_stat_plans.calls+|
+ | | | -> Hash Join +|
+ | | | Hash Cond: (pg_stat_plans.userid = pg_authid.oid) +|
+ | | | -> Function Scan on pg_stat_plans +|
+ | | | -> Hash +|
+ | | | -> Seq Scan on pg_authid |
+ regress_stats_user1 | t | t | Result | 1
+(4 rows)
+
+--
+-- regress_stats_user1 has no privileges to read the plan text, queryid
+-- or planid of queries executed by others but can see statistics
+-- like calls and rows.
+--
+SET ROLE regress_stats_user1;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+ rolname | queryid_bool | planid_bool | plan | calls
+-------------------------+--------------+-------------+------------------------------------------------------------------------------------+-------
+ regress_stats_superuser | | | <insufficient privilege> | 1
+ regress_stats_superuser | | | <insufficient privilege> | 1
+ regress_stats_superuser | | | <insufficient privilege> | 1
+ regress_stats_user1 | t | t | Result | 1
+ regress_stats_user1 | t | t | Sort +| 0
+ | | | Sort Key: pg_authid.rolname, pg_stat_plans.plan COLLATE "C", pg_stat_plans.calls+|
+ | | | -> Hash Join +|
+ | | | Hash Cond: (pg_stat_plans.userid = pg_authid.oid) +|
+ | | | -> Function Scan on pg_stat_plans +|
+ | | | -> Hash +|
+ | | | -> Seq Scan on pg_authid |
+(5 rows)
+
+--
+-- regress_stats_user2, with pg_read_all_stats role privileges, can
+-- read all columns, including plan text, queryid and planid, of queries
+-- executed by others.
+--
+SET ROLE regress_stats_user2;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+ rolname | queryid_bool | planid_bool | plan | calls
+-------------------------+--------------+-------------+------------------------------------------------------------------------------------+-------
+ regress_stats_superuser | t | t | Result | 1
+ regress_stats_superuser | t | t | Result | 1
+ regress_stats_superuser | t | t | Sort +| 1
+ | | | Sort Key: pg_authid.rolname, pg_stat_plans.plan COLLATE "C", pg_stat_plans.calls+|
+ | | | -> Hash Join +|
+ | | | Hash Cond: (pg_stat_plans.userid = pg_authid.oid) +|
+ | | | -> Function Scan on pg_stat_plans +|
+ | | | -> Hash +|
+ | | | -> Seq Scan on pg_authid |
+ regress_stats_user1 | t | t | Result | 1
+ regress_stats_user1 | t | t | Sort +| 1
+ | | | Sort Key: pg_authid.rolname, pg_stat_plans.plan COLLATE "C", pg_stat_plans.calls+|
+ | | | -> Hash Join +|
+ | | | Hash Cond: (pg_stat_plans.userid = pg_authid.oid) +|
+ | | | -> Function Scan on pg_stat_plans +|
+ | | | -> Hash +|
+ | | | -> Seq Scan on pg_authid |
+ regress_stats_user2 | t | t | Sort +| 0
+ | | | Sort Key: pg_authid.rolname, pg_stat_plans.plan COLLATE "C", pg_stat_plans.calls+|
+ | | | -> Hash Join +|
+ | | | Hash Cond: (pg_stat_plans.userid = pg_authid.oid) +|
+ | | | -> Function Scan on pg_stat_plans +|
+ | | | -> Hash +|
+ | | | -> Seq Scan on pg_authid |
+(6 rows)
+
+--
+-- cleanup
+--
+RESET ROLE;
+DROP ROLE regress_stats_superuser;
+DROP ROLE regress_stats_user1;
+DROP ROLE regress_stats_user2;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
diff --git a/contrib/pg_stat_plans/expected/select.out b/contrib/pg_stat_plans/expected/select.out
new file mode 100644
index 0000000000..906d8ce90d
--- /dev/null
+++ b/contrib/pg_stat_plans/expected/select.out
@@ -0,0 +1,262 @@
+--
+-- SELECT statements
+--
+CREATE EXTENSION pg_stat_plans;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
+--
+-- simple statements
+--
+SELECT 1 FROM pg_class LIMIT 1;
+ ?column?
+----------
+ 1
+(1 row)
+
+SELECT 1 FROM pg_class WHERE relname = 'pg_class';
+ ?column?
+----------
+ 1
+(1 row)
+
+SET enable_indexscan = off;
+SELECT 1 FROM pg_class WHERE relname = 'pg_class';
+ ?column?
+----------
+ 1
+(1 row)
+
+SET enable_indexscan = on;
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+ plan | calls
+--------------------------------------------------------------+-------
+ Bitmap Heap Scan on pg_class +| 1
+ Recheck Cond: (relname = 'pg_class'::name) +|
+ -> Bitmap Index Scan on pg_class_relname_nsp_index +|
+ Index Cond: (relname = 'pg_class'::name) |
+ Index Only Scan using pg_class_relname_nsp_index on pg_class+| 1
+ Index Cond: (relname = 'pg_class'::name) |
+ Limit +| 1
+ -> Seq Scan on pg_class |
+ Result | 1
+ Sort +| 0
+ Sort Key: pg_stat_plans.plan COLLATE "C" +|
+ -> Function Scan on pg_stat_plans |
+(5 rows)
+
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
+--
+-- subplans and CTEs
+--
+WITH x AS MATERIALIZED (SELECT 1)
+SELECT * FROM x;
+ ?column?
+----------
+ 1
+(1 row)
+
+SELECT a.attname,
+ (SELECT pg_catalog.pg_get_expr(d.adbin, d.adrelid)
+ FROM pg_catalog.pg_attrdef d
+ WHERE d.adrelid = a.attrelid AND d.adnum = a.attnum AND a.atthasdef)
+ FROM pg_catalog.pg_attribute a
+ WHERE a.attrelid = 'pg_class'::regclass
+ ORDER BY attnum LIMIT 1;
+ attname | pg_get_expr
+----------+-------------
+ tableoid |
+(1 row)
+
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+ plan | calls
+-------------------------------------------------------------------------------+-------
+ CTE Scan on x +| 1
+ CTE x +|
+ -> Result |
+ Limit +| 1
+ -> Index Scan using pg_attribute_relid_attnum_index on pg_attribute a +|
+ Index Cond: (attrelid = '1259'::oid) +|
+ SubPlan 1 +|
+ -> Result +|
+ One-Time Filter: a.atthasdef +|
+ -> Seq Scan on pg_attrdef d +|
+ Filter: ((adrelid = a.attrelid) AND (adnum = a.attnum)) |
+ Result | 1
+ Sort +| 0
+ Sort Key: pg_stat_plans.plan COLLATE "C" +|
+ -> Function Scan on pg_stat_plans |
+(4 rows)
+
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
+--
+-- partitoning
+--
+create table lp (a char) partition by list (a);
+create table lp_default partition of lp default;
+create table lp_ef partition of lp for values in ('e', 'f');
+create table lp_ad partition of lp for values in ('a', 'd');
+create table lp_bc partition of lp for values in ('b', 'c');
+create table lp_g partition of lp for values in ('g');
+create table lp_null partition of lp for values in (null);
+select * from lp;
+ a
+---
+(0 rows)
+
+select * from lp where a > 'a' and a < 'd';
+ a
+---
+(0 rows)
+
+select * from lp where a > 'a' and a <= 'd';
+ a
+---
+(0 rows)
+
+select * from lp where a = 'a';
+ a
+---
+(0 rows)
+
+select * from lp where 'a' = a; /* commuted */
+ a
+---
+(0 rows)
+
+select * from lp where a is not null;
+ a
+---
+(0 rows)
+
+select * from lp where a is null;
+ a
+---
+(0 rows)
+
+select * from lp where a = 'a' or a = 'c';
+ a
+---
+(0 rows)
+
+select * from lp where a is not null and (a = 'a' or a = 'c');
+ a
+---
+(0 rows)
+
+select * from lp where a <> 'g';
+ a
+---
+(0 rows)
+
+select * from lp where a <> 'a' and a <> 'd';
+ a
+---
+(0 rows)
+
+select * from lp where a not in ('a', 'd');
+ a
+---
+(0 rows)
+
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+ plan | calls
+--------------------------------------------------------------------------------+-------
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ Filter: ((a = 'a'::bpchar) OR (a = 'c'::bpchar)) +|
+ -> Seq Scan on lp_bc lp_2 +|
+ Filter: ((a = 'a'::bpchar) OR (a = 'c'::bpchar)) |
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ Filter: ((a > 'a'::bpchar) AND (a <= 'd'::bpchar)) +|
+ -> Seq Scan on lp_bc lp_2 +|
+ Filter: ((a > 'a'::bpchar) AND (a <= 'd'::bpchar)) +|
+ -> Seq Scan on lp_default lp_3 +|
+ Filter: ((a > 'a'::bpchar) AND (a <= 'd'::bpchar)) |
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ Filter: ((a IS NOT NULL) AND ((a = 'a'::bpchar) OR (a = 'c'::bpchar)))+|
+ -> Seq Scan on lp_bc lp_2 +|
+ Filter: ((a IS NOT NULL) AND ((a = 'a'::bpchar) OR (a = 'c'::bpchar))) |
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ Filter: (a <> 'g'::bpchar) +|
+ -> Seq Scan on lp_bc lp_2 +|
+ Filter: (a <> 'g'::bpchar) +|
+ -> Seq Scan on lp_ef lp_3 +|
+ Filter: (a <> 'g'::bpchar) +|
+ -> Seq Scan on lp_default lp_4 +|
+ Filter: (a <> 'g'::bpchar) |
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ Filter: (a IS NOT NULL) +|
+ -> Seq Scan on lp_bc lp_2 +|
+ Filter: (a IS NOT NULL) +|
+ -> Seq Scan on lp_ef lp_3 +|
+ Filter: (a IS NOT NULL) +|
+ -> Seq Scan on lp_g lp_4 +|
+ Filter: (a IS NOT NULL) +|
+ -> Seq Scan on lp_default lp_5 +|
+ Filter: (a IS NOT NULL) |
+ Append +| 1
+ -> Seq Scan on lp_ad lp_1 +|
+ -> Seq Scan on lp_bc lp_2 +|
+ -> Seq Scan on lp_ef lp_3 +|
+ -> Seq Scan on lp_g lp_4 +|
+ -> Seq Scan on lp_null lp_5 +|
+ -> Seq Scan on lp_default lp_6 |
+ Append +| 1
+ -> Seq Scan on lp_bc lp_1 +|
+ Filter: ((a <> 'a'::bpchar) AND (a <> 'd'::bpchar)) +|
+ -> Seq Scan on lp_ef lp_2 +|
+ Filter: ((a <> 'a'::bpchar) AND (a <> 'd'::bpchar)) +|
+ -> Seq Scan on lp_g lp_3 +|
+ Filter: ((a <> 'a'::bpchar) AND (a <> 'd'::bpchar)) +|
+ -> Seq Scan on lp_default lp_4 +|
+ Filter: ((a <> 'a'::bpchar) AND (a <> 'd'::bpchar)) |
+ Append +| 1
+ -> Seq Scan on lp_bc lp_1 +|
+ Filter: ((a > 'a'::bpchar) AND (a < 'd'::bpchar)) +|
+ -> Seq Scan on lp_default lp_2 +|
+ Filter: ((a > 'a'::bpchar) AND (a < 'd'::bpchar)) |
+ Append +| 1
+ -> Seq Scan on lp_bc lp_1 +|
+ Filter: (a <> ALL ('{a,d}'::bpchar[])) +|
+ -> Seq Scan on lp_ef lp_2 +|
+ Filter: (a <> ALL ('{a,d}'::bpchar[])) +|
+ -> Seq Scan on lp_g lp_3 +|
+ Filter: (a <> ALL ('{a,d}'::bpchar[])) +|
+ -> Seq Scan on lp_default lp_4 +|
+ Filter: (a <> ALL ('{a,d}'::bpchar[])) |
+ Result | 1
+ Seq Scan on lp_ad lp +| 1
+ Filter: ('a'::bpchar = a) |
+ Seq Scan on lp_ad lp +| 1
+ Filter: (a = 'a'::bpchar) |
+ Seq Scan on lp_null lp +| 1
+ Filter: (a IS NULL) |
+ Sort +| 0
+ Sort Key: pg_stat_plans.plan COLLATE "C" +|
+ -> Function Scan on pg_stat_plans |
+(14 rows)
+
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+ t
+---
+ t
+(1 row)
+
diff --git a/contrib/pg_stat_plans/meson.build b/contrib/pg_stat_plans/meson.build
new file mode 100644
index 0000000000..3bd884d960
--- /dev/null
+++ b/contrib/pg_stat_plans/meson.build
@@ -0,0 +1,43 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+pg_stat_plans_sources = files(
+ 'pg_stat_plans.c',
+)
+
+if host_system == 'windows'
+ pg_stat_plans_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'pg_stat_plans',
+ '--FILEDESC', 'pg_stat_plans - track per-plan call counts, execution times and EXPLAIN texts',])
+endif
+
+pg_stat_plans = shared_module('pg_stat_plans',
+ pg_stat_plans_sources,
+ kwargs: contrib_mod_args + {
+ 'dependencies': contrib_mod_args['dependencies'],
+ },
+)
+contrib_targets += pg_stat_plans
+
+install_data(
+ 'pg_stat_plans.control',
+ 'pg_stat_plans--1.0.sql',
+ kwargs: contrib_data_args,
+)
+
+tests += {
+ 'name': 'pg_stat_plans',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'regress': {
+ 'sql': [
+ 'select',
+ 'privileges',
+ 'cleanup',
+ ],
+ 'regress_args': ['--temp-config', files('pg_stat_plans.conf')],
+ # Disabled because these tests require
+ # "shared_preload_libraries=pg_stat_plans", which typical
+ # runningcheck users do not have (e.g. buildfarm clients).
+ 'runningcheck': false,
+ }
+}
diff --git a/contrib/pg_stat_plans/pg_stat_plans--1.0.sql b/contrib/pg_stat_plans/pg_stat_plans--1.0.sql
new file mode 100644
index 0000000000..f08452b274
--- /dev/null
+++ b/contrib/pg_stat_plans/pg_stat_plans--1.0.sql
@@ -0,0 +1,33 @@
+/* contrib/pg_stat_plans/pg_stat_plans--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION pg_stat_plans" to load this file. \quit
+
+-- Register functions.
+CREATE FUNCTION pg_stat_plans_reset()
+RETURNS void
+AS 'MODULE_PATHNAME'
+LANGUAGE C PARALLEL SAFE;
+
+CREATE FUNCTION pg_stat_plans(IN showplan boolean,
+ OUT userid oid,
+ OUT dbid oid,
+ OUT toplevel bool,
+ OUT queryid bigint,
+ OUT planid bigint,
+ OUT calls int8,
+ OUT total_exec_time float8,
+ OUT plan text
+)
+RETURNS SETOF record
+AS 'MODULE_PATHNAME', 'pg_stat_plans_1_0'
+LANGUAGE C STRICT VOLATILE PARALLEL SAFE;
+
+-- Register a view on the function for ease of use.
+CREATE VIEW pg_stat_plans AS
+ SELECT * FROM pg_stat_plans(true);
+
+GRANT SELECT ON pg_stat_plans TO PUBLIC;
+
+-- Don't want this to be available to non-superusers.
+REVOKE ALL ON FUNCTION pg_stat_plans_reset() FROM PUBLIC;
diff --git a/contrib/pg_stat_plans/pg_stat_plans.c b/contrib/pg_stat_plans/pg_stat_plans.c
new file mode 100644
index 0000000000..65da41eedb
--- /dev/null
+++ b/contrib/pg_stat_plans/pg_stat_plans.c
@@ -0,0 +1,779 @@
+/*--------------------------------------------------------------------------
+ *
+ * pg_stat_plans.c
+ * Track per-plan call counts, execution times and EXPLAIN texts
+ * across a whole database cluster.
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * contrib/pg_stat_plans/pg_stat_plans.c
+ *
+ * -------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/parallel.h"
+#include "catalog/pg_authid.h"
+#include "commands/explain.h"
+#include "common/hashfn.h"
+#include "funcapi.h"
+#include "mb/pg_wchar.h"
+#include "nodes/jumble.h"
+#include "pgstat.h"
+#include "optimizer/planner.h"
+#include "utils/acl.h"
+#include "utils/builtins.h"
+#include "utils/guc.h"
+#include "utils/pgstat_internal.h"
+#include "utils/snapmgr.h"
+
+PG_MODULE_MAGIC;
+
+/* Current nesting depth of planner/ExecutorRun/ProcessUtility calls */
+static int nesting_level = 0;
+
+/* Saved hook values */
+static planner_hook_type prev_planner_hook = NULL;
+static ExecutorStart_hook_type prev_ExecutorStart = NULL;
+static ExecutorRun_hook_type prev_ExecutorRun = NULL;
+static ExecutorFinish_hook_type prev_ExecutorFinish = NULL;
+static ExecutorEnd_hook_type prev_ExecutorEnd = NULL;
+
+/*---- GUC variables ----*/
+
+typedef enum
+{
+ PGSP_TRACK_NONE, /* track no plans */
+ PGSP_TRACK_TOP, /* only plans for top level statements */
+ PGSP_TRACK_ALL, /* all plans, including for nested statements */
+} PGSPTrackLevel;
+
+static const struct config_enum_entry track_options[] =
+{
+ {"none", PGSP_TRACK_NONE, false},
+ {"top", PGSP_TRACK_TOP, false},
+ {"all", PGSP_TRACK_ALL, false},
+ {NULL, 0, false}
+};
+
+static int pgsp_max = 5000; /* max # plans to track */
+static int pgsp_max_size = 2048; /* max size of plan text to track (in
+ * bytes) */
+static int pgsp_track = PGSP_TRACK_TOP; /* tracking level */
+
+#define pgsp_enabled(level) \
+ (!IsParallelWorker() && \
+ (compute_plan_id != COMPUTE_PLAN_ID_OFF) && \
+ (pgsp_track == PGSP_TRACK_ALL || \
+ (pgsp_track == PGSP_TRACK_TOP && (level) == 0)))
+
+#define USAGE_INCREASE 0.5 /* increase by this each time we report
+ * stats */
+#define USAGE_DECREASE_FACTOR (0.99) /* decreased every
+ * pgstat_dealloc_plans */
+#define USAGE_DEALLOC_PERCENT 5 /* free this % of entries at once */
+
+/*---- Function declarations ----*/
+
+PG_FUNCTION_INFO_V1(pg_stat_plans_reset);
+PG_FUNCTION_INFO_V1(pg_stat_plans_1_0);
+
+/* Structures for statistics of plans */
+typedef struct PgStatShared_PlanInfo
+{
+ /* key elements that identify a plan (together with the dboid) */
+ uint64 planid;
+ uint64 queryid;
+ Oid userid; /* userid is tracked to allow users to see
+ * their own query plans */
+ bool toplevel; /* query executed at top level */
+
+ dsa_pointer plan_text; /* pointer to DSA memory containing plan text */
+ int plan_encoding; /* plan text encoding */
+} PgStatShared_PlanInfo;
+
+typedef struct PgStat_StatPlanEntry
+{
+ PgStat_Counter exec_count;
+ double exec_time;
+ double usage; /* Usage factor of the entry, used to
+ * prioritize which plans to age out */
+
+ /* Only used in shared structure, not in local pending stats */
+ PgStatShared_PlanInfo info;
+} PgStat_StatPlanEntry;
+
+typedef struct PgStatShared_Plan
+{
+ PgStatShared_Common header;
+ PgStat_StatPlanEntry stats;
+} PgStatShared_Plan;
+
+static bool plan_stats_flush_cb(PgStat_EntryRef *entry_ref, bool nowait);
+
+static const PgStat_KindInfo plan_stats = {
+ .name = "plan_stats",
+ .fixed_amount = false,
+
+ /*
+ * We currently don't write to a file since plan texts would get lost (and
+ * just the stats on their own aren't that useful)
+ */
+ .write_to_file = false,
+
+ /*
+ * Plan statistics are available system-wide to simplify monitoring
+ * scripts
+ */
+ .accessed_across_databases = true,
+
+ .shared_size = sizeof(PgStatShared_Plan),
+ .shared_data_off = offsetof(PgStatShared_Plan, stats),
+ .shared_data_len = sizeof(((PgStatShared_Plan *) 0)->stats),
+ .pending_size = sizeof(PgStat_StatPlanEntry),
+ .flush_pending_cb = plan_stats_flush_cb,
+};
+
+/*
+ * Compute stats entry idx from query ID and plan ID with an 8-byte hash.
+ *
+ * Whilst we could theorically just use the plan ID here, we intentionally
+ * add the query ID into the mix to ease interpreting the data in combination
+ * with pg_stat_statements.
+ */
+#define PGSTAT_PLAN_IDX(query_id, plan_id, user_id, toplevel) hash_combine64(toplevel, hash_combine64(query_id, hash_combine64(plan_id, user_id)))
+
+/*
+ * Kind ID reserved for statistics of plans.
+ */
+#define PGSTAT_KIND_PLANS PGSTAT_KIND_EXPERIMENTAL /* TODO: Assign */
+
+/*
+ * Callback for stats handling
+ */
+static bool
+plan_stats_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
+{
+ PgStat_StatPlanEntry *localent;
+ PgStatShared_Plan *shfuncent;
+
+ localent = (PgStat_StatPlanEntry *) entry_ref->pending;
+ shfuncent = (PgStatShared_Plan *) entry_ref->shared_stats;
+
+ if (!pgstat_lock_entry(entry_ref, nowait))
+ return false;
+
+ shfuncent->stats.exec_count += localent->exec_count;
+ shfuncent->stats.exec_time += localent->exec_time;
+ shfuncent->stats.usage += localent->usage;
+
+ pgstat_unlock_entry(entry_ref);
+
+ return true;
+}
+
+static char *
+pgsp_explain_plan(QueryDesc *queryDesc)
+{
+ ExplainState *es;
+ StringInfo es_str;
+
+ es = NewExplainState();
+ es_str = es->str;
+
+ /*
+ * We turn off COSTS since identical planids may have very different
+ * costs, and it could be misleading to only show the first recorded
+ * plan's costs.
+ */
+ es->costs = false;
+ es->format = EXPLAIN_FORMAT_TEXT;
+
+ ExplainBeginOutput(es);
+ ExplainPrintPlan(es, queryDesc);
+ ExplainEndOutput(es);
+
+ return es_str->data;
+}
+
+static void
+pgstat_gc_plan_memory()
+{
+ dshash_seq_status hstat;
+ PgStatShared_HashEntry *p;
+
+ /* dshash entry is not modified, take shared lock */
+ dshash_seq_init(&hstat, pgStatLocal.shared_hash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ PgStatShared_Common *header;
+ PgStat_StatPlanEntry *statent;
+
+ if (!p->dropped || p->key.kind != PGSTAT_KIND_PLANS)
+ continue;
+
+ header = dsa_get_address(pgStatLocal.dsa, p->body);
+
+ if (!LWLockConditionalAcquire(&header->lock, LW_EXCLUSIVE))
+ continue;
+
+ statent = (PgStat_StatPlanEntry *) pgstat_get_entry_data(PGSTAT_KIND_PLANS, header);
+
+ /*
+ * Clean up this entry's plan text allocation, if we haven't done so
+ * already
+ */
+ if (DsaPointerIsValid(statent->info.plan_text))
+ {
+ dsa_free(pgStatLocal.dsa, statent->info.plan_text);
+ statent->info.plan_text = InvalidDsaPointer;
+
+ /* Allow removal of the shared stats entry */
+ pg_atomic_fetch_sub_u32(&p->refcount, 1);
+ }
+
+ LWLockRelease(&header->lock);
+ }
+ dshash_seq_term(&hstat);
+
+ /* Encourage other backends to clean up dropped entry refs */
+ pgstat_request_entry_refs_gc();
+}
+
+typedef struct PlanDeallocEntry
+{
+ PgStat_HashKey key;
+ double usage;
+} PlanDeallocEntry;
+
+/*
+ * list sort comparator for sorting into decreasing usage order
+ */
+static int
+entry_cmp_lru(const union ListCell *lhs, const union ListCell *rhs)
+{
+ double l_usage = ((PlanDeallocEntry *) lfirst(lhs))->usage;
+ double r_usage = ((PlanDeallocEntry *) lfirst(rhs))->usage;
+
+ if (l_usage > r_usage)
+ return -1;
+ else if (l_usage < r_usage)
+ return +1;
+ else
+ return 0;
+}
+
+static void
+pgstat_dealloc_plans()
+{
+ dshash_seq_status hstat;
+ PgStatShared_HashEntry *p;
+ List *entries = NIL;
+ ListCell *lc;
+ int nvictims;
+
+ /* dshash entry is not modified, take shared lock */
+ dshash_seq_init(&hstat, pgStatLocal.shared_hash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ PgStatShared_Common *header;
+ PgStat_StatPlanEntry *statent;
+ PlanDeallocEntry *entry;
+
+ if (p->dropped || p->key.kind != PGSTAT_KIND_PLANS)
+ continue;
+
+ header = dsa_get_address(pgStatLocal.dsa, p->body);
+
+ if (!LWLockConditionalAcquire(&header->lock, LW_EXCLUSIVE))
+ continue;
+
+ statent = (PgStat_StatPlanEntry *) pgstat_get_entry_data(PGSTAT_KIND_PLANS, header);
+ statent->usage *= USAGE_DECREASE_FACTOR;
+
+ entry = palloc(sizeof(PlanDeallocEntry));
+ entry->key = p->key;
+ entry->usage = statent->usage;
+
+ LWLockRelease(&header->lock);
+
+ entries = lappend(entries, entry);
+ }
+ dshash_seq_term(&hstat);
+
+ /* Sort by usage ascending (lowest used entries are last) */
+ list_sort(entries, entry_cmp_lru);
+
+ /* At a minimum, deallocate 10 entries to make it worth our while */
+ nvictims = Max(10, list_length(entries) * USAGE_DEALLOC_PERCENT / 100);
+ nvictims = Min(nvictims, list_length(entries));
+
+ /* Actually drop the entries */
+ for_each_from(lc, entries, list_length(entries) - nvictims)
+ {
+ PlanDeallocEntry *entry = lfirst(lc);
+
+ pgstat_drop_entry(entry->key.kind, entry->key.dboid, entry->key.objid);
+ }
+
+ /* Clean up our working memory immediately */
+ foreach(lc, entries)
+ {
+ PlanDeallocEntry *entry = lfirst(lc);
+
+ pfree(entry);
+ }
+ pfree(entries);
+}
+
+static void
+pgstat_gc_plans()
+{
+ dshash_seq_status hstat;
+ PgStatShared_HashEntry *p;
+ bool have_dropped_entries = false;
+ size_t plan_entry_count = 0;
+
+ /* TODO: Prevent concurrent GC cycles - flag an active GC run somehow */
+
+ /*
+ * Count our active entries, and whether there are any dropped entries we
+ * may need to clean up at the end.
+ */
+ dshash_seq_init(&hstat, pgStatLocal.shared_hash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ if (p->key.kind != PGSTAT_KIND_PLANS)
+ continue;
+
+ if (p->dropped)
+ have_dropped_entries = true;
+ else
+ plan_entry_count++;
+ }
+ dshash_seq_term(&hstat);
+
+ /*
+ * If we're over the limit, delete entries with lowest usage factor.
+ */
+ if (plan_entry_count > pgsp_max)
+ {
+ pgstat_dealloc_plans();
+ have_dropped_entries = true; /* Assume we did some work */
+ }
+
+ /* If there are dropped entries, clean up their plan memory if needed */
+ if (have_dropped_entries)
+ pgstat_gc_plan_memory();
+}
+
+static void
+pgstat_report_plan_stats(QueryDesc *queryDesc,
+ PgStat_Counter exec_count,
+ double exec_time)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_Plan *shstatent;
+ PgStat_StatPlanEntry *statent;
+ bool newly_created;
+ uint64 queryId = queryDesc->plannedstmt->queryId;
+ uint64 planId = queryDesc->plannedstmt->planId;
+ Oid userid = GetUserId();
+ bool toplevel = (nesting_level == 0);
+
+ entry_ref = pgstat_prep_pending_entry(PGSTAT_KIND_PLANS, MyDatabaseId,
+ PGSTAT_PLAN_IDX(queryId, planId, userid, toplevel), &newly_created);
+
+ shstatent = (PgStatShared_Plan *) entry_ref->shared_stats;
+ statent = &shstatent->stats;
+
+ if (newly_created)
+ {
+ char *plan = pgsp_explain_plan(queryDesc);
+ size_t plan_size = Min(strlen(plan), pgsp_max_size);
+
+ (void) pgstat_lock_entry(entry_ref, false);
+
+ /*
+ * We may be over the limit, so run GC now before saving entry (we do
+ * this whilst holding the lock on the new entry so we don't remove it
+ * by accident)
+ */
+ pgstat_gc_plans();
+
+ shstatent->stats.info.planid = planId;
+ shstatent->stats.info.queryid = queryId;
+ shstatent->stats.info.userid = userid;
+ shstatent->stats.info.toplevel = toplevel;
+ shstatent->stats.info.plan_text = dsa_allocate(pgStatLocal.dsa, plan_size);
+ strlcpy(dsa_get_address(pgStatLocal.dsa, shstatent->stats.info.plan_text), plan, plan_size);
+
+ shstatent->stats.info.plan_encoding = GetDatabaseEncoding();
+
+ /*
+ * Increase refcount here so entry can't get released without us
+ * dropping the plan text
+ */
+ pg_atomic_fetch_add_u32(&entry_ref->shared_entry->refcount, 1);
+
+ pgstat_unlock_entry(entry_ref);
+
+ pfree(plan);
+ }
+
+ statent->exec_count += exec_count;
+ statent->exec_time += exec_time;
+ statent->usage += USAGE_INCREASE;
+}
+
+/*
+ * Planner hook: forward to regular planner, but increase plan count and
+ * record query plan if needed.
+ */
+static PlannedStmt *
+pgsp_planner(Query *parse,
+ const char *query_string,
+ int cursorOptions,
+ ParamListInfo boundParams)
+{
+ PlannedStmt *result;
+
+ /*
+ * Increment the nesting level, to ensure that functions evaluated during
+ * planning are not seen as top-level calls.
+ */
+ nesting_level++;
+ PG_TRY();
+ {
+ if (prev_planner_hook)
+ result = prev_planner_hook(parse, query_string, cursorOptions,
+ boundParams);
+ else
+ result = standard_planner(parse, query_string, cursorOptions,
+ boundParams);
+ }
+ PG_FINALLY();
+ {
+ nesting_level--;
+ }
+ PG_END_TRY();
+
+ return result;
+}
+
+/*
+ * ExecutorStart hook: start up tracking if needed
+ */
+static void
+pgsp_ExecutorStart(QueryDesc *queryDesc, int eflags)
+{
+ uint64 queryId = queryDesc->plannedstmt->queryId;
+ uint64 planId = queryDesc->plannedstmt->planId;
+
+ if (prev_ExecutorStart)
+ prev_ExecutorStart(queryDesc, eflags);
+ else
+ standard_ExecutorStart(queryDesc, eflags);
+
+ if (queryId != UINT64CONST(0) && planId != UINT64CONST(0) &&
+ pgsp_enabled(nesting_level))
+ {
+ /*
+ * Record initial entry now, so plan text is available for currently
+ * running queries
+ */
+ pgstat_report_plan_stats(queryDesc,
+ 0, /* executions are counted in
+ * pgsp_ExecutorEnd */
+ 0.0);
+
+ /*
+ * Set up to track total elapsed time in ExecutorRun. Make sure the
+ * space is allocated in the per-query context so it will go away at
+ * ExecutorEnd.
+ */
+ if (queryDesc->totaltime == NULL)
+ {
+ MemoryContext oldcxt;
+
+ oldcxt = MemoryContextSwitchTo(queryDesc->estate->es_query_cxt);
+ queryDesc->totaltime = InstrAlloc(1, INSTRUMENT_ALL, false);
+ MemoryContextSwitchTo(oldcxt);
+ }
+ }
+}
+
+/*
+ * ExecutorRun hook: all we need do is track nesting depth
+ */
+static void
+pgsp_ExecutorRun(QueryDesc *queryDesc, ScanDirection direction, uint64 count)
+{
+ nesting_level++;
+ PG_TRY();
+ {
+ if (prev_ExecutorRun)
+ prev_ExecutorRun(queryDesc, direction, count);
+ else
+ standard_ExecutorRun(queryDesc, direction, count);
+ }
+ PG_FINALLY();
+ {
+ nesting_level--;
+ }
+ PG_END_TRY();
+}
+
+/*
+ * ExecutorFinish hook: all we need do is track nesting depth
+ */
+static void
+pgsp_ExecutorFinish(QueryDesc *queryDesc)
+{
+ nesting_level++;
+ PG_TRY();
+ {
+ if (prev_ExecutorFinish)
+ prev_ExecutorFinish(queryDesc);
+ else
+ standard_ExecutorFinish(queryDesc);
+ }
+ PG_FINALLY();
+ {
+ nesting_level--;
+ }
+ PG_END_TRY();
+}
+
+/*
+ * ExecutorEnd hook: store results if needed
+ */
+static void
+pgsp_ExecutorEnd(QueryDesc *queryDesc)
+{
+ uint64 queryId = queryDesc->plannedstmt->queryId;
+ uint64 planId = queryDesc->plannedstmt->planId;
+
+ if (queryId != UINT64CONST(0) && planId != UINT64CONST(0) &&
+ queryDesc->totaltime && pgsp_enabled(nesting_level))
+ {
+ /*
+ * Make sure stats accumulation is done. (Note: it's okay if several
+ * levels of hook all do this.)
+ */
+ InstrEndLoop(queryDesc->totaltime);
+
+ pgstat_report_plan_stats(queryDesc,
+ 1,
+ queryDesc->totaltime->total * 1000.0 /* convert to msec */ );
+ }
+
+ if (prev_ExecutorEnd)
+ prev_ExecutorEnd(queryDesc);
+ else
+ standard_ExecutorEnd(queryDesc);
+}
+
+/*
+ * Module load callback
+ */
+void
+_PG_init(void)
+{
+ /*
+ * In order to register for shared memory stats, we have to be loaded via
+ * shared_preload_libraries. If not, fall out without hooking into any of
+ * the main system. (We don't throw error here because it seems useful to
+ * allow the pg_stat_plans functions to be created even when the module
+ * isn't active. The functions must protect themselves against being
+ * called then, however.)
+ */
+ if (!process_shared_preload_libraries_in_progress)
+ return;
+
+ /*
+ * Inform the postmaster that we want to enable query_id calculation if
+ * compute_query_id is set to auto, as well as plan_id calculation if
+ * compute_plan_id is set to auto.
+ */
+ EnableQueryId();
+ EnablePlanId();
+
+ /*
+ * Define (or redefine) custom GUC variables.
+ */
+ DefineCustomIntVariable("pg_stat_plans.max",
+ "Sets the maximum number of plans tracked by pg_stat_plans in shared memory.",
+ NULL,
+ &pgsp_max,
+ 5000,
+ 100,
+ INT_MAX / 2,
+ PGC_SIGHUP,
+ 0,
+ NULL,
+ NULL,
+ NULL);
+
+ DefineCustomIntVariable("pg_stat_plans.max_size",
+ "Sets the maximum size of plan texts tracked by pg_stat_plans in shared memory.",
+ NULL,
+ &pgsp_max_size,
+ 2048,
+ 100,
+ 1048576, /* 1MB hard limit */
+ PGC_SUSET,
+ 0,
+ NULL,
+ NULL,
+ NULL);
+
+ DefineCustomEnumVariable("pg_stat_plans.track",
+ "Selects which plans are tracked by pg_stat_plans.",
+ NULL,
+ &pgsp_track,
+ PGSP_TRACK_TOP,
+ track_options,
+ PGC_SUSET,
+ 0,
+ NULL,
+ NULL,
+ NULL);
+
+ MarkGUCPrefixReserved("pg_stat_plans");
+
+ /*
+ * Install hooks.
+ */
+ prev_planner_hook = planner_hook;
+ planner_hook = pgsp_planner;
+ prev_ExecutorStart = ExecutorStart_hook;
+ ExecutorStart_hook = pgsp_ExecutorStart;
+ prev_ExecutorRun = ExecutorRun_hook;
+ ExecutorRun_hook = pgsp_ExecutorRun;
+ prev_ExecutorFinish = ExecutorFinish_hook;
+ ExecutorFinish_hook = pgsp_ExecutorFinish;
+ prev_ExecutorEnd = ExecutorEnd_hook;
+ ExecutorEnd_hook = pgsp_ExecutorEnd;
+
+ pgstat_register_kind(PGSTAT_KIND_PLANS, &plan_stats);
+}
+
+static bool
+match_plans_entries(PgStatShared_HashEntry *entry, Datum match_data)
+{
+ return entry->key.kind == PGSTAT_KIND_PLANS;
+}
+
+/*
+ * Reset statement statistics.
+ */
+Datum
+pg_stat_plans_reset(PG_FUNCTION_ARGS)
+{
+ pgstat_drop_matching_entries(match_plans_entries, 0);
+
+ /* Free plan text memory and allow cleanup of dropped entries */
+ pgstat_gc_plan_memory();
+
+ PG_RETURN_VOID();
+}
+
+#define PG_STAT_PLANS_COLS 8
+
+Datum
+pg_stat_plans_1_0(PG_FUNCTION_ARGS)
+{
+ bool showplan = PG_GETARG_BOOL(0);
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ Oid userid = GetUserId();
+ bool is_allowed_role = false;
+
+ dshash_seq_status hstat;
+ PgStatShared_HashEntry *p;
+
+ /*
+ * Superusers or roles with the privileges of pg_read_all_stats members
+ * are allowed
+ */
+ is_allowed_role = has_privs_of_role(userid, ROLE_PG_READ_ALL_STATS);
+
+ /* stats kind must be registered already */
+ if (!pgstat_get_kind_info(PGSTAT_KIND_PLANS))
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("pg_stat_plans must be loaded via \"shared_preload_libraries\"")));
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ /* dshash entry is not modified, take shared lock */
+ dshash_seq_init(&hstat, pgStatLocal.shared_hash, false);
+ while ((p = dshash_seq_next(&hstat)) != NULL)
+ {
+ PgStat_StatPlanEntry *statent;
+ Datum values[PG_STAT_PLANS_COLS];
+ bool nulls[PG_STAT_PLANS_COLS];
+ int i = 0;
+
+ if (p->dropped || p->key.kind != PGSTAT_KIND_PLANS)
+ continue;
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+
+ statent = pgstat_get_entry_data(p->key.kind, dsa_get_address(pgStatLocal.dsa, p->body));
+
+ values[i++] = ObjectIdGetDatum(statent->info.userid);
+ values[i++] = ObjectIdGetDatum(p->key.dboid);
+ values[i++] = BoolGetDatum(statent->info.toplevel);
+ if (is_allowed_role || statent->info.userid == userid)
+ {
+ int64 queryid = statent->info.queryid;
+ int64 planid = statent->info.planid;
+
+ values[i++] = Int64GetDatumFast(queryid);
+ values[i++] = Int64GetDatumFast(planid);
+ }
+ else
+ {
+ nulls[i++] = true;
+ nulls[i++] = true;
+ }
+ values[i++] = Int64GetDatumFast(statent->exec_count);
+ values[i++] = Float8GetDatumFast(statent->exec_time);
+
+ if (showplan && (is_allowed_role || statent->info.userid == userid))
+ {
+ char *pstr = DsaPointerIsValid(statent->info.plan_text) ? dsa_get_address(pgStatLocal.dsa, statent->info.plan_text) : NULL;
+
+ if (pstr)
+ {
+ char *enc = pg_any_to_server(pstr, strlen(pstr), statent->info.plan_encoding);
+
+ values[i++] = CStringGetTextDatum(enc);
+
+ if (enc != pstr)
+ pfree(enc);
+ }
+ else
+ {
+ nulls[i++] = true;
+ }
+ }
+ else if (showplan)
+ {
+ values[i++] = CStringGetTextDatum("<insufficient privilege>");
+ }
+ else
+ {
+ nulls[i++] = true;
+ }
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
+ }
+ dshash_seq_term(&hstat);
+
+ return (Datum) 0;
+}
diff --git a/contrib/pg_stat_plans/pg_stat_plans.conf b/contrib/pg_stat_plans/pg_stat_plans.conf
new file mode 100644
index 0000000000..6750b3e2cc
--- /dev/null
+++ b/contrib/pg_stat_plans/pg_stat_plans.conf
@@ -0,0 +1 @@
+shared_preload_libraries = 'pg_stat_plans'
diff --git a/contrib/pg_stat_plans/pg_stat_plans.control b/contrib/pg_stat_plans/pg_stat_plans.control
new file mode 100644
index 0000000000..4db3a47239
--- /dev/null
+++ b/contrib/pg_stat_plans/pg_stat_plans.control
@@ -0,0 +1,5 @@
+# pg_stat_plans extension
+comment = 'track per-plan call counts, execution times and EXPLAIN texts'
+default_version = '1.0'
+module_pathname = '$libdir/pg_stat_plans'
+relocatable = true
diff --git a/contrib/pg_stat_plans/sql/cleanup.sql b/contrib/pg_stat_plans/sql/cleanup.sql
new file mode 100644
index 0000000000..51565617ce
--- /dev/null
+++ b/contrib/pg_stat_plans/sql/cleanup.sql
@@ -0,0 +1 @@
+DROP EXTENSION pg_stat_plans;
diff --git a/contrib/pg_stat_plans/sql/privileges.sql b/contrib/pg_stat_plans/sql/privileges.sql
new file mode 100644
index 0000000000..aaad72a655
--- /dev/null
+++ b/contrib/pg_stat_plans/sql/privileges.sql
@@ -0,0 +1,59 @@
+--
+-- Only superusers and roles with privileges of the pg_read_all_stats role
+-- are allowed to see the plan text, queryid and planid of queries executed by
+-- other users. Other users can see the statistics.
+--
+
+CREATE ROLE regress_stats_superuser SUPERUSER;
+CREATE ROLE regress_stats_user1;
+CREATE ROLE regress_stats_user2;
+GRANT pg_read_all_stats TO regress_stats_user2;
+
+SET ROLE regress_stats_superuser;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+SELECT 1 AS "ONE";
+
+SET ROLE regress_stats_user1;
+SELECT 1+1 AS "TWO";
+
+--
+-- A superuser can read all columns of queries executed by others,
+-- including plan text, queryid and planid.
+--
+
+SET ROLE regress_stats_superuser;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+
+--
+-- regress_stats_user1 has no privileges to read the plan text, queryid
+-- or planid of queries executed by others but can see statistics
+-- like calls and rows.
+--
+
+SET ROLE regress_stats_user1;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+
+--
+-- regress_stats_user2, with pg_read_all_stats role privileges, can
+-- read all columns, including plan text, queryid and planid, of queries
+-- executed by others.
+--
+
+SET ROLE regress_stats_user2;
+SELECT r.rolname, ss.queryid <> 0 AS queryid_bool, ss.planid <> 0 AS planid_bool, ss.plan, ss.calls
+ FROM pg_stat_plans ss JOIN pg_roles r ON ss.userid = r.oid
+ ORDER BY r.rolname, ss.plan COLLATE "C", ss.calls;
+
+--
+-- cleanup
+--
+
+RESET ROLE;
+DROP ROLE regress_stats_superuser;
+DROP ROLE regress_stats_user1;
+DROP ROLE regress_stats_user2;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
diff --git a/contrib/pg_stat_plans/sql/select.sql b/contrib/pg_stat_plans/sql/select.sql
new file mode 100644
index 0000000000..f0e803ad70
--- /dev/null
+++ b/contrib/pg_stat_plans/sql/select.sql
@@ -0,0 +1,67 @@
+--
+-- SELECT statements
+--
+
+CREATE EXTENSION pg_stat_plans;
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+
+--
+-- simple statements
+--
+
+SELECT 1 FROM pg_class LIMIT 1;
+
+SELECT 1 FROM pg_class WHERE relname = 'pg_class';
+
+SET enable_indexscan = off;
+SELECT 1 FROM pg_class WHERE relname = 'pg_class';
+SET enable_indexscan = on;
+
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+
+--
+-- subplans and CTEs
+--
+
+WITH x AS MATERIALIZED (SELECT 1)
+SELECT * FROM x;
+
+SELECT a.attname,
+ (SELECT pg_catalog.pg_get_expr(d.adbin, d.adrelid)
+ FROM pg_catalog.pg_attrdef d
+ WHERE d.adrelid = a.attrelid AND d.adnum = a.attnum AND a.atthasdef)
+ FROM pg_catalog.pg_attribute a
+ WHERE a.attrelid = 'pg_class'::regclass
+ ORDER BY attnum LIMIT 1;
+
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
+
+--
+-- partitoning
+--
+
+create table lp (a char) partition by list (a);
+create table lp_default partition of lp default;
+create table lp_ef partition of lp for values in ('e', 'f');
+create table lp_ad partition of lp for values in ('a', 'd');
+create table lp_bc partition of lp for values in ('b', 'c');
+create table lp_g partition of lp for values in ('g');
+create table lp_null partition of lp for values in (null);
+
+select * from lp;
+select * from lp where a > 'a' and a < 'd';
+select * from lp where a > 'a' and a <= 'd';
+select * from lp where a = 'a';
+select * from lp where 'a' = a; /* commuted */
+select * from lp where a is not null;
+select * from lp where a is null;
+select * from lp where a = 'a' or a = 'c';
+select * from lp where a is not null and (a = 'a' or a = 'c');
+select * from lp where a <> 'g';
+select * from lp where a <> 'a' and a <> 'd';
+select * from lp where a not in ('a', 'd');
+
+SELECT plan, calls FROM pg_stat_plans ORDER BY plan COLLATE "C";
+SELECT pg_stat_plans_reset() IS NOT NULL AS t;
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index 7c381949a5..4a5a02c704 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -157,6 +157,7 @@ CREATE EXTENSION <replaceable>extension_name</replaceable>;
&pglogicalinspect;
&pgprewarm;
&pgrowlocks;
+ &pgstatplans;
&pgstatstatements;
&pgstattuple;
&pgsurgery;
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 66e6dccd4c..b0afb33ce2 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -146,6 +146,7 @@
<!ENTITY pglogicalinspect SYSTEM "pglogicalinspect.sgml">
<!ENTITY pgprewarm SYSTEM "pgprewarm.sgml">
<!ENTITY pgrowlocks SYSTEM "pgrowlocks.sgml">
+<!ENTITY pgstatplans SYSTEM "pgstatplans.sgml">
<!ENTITY pgstatstatements SYSTEM "pgstatstatements.sgml">
<!ENTITY pgstattuple SYSTEM "pgstattuple.sgml">
<!ENTITY pgsurgery SYSTEM "pgsurgery.sgml">
diff --git a/doc/src/sgml/pgstatplans.sgml b/doc/src/sgml/pgstatplans.sgml
new file mode 100644
index 0000000000..2d5faff15a
--- /dev/null
+++ b/doc/src/sgml/pgstatplans.sgml
@@ -0,0 +1,425 @@
+<!-- doc/src/sgml/pgstatplans.sgml -->
+
+<sect1 id="pgstatplans" xreflabel="pg_stat_plans">
+ <title>pg_stat_plans — track per-plan call counts, execution times and EXPLAIN texts</title>
+
+ <indexterm zone="pgstatplans">
+ <primary>pg_stat_plans</primary>
+ </indexterm>
+
+ <para>
+ The <filename>pg_stat_plans</filename> module provides a means for
+ tracking per-plan statistics and plan texts of all SQL statements executed by
+ a server.
+ </para>
+
+ <para>
+ The module must be loaded by adding <literal>pg_stat_plans</literal> to
+ <xref linkend="guc-shared-preload-libraries"/> in
+ <filename>postgresql.conf</filename>, because it requires additional shared memory.
+ This means that a server restart is needed to add or remove the module.
+ In addition, query and plan identifier calculation must be enabled in order for the
+ module to be active by setting both <xref linkend="guc-compute-plan-id"/> to
+ <literal>auto</literal> or <literal>on</literal> and
+ <xref linkend="guc-compute-query-id"/> to <literal>auto</literal> or <literal>on</literal>.
+ </para>
+
+ <para>
+ When <filename>pg_stat_plans</filename> is active, it tracks
+ statistics across all databases of the server. To access and manipulate
+ these statistics, the module provides the <structname>pg_stat_plans</structname>
+ view and the utility functions <function>pg_stat_plans_reset</function> and
+ <function>pg_stat_plans</function>. These are not available globally but
+ can be enabled for a specific database with
+ <command>CREATE EXTENSION pg_stat_plans</command>.
+ </para>
+
+ <sect2 id="pgstatplans-pg-stat-plans">
+ <title>The <structname>pg_stat_plans</structname> View</title>
+
+ <para>
+ The statistics gathered by the module are made available via a
+ view named <structname>pg_stat_plans</structname>. This view
+ contains one row for each distinct combination of database ID, user
+ ID, whether it's a top-level statement or not, query ID and plan ID
+ (up to the maximum number of distinct plans that the module can track).
+ The columns of the view are shown in <xref linkend="pgstatplans-columns"/>.
+ </para>
+
+ <table id="pgstatplans-columns">
+ <title><structname>pg_stat_plans</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>userid</structfield> <type>oid</type>
+ (references <link linkend="catalog-pg-authid"><structname>pg_authid</structname></link>.<structfield>oid</structfield>)
+ </para>
+ <para>
+ OID of user who executed the statement
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>dbid</structfield> <type>oid</type>
+ (references <link linkend="catalog-pg-database"><structname>pg_database</structname></link>.<structfield>oid</structfield>)
+ </para>
+ <para>
+ OID of database in which the statement was executed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>toplevel</structfield> <type>bool</type>
+ </para>
+ <para>
+ True if the query was executed as a top-level statement
+ (always true if <varname>pg_stat_plans.track</varname> is set to
+ <literal>top</literal>)
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>queryid</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Hash code to identify identical normalized queries.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>planid</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Hash code to identify identical plan shapes.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>calls</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Number of times the plan was executed
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>total_exec_time</structfield> <type>double precision</type>
+ </para>
+ <para>
+ Total time spent executing the plan, in milliseconds
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>plan</structfield> <type>text</type>
+ </para>
+ <para>
+ Plan text of a representative plan. This is similar to the output of
+ <literal>EXPLAIN (COSTS OFF)</literal>. Note the plan text will contain constant
+ values of the first plan recorded, but subsequent executions of the
+ same plan hash code (<structfield>planid</structfield>) with different
+ constant values will be tracked under the same entry.
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ For security reasons, only superusers and roles with privileges of the
+ <literal>pg_read_all_stats</literal> role are allowed to see the plan text,
+ <structfield>queryid</structfield> and <structfield>planid</structfield>
+ of queries executed by other users. Other users can see the statistics,
+ however, if the view has been installed in their database.
+ </para>
+
+ <para>
+ Plannable queries (that is, <command>SELECT</command>, <command>INSERT</command>,
+ <command>UPDATE</command>, <command>DELETE</command>, and <command>MERGE</command>)
+ will have the same <structfield>planid</structfield> whenever they have identical plan
+ structures according to an internal hash calculation. Typically, two plans will be
+ considered the same for this purpose if they have the same
+ <literal>EXPLAIN (COSTS OFF)</literal> output and are semantically equivalent except
+ for the values of literal constants appearing in the query plan.
+ </para>
+
+ <para>
+ Note that queries that have not finished executing yet will show in
+ <structname>pg_stat_plans</structname> with their plan text, but without
+ the <structname>calls</structname> field being incremented. This can be
+ used to identify the query plan for a currently running statement by joining
+ <link linkend="monitoring-pg-stat-activity-view">pg_stat_activity</link>
+ with <structname>pg_stat_plans</structname>, see example usage in
+ <xref linkend="pgstatplans-sample-output"/>.
+ </para>
+
+ <para>
+ Consumers of <structname>pg_stat_plans</structname> should use
+ <structfield>planid</structfield> in combination with
+ <structfield>queryid</structfield>, <structfield>dbid</structfield>,
+ <structfield>userid</structfield> and <structfield>toplevel</structfield>
+ as a stable and reliable identifier for each entry, instead of using its
+ plan text. However, it is important to understand that there are only limited
+ guarantees around the stability of the <structfield>planid</structfield>
+ hash value. Since the identifier is derived from the plan tree, its value
+ is a function of, among other things, the internal object identifiers
+ appearing in this representation. This has some counterintuitive implications.
+ For example, <filename>pg_stat_plans</filename> will consider two
+ apparently-identical plans to be distinct, if they reference a table
+ that was dropped and recreated between the creation of the two plans.
+ The hashing process is also sensitive to differences in
+ machine architecture and other facets of the platform.
+ Furthermore, it is not safe to assume that <structfield>planid</structfield>
+ will be stable across major versions of <productname>PostgreSQL</productname>.
+ </para>
+
+ <para>
+ Two servers participating in replication based on physical WAL replay can
+ be expected to have identical <structfield>planid</structfield> values for
+ the same plan. However, logical replication schemes do not promise to
+ keep replicas identical in all relevant details, so
+ <structfield>planid</structfield> will not be a useful identifier for
+ accumulating costs across a set of logical replicas.
+ If in doubt, direct testing is recommended.
+ </para>
+
+ <para>
+ Plan texts are stored in shared memory, and limited in length. To increase
+ the maximum length of stored plan texts you can increase
+ <varname>pg_stat_plans.max_size</varname>. This value can be changed for
+ an individual connection, or set as a server-wide setting.
+ </para>
+ </sect2>
+
+ <sect2 id="pgstatplans-funcs">
+ <title>Functions</title>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ <function>pg_stat_plans_reset() returns void</function>
+ <indexterm>
+ <primary>pg_stat_plans_reset</primary>
+ </indexterm>
+ </term>
+
+ <listitem>
+ <para>
+ <function>pg_stat_plans_reset</function> discards statistics and plan texts
+ gathered so far by <filename>pg_stat_plans</filename>.
+ By default, this function can only be executed by superusers.
+ Access may be granted to others using <command>GRANT</command>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <function>pg_stat_plans(showplan boolean) returns setof record</function>
+ <indexterm>
+ <primary>pg_stat_plans</primary>
+ <secondary>function</secondary>
+ </indexterm>
+ </term>
+
+ <listitem>
+ <para>
+ The <structname>pg_stat_plans</structname> view is defined in
+ terms of a function also named <function>pg_stat_plans</function>.
+ It is possible for clients to call
+ the <function>pg_stat_plans</function> function directly, and by
+ specifying <literal>showplan := false</literal> have plan texts be
+ omitted (that is, the <literal>OUT</literal> argument that corresponds
+ to the view's <structfield>plan</structfield> column will return nulls). This
+ feature is intended to support external tools that might wish to avoid
+ the overhead of repeatedly retrieving plan texts of indeterminate
+ length. Such tools can instead cache the first plan text observed
+ for each entry themselves, since that is
+ all <filename>pg_stat_plans</filename> itself does, and then retrieve
+ plan texts only as needed.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect2>
+
+ <sect2 id="pgstatplans-config-params">
+ <title>Configuration Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term>
+ <varname>pg_stat_plans.max</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>pg_stat_plans.max</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+
+ <listitem>
+ <para>
+ <varname>pg_stat_plans.max</varname> is the maximum number of
+ plans tracked by the module (i.e., the maximum number of rows
+ in the <structname>pg_stat_plans</structname> view). If more distinct
+ plans than that are observed, information about the least-executed
+ plans is discarded. The default value is 5000.
+ Only superusers can change this setting. Changing the setting requires
+ a reload of the server.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <varname>pg_stat_plans.max_size</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>pg_stat_plans.max_size</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+
+ <listitem>
+ <para>
+ <varname>pg_stat_plans.max_size</varname> is the maximum length of
+ each plan text tracked by the module in bytes. Longer plan texts will be truncated.
+ The default value is 2048 (2kB).
+ Only superusers can change this setting.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <varname>pg_stat_plans.track</varname> (<type>enum</type>)
+ <indexterm>
+ <primary><varname>pg_stat_plans.track</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+
+ <listitem>
+ <para>
+ <varname>pg_stat_plans.track</varname> controls which plans
+ are counted by the module.
+ Specify <literal>top</literal> to track plans by top-level statements (those issued
+ directly by clients), <literal>all</literal> to also track nested statements
+ (such as statements invoked within functions), or <literal>none</literal> to
+ disable plan statistics collection.
+ The default value is <literal>top</literal>.
+ Only superusers can change this setting.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ <para>
+ The module requires additional shared memory proportional to
+ <varname>pg_stat_plans.max</varname> for statistics, as well as
+ <varname>pg_stat_plans.max</varname> multiplied by
+ <varname>pg_stat_plans.max_size</varname> for plan texts. Note that this
+ memory is only consumed when entries are created, and not if
+ <varname>pg_stat_plans.track</varname> is set to <literal>none</literal>.
+ </para>
+
+ <para>
+ These parameters must be set in <filename>postgresql.conf</filename>.
+ Typical usage might be:
+
+<programlisting>
+# postgresql.conf
+shared_preload_libraries = 'pg_stat_plans'
+
+compute_query_id = on
+compute_plan_id = on
+pg_stat_plans.max = 10000
+pg_stat_plans.max_size = 4096
+pg_stat_plans.track = all
+</programlisting>
+ </para>
+ </sect2>
+
+ <sect2 id="pgstatplans-sample-output">
+ <title>Sample Output</title>
+
+<screen>
+bench=# SELECT pg_stat_plans_reset();
+
+$ pgbench -i bench
+$ pgbench -c10 -t300 bench
+
+bench=# \x
+bench=# SELECT plan, calls, total_exec_time
+ FROM pg_stat_plans ORDER BY total_exec_time DESC LIMIT 5;
+-[ RECORD 1 ]---+-----------------------------------------------------------------
+plan | Update on pgbench_tellers +
+ | -> Seq Scan on pgbench_tellers +
+ | Filter: (tid = 5)
+calls | 3000
+total_exec_time | 642.8880919999993
+-[ RECORD 2 ]---+-----------------------------------------------------------------
+plan | Update on pgbench_branches +
+ | -> Seq Scan on pgbench_branches +
+ | Filter: (bid = 1)
+calls | 1813
+total_exec_time | 476.64152700000005
+-[ RECORD 3 ]---+-----------------------------------------------------------------
+plan | Update on pgbench_branches +
+ | -> Index Scan using pgbench_branches_pkey on pgbench_branches+
+ | Index Cond: (bid = 1)
+calls | 1187
+total_exec_time | 326.1257549999999
+-[ RECORD 4 ]---+-----------------------------------------------------------------
+plan | Update on pgbench_accounts +
+ | -> Index Scan using pgbench_accounts_pkey on pgbench_accounts+
+ | Index Cond: (aid = 48793)
+calls | 3000
+total_exec_time | 21.664690000000093
+-[ RECORD 5 ]---+-----------------------------------------------------------------
+plan | Insert on pgbench_history +
+ | -> Result
+calls | 3000
+total_exec_time | 4.365250999999957
+
+session 1:
+
+bench# SELECT pg_sleep(100), COUNT(*) FROM pgbench_accounts;
+
+session 2:
+
+bench=# SELECT query, plan FROM pg_stat_activity
+ JOIN pg_stat_plans ON (usesysid = userid AND datid = dbid AND query_id = queryid AND plan_id = planid)
+ WHERE query LIKE 'SELECT pg_sleep%';
+ query | plan
+-------------------------------------------------------+------------------------------------
+ SELECT pg_sleep(100), COUNT(*) FROM pgbench_accounts; | Aggregate +
+ | -> Seq Scan on pgbench_accounts
+(1 row)
+
+</screen>
+ </sect2>
+
+ <sect2 id="pgstatplans-authors">
+ <title>Authors</title>
+
+ <para>
+ Lukas Fittl <email>lukas@fittl.com</email>.
+ </para>
+ </sect2>
+
+</sect1>
--
2.47.1
v5-0003-Optionally-record-a-plan_id-in-PlannedStmt-to-ide.patchapplication/octet-stream; name=v5-0003-Optionally-record-a-plan_id-in-PlannedStmt-to-ide.patchDownload
From 6d3e917b9fd348dfa3384be32ea1312c4a1c2642 Mon Sep 17 00:00:00 2001
From: Lukas Fittl <lukas@fittl.com>
Date: Fri, 7 Feb 2025 01:18:28 +0000
Subject: [PATCH v5 3/4] Optionally record a plan_id in PlannedStmt to identify
plan shape
When enabled via the new compute_plan_id GUC (default off), this utilizes
the existing treewalk in setrefs.c after planning to calculate a hash
(the "plan_id", or plan identifier) that can be used to identify
which plan was chosen.
The plan_id generally intends to be the same if a given EXPLAIN (without
ANALYZE) output is the same. The plan_id includes both the top-level plan
as well as all subplans. Execution statistics are excluded.
If enabled, the plan_id is shown for currently running queries in
pg_stat_activity, as well as recorded in EXPLAIN and auto_explain output.
Other in core users or extensions can use this facility to show or
accumulate statistics about the plans used by queries, to help identify
plan regressions, or drive plan management decisions.
Note that this commit intentionally does not include a facility to map
a given plan_id to the EXPLAIN text output - it is a assumed that users
can utilize the auto_explain extension to establish this mapping as
needed, or extensions can record this via the existing planner hook.
---
doc/src/sgml/config.sgml | 34 +++++++++
doc/src/sgml/monitoring.sgml | 16 +++++
src/backend/catalog/system_views.sql | 1 +
src/backend/commands/explain.c | 16 +++++
src/backend/executor/execMain.c | 10 +--
src/backend/executor/execParallel.c | 1 +
src/backend/nodes/gen_node_support.pl | 36 +++++++++-
src/backend/nodes/jumblefuncs.c | 65 +++++++++++++++++
src/backend/optimizer/plan/planner.c | 19 +++++
src/backend/optimizer/plan/setrefs.c | 9 +++
src/backend/postmaster/launch_backend.c | 3 +
src/backend/tcop/postgres.c | 1 +
src/backend/utils/activity/backend_status.c | 70 ++++++++++++++++++-
src/backend/utils/adt/pgstatfuncs.c | 7 +-
src/backend/utils/misc/guc_tables.c | 28 ++++++++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/catalog/pg_proc.dat | 6 +-
src/include/nodes/jumble.h | 28 ++++++++
src/include/nodes/pathnodes.h | 3 +
src/include/nodes/plannodes.h | 51 +++++++-------
src/include/nodes/primnodes.h | 7 +-
src/include/utils/backend_status.h | 5 ++
src/test/regress/expected/explain.out | 11 +++
src/test/regress/expected/rules.out | 9 +--
src/test/regress/sql/explain.sql | 4 ++
25 files changed, 396 insertions(+), 45 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 38244409e3..4c26f9eb13 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -8406,6 +8406,40 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-compute-plan-id" xreflabel="compute_plan_id">
+ <term><varname>compute_plan_id</varname> (<type>enum</type>)
+ <indexterm>
+ <primary><varname>compute_plan_id</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Enables in-core computation of a plan identifier.
+ Plan identifiers can be displayed in the <link
+ linkend="monitoring-pg-stat-activity-view"><structname>pg_stat_activity</structname></link>
+ view or using <command>EXPLAIN</command>.
+ Note that an external module can alternatively be used if the
+ in-core plan identifier computation method is not acceptable.
+ In this case, in-core computation must be always disabled.
+ Valid values are <literal>off</literal> (always disabled),
+ <literal>on</literal> (always enabled), <literal>auto</literal>,
+ which lets modules that utilize plan identifiers enable
+ it automatically, and <literal>regress</literal> which
+ has the same effect as <literal>on</literal>, except that the
+ query identifier is not shown in the <literal>EXPLAIN</literal> output
+ in order to facilitate automated regression testing.
+ The default is <literal>auto</literal>.
+ </para>
+ <note>
+ <para>
+ To ensure that only one plan identifier is calculated and
+ displayed, extensions that calculate plan identifiers should
+ throw an error if a plan identifier has already been computed.
+ </para>
+ </note>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-log-statement-stats">
<term><varname>log_statement_stats</varname> (<type>boolean</type>)
<indexterm>
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index edc2470bcf..64e62a0905 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -972,6 +972,22 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>plan_id</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Identifier of this backend's most recent query plan. If
+ <structfield>state</structfield> is <literal>active</literal> this
+ field shows the identifier of the currently executing query plan. In
+ all other states, it shows the identifier of last query plan that
+ was executed. Plan identifiers are not computed by default so this
+ field will be null unless <xref linkend="guc-compute-plan-id"/>
+ parameter is enabled or a third-party module that computes plan
+ identifiers is configured.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>query</structfield> <type>text</type>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index cddc3ea9b5..d3d5578d2c 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -889,6 +889,7 @@ CREATE VIEW pg_stat_activity AS
S.backend_xid,
s.backend_xmin,
S.query_id,
+ S.plan_id,
S.query,
S.backend_type
FROM pg_stat_get_activity(NULL) AS S
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index c24e66f82e..31a1761f53 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -966,6 +966,22 @@ ExplainPrintPlan(ExplainState *es, QueryDesc *queryDesc)
ExplainPropertyInteger("Query Identifier", NULL, (int64)
queryDesc->plannedstmt->queryId, es);
}
+
+ /*
+ * COMPUTE_PLAN_ID_REGRESS means COMPUTE_PLAN_ID_YES, but we don't show
+ * the queryid in any of the EXPLAIN plans to keep stable the results
+ * generated by regression test suites.
+ */
+ if (es->verbose && queryDesc->plannedstmt->planId != UINT64CONST(0) &&
+ compute_plan_id != COMPUTE_PLAN_ID_REGRESS)
+ {
+ /*
+ * Output the queryid as an int64 rather than a uint64 so we match
+ * what would be seen in the BIGINT pg_stat_activity.plan_id column.
+ */
+ ExplainPropertyInteger("Plan Identifier", NULL, (int64)
+ queryDesc->plannedstmt->planId, es);
+ }
}
/*
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index fa6002d5f9..920c63f4cf 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -121,13 +121,15 @@ ExecutorStart(QueryDesc *queryDesc, int eflags)
{
/*
* In some cases (e.g. an EXECUTE statement or an execute message with the
- * extended query protocol) the query_id won't be reported, so do it now.
+ * extended query protocol) the query_id and plan_id won't be reported, so
+ * do it now.
*
- * Note that it's harmless to report the query_id multiple times, as the
- * call will be ignored if the top level query_id has already been
- * reported.
+ * Note that it's harmless to report the identifiers multiple times, as
+ * the call will be ignored if the top level query_id / plan_id has
+ * already been reported.
*/
pgstat_report_query_id(queryDesc->plannedstmt->queryId, false);
+ pgstat_report_plan_id(queryDesc->plannedstmt->planId, queryDesc->plannedstmt->queryId, false);
if (ExecutorStart_hook)
(*ExecutorStart_hook) (queryDesc, eflags);
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 9c313d8131..da2279579a 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -174,6 +174,7 @@ ExecSerializePlan(Plan *plan, EState *estate)
pstmt = makeNode(PlannedStmt);
pstmt->commandType = CMD_SELECT;
pstmt->queryId = pgstat_get_my_query_id();
+ pstmt->planId = pgstat_get_my_plan_id();
pstmt->hasReturning = false;
pstmt->hasModifyingCTE = false;
pstmt->canSetTag = true;
diff --git a/src/backend/nodes/gen_node_support.pl b/src/backend/nodes/gen_node_support.pl
index 26ec7e0d59..2d79bdc4b3 100644
--- a/src/backend/nodes/gen_node_support.pl
+++ b/src/backend/nodes/gen_node_support.pl
@@ -1281,13 +1281,18 @@ _jumble${n}(JumbleState *jstate, Node *node)
{
my $t = $node_type_info{$n}->{field_types}{$f};
my @a = @{ $node_type_info{$n}->{field_attrs}{$f} };
+ my $array_size_field;
my $query_jumble_ignore = $struct_no_query_jumble;
my $query_jumble_location = 0;
# extract per-field attributes
foreach my $a (@a)
{
- if ($a eq 'query_jumble_ignore')
+ if ($a =~ /^array_size\(([\w.]+)\)$/)
+ {
+ $array_size_field = $1;
+ }
+ elsif ($a eq 'query_jumble_ignore')
{
$query_jumble_ignore = 1;
}
@@ -1297,8 +1302,15 @@ _jumble${n}(JumbleState *jstate, Node *node)
}
}
+ next if $query_jumble_ignore;
+
+ if ($t eq 'Bitmapset*')
+ {
+ print $jff "\tJUMBLE_BITMAPSET($f);\n"
+ unless $query_jumble_ignore;
+ }
# node type
- if (($t =~ /^(\w+)\*$/ or $t =~ /^struct\s+(\w+)\*$/)
+ elsif (($t =~ /^(\w+)\*$/ or $t =~ /^struct\s+(\w+)\*$/)
and elem $1, @node_types)
{
print $jff "\tJUMBLE_NODE($f);\n"
@@ -1318,6 +1330,26 @@ _jumble${n}(JumbleState *jstate, Node *node)
print $jff "\tJUMBLE_STRING($f);\n"
unless $query_jumble_ignore;
}
+ elsif ($t =~ /^(\w+)(\*|\[\w+\])$/ and elem $1, @scalar_types)
+ {
+ if (!defined $array_size_field)
+ {
+ die "no array size defined for $n.$f of type $t\n";
+ }
+ if ($node_type_info{$n}->{field_types}{$array_size_field} eq
+ 'List*')
+ {
+ print $jff
+ "\tJUMBLE_ARRAY($f, list_length(expr->$array_size_field));\n"
+ unless $query_jumble_ignore;
+ }
+ else
+ {
+ print $jff
+ "\tJUMBLE_ARRAY($f, expr->$array_size_field);\n"
+ unless $query_jumble_ignore;
+ }
+ }
else
{
print $jff "\tJUMBLE_FIELD($f);\n"
diff --git a/src/backend/nodes/jumblefuncs.c b/src/backend/nodes/jumblefuncs.c
index fcf34ae479..df1acbcb96 100644
--- a/src/backend/nodes/jumblefuncs.c
+++ b/src/backend/nodes/jumblefuncs.c
@@ -39,12 +39,14 @@
#include "common/hashfn.h"
#include "miscadmin.h"
#include "nodes/jumble.h"
+#include "parser/parsetree.h"
#include "parser/scansup.h"
#define JUMBLE_SIZE 1024 /* query serialization buffer size */
/* GUC parameters */
int compute_query_id = COMPUTE_QUERY_ID_AUTO;
+int compute_plan_id = COMPUTE_PLAN_ID_AUTO;
/*
* True when compute_query_id is ON or AUTO, and a module requests them.
@@ -55,6 +57,15 @@ int compute_query_id = COMPUTE_QUERY_ID_AUTO;
*/
bool query_id_enabled = false;
+/*
+ * True when compute_plan_id is ON or AUTO, and a module requests them.
+ *
+ * Note that IsPlanIdEnabled() should be used instead of checking
+ * plan_id_enabled or plan_query_id directly when we want to know
+ * whether plan identifiers are computed in the core or not.
+ */
+bool plan_id_enabled = false;
+
static void RecordConstLocation(JumbleState *jstate, int location);
static void _jumbleA_Const(JumbleState *jstate, Node *node);
static void _jumbleList(JumbleState *jstate, Node *node);
@@ -175,6 +186,19 @@ EnableQueryId(void)
query_id_enabled = true;
}
+/*
+ * Enables plan identifier computation.
+ *
+ * Third-party plugins can use this function to inform core that they require
+ * a query identifier to be computed.
+ */
+void
+EnablePlanId(void)
+{
+ if (compute_plan_id != COMPUTE_PLAN_ID_OFF)
+ plan_id_enabled = true;
+}
+
/*
* AppendJumble: Append a value that is substantive in a given query to
* the current jumble.
@@ -244,6 +268,13 @@ RecordConstLocation(JumbleState *jstate, int location)
RecordConstLocation(jstate, expr->location)
#define JUMBLE_FIELD(item) \
AppendJumble(jstate, (const unsigned char *) &(expr->item), sizeof(expr->item))
+#define JUMBLE_BITMAPSET(item) \
+do { \
+ if (expr->item) \
+ AppendJumble(jstate, (const unsigned char *) expr->item->words, sizeof(bitmapword) * expr->item->nwords); \
+} while(0)
+#define JUMBLE_ARRAY(item, len) \
+ AppendJumble(jstate, (const unsigned char *) expr->item, sizeof(*(expr->item)) * len)
#define JUMBLE_FIELD_SINGLE(item) \
AppendJumble(jstate, (const unsigned char *) &(item), sizeof(item))
#define JUMBLE_STRING(str) \
@@ -392,3 +423,37 @@ _jumbleVariableSetStmt(JumbleState *jstate, Node *node)
JUMBLE_FIELD(is_local);
JUMBLE_LOCATION(location);
}
+
+/*
+ * Jumble the entries in the rangle table to map RT indexes to relations
+ *
+ * This ensures jumbled RT indexes (e.g. in a Scan or Modify node), are
+ * distinguished by the target of the RT entry, even if the index is the same.
+ */
+void
+JumbleRangeTable(JumbleState *jstate, List *rtable)
+{
+ ListCell *lc;
+
+ foreach(lc, rtable)
+ {
+ RangeTblEntry *expr = lfirst_node(RangeTblEntry, lc);
+
+ switch (expr->rtekind)
+ {
+ case RTE_RELATION:
+ JUMBLE_FIELD(relid);
+ break;
+ case RTE_CTE:
+ JUMBLE_STRING(ctename);
+ break;
+ default:
+
+ /*
+ * Ignore other targets, the jumble includes something identifying
+ * about them already
+ */
+ break;
+ }
+ }
+}
\ No newline at end of file
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index ffd7517ea9..a6ff95b43c 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -37,6 +37,7 @@
#ifdef OPTIMIZER_DEBUG
#include "nodes/print.h"
#endif
+#include "nodes/jumble.h"
#include "nodes/supportnodes.h"
#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
@@ -532,6 +533,16 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions,
Assert(glob->finalrowmarks == NIL);
Assert(glob->resultRelations == NIL);
Assert(glob->appendRelations == NIL);
+
+ /*
+ * Initialize plan identifier jumble if needed
+ *
+ * Note the actual jumbling is done in the tree walk in
+ * set_plan_references
+ */
+ if (IsPlanIdEnabled())
+ glob->plan_jumble_state = InitializeJumbleState(false);
+
top_plan = set_plan_references(root, top_plan);
/* ... and the subplans (both regular subplans and initplans) */
Assert(list_length(glob->subplans) == list_length(glob->subroots));
@@ -596,6 +607,14 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions,
result->jitFlags |= PGJIT_DEFORM;
}
+ if (IsPlanIdEnabled())
+ {
+ JumbleRangeTable(glob->plan_jumble_state, glob->finalrtable);
+ result->planId = HashJumbleState(glob->plan_jumble_state);
+ pfree(glob->plan_jumble_state->jumble);
+ pfree(glob->plan_jumble_state);
+ }
+
if (glob->partition_directory != NULL)
DestroyPartitionDirectory(glob->partition_directory);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 0868249be9..c5434dd316 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -19,6 +19,7 @@
#include "catalog/pg_type.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
+#include "nodes/jumble.h"
#include "optimizer/optimizer.h"
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
@@ -1306,6 +1307,14 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset)
plan->lefttree = set_plan_refs(root, plan->lefttree, rtoffset);
plan->righttree = set_plan_refs(root, plan->righttree, rtoffset);
+ /*
+ * If enabled, append significant information to the plan identifier
+ * jumble (we do this here since we're already walking the tree in a
+ * near-final state)
+ */
+ if (IsPlanIdEnabled())
+ JumbleNode(root->glob->plan_jumble_state, (Node *) plan);
+
return plan;
}
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index b127b1bef3..3d34784c04 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -115,6 +115,7 @@ typedef struct
bool redirection_done;
bool IsBinaryUpgrade;
bool query_id_enabled;
+ bool plan_id_enabled;
int max_safe_fds;
int MaxBackends;
int num_pmchild_slots;
@@ -744,6 +745,7 @@ save_backend_variables(BackendParameters *param,
param->redirection_done = redirection_done;
param->IsBinaryUpgrade = IsBinaryUpgrade;
param->query_id_enabled = query_id_enabled;
+ param->plan_id_enabled = plan_id_enabled;
param->max_safe_fds = max_safe_fds;
param->MaxBackends = MaxBackends;
@@ -1004,6 +1006,7 @@ restore_backend_variables(BackendParameters *param)
redirection_done = param->redirection_done;
IsBinaryUpgrade = param->IsBinaryUpgrade;
query_id_enabled = param->query_id_enabled;
+ plan_id_enabled = param->plan_id_enabled;
max_safe_fds = param->max_safe_fds;
MaxBackends = param->MaxBackends;
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 5655348a2e..6d8947bae9 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -1106,6 +1106,7 @@ exec_simple_query(const char *query_string)
size_t cmdtaglen;
pgstat_report_query_id(0, true);
+ pgstat_report_plan_id(0, 0, true);
/*
* Get the command name for use in status display (it also becomes the
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index 731342799a..1dfb7a58f8 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -379,6 +379,7 @@ pgstat_bestart(void)
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = UINT64CONST(0);
+ lbeentry.st_plan_id = UINT64CONST(0);
/*
* we don't zero st_progress_param here to save cycles; nobody should
@@ -533,6 +534,7 @@ pgstat_report_activity(BackendState state, const char *cmd_str)
/* st_xact_start_timestamp and wait_event_info are also disabled */
beentry->st_xact_start_timestamp = 0;
beentry->st_query_id = UINT64CONST(0);
+ beentry->st_plan_id = UINT64CONST(0);
proc->wait_event_info = 0;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -588,12 +590,15 @@ pgstat_report_activity(BackendState state, const char *cmd_str)
beentry->st_state_start_timestamp = current_timestamp;
/*
- * If a new query is started, we reset the query identifier as it'll only
- * be known after parse analysis, to avoid reporting last query's
- * identifier.
+ * If a new query is started, we reset the query and plan identifier as
+ * it'll only be known after parse analysis / planning, to avoid reporting
+ * last query's identifiers.
*/
if (state == STATE_RUNNING)
+ {
beentry->st_query_id = UINT64CONST(0);
+ beentry->st_plan_id = UINT64CONST(0);
+ }
if (cmd_str != NULL)
{
@@ -644,6 +649,45 @@ pgstat_report_query_id(uint64 query_id, bool force)
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
+/* --------
+ * pgstat_report_plan_id() -
+ *
+ * Called to update top-level plan identifier.
+ * --------
+ */
+void
+pgstat_report_plan_id(uint64 plan_id, uint64 query_id, bool force)
+{
+ volatile PgBackendStatus *beentry = MyBEEntry;
+
+ /*
+ * if track_activities is disabled, st_plan_id should already have been
+ * reset
+ */
+ if (!beentry || !pgstat_track_activities)
+ return;
+
+ /*
+ * We only report the top-level plan identifiers. The stored plan_id is
+ * reset when a backend calls pgstat_report_activity(STATE_RUNNING), or
+ * with an explicit call to this function using the force flag. If the
+ * saved plan identifier is not zero or the query identifier is 0, it
+ * means that it's not a top-level command, so ignore the one provided
+ * unless it's an explicit call to reset the identifier.
+ */
+ if ((beentry->st_plan_id != 0 || query_id == 0) && !force)
+ return;
+
+ /*
+ * Update my status entry, following the protocol of bumping
+ * st_changecount before and after. We use a volatile pointer here to
+ * ensure the compiler doesn't try to get cute.
+ */
+ PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
+ beentry->st_plan_id = plan_id;
+ PGSTAT_END_WRITE_ACTIVITY(beentry);
+}
+
/* ----------
* pgstat_report_appname() -
@@ -1040,6 +1084,26 @@ pgstat_get_my_query_id(void)
return MyBEEntry->st_query_id;
}
+/* ----------
+ * pgstat_get_my_plan_id() -
+ *
+ * Return current backend's plan identifier.
+ */
+uint64
+pgstat_get_my_plan_id(void)
+{
+ if (!MyBEEntry)
+ return 0;
+
+ /*
+ * There's no need for a lock around pgstat_begin_read_activity /
+ * pgstat_end_read_activity here as it's only called from
+ * pg_stat_get_activity which is already protected, or from the same
+ * backend which means that there won't be concurrent writes.
+ */
+ return MyBEEntry->st_plan_id;
+}
+
/* ----------
* pgstat_get_backend_type_by_proc_number() -
*
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index e9096a8849..a3d5592beb 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -330,7 +330,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum
pg_stat_get_activity(PG_FUNCTION_ARGS)
{
-#define PG_STAT_GET_ACTIVITY_COLS 31
+#define PG_STAT_GET_ACTIVITY_COLS 32
int num_backends = pgstat_fetch_stat_numbackends();
int curr_backend;
int pid = PG_ARGISNULL(0) ? -1 : PG_GETARG_INT32(0);
@@ -641,6 +641,10 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
nulls[30] = true;
else
values[30] = UInt64GetDatum(beentry->st_query_id);
+ if (beentry->st_plan_id == 0)
+ nulls[31] = true;
+ else
+ values[31] = UInt64GetDatum(beentry->st_plan_id);
}
else
{
@@ -670,6 +674,7 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
nulls[28] = true;
nulls[29] = true;
nulls[30] = true;
+ nulls[31] = true;
}
tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 9a22512fef..8abb9bead7 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -309,6 +309,24 @@ static const struct config_enum_entry compute_query_id_options[] = {
{NULL, 0, false}
};
+/*
+ * Although only "on" and "off" are documented, we accept
+ * all the likely variants of "on" and "off".
+ */
+static const struct config_enum_entry compute_plan_id_options[] = {
+ {"auto", COMPUTE_PLAN_ID_AUTO, false},
+ {"regress", COMPUTE_PLAN_ID_REGRESS, false},
+ {"on", COMPUTE_PLAN_ID_ON, false},
+ {"off", COMPUTE_PLAN_ID_OFF, false},
+ {"true", COMPUTE_PLAN_ID_ON, true},
+ {"false", COMPUTE_PLAN_ID_OFF, true},
+ {"yes", COMPUTE_PLAN_ID_ON, true},
+ {"no", COMPUTE_PLAN_ID_OFF, true},
+ {"1", COMPUTE_PLAN_ID_ON, true},
+ {"0", COMPUTE_PLAN_ID_OFF, true},
+ {NULL, 0, false}
+};
+
/*
* Although only "on", "off", and "partition" are documented, we
* accept all the likely variants of "on" and "off".
@@ -4882,6 +4900,16 @@ struct config_enum ConfigureNamesEnum[] =
NULL, NULL, NULL
},
+ {
+ {"compute_plan_id", PGC_SUSET, STATS_MONITORING,
+ gettext_noop("Enables in-core computation of plan identifiers."),
+ NULL
+ },
+ &compute_plan_id,
+ COMPUTE_PLAN_ID_AUTO, compute_plan_id_options,
+ NULL, NULL, NULL
+ },
+
{
{"constraint_exclusion", PGC_USERSET, QUERY_TUNING_OTHER,
gettext_noop("Enables the planner to use constraints to optimize queries."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index c40b7a3121..8b7e802b99 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -641,6 +641,7 @@
# - Monitoring -
#compute_query_id = auto
+#compute_plan_id = auto
#log_statement_stats = off
#log_parser_stats = off
#log_planner_stats = off
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 5b8c2ad2a5..5547db7d2f 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5568,9 +5568,9 @@
proname => 'pg_stat_get_activity', prorows => '100', proisstrict => 'f',
proretset => 't', provolatile => 's', proparallel => 'r',
prorettype => 'record', proargtypes => 'int4',
- proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,bool,int4,int8}',
- proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,gss_delegation,leader_pid,query_id}',
+ proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,bool,int4,int8,int8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,gss_delegation,leader_pid,query_id,plan_id}',
prosrc => 'pg_stat_get_activity' },
{ oid => '6318', descr => 'describe wait events',
proname => 'pg_get_wait_events', procost => '10', prorows => '250',
diff --git a/src/include/nodes/jumble.h b/src/include/nodes/jumble.h
index 7587c9f708..56c65310dd 100644
--- a/src/include/nodes/jumble.h
+++ b/src/include/nodes/jumble.h
@@ -15,6 +15,7 @@
#define JUMBLE_H
#include "nodes/parsenodes.h"
+#include "nodes/pathnodes.h"
/*
* Struct for tracking locations/lengths of constants during normalization
@@ -59,15 +60,27 @@ enum ComputeQueryIdType
COMPUTE_QUERY_ID_REGRESS,
};
+/* Values for the compute_plan_id GUC */
+enum ComputePlanIdType
+{
+ COMPUTE_PLAN_ID_OFF,
+ COMPUTE_PLAN_ID_ON,
+ COMPUTE_PLAN_ID_AUTO,
+ COMPUTE_PLAN_ID_REGRESS,
+};
+
/* GUC parameters */
extern PGDLLIMPORT int compute_query_id;
+extern PGDLLIMPORT int compute_plan_id;
extern const char *CleanQuerytext(const char *query, int *location, int *len);
extern JumbleState *JumbleQuery(Query *query);
extern void EnableQueryId(void);
+extern void EnablePlanId(void);
extern PGDLLIMPORT bool query_id_enabled;
+extern PGDLLIMPORT bool plan_id_enabled;
/*
* Returns whether query identifier computation has been enabled, either
@@ -83,10 +96,25 @@ IsQueryIdEnabled(void)
return query_id_enabled;
}
+/*
+ * Returns whether plan identifier computation has been enabled, either
+ * directly in the GUC or by a module when the setting is 'auto'.
+ */
+static inline bool
+IsPlanIdEnabled(void)
+{
+ if (compute_plan_id == COMPUTE_PLAN_ID_OFF)
+ return false;
+ if (compute_plan_id == COMPUTE_PLAN_ID_ON)
+ return true;
+ return plan_id_enabled;
+}
+
/* Functions called for plan jumbling or extensions doing their own jumbling */
extern JumbleState *InitializeJumbleState(bool record_clocations);
extern void AppendJumble(JumbleState *jstate, const unsigned char *item, Size size);
extern void JumbleNode(JumbleState *jstate, Node *node);
+extern void JumbleRangeTable(JumbleState *jstate, List *rtable);
extern uint64 HashJumbleState(JumbleState *jstate);
#endif /* JUMBLE_H */
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 52d44f4302..4baf829f64 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -166,6 +166,9 @@ typedef struct PlannerGlobal
/* partition descriptors */
PartitionDirectory partition_directory pg_node_attr(read_write_ignore);
+
+ /* optional jumble state for plan identifier calculation */
+ struct JumbleState *plan_jumble_state pg_node_attr(read_write_ignore);
} PlannerGlobal;
/* macro for fetching the Plan associated with a SubPlan node */
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 1e082bfdcd..6a9795f6a2 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -55,6 +55,9 @@ typedef struct PlannedStmt
/* query identifier (copied from Query) */
uint64 queryId;
+ /* plan identifier (set when compute_plan_id is enabled or by plugins) */
+ uint64 planId;
+
/* is it insert|update|delete|merge RETURNING? */
bool hasReturning;
@@ -142,7 +145,7 @@ typedef struct PlannedStmt
*/
typedef struct Plan
{
- pg_node_attr(abstract, no_equal, no_query_jumble)
+ pg_node_attr(abstract, no_equal)
NodeTag type;
@@ -150,19 +153,19 @@ typedef struct Plan
* estimated execution costs for plan (see costsize.c for more info)
*/
/* count of disabled nodes */
- int disabled_nodes;
+ int disabled_nodes pg_node_attr(query_jumble_ignore);
/* cost expended before fetching any tuples */
- Cost startup_cost;
+ Cost startup_cost pg_node_attr(query_jumble_ignore);
/* total cost (assuming all tuples fetched) */
- Cost total_cost;
+ Cost total_cost pg_node_attr(query_jumble_ignore);
/*
* planner's estimate of result size of this plan step
*/
/* number of rows plan is expected to emit */
- Cardinality plan_rows;
+ Cardinality plan_rows pg_node_attr(query_jumble_ignore);
/* average row width in bytes */
- int plan_width;
+ int plan_width pg_node_attr(query_jumble_ignore);
/*
* information needed for parallel query
@@ -188,10 +191,10 @@ typedef struct Plan
/* implicitly-ANDed qual conditions */
List *qual;
/* input plan tree(s) */
- struct Plan *lefttree;
- struct Plan *righttree;
+ struct Plan *lefttree pg_node_attr(query_jumble_ignore);
+ struct Plan *righttree pg_node_attr(query_jumble_ignore);
/* Init Plan nodes (un-correlated expr subselects) */
- List *initPlan;
+ List *initPlan pg_node_attr(query_jumble_ignore);
/*
* Information for management of parameter-change-driven rescanning
@@ -289,7 +292,7 @@ typedef struct ModifyTable
/* per-target-table RETURNING tlists */
List *returningLists;
/* per-target-table FDW private data lists */
- List *fdwPrivLists;
+ List *fdwPrivLists pg_node_attr(query_jumble_ignore);
/* indices of FDW DM plans */
Bitmapset *fdwDirectModifyPlans;
/* PlanRowMarks (non-locking only) */
@@ -328,7 +331,7 @@ typedef struct Append
Plan plan;
/* RTIs of appendrel(s) formed by this node */
Bitmapset *apprelids;
- List *appendplans;
+ List *appendplans pg_node_attr(query_jumble_ignore);
/* # of asynchronous plans */
int nasyncplans;
@@ -358,7 +361,7 @@ typedef struct MergeAppend
/* RTIs of appendrel(s) formed by this node */
Bitmapset *apprelids;
- List *mergeplans;
+ List *mergeplans pg_node_attr(query_jumble_ignore);
/* these fields are just like the sort-key info in struct Sort: */
@@ -413,7 +416,7 @@ typedef struct RecursiveUnion
Oid *dupCollations pg_node_attr(array_size(numCols));
/* estimated number of groups in input */
- long numGroups;
+ long numGroups pg_node_attr(query_jumble_ignore);
} RecursiveUnion;
/* ----------------
@@ -427,7 +430,7 @@ typedef struct RecursiveUnion
typedef struct BitmapAnd
{
Plan plan;
- List *bitmapplans;
+ List *bitmapplans pg_node_attr(query_jumble_ignore);
} BitmapAnd;
/* ----------------
@@ -442,7 +445,7 @@ typedef struct BitmapOr
{
Plan plan;
bool isshared;
- List *bitmapplans;
+ List *bitmapplans pg_node_attr(query_jumble_ignore);
} BitmapOr;
/*
@@ -690,7 +693,7 @@ typedef enum SubqueryScanStatus
typedef struct SubqueryScan
{
Scan scan;
- Plan *subplan;
+ Plan *subplan pg_node_attr(query_jumble_ignore);
SubqueryScanStatus scanstatus;
} SubqueryScan;
@@ -818,7 +821,7 @@ typedef struct ForeignScan
/* expressions that FDW may evaluate */
List *fdw_exprs;
/* private data for FDW */
- List *fdw_private;
+ List *fdw_private pg_node_attr(query_jumble_ignore);
/* optional tlist describing scan tuple */
List *fdw_scan_tlist;
/* original quals not in scan.plan.qual */
@@ -856,7 +859,7 @@ typedef struct CustomScan
/* expressions that custom code may evaluate */
List *custom_exprs;
/* private data for custom code */
- List *custom_private;
+ List *custom_private pg_node_attr(query_jumble_ignore);
/* optional tlist describing scan tuple */
List *custom_scan_tlist;
/* RTIs generated by this scan */
@@ -867,7 +870,7 @@ typedef struct CustomScan
* static table of callback functions. So we don't copy the table itself,
* just reference the original one.
*/
- const struct CustomScanMethods *methods;
+ const struct CustomScanMethods *methods pg_node_attr(query_jumble_ignore);
} CustomScan;
/*
@@ -929,7 +932,7 @@ typedef struct NestLoop
typedef struct NestLoopParam
{
- pg_node_attr(no_equal, no_query_jumble)
+ pg_node_attr(no_equal)
NodeTag type;
/* number of the PARAM_EXEC Param to set */
@@ -1037,7 +1040,7 @@ typedef struct Memoize
* The maximum number of entries that the planner expects will fit in the
* cache, or 0 if unknown
*/
- uint32 est_entries;
+ uint32 est_entries pg_node_attr(query_jumble_ignore);
/* paramids from param_exprs */
Bitmapset *keyparamids;
@@ -1134,7 +1137,7 @@ typedef struct Agg
Oid *grpCollations pg_node_attr(array_size(numCols));
/* estimated number of groups in input */
- long numGroups;
+ long numGroups pg_node_attr(query_jumble_ignore);
/* for pass-by-ref transition data */
uint64 transitionSpace;
@@ -1339,7 +1342,7 @@ typedef struct Hash
bool skewInherit;
/* all other info is in the parent HashJoin node */
/* estimate total rows if parallel_aware */
- Cardinality rows_total;
+ Cardinality rows_total pg_node_attr(query_jumble_ignore);
} Hash;
/* ----------------
@@ -1370,7 +1373,7 @@ typedef struct SetOp
bool *cmpNullsFirst pg_node_attr(array_size(numCols));
/* estimated number of groups in left input */
- long numGroups;
+ long numGroups pg_node_attr(query_jumble_ignore);
} SetOp;
/* ----------------
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
index 839e71d52f..85702f7507 100644
--- a/src/include/nodes/primnodes.h
+++ b/src/include/nodes/primnodes.h
@@ -1075,8 +1075,6 @@ typedef struct SubLink
*/
typedef struct SubPlan
{
- pg_node_attr(no_query_jumble)
-
Expr xpr;
/* Fields copied from original SubLink: */
SubLinkType subLinkType; /* see above */
@@ -1107,8 +1105,9 @@ typedef struct SubPlan
List *parParam; /* indices of input Params from parent plan */
List *args; /* exprs to pass as parParam values */
/* Estimated execution costs: */
- Cost startup_cost; /* one-time setup cost */
- Cost per_call_cost; /* cost for each subplan evaluation */
+ Cost startup_cost pg_node_attr(query_jumble_ignore); /* one-time setup cost */
+ Cost per_call_cost pg_node_attr(query_jumble_ignore); /* cost for each subplan
+ * evaluation */
} SubPlan;
/*
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index d3d4ff6c5c..437a4cec5b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -170,6 +170,9 @@ typedef struct PgBackendStatus
/* query identifier, optionally computed using post_parse_analyze_hook */
uint64 st_query_id;
+
+ /* plan identifier, optionally computed after planning */
+ uint64 st_plan_id;
} PgBackendStatus;
@@ -316,6 +319,7 @@ extern void pgstat_clear_backend_activity_snapshot(void);
/* Activity reporting functions */
extern void pgstat_report_activity(BackendState state, const char *cmd_str);
extern void pgstat_report_query_id(uint64 query_id, bool force);
+extern void pgstat_report_plan_id(uint64 query_id, uint64 plan_id, bool force);
extern void pgstat_report_tempfile(size_t filesize);
extern void pgstat_report_appname(const char *appname);
extern void pgstat_report_xact_timestamp(TimestampTz tstamp);
@@ -323,6 +327,7 @@ extern const char *pgstat_get_backend_current_activity(int pid, bool checkUser);
extern const char *pgstat_get_crashed_backend_activity(int pid, char *buffer,
int buflen);
extern uint64 pgstat_get_my_query_id(void);
+extern uint64 pgstat_get_my_plan_id(void);
extern BackendType pgstat_get_backend_type_by_proc_number(ProcNumber procNumber);
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index ee31e41d50..8bfa3c1a5f 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -699,6 +699,17 @@ select explain_filter('explain (verbose) create table test_ctas as select 1');
Query Identifier: N
(3 rows)
+-- Test compute_plan_id
+set compute_plan_id = on;
+select explain_filter('explain (verbose) select * from int8_tbl i8');
+ explain_filter
+----------------------------------------------------------------
+ Seq Scan on public.int8_tbl i8 (cost=N.N..N.N rows=N width=N)
+ Output: q1, q2
+ Query Identifier: N
+ Plan Identifier: N
+(4 rows)
+
-- Test SERIALIZE option
select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
explain_filter
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 3361f6a69c..6778b79e65 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1760,9 +1760,10 @@ pg_stat_activity| SELECT s.datid,
s.backend_xid,
s.backend_xmin,
s.query_id,
+ s.plan_id,
s.query,
s.backend_type
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
LEFT JOIN pg_database d ON ((s.datid = d.oid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_all_indexes| SELECT c.oid AS relid,
@@ -1890,7 +1891,7 @@ pg_stat_gssapi| SELECT pid,
gss_princ AS principal,
gss_enc AS encrypted,
gss_delegation AS credentials_delegated
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
WHERE (client_port IS NOT NULL);
pg_stat_io| SELECT backend_type,
object,
@@ -2096,7 +2097,7 @@ pg_stat_replication| SELECT s.pid,
w.sync_priority,
w.sync_state,
w.reply_time
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
JOIN pg_stat_get_wal_senders() w(pid, state, sent_lsn, write_lsn, flush_lsn, replay_lsn, write_lag, flush_lag, replay_lag, sync_priority, sync_state, reply_time) ON ((s.pid = w.pid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_replication_slots| SELECT s.slot_name,
@@ -2130,7 +2131,7 @@ pg_stat_ssl| SELECT pid,
ssl_client_dn AS client_dn,
ssl_client_serial AS client_serial,
ssl_issuer_dn AS issuer_dn
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
WHERE (client_port IS NOT NULL);
pg_stat_subscription| SELECT su.oid AS subid,
su.subname,
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index 0bafa87049..d787ad2cda 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -167,6 +167,10 @@ select explain_filter('explain (verbose) select * from int8_tbl i8');
select explain_filter('explain (verbose) declare test_cur cursor for select * from int8_tbl');
select explain_filter('explain (verbose) create table test_ctas as select 1');
+-- Test compute_plan_id
+set compute_plan_id = on;
+select explain_filter('explain (verbose) select * from int8_tbl i8');
+
-- Test SERIALIZE option
select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
select explain_filter('explain (analyze,serialize text,buffers,timing off) select * from int8_tbl i8');
--
2.47.1
On Thu, Feb 06, 2025 at 07:52:53PM -0600, Sami Imseih wrote:
This fixes the long comments in plannodes.h to make it easier to add the
attribute annotation. It made the most sense to make this the first patch
in the set.
A commit that happened last Friday made also this to have conflict.
Done. Also rewrote the header comment in jumblefuncs.c to describe
a more generic node jumbling mechanism that this file now offers.Yes, after getting my hands on this, I agree with you. It made more sense
to keep all the jumbling work in jumblefuncs.c
-static void AppendJumble(JumbleState *jstate,
- const unsigned char *item, Size size
I don't understand why there is a need for publishing AppendJumble()
while it remains statis in jumblefuncs.c. This is not needed in 0003
and 0004, either.
Should we use more generic names for the existing custom_query_jumble,
no_query_jumble, query_jumble_ignore and query_jumble_location? Last
time I've talked about that with Peter E, "jumble" felt too generic,
so perhaps we're looking for a completely new term? This impacts as
well the naming of the existing queryjumblefuncs.c. The simplest term
that may be possible here is "hash", actually, because that's what we
are doing with all these node structures? That's also a very generic
term. The concept of location does not apply to plans, based on the
current proposal, so perhaps we should talk about "query normalization
location"?
Point is that query_jumble_ignore is used in the planner nodes, which
feels inconsistent, so perhaps we could rename query_jumble_ignore and
no_query_jumble to "hash_ignore" and/or "no_hash", or something like
that? This may point towards the need of a split, not sure, still the
picture is incomplete.
v5-0003 and v5-0004 introduce the planId in core and pg_stat_plans. These
needed rebasing only; but I have not yet looked at this thoroughly.We should aim to get 0001 and 0002 committed next.
Yeah. I didn't see any reasons why 0001 should not happen now, as it
makes the whole easier while making the header styles a bit more
consistent. Perhaps also if somebody forks the code and adds some
pg_node_attr() properties?
v5-0003 and v5-0004, not sure yet. The intrisincs of the planner make
putting a strict definition of what a hash means hard to set down,
we should work towards studying that more first. I don't see this
happen until the next release freeze.
--
Michael
This fixes the long comments in plannodes.h to make it easier to add the
attribute annotation. It made the most sense to make this the first patch
in the set.
A commit that happened last Friday made also this to have conflict.
Thanks for committing v5-0001. I am not sure why there is comment
that is not correctly indented. Attached is a fix for that
I don't understand why there is a need for publishing AppendJumble()
while it remains statis in jumblefuncs.c. This is not needed in 0003
and 0004, either.
In v5-0002, AppendJumble is no longer static.
-static void
+void
AppendJumble(JumbleState *jstate, const unsigned char *item, Size size)
{
Maybe I am missing something?
Should we use more generic names for the existing custom_query_jumble,
no_query_jumble, query_jumble_ignore and query_jumble_location? Last
time I've talked about that with Peter E, "jumble" felt too generic,
so perhaps we're looking for a completely new term? This impacts as
well the naming of the existing queryjumblefuncs.c. The simplest term
that may be possible here is "hash", actually, because that's what we
Point is that query_jumble_ignore is used in the planner nodes, which
feels inconsistent, so perhaps we could rename query_jumble_ignore and
no_query_jumble to "hash_ignore" and/or "no_hash", or something like
that? This may point towards the need of a split, not sure, still the
picture is incomplete.
I was thinking about this as I was reworking the comments in jumblefuncs.c
for v5-0002.
I am OK with moving away from "jumble" in-lieu of something else, but
my thoughts are we should actually call this process "fingerprint"
( a term we already use in the queryjumblefuncs.c comment ).
A fingerprint consists of all the interesting parts of a node tree that are
appended and the final product is a hash of this fingerprint ( i.e. queryId )
For node attributes we can specify "fingerprint_ignore"
or "no_fingerprint". What do you think?
The concept of location does not apply to plans, based on the
current proposal, so perhaps we should talk about "query normalization
location"?
Are you referring to JUMBLE_LOCATION? and whether to keep it in
queryjumblefuncs.c ( or jumblefuncs.c as is being proposed )?
Regards,
Sami
Attachments:
v1-0001-Fix-comment-indentation.patchapplication/octet-stream; name=v1-0001-Fix-comment-indentation.patchDownload
From a256006691abed3eb3e401e5fb9fb30e08d1f19a Mon Sep 17 00:00:00 2001
From: "Sami Imseih (AWS)"
<simseih@dev-dsk-simseih-1d-3940b79e.us-east-1.amazon.com>
Date: Mon, 10 Feb 2025 18:01:31 +0000
Subject: [PATCH v1 1/2] Fix comment indentation
Fix an unindented comment introduced in
3d17d7d7fb7
---
src/include/nodes/plannodes.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 67e4040a70..bf1f25c0db 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -110,7 +110,7 @@ typedef struct PlannedStmt
*/
List *subplans;
-/* indices of subplans that require REWIND */
+ /* indices of subplans that require REWIND */
Bitmapset *rewindPlanIDs;
/* a list of PlanRowMark's */
--
2.47.1
Another thought that I have is that If we mention that extensions can use
these jumbling ( or whatever the final name is ) functions outside of
core, it makes
sense to actually show an example of this. What do you think?
--
Sami
On Mon, Feb 10, 2025 at 02:02:10PM -0600, Sami Imseih wrote:
Thanks for committing v5-0001. I am not sure why there is comment
that is not correctly indented. Attached is a fix for that
Thanks, fixed. The reason behind that is likely that I have fat
fingers.
--
Michael
On Mon, Feb 10, 2025 at 02:14:09PM -0600, Sami Imseih wrote:
Another thought that I have is that If we mention that extensions can use
these jumbling ( or whatever the final name is ) functions outside of
core, it makes
sense to actually show an example of this. What do you think?
Not sure. Do you have anything specific in mind that pgss is not able
to achieve with its jumbling based on the query strings?
--
Michael
On Mon, Feb 10, 2025 at 02:02:10PM -0600, Sami Imseih wrote:
I am OK with moving away from "jumble" in-lieu of something else, but
my thoughts are we should actually call this process "fingerprint"
( a term we already use in the queryjumblefuncs.c comment ).
A fingerprint consists of all the interesting parts of a node tree that are
appended and the final product is a hash of this fingerprint ( i.e. queryId )
For node attributes we can specify "fingerprint_ignore"
or "no_fingerprint". What do you think?
I think that I have a long history of showing a bad naming sense, that
I've done some follow-up API renames even on stable branches because
folks didn't like some names, and that I have a reputation for that on
these lists. :D
Wikipedia seems to agree with you that "fingerprint" would fit for
this purpose, though:
https://en.wikipedia.org/wiki/Fingerprint_(computing)
Has anybody any comments about that? That would be a large renaming,
but in the long term is makes sense if we want to apply that to more
than just parse nodes and query strings. If you do that, it impacts
the file names and the properties, that are hidden in the backend for
most of it, except the entry API and JumbleState. This last part
impacts some extensions and I have been maintaining one a bit
(pg_hint_plan).
The concept of location does not apply to plans, based on the
current proposal, so perhaps we should talk about "query normalization
location"?Are you referring to JUMBLE_LOCATION? and whether to keep it in
queryjumblefuncs.c ( or jumblefuncs.c as is being proposed )?
Yes, I am referring to the existing jumble location. I don't quite
see how it fits with the plan part because we don't really have
locations to track.
Point worth noting, Alvaro has mentioned that he was planning to look
at the pgss patch with IN clauses:
/messages/by-id/202502111214.fcfodex6t3sy@alvherre.pgsql
Adding him in CC here for awareness, but I can see that both of you
are involved on the other thread, as well. Also adding Julien in CC,
as he has some out-of-core extension code that depends on the jumbling
structures if I recall correctly.
--
Michael
On Wed, Feb 12, 2025 at 09:08:00AM +0900, Michael Paquier wrote:
Wikipedia seems to agree with you that "fingerprint" would fit for
this purpose, though:
https://en.wikipedia.org/wiki/Fingerprint_(computing)Has anybody any comments about that? That would be a large renaming,
but in the long term is makes sense if we want to apply that to more
than just parse nodes and query strings. If you do that, it impacts
the file names and the properties, that are hidden in the backend for
most of it, except the entry API and JumbleState. This last part
impacts some extensions and I have been maintaining one a bit
(pg_hint_plan).
I agree that fingerprint is a good improvement.
Also adding Julien in CC,
as he has some out-of-core extension code that depends on the jumbling
structures if I recall correctly.
I do have an extension to support custom fingerprinting logic, but the
introduction of the pg_node_attr based jumbling kind of broke it.
FTR my main motivation was to be able to deal with queries referencing
temporary relations, as if your application creates a lot of those it basically
means that you cannot use pg_stat_statements anymore.
On Wed, Feb 12, 2025 at 09:20:53AM +0800, Julien Rouhaud wrote:
On Wed, Feb 12, 2025 at 09:08:00AM +0900, Michael Paquier wrote:
Wikipedia seems to agree with you that "fingerprint" would fit for
this purpose, though:
https://en.wikipedia.org/wiki/Fingerprint_(computing)Has anybody any comments about that? That would be a large renaming,
but in the long term is makes sense if we want to apply that to more
than just parse nodes and query strings. If you do that, it impacts
the file names and the properties, that are hidden in the backend for
most of it, except the entry API and JumbleState. This last part
impacts some extensions and I have been maintaining one a bit
(pg_hint_plan).I agree that fingerprint is a good improvement.
Okay, thanks. So this would mean something for the file names, the
node_attr names, the structures and the APIs if we put all that under
the same label.
Also adding Julien in CC,
as he has some out-of-core extension code that depends on the jumbling
structures if I recall correctly.I do have an extension to support custom fingerprinting logic, but the
introduction of the pg_node_attr based jumbling kind of broke it.FTR my main motivation was to be able to deal with queries referencing
temporary relations, as if your application creates a lot of those it basically
means that you cannot use pg_stat_statements anymore.
Do you have an issue more details about your problem? If we can
improve the situation in core without impacting the existing cases
that we need to support in pgss, that may be worth looking at.
--
Michael
On Wed, Feb 12, 2025 at 10:59:04AM +0900, Michael Paquier wrote:
On Wed, Feb 12, 2025 at 09:20:53AM +0800, Julien Rouhaud wrote:
FTR my main motivation was to be able to deal with queries referencing
temporary relations, as if your application creates a lot of those it basically
means that you cannot use pg_stat_statements anymore.Do you have an issue more details about your problem? If we can
improve the situation in core without impacting the existing cases
that we need to support in pgss, that may be worth looking at.
I thought this was a well known limitation. The basic is that if you rely on
temp tables, you usually end up with a virtually infinite number of queryids
since all temp tables get a different oid and that oid is used in the queryid
computation. And in that case the overhead of pg_stat_statements is insanely
high. The last figures I saw was by Andres many years ago, with a mention 40%
overhead, and I don't think it's hard to get way worse overhead than that if
you have lengthier query texts.
As a prototype in my extension I think I just entirely ignored such queries,
but another (and probably friendlier for the actual pg_stat_statements
statistics) approach would be to use the relation name to compute the queryid
rather than its oid. This would add some overhead, but I think it would have
very limited impact especially compared to the current situation.
Of course some people may want to keep the current behavior, if they have
limited number of temp tables or similar, so I had a GUC for that. I don't
think that the community would really welcome such GUC for core-postgres,
especially since it wouldn't be pg_stat_statements specific.
On Tue, Feb 11, 2025 at 7:08 PM Michael Paquier <michael@paquier.xyz> wrote:
On Mon, Feb 10, 2025 at 02:02:10PM -0600, Sami Imseih wrote:
I am OK with moving away from "jumble" in-lieu of something else, but my
thoughts are we should actually call this process "fingerprint"
I agree fingerprint is the right final word. But "jumble" conveys the
*process* better than "fingerprinting". I view it as jumbling produces an
object that can be fingerprinted.
For node attributes we can specify "fingerprint_ignore" or
"no_fingerprint". What do you think?
Still should be jumble_ignore.
Cheers,
Greg
--
Crunchy Data - https://www.crunchydata.com
Enterprise Postgres Software Products & Tech Support
Of course some people may want to keep the current behavior, if they have
limited number of temp tables or similar, so I had a GUC for that. I don't
think that the community would really welcome such GUC for core-postgres,
especially since it wouldn't be pg_stat_statements specific.
FWIW, I think options to tweak queryId computation is something
that should be in core. It was discussed earlier in the context
of IN list merging; the patch for this currently has the guc
for the feature in pg_stat_statements, but there was a discussion
about actually moving this to core [1]/messages/by-id/202502111852.btskmr7nhien@alvherre.pgsql Giving the user a way
to control certain behavior about the queryId computation
is a good thing to do in core; especially queryId is no longer
just consumed in pg_stat_statements. Maybe the right answer
is an enum GUC, not sure yet.
Specifically for the use-case you mention, using names vs OIDs in
queryId computation is a valid use case for more than temporary tables,
I can also think of upgrade, dump/restore, logical replication cases which
can then allow for a consistent queryId.
[1]: /messages/by-id/202502111852.btskmr7nhien@alvherre.pgsql
--
Sami
On Tue, Feb 11, 2025 at 08:57:46PM -0600, Sami Imseih wrote:
Of course some people may want to keep the current behavior, if they have
limited number of temp tables or similar, so I had a GUC for that. I don't
think that the community would really welcome such GUC for core-postgres,
especially since it wouldn't be pg_stat_statements specific.FWIW, I think options to tweak queryId computation is something
that should be in core. It was discussed earlier in the context
of IN list merging; the patch for this currently has the guc
for the feature in pg_stat_statements, but there was a discussion
about actually moving this to core [1] Giving the user a way
to control certain behavior about the queryId computation
is a good thing to do in core; especially queryId is no longer
just consumed in pg_stat_statements. Maybe the right answer
is an enum GUC, not sure yet.Specifically for the use-case you mention, using names vs OIDs in
queryId computation is a valid use case for more than temporary tables,
I can also think of upgrade, dump/restore, logical replication cases which
can then allow for a consistent queryId.
Well, the ability for extensions to override the actual queryid calculation was
the result of more than half a decade of strong disagreements about it. And
I'm definitely not volunteering to reopen that topic :)
On 2025-Feb-12, Julien Rouhaud wrote:
FWIW, I think options to tweak queryId computation is something that
should be in core. It was discussed earlier in the context of IN
list merging; the patch for this currently has the guc for the
feature in pg_stat_statements, but there was a discussion about
actually moving this to core [1] Giving the user a way to control
certain behavior about the queryId computation is a good thing to do
in core; especially queryId is no longer just consumed in
pg_stat_statements. Maybe the right answer is an enum GUC, not sure
yet.
Well, the ability for extensions to override the actual queryid
calculation was the result of more than half a decade of strong
disagreements about it. And I'm definitely not volunteering to
reopen that topic :)
Sorry, Michael already did.
Anyway, I think that's different. We do support compute_query_id=off as
a way for a custom module to compute completely different query IDs
using their own algorithm, which I think is what you're referring to.
However, the ability to affect the way the in-core algorithm works is a
different thing: you still want in-core code to compute the query ID.
Right now, the proposal in the other thread is that if you want to
affect that algorithm in order to merge arrays to be considered a single
query element regardless of its length, you set the GUC for that.
Initially the GUC was in the core code. Then, based on review, the GUC
was moved to the extension, _BUT_ the implementation was still in the
core code: in order to activate it, the extension calls a function that
modifies core code behavior. So there are more moving parts than
before, and if you for whatever reason want that behavior but not the
extension, then you need to write a C function. To me this is absurd.
So what I suggest we do is return to having the GUC in the core code.
Now I admit I'm not sure what the solution would be for the problem
discussed in this subthread. Apparently the problem is related to temp
tables and their changing OIDs. I'm not sure what exactly the proposal
for a GUC is. I mean, what would the behavior change be? Maybe what
you want is something like "if this table reference here is to a temp
table, then instead of jumbling the OID then jumble the string
'pg_temp.tablename' instead", which would make the query ID be the same
for all occurrences of that query in whatever backend return the same
number, regardless both of what OID the temp schema for that backend is,
and the table OID itself. Is there more to it than that? (The only
difficulty I see here is how to get the table name when the only thing
you have is the RangeTblEntry, which doesn't have the name but just the
OID. I see in [1]https://github.com/rjuju/pg_queryid/blob/master/pg_queryid.c#L941 that you simply do a syscache lookup, but it would be
good to avoid that.)
Maybe that sounds pretty obscure if you try to describe it too
precisely, but if you don't think too hard about it it probably natural
-- at least to me. So my next question is, do we really need this
behavior to be configurable? Wouldn't it be better to make the default
way to deal with temp tables in all cases? The current behavior seems
rather unhelpful. I do note that what you do in pg_queryid, which is
simply to ignore the table altogether, is probably not a great idea.
Anyway, assuming we make a GUC of it (a big if!), let me talk a bit
about GUC names. In the other thread, the list of GUC names in the
submitted patch plus the ones I suggested are:
query_id_const_merge
query_id_merge_values
query_id_merge_value_lists
query_id_squash_constant_lists
so maybe here I would consider something like
query_id_merge_temp_tables
query_id_squash_temporary_tables
[1]: https://github.com/rjuju/pg_queryid/blob/master/pg_queryid.c#L941
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"The important things in the world are problems with society that we don't
understand at all. The machines will become more complicated but they won't
be more complicated than the societies that run them." (Freeman Dyson)
On Wed, Feb 12, 2025 at 01:57:47PM +0100, Alvaro Herrera wrote:
On 2025-Feb-12, Julien Rouhaud wrote:
FWIW, I think options to tweak queryId computation is something that
should be in core. It was discussed earlier in the context of IN
list merging; the patch for this currently has the guc for the
feature in pg_stat_statements, but there was a discussion about
actually moving this to core [1] Giving the user a way to control
certain behavior about the queryId computation is a good thing to do
in core; especially queryId is no longer just consumed in
pg_stat_statements. Maybe the right answer is an enum GUC, not sure
yet.Well, the ability for extensions to override the actual queryid
calculation was the result of more than half a decade of strong
disagreements about it. And I'm definitely not volunteering to
reopen that topic :)Anyway, I think that's different. We do support compute_query_id=off as
a way for a custom module to compute completely different query IDs
using their own algorithm, which I think is what you're referring to.
However, the ability to affect the way the in-core algorithm works is a
different thing: you still want in-core code to compute the query ID.
I don't think that's the actual behavior, or at least not what it was supposed
to be.
What we should have is the ability to compute queryid, which can be either in
core or done by an external module, but one only one can / should be done. And
then you have stuff that use that queryid, e.g. pg_stat_statements,
pg_stat_activity and whatnot, no matter what generated it. That's per the
original commit 5fd9dfa5f50e message:
Add compute_query_id GUC to control whether a query identifier should be
computed by the core (off by default). It's thefore now possible to
disable core queryid computation and use pg_stat_statements with a
different algorithm to compute the query identifier by using a
third-party module.
To ensure that a single source of query identifier can be used and is
well defined, modules that calculate a query identifier should throw an
error if compute_query_id specified to compute a query id and if a query
idenfitier was already calculated.
Right now, the proposal in the other thread is that if you want to
affect that algorithm in order to merge arrays to be considered a single
query element regardless of its length, you set the GUC for that.
Initially the GUC was in the core code. Then, based on review, the GUC
was moved to the extension, _BUT_ the implementation was still in the
core code: in order to activate it, the extension calls a function that
modifies core code behavior. So there are more moving parts than
before, and if you for whatever reason want that behavior but not the
extension, then you need to write a C function. To me this is absurd.
So what I suggest we do is return to having the GUC in the core code.
I agree, although that probably breaks the queryid extensibility. I haven't
read the patch but IIUC if you want the feature to work you need to both change
the queryid calculation but also the way the constants are recorded and the
query text is normalized, and I don't know if extensions have access to it. If
they have access and fail to do what the GUC asked then of course that's just a
bug in that extension.
Now I admit I'm not sure what the solution would be for the problem
discussed in this subthread. Apparently the problem is related to temp
tables and their changing OIDs. I'm not sure what exactly the proposal
for a GUC is.
I'm not proposing anything, just explaining why pg_stat_statements is generally
useless if you use temp tables as someone asked.
I do note that what you do in pg_queryid, which is
simply to ignore the table altogether, is probably not a great idea.
Yeah, that's also why I said in my previous message that using its name for the
queryid would be better. Note that in pg_queryid it's already possible to use
relation names rather than oid for the queryid (which I wouldn't recommend, but
it's good for testing). I just never implemented it for a temp-only
granularity.
On 2025-Feb-12, Julien Rouhaud wrote:
On Wed, Feb 12, 2025 at 01:57:47PM +0100, Alvaro Herrera wrote:
Anyway, I think that's different. We do support compute_query_id=off as
a way for a custom module to compute completely different query IDs
using their own algorithm, which I think is what you're referring to.
However, the ability to affect the way the in-core algorithm works is a
different thing: you still want in-core code to compute the query ID.I don't think that's the actual behavior, or at least not what it was
supposed to be.What we should have is the ability to compute queryid, which can be
either in core or done by an external module, but one only one can /
should be done.
Yes, that's what I tried to say, but I don't understand why you say I
said something different.
Right now, the proposal in the other thread is that if you want to
affect that algorithm in order to merge arrays to be considered a single
query element regardless of its length, you set the GUC for that.
Initially the GUC was in the core code. Then, based on review, the GUC
was moved to the extension, _BUT_ the implementation was still in the
core code: in order to activate it, the extension calls a function that
modifies core code behavior. So there are more moving parts than
before, and if you for whatever reason want that behavior but not the
extension, then you need to write a C function. To me this is absurd.
So what I suggest we do is return to having the GUC in the core code.I agree, although that probably breaks the queryid extensibility.
It does?
I haven't read the patch but IIUC if you want the feature to work you
need to both change the queryid calculation but also the way the
constants are recorded and the query text is normalized, and I don't
know if extensions have access to it.
Hmm. As for the query text: with Andrey's feature with the GUC in core,
a query like this
SELECT foo FROM tab WHERE col1 IN (1,2,3,4)
will have in pg_stat_activity an identical query_id to a query like this
SELECT foo WHERE tab WHERE col1 IN (1,2,3,4,5)
even though the query texts differ (in the number of elements in the
array). I don't think this is a problem. This means that the query_id
for two different queries can be identical, but that should be no
surprise, precisely because the GUC that controls it is documented to do
that.
If pg_stat_statements is enabled with Andrey's patch, then the same
query_id will have a single entry (which has stats for both execution of
those queries) with that query_id, with a normalized query text that is
going to be different from those two above; without Andrey's feature,
the text would be
SELECT foo WHERE tab WHERE col1 IN ($1,$2,$3,$4);
SELECT foo WHERE tab WHERE col1 IN ($1,$2,$3,$4,$5);
(that is, pg_stat_statements transformed the values into placeholders,
but using exactly the same number of items in the array as the original
queries). With Andrey's feature, it will be
SELECT foo WHERE tab WHERE col1 IN (...);
that is, the query text has been modified and no longer matches exactly
any of the queries in pg_stat_activity. But note that the query text
already does not match what's in pg_stat_activity, even before Andrey's
patch.
I don't understand what you mean with "the way the constants are
recorded". What constants are you talking about? pg_stat_statements
purposefully discards any constants used in the query (obviously).
If they have access and fail to do what the GUC asked then of course
that's just a bug in that extension.
I don't understand what bug are you thinking that such hypothetical
extension would have. (pg_stat_statements does of course have access to
the query text and to the location of all constants).
Now I admit I'm not sure what the solution would be for the problem
discussed in this subthread. Apparently the problem is related to temp
tables and their changing OIDs. I'm not sure what exactly the proposal
for a GUC is.I'm not proposing anything, just explaining why pg_stat_statements is
generally useless if you use temp tables as someone asked.
Ah, okay. Well, where you see a deficiency, I see an opportunity for
improvement :-)
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
On Mon, Feb 10, 2025 at 02:02:10PM -0600, Sami Imseih wrote:
I am OK with moving away from "jumble" in-lieu of something else, but my thoughts are we should actually call this process "fingerprint"
I agree fingerprint is the right final word. But "jumble" conveys the *process* better than "fingerprinting".
I view it as jumbling produces an object that can be fingerprinted.
hmm, "jumble" describes something that is scrambled
or not in order, such as the 64-bit hash produced. It
sounds like the final product.
Fingerprinting on the other hand [1]https://en.wikipedia.org/wiki/Fingerprint_(computing) sounds more of the process
to add all the pieces that will eventually be hashed ( or jumbled ).
hash and jumble are synonyms according to Merriam-Webster [2]https://www.merriam-webster.com/thesaurus/hash
--
Sami
[1]: https://en.wikipedia.org/wiki/Fingerprint_(computing)
[2]: https://www.merriam-webster.com/thesaurus/hash
On 2025-Feb-12, Sami Imseih wrote:
Greg S. Mullane wrote:
I agree fingerprint is the right final word. But "jumble" conveys
the *process* better than "fingerprinting". I view it as jumbling
produces an object that can be fingerprinted.hmm, "jumble" describes something that is scrambled
or not in order, such as the 64-bit hash produced. It
sounds like the final product.
I don't understand why we would change any naming here at all. I think
you should be looking at a much broader consensus and plus-ones that a
renaming is needed. -1 from me.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"No renuncies a nada. No te aferres a nada."
I don't understand why we would change any naming here at all. I think
you should be looking at a much broader consensus and plus-ones that a
renaming is needed. -1 from me.
The reason for the change is because "query jumble" will no longer
make sense if the jumble code can now be used for other types of
trees, such as Plan.
I do agree that this needs a single-threaded discussion to achieve a
consensus.
--
Sami
On Thu, Feb 13, 2025 at 10:44:33AM -0600, Sami Imseih wrote:
The reason for the change is because "query jumble" will no longer
make sense if the jumble code can now be used for other types of
trees, such as Plan.I do agree that this needs a single-threaded discussion to achieve a
consensus.
FWIW, I was playing with a sub-project where I was jumbling a portion
of nodes other than Query, and it is annoying to not have a direct
access to jumbleNode(). So, how about doing the refactoring proposed
in v5-0002 with an initialization routine and JumbleNode() as the
entry point for the jumbling, but not rename the existing files
queryjumblefuncs.c and queryjumble.h? That seems doable for this
release, at least.
I don't think that we should expose AppendJumble(), either.
--
Michael
On 3/18/25 08:31, Michael Paquier wrote:
On Thu, Feb 13, 2025 at 10:44:33AM -0600, Sami Imseih wrote:
The reason for the change is because "query jumble" will no longer
make sense if the jumble code can now be used for other types of
trees, such as Plan.I do agree that this needs a single-threaded discussion to achieve a
consensus.FWIW, I was playing with a sub-project where I was jumbling a portion
of nodes other than Query, and it is annoying to not have a direct
access to jumbleNode(). So, how about doing the refactoring proposed
in v5-0002 with an initialization routine and JumbleNode() as the
entry point for the jumbling, but not rename the existing files
queryjumblefuncs.c and queryjumble.h? That seems doable for this
release, at least.
It seems pretty helpful to me. Having a code for hashing an expression
or subquery, we may design new optimisations. I personally have such a
necessity in a couple of planner extensions.
At the same time, generalising jumbling code we may decide to work on
the JumbleState structure: code related to constant locations may be
replaced with callbacks - let the caller decide what action to take on
each node (not only constants). Of course, it is not for current release.
I don't think that we should expose AppendJumble(), either.
Agree
--
regards, Andrei Lepikhov
On 12.02.2025 19:00, Alvaro Herrera wrote:
On 2025-Feb-12, Julien Rouhaud wrote:
On Wed, Feb 12, 2025 at 01:57:47PM +0100, Alvaro Herrera wrote:
Anyway, I think that's different. We do support compute_query_id=off as
a way for a custom module to compute completely different query IDs
using their own algorithm, which I think is what you're referring to.
However, the ability to affect the way the in-core algorithm works is a
different thing: you still want in-core code to compute the query ID.I don't think that's the actual behavior, or at least not what it was
supposed to be.What we should have is the ability to compute queryid, which can be
either in core or done by an external module, but one only one can /
should be done.Yes, that's what I tried to say, but I don't understand why you say I
said something different.Right now, the proposal in the other thread is that if you want to
affect that algorithm in order to merge arrays to be considered a single
query element regardless of its length, you set the GUC for that.
Initially the GUC was in the core code. Then, based on review, the GUC
was moved to the extension, _BUT_ the implementation was still in the
core code: in order to activate it, the extension calls a function that
modifies core code behavior. So there are more moving parts than
before, and if you for whatever reason want that behavior but not the
extension, then you need to write a C function. To me this is absurd.
So what I suggest we do is return to having the GUC in the core code.I agree, although that probably breaks the queryid extensibility.
It does?
I haven't read the patch but IIUC if you want the feature to work you
need to both change the queryid calculation but also the way the
constants are recorded and the query text is normalized, and I don't
know if extensions have access to it.Hmm. As for the query text: with Andrey's feature with the GUC in core,
a query like this
SELECT foo FROM tab WHERE col1 IN (1,2,3,4)
will have in pg_stat_activity an identical query_id to a query like this
SELECT foo WHERE tab WHERE col1 IN (1,2,3,4,5)
even though the query texts differ (in the number of elements in the
array). I don't think this is a problem. This means that the query_id
for two different queries can be identical, but that should be no
surprise, precisely because the GUC that controls it is documented to do
that.If pg_stat_statements is enabled with Andrey's patch, then the same
query_id will have a single entry (which has stats for both execution of
those queries) with that query_id, with a normalized query text that is
going to be different from those two above; without Andrey's feature,
the text would be
SELECT foo WHERE tab WHERE col1 IN ($1,$2,$3,$4);
SELECT foo WHERE tab WHERE col1 IN ($1,$2,$3,$4,$5);
(that is, pg_stat_statements transformed the values into placeholders,
but using exactly the same number of items in the array as the original
queries). With Andrey's feature, it will be
SELECT foo WHERE tab WHERE col1 IN (...);
that is, the query text has been modified and no longer matches exactly
any of the queries in pg_stat_activity. But note that the query text
already does not match what's in pg_stat_activity, even before Andrey's
patch.I don't understand what you mean with "the way the constants are
recorded". What constants are you talking about? pg_stat_statements
purposefully discards any constants used in the query (obviously).If they have access and fail to do what the GUC asked then of course
that's just a bug in that extension.I don't understand what bug are you thinking that such hypothetical
extension would have. (pg_stat_statements does of course have access to
the query text and to the location of all constants).Now I admit I'm not sure what the solution would be for the problem
discussed in this subthread. Apparently the problem is related to temp
tables and their changing OIDs. I'm not sure what exactly the proposal
for a GUC is.I'm not proposing anything, just explaining why pg_stat_statements is
generally useless if you use temp tables as someone asked.Ah, okay. Well, where you see a deficiency, I see an opportunity for
improvement :-)
Hi everyone,
I support the idea of computing the planid for temporary tables using
'pg_temp.rel_name'. Moreover, we have already started using this
approach for computing queryid [0]/messages/by-id/Z9mkqplmUpQ4xG52@msg.df7cb.de. It seems reasonable to apply the
same logic to the planid calculation as well.
[0]: /messages/by-id/Z9mkqplmUpQ4xG52@msg.df7cb.de
/messages/by-id/Z9mkqplmUpQ4xG52@msg.df7cb.de
--
Best regards,
Ilia Evdokimov,
Tantor Labs LLC.
Attachments:
v6-0002-Fix-jumbling-of-empty-arrays-in-plan-nodes.patchtext/x-diff; name=v6-0002-Fix-jumbling-of-empty-arrays-in-plan-nodes.patchDownload
From 70c9720a8edd84fd31c4e9a9b1978765df5dbaea Mon Sep 17 00:00:00 2001
From: Andrey Kazachkov <andrey.kazachkov@tantorlabs.ru>
Date: Wed, 24 Dec 2025 18:47:34 +0300
Subject: [PATCH v6 2/2] Fix jumbling of empty arrays in plan nodes
---
src/backend/nodes/queryjumblefuncs.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/src/backend/nodes/queryjumblefuncs.c b/src/backend/nodes/queryjumblefuncs.c
index 5717480dbf5..a0d28827269 100644
--- a/src/backend/nodes/queryjumblefuncs.c
+++ b/src/backend/nodes/queryjumblefuncs.c
@@ -555,7 +555,10 @@ do { \
AppendJumble(jstate, (const unsigned char *) expr->item->words, sizeof(bitmapword) * expr->item->nwords); \
} while(0)
#define JUMBLE_ARRAY(item, len) \
- AppendJumble(jstate, (const unsigned char *) expr->item, sizeof(*(expr->item)) * len)
+ do { \
+ if (len) \
+ AppendJumble(jstate, (const unsigned char *) expr->item, sizeof(*(expr->item)) * len); \
+ } while (0)
#define JUMBLE_STRING(str) \
do { \
if (expr->str) \
--
2.43.0
v6-0001-Added-plan_id-to-the-core.patchtext/x-diff; name=v6-0001-Added-plan_id-to-the-core.patchDownload
From 49518713775c3d24ba8e7a802c817042143e548a Mon Sep 17 00:00:00 2001
From: Andrey Kazachkov <andrey.kazachkov@tantorlabs.ru>
Date: Tue, 16 Dec 2025 19:52:39 +0300
Subject: [PATCH v6 1/2] Added plan_id to the core.
1. Changed a jumbling initialization routine
2. JumbleNode() is an entry point for jumbling
3. AppendJumble() remains static
4. guc-compute-plan-id has been added
5. gen_node_support.pl changed to process plan node structures
6. pg_node_attr has been added to certain structures/fields
---
doc/src/sgml/config.sgml | 34 ++++
doc/src/sgml/monitoring.sgml | 16 ++
src/backend/catalog/system_views.sql | 1 +
src/backend/commands/explain.c | 17 ++
src/backend/executor/execMain.c | 10 +-
src/backend/nodes/gen_node_support.pl | 34 +++-
src/backend/nodes/queryjumblefuncs.c | 180 +++++++++++-------
src/backend/optimizer/plan/planner.c | 19 ++
src/backend/optimizer/plan/setrefs.c | 9 +
src/backend/postmaster/launch_backend.c | 3 +
src/backend/utils/adt/pgstatfuncs.c | 7 +-
src/backend/utils/misc/guc_parameters.dat | 7 +
src/backend/utils/misc/guc_tables.c | 18 ++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/catalog/pg_proc.dat | 6 +-
src/include/nodes/pathnodes.h | 3 +
src/include/nodes/plannodes.h | 50 ++---
src/include/nodes/primnodes.h | 4 +-
src/include/nodes/queryjumble.h | 35 +++-
src/include/utils/backend_status.h | 2 +-
src/test/regress/expected/explain.out | 11 ++
src/test/regress/expected/rules.out | 9 +-
src/test/regress/sql/explain.sql | 4 +
23 files changed, 371 insertions(+), 109 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 405c9689bd0..4cdaf9c49ba 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -8864,6 +8864,40 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-compute-plan-id" xreflabel="compute_plan_id">
+ <term><varname>compute_plan_id</varname> (<type>enum</type>)
+ <indexterm>
+ <primary><varname>compute_plan_id</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Enables in-core computation of a plan identifier.
+ Plan identifiers can be displayed in the <link
+ linkend="monitoring-pg-stat-activity-view"><structname>pg_stat_activity</structname></link>
+ view or using <command>EXPLAIN</command>.
+ Note that an external module can alternatively be used if the
+ in-core plan identifier computation method is not acceptable.
+ In this case, in-core computation must be always disabled.
+ Valid values are <literal>off</literal> (always disabled),
+ <literal>on</literal> (always enabled), <literal>auto</literal>,
+ which lets modules that utilize plan identifiers enable
+ it automatically, and <literal>regress</literal> which
+ has the same effect as <literal>on</literal>, except that the
+ query identifier is not shown in the <literal>EXPLAIN</literal> output
+ in order to facilitate automated regression testing.
+ The default is <literal>auto</literal>.
+ </para>
+ <note>
+ <para>
+ To ensure that only one plan identifier is calculated and
+ displayed, extensions that calculate plan identifiers should
+ throw an error if a plan identifier has already been computed.
+ </para>
+ </note>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-log-statement-stats">
<term><varname>log_statement_stats</varname> (<type>boolean</type>)
<indexterm>
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 817fd9f4ca7..9e98fd2c198 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -983,6 +983,22 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>plan_id</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Identifier of this backend's most recent query plan. If
+ <structfield>state</structfield> is <literal>active</literal> this
+ field shows the identifier of the currently executing query plan. In
+ all other states, it shows the identifier of last query plan that
+ was executed. Plan identifiers are not computed by default so this
+ field will be null unless <xref linkend="guc-compute-plan-id"/>
+ parameter is enabled or a third-party module that computes plan
+ identifiers is configured.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>query</structfield> <type>text</type>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 0a0f95f6bb9..35744a82f0b 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -919,6 +919,7 @@ CREATE VIEW pg_stat_activity AS
S.backend_xid,
S.backend_xmin,
S.query_id,
+ S.plan_id,
S.query,
S.backend_type
FROM pg_stat_get_activity(NULL) AS S
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 5a6390631eb..b21cd142111 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -818,6 +818,23 @@ ExplainPrintPlan(ExplainState *es, QueryDesc *queryDesc)
ExplainPropertyInteger("Query Identifier", NULL,
queryDesc->plannedstmt->queryId, es);
}
+
+ /*
+ * COMPUTE_PLAN_ID_REGRESS means COMPUTE_PLAN_ID_YES, but we don't show
+ * the queryid in any of the EXPLAIN plans to keep stable the results
+ * generated by regression test suites.
+ */
+ if (es->verbose && queryDesc->plannedstmt->planId != UINT64CONST(0) &&
+ compute_plan_id != COMPUTE_PLAN_ID_REGRESS)
+ {
+ /*
+ * Output the queryid as an int64 rather than a uint64 so we match
+ * what would be seen in the BIGINT pg_stat_activity.plan_id column.
+ */
+ ExplainPropertyInteger("Plan Identifier", NULL,
+ queryDesc->plannedstmt->planId, es);
+ }
+
}
/*
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 797d8b1ca1c..14f107d45b1 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -123,13 +123,15 @@ ExecutorStart(QueryDesc *queryDesc, int eflags)
{
/*
* In some cases (e.g. an EXECUTE statement or an execute message with the
- * extended query protocol) the query_id won't be reported, so do it now.
+ * extended query protocol) the query_id and plan_id won't be reported, so
+ * do it now.
*
- * Note that it's harmless to report the query_id multiple times, as the
- * call will be ignored if the top level query_id has already been
- * reported.
+ * Note that it's harmless to report the identifiers multiple times, as
+ * the call will be ignored if the top level query_id or plan_id has
+ * already been reported.
*/
pgstat_report_query_id(queryDesc->plannedstmt->queryId, false);
+ pgstat_report_plan_id(queryDesc->plannedstmt->planId, false);
if (ExecutorStart_hook)
(*ExecutorStart_hook) (queryDesc, eflags);
diff --git a/src/backend/nodes/gen_node_support.pl b/src/backend/nodes/gen_node_support.pl
index 9ecddb14231..eab144bd962 100644
--- a/src/backend/nodes/gen_node_support.pl
+++ b/src/backend/nodes/gen_node_support.pl
@@ -1291,6 +1291,7 @@ _jumble${n}(JumbleState *jstate, Node *node)
{
my $t = $node_type_info{$n}->{field_types}{$f};
my @a = @{ $node_type_info{$n}->{field_attrs}{$f} };
+ my $array_size_field;
my $query_jumble_ignore = $struct_no_query_jumble;
my $query_jumble_custom = 0;
my $query_jumble_location = 0;
@@ -1303,7 +1304,11 @@ _jumble${n}(JumbleState *jstate, Node *node)
{
$query_jumble_custom = 1;
}
- if ($a eq 'query_jumble_ignore')
+ elsif ($a =~ /^array_size\(([\w.]+)\)$/)
+ {
+ $array_size_field = $1;
+ }
+ elsif ($a eq 'query_jumble_ignore')
{
$query_jumble_ignore = 1;
}
@@ -1317,12 +1322,19 @@ _jumble${n}(JumbleState *jstate, Node *node)
}
}
+ next if $query_jumble_ignore;
+
if ($query_jumble_custom)
{
# Custom function that applies to one field of a node.
print $jff "\tJUMBLE_CUSTOM($n, $f);\n"
unless $query_jumble_ignore;
}
+ elsif ($t eq 'Bitmapset*')
+ {
+ print $jff "\tJUMBLE_BITMAPSET($f);\n"
+ unless $query_jumble_ignore;
+ }
elsif (($t =~ /^(\w+)\*$/ or $t =~ /^struct\s+(\w+)\*$/)
and elem $1, @node_types)
{
@@ -1352,6 +1364,26 @@ _jumble${n}(JumbleState *jstate, Node *node)
print $jff "\tJUMBLE_STRING($f);\n"
unless $query_jumble_ignore;
}
+ elsif ($t =~ /^(\w+)(\*|\[\w+\])$/ and elem $1, @scalar_types)
+ {
+ if (!defined $array_size_field)
+ {
+ die "no array size defined for $n.$f of type $t\n";
+ }
+ if ($node_type_info{$n}->{field_types}{$array_size_field} eq
+ 'List*')
+ {
+ print $jff
+ "\tJUMBLE_ARRAY($f, list_length(expr->$array_size_field));\n"
+ unless $query_jumble_ignore;
+ }
+ else
+ {
+ print $jff
+ "\tJUMBLE_ARRAY($f, expr->$array_size_field);\n"
+ unless $query_jumble_ignore;
+ }
+ }
else
{
print $jff "\tJUMBLE_FIELD($f);\n"
diff --git a/src/backend/nodes/queryjumblefuncs.c b/src/backend/nodes/queryjumblefuncs.c
index ffc230af427..5717480dbf5 100644
--- a/src/backend/nodes/queryjumblefuncs.c
+++ b/src/backend/nodes/queryjumblefuncs.c
@@ -3,32 +3,31 @@
* queryjumblefuncs.c
* Query normalization and fingerprinting.
*
- * Normalization is a process whereby similar queries, typically differing only
- * in their constants (though the exact rules are somewhat more subtle than
- * that) are recognized as equivalent, and are tracked as a single entry. This
- * is particularly useful for non-prepared queries.
+ * Fingerprinting selectively serializes key fields within a tree structure,
+ * such as a Query or Plan tree, to create a unique identifier while ignoring
+ * extraneous details. These essential fields are concatenated into a jumble,
+ * from which a 64-bit hash is computed. Unlike regular serialization, this
+ * approach excludes irrelevant information.
+ *
+ * Use Cases:
*
- * Normalization is implemented by fingerprinting queries, selectively
- * serializing those fields of each query tree's nodes that are judged to be
- * essential to the query. This is referred to as a query jumble. This is
- * distinct from a regular serialization in that various extraneous
- * information is ignored as irrelevant or not essential to the query, such
- * as the collations of Vars and, most notably, the values of constants.
- *
- * This jumble is acquired at the end of parse analysis of each query, and
- * a 64-bit hash of it is stored into the query's Query.queryId field.
- * The server then copies this value around, making it available in plan
- * tree(s) generated from the query. The executor can then use this value
- * to blame query costs on the proper queryId.
- *
- * Arrays of two or more constants and PARAM_EXTERN parameters are "squashed"
- * and contribute only once to the jumble. This has the effect that queries
- * that differ only on the length of such lists have the same queryId.
- *
- *
- * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
- * Portions Copyright (c) 1994, Regents of the University of California
+ * 1. In-Core Query Normalization & Identification
*
+ * Fingerprinting is used to normalize query trees by generating a hash stored
+ * in the Query.queryId field. This ID is propagated to plan tree(s), allowing
+ * the executor to attribute query costs on the proper queryId. The process
+ * excludes information like typmod, collation, and most notably, the values
+ * of constants.
+ *
+ * Example: The following queries produce the same queryId:
+ *
+ * SELECT t.* FROM s1.t WHERE c1 = 1;
+ * SELECT t.* FROM s1.t WHERE c1 = 2;
+ *
+ * 2. Modified jumbling logic for extensions
+ *
+ * Extensions can modify the fingerprinting logic for queryId, or fingerprint
+ * other types of trees, such as a plan tree, to compute a plan identifier.
*
* IDENTIFICATION
* src/backend/nodes/queryjumblefuncs.c
@@ -50,6 +49,7 @@
/* GUC parameters */
int compute_query_id = COMPUTE_QUERY_ID_AUTO;
+int compute_plan_id = COMPUTE_PLAN_ID_AUTO;
/*
* True when compute_query_id is ON or AUTO, and a module requests them.
@@ -60,15 +60,21 @@ int compute_query_id = COMPUTE_QUERY_ID_AUTO;
*/
bool query_id_enabled = false;
-static JumbleState *InitJumble(void);
-static int64 DoJumble(JumbleState *jstate, Node *node);
+/*
+ * True when compute_plan_id is ON or AUTO, and a module requests them.
+ *
+ * Note that IsPlanIdEnabled() should be used instead of checking
+ * plan_id_enabled or compute_plan_id directly when we want to know
+ * whether query identifiers are computed in the core or not.
+ */
+bool plan_id_enabled = false;
+
static void AppendJumble(JumbleState *jstate,
const unsigned char *value, Size size);
static void FlushPendingNulls(JumbleState *jstate);
static void RecordConstLocation(JumbleState *jstate,
bool extern_param,
int location, int len);
-static void _jumbleNode(JumbleState *jstate, Node *node);
static void _jumbleList(JumbleState *jstate, Node *node);
static void _jumbleElements(JumbleState *jstate, List *elements, Node *node);
static void _jumbleParam(JumbleState *jstate, Node *node);
@@ -136,13 +142,13 @@ CleanQuerytext(const char *query, int *location, int *len)
JumbleState *
JumbleQuery(Query *query)
{
- JumbleState *jstate;
+ JumbleState *jstate = InitializeJumbleState(true);;
Assert(IsQueryIdEnabled());
- jstate = InitJumble();
-
- query->queryId = DoJumble(jstate, (Node *) query);
+ /* Compute query ID and mark the Query node with it */
+ JumbleNode(jstate, (Node *) query);
+ query->queryId = HashJumbleState(jstate);
/*
* If we are unlucky enough to get a hash of zero, use 1 instead for
@@ -173,44 +179,44 @@ EnableQueryId(void)
}
/*
- * InitJumble
- * Allocate a JumbleState object and make it ready to jumble.
+ * Enables plan identifier computation.
+ *
+ * Third-party plugins can use this function to inform core that they require
+ * a query identifier to be computed.
*/
-static JumbleState *
-InitJumble(void)
+void
+EnablePlanId(void)
{
- JumbleState *jstate;
+ if (compute_plan_id != COMPUTE_PLAN_ID_OFF)
+ plan_id_enabled = true;
+}
- jstate = palloc_object(JumbleState);
+/*
+ * InitializeJumbleState
+ * Allocate a JumbleState object and make it ready to jumble.
+ */
+JumbleState *
+InitializeJumbleState(bool record_clocations)
+{
+ JumbleState *jstate = (JumbleState *) palloc0(sizeof(JumbleState));
/* Set up workspace for query jumbling */
jstate->jumble = (unsigned char *) palloc(JUMBLE_SIZE);
jstate->jumble_len = 0;
- jstate->clocations_buf_size = 32;
- jstate->clocations = (LocationLen *) palloc(jstate->clocations_buf_size *
- sizeof(LocationLen));
- jstate->clocations_count = 0;
- jstate->highest_extern_param_id = 0;
- jstate->pending_nulls = 0;
- jstate->has_squashed_lists = false;
-#ifdef USE_ASSERT_CHECKING
- jstate->total_jumble_len = 0;
-#endif
+
+ if (record_clocations)
+ {
+ jstate->clocations_buf_size = 32;
+ jstate->clocations = (LocationLen *)
+ palloc(jstate->clocations_buf_size * sizeof(LocationLen));
+ }
return jstate;
}
-/*
- * DoJumble
- * Jumble the given Node using the given JumbleState and return the resulting
- * jumble hash.
- */
-static int64
-DoJumble(JumbleState *jstate, Node *node)
+uint64
+HashJumbleState(JumbleState *jstate)
{
- /* Jumble the given node */
- _jumbleNode(jstate, node);
-
/* Flush any pending NULLs before doing the final hash */
if (jstate->pending_nulls > 0)
FlushPendingNulls(jstate);
@@ -219,10 +225,9 @@ DoJumble(JumbleState *jstate, Node *node)
if (jstate->has_squashed_lists)
jstate->highest_extern_param_id = 0;
- /* Process the jumble buffer and produce the hash value */
- return DatumGetInt64(hash_any_extended(jstate->jumble,
- jstate->jumble_len,
- 0));
+ return DatumGetUInt64(hash_any_extended(jstate->jumble,
+ jstate->jumble_len,
+ 0));
}
/*
@@ -398,7 +403,7 @@ static void
RecordConstLocation(JumbleState *jstate, bool extern_param, int location, int len)
{
/* -1 indicates unknown or undefined location */
- if (location >= 0)
+ if (location >= 0 && jstate->clocations_buf_size > 0)
{
/* enlarge array if needed */
if (jstate->clocations_count >= jstate->clocations_buf_size)
@@ -526,7 +531,7 @@ IsSquashableConstantList(List *elements)
}
#define JUMBLE_NODE(item) \
- _jumbleNode(jstate, (Node *) expr->item)
+ JumbleNode(jstate, (Node *) expr->item)
#define JUMBLE_ELEMENTS(list, node) \
_jumbleElements(jstate, (List *) expr->list, node)
#define JUMBLE_LOCATION(location) \
@@ -544,6 +549,13 @@ do { \
else \
AppendJumble(jstate, (const unsigned char *) &(expr->item), sizeof(expr->item)); \
} while (0)
+#define JUMBLE_BITMAPSET(item) \
+do { \
+ if (expr->item) \
+ AppendJumble(jstate, (const unsigned char *) expr->item->words, sizeof(bitmapword) * expr->item->nwords); \
+} while(0)
+#define JUMBLE_ARRAY(item, len) \
+ AppendJumble(jstate, (const unsigned char *) expr->item, sizeof(*(expr->item)) * len)
#define JUMBLE_STRING(str) \
do { \
if (expr->str) \
@@ -557,8 +569,8 @@ do { \
#include "queryjumblefuncs.funcs.c"
-static void
-_jumbleNode(JumbleState *jstate, Node *node)
+void
+JumbleNode(JumbleState *jstate, Node *node)
{
Node *expr = node;
#ifdef USE_ASSERT_CHECKING
@@ -612,7 +624,7 @@ _jumbleList(JumbleState *jstate, Node *node)
{
case T_List:
foreach(l, expr)
- _jumbleNode(jstate, lfirst(l));
+ JumbleNode(jstate, lfirst(l));
break;
case T_IntList:
foreach(l, expr)
@@ -668,7 +680,7 @@ _jumbleElements(JumbleState *jstate, List *elements, Node *node)
if (!normalize_list)
{
- _jumbleNode(jstate, (Node *) elements);
+ JumbleNode(jstate, (Node *) elements);
}
}
@@ -758,6 +770,40 @@ _jumbleVariableSetStmt(JumbleState *jstate, Node *node)
JUMBLE_LOCATION(location);
}
+/*
+ * Jumble the entries in the rangle table to map RT indexes to relations
+ *
+ * This ensures jumbled RT indexes (e.g. in a Scan or Modify node), are
+ * distinguished by the target of the RT entry, even if the index is the same.
+ */
+void
+JumbleRangeTable(JumbleState *jstate, List *rtable)
+{
+ ListCell *lc;
+
+ foreach(lc, rtable)
+ {
+ RangeTblEntry *expr = lfirst_node(RangeTblEntry, lc);
+
+ switch (expr->rtekind)
+ {
+ case RTE_RELATION:
+ JUMBLE_FIELD(relid);
+ break;
+ case RTE_CTE:
+ JUMBLE_STRING(ctename);
+ break;
+ default:
+
+ /*
+ * Ignore other targets, the jumble includes something identifying
+ * about them already
+ */
+ break;
+ }
+ }
+}
+
/*
* Custom query jumble function for RangeTblEntry.eref.
*/
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 8b22c30559b..0309566f602 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -37,6 +37,7 @@
#ifdef OPTIMIZER_DEBUG
#include "nodes/print.h"
#endif
+#include "nodes/queryjumble.h"
#include "nodes/supportnodes.h"
#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
@@ -578,6 +579,16 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions,
Assert(glob->finalrowmarks == NIL);
Assert(glob->resultRelations == NIL);
Assert(glob->appendRelations == NIL);
+
+ /*
+ * Initialize plan identifier jumble if needed
+ *
+ * Note the actual jumbling is done in the tree walk in
+ * set_plan_references
+ */
+ if (IsPlanIdEnabled())
+ glob->plan_jumble_state = InitializeJumbleState(false);
+
top_plan = set_plan_references(root, top_plan);
/* ... and the subplans (both regular subplans and initplans) */
Assert(list_length(glob->subplans) == list_length(glob->subroots));
@@ -645,6 +656,14 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions,
result->jitFlags |= PGJIT_DEFORM;
}
+ if (IsPlanIdEnabled())
+ {
+ JumbleRangeTable(glob->plan_jumble_state, glob->finalrtable);
+ result->planId = HashJumbleState(glob->plan_jumble_state);
+ pfree(glob->plan_jumble_state->jumble);
+ pfree(glob->plan_jumble_state);
+ }
+
/* Allow plugins to take control before we discard "glob" */
if (planner_shutdown_hook)
(*planner_shutdown_hook) (glob, parse, query_string, result);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index cd7ea1e6b58..947bd980c66 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -19,6 +19,7 @@
#include "catalog/pg_type.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
+#include "nodes/queryjumble.h"
#include "optimizer/optimizer.h"
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
@@ -1338,6 +1339,14 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset)
plan->lefttree = set_plan_refs(root, plan->lefttree, rtoffset);
plan->righttree = set_plan_refs(root, plan->righttree, rtoffset);
+ /*
+ * If enabled, append significant information to the plan identifier
+ * jumble (we do this here since we're already walking the tree in a
+ * near-final state)
+ */
+ if (IsPlanIdEnabled())
+ JumbleNode(root->glob->plan_jumble_state, (Node *) plan);
+
return plan;
}
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 98f7c4848c9..b64c88db200 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -118,6 +118,7 @@ typedef struct
bool redirection_done;
bool IsBinaryUpgrade;
bool query_id_enabled;
+ bool plan_id_enabled;
int max_safe_fds;
int MaxBackends;
int num_pmchild_slots;
@@ -757,6 +758,7 @@ save_backend_variables(BackendParameters *param,
param->redirection_done = redirection_done;
param->IsBinaryUpgrade = IsBinaryUpgrade;
param->query_id_enabled = query_id_enabled;
+ param->plan_id_enabled = plan_id_enabled;
param->max_safe_fds = max_safe_fds;
param->MaxBackends = MaxBackends;
@@ -1019,6 +1021,7 @@ restore_backend_variables(BackendParameters *param)
redirection_done = param->redirection_done;
IsBinaryUpgrade = param->IsBinaryUpgrade;
query_id_enabled = param->query_id_enabled;
+ plan_id_enabled = param->plan_id_enabled;
max_safe_fds = param->max_safe_fds;
MaxBackends = param->MaxBackends;
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index ef6fffe60b9..854c9f8bffd 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -351,7 +351,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum
pg_stat_get_activity(PG_FUNCTION_ARGS)
{
-#define PG_STAT_GET_ACTIVITY_COLS 31
+#define PG_STAT_GET_ACTIVITY_COLS 32
int num_backends = pgstat_fetch_stat_numbackends();
int curr_backend;
int pid = PG_ARGISNULL(0) ? -1 : PG_GETARG_INT32(0);
@@ -665,6 +665,10 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
nulls[30] = true;
else
values[30] = Int64GetDatum(beentry->st_query_id);
+ if (beentry->st_plan_id == INT64CONST(0))
+ nulls[31] = true;
+ else
+ values[31] = UInt64GetDatum(beentry->st_plan_id);
}
else
{
@@ -694,6 +698,7 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
nulls[28] = true;
nulls[29] = true;
nulls[30] = true;
+ nulls[31] = true;
}
tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index 3b9d8349078..d9f5cf31c34 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -462,6 +462,13 @@
check_hook => 'check_commit_ts_buffers',
},
+{ name => 'compute_plan_id', type => 'enum', context => 'PGC_SUSET', group => 'STATS_MONITORING',
+ short_desc => 'Enables in-core computation of plan identifiers.',
+ variable => 'compute_plan_id',
+ boot_val => 'COMPUTE_PLAN_ID_AUTO',
+ options => 'compute_plan_id_options',
+},
+
{ name => 'compute_query_id', type => 'enum', context => 'PGC_SUSET', group => 'STATS_MONITORING',
short_desc => 'Enables in-core computation of query identifiers.',
variable => 'compute_query_id',
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f87b558c2c6..3a5d7ee32ab 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -318,6 +318,24 @@ static const struct config_enum_entry compute_query_id_options[] = {
{NULL, 0, false}
};
+/*
+ * Although only "on" and "off" are documented, we accept
+ * all the likely variants of "on" and "off".
+ */
+static const struct config_enum_entry compute_plan_id_options[] = {
+ {"auto", COMPUTE_PLAN_ID_AUTO, false},
+ {"regress", COMPUTE_PLAN_ID_REGRESS, false},
+ {"on", COMPUTE_PLAN_ID_ON, false},
+ {"off", COMPUTE_PLAN_ID_OFF, false},
+ {"true", COMPUTE_PLAN_ID_ON, true},
+ {"false", COMPUTE_PLAN_ID_OFF, true},
+ {"yes", COMPUTE_PLAN_ID_ON, true},
+ {"no", COMPUTE_PLAN_ID_OFF, true},
+ {"1", COMPUTE_PLAN_ID_ON, true},
+ {"0", COMPUTE_PLAN_ID_OFF, true},
+ {NULL, 0, false}
+};
+
/*
* Although only "on", "off", and "partition" are documented, we
* accept all the likely variants of "on" and "off".
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index dc9e2255f8a..3d0e42f083b 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -674,6 +674,7 @@
# - Monitoring -
#compute_query_id = auto
+#compute_plan_id = auto
#log_statement_stats = off
#log_parser_stats = off
#log_planner_stats = off
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index fd9448ec7b9..2bf284e2732 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5653,9 +5653,9 @@
proname => 'pg_stat_get_activity', prorows => '100', proisstrict => 'f',
proretset => 't', provolatile => 's', proparallel => 'r',
prorettype => 'record', proargtypes => 'int4',
- proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,bool,int4,int8}',
- proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,gss_delegation,leader_pid,query_id}',
+ proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,bool,int4,int8,int8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,gss_delegation,leader_pid,query_id,plan_id}',
prosrc => 'pg_stat_get_activity' },
{ oid => '6318', descr => 'describe wait events',
proname => 'pg_get_wait_events', procost => '10', prorows => '250',
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index b5ff456ef7f..c98f7b97283 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -189,6 +189,9 @@ typedef struct PlannerGlobal
/* extension state */
void **extension_state pg_node_attr(read_write_ignore);
int extension_state_allocated;
+
+ /* optional jumble state for plan identifier calculation */
+ struct JumbleState *plan_jumble_state pg_node_attr(read_write_ignore);
} PlannerGlobal;
/* macro for fetching the Plan associated with a SubPlan node */
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index c4393a94321..a5a2ccea1f7 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -70,7 +70,7 @@ typedef struct PlannedStmt
/* query identifier (copied from Query) */
int64 queryId;
- /* plan identifier (can be set by plugins) */
+ /* plan identifier (set when compute_plan_id is enabled or by plugins) */
int64 planId;
/* origin of plan */
@@ -184,7 +184,7 @@ typedef struct PlannedStmt
*/
typedef struct Plan
{
- pg_node_attr(abstract, no_equal, no_query_jumble)
+ pg_node_attr(abstract, no_equal)
NodeTag type;
@@ -192,19 +192,19 @@ typedef struct Plan
* estimated execution costs for plan (see costsize.c for more info)
*/
/* count of disabled nodes */
- int disabled_nodes;
+ int disabled_nodes pg_node_attr(query_jumble_ignore);
/* cost expended before fetching any tuples */
- Cost startup_cost;
+ Cost startup_cost pg_node_attr(query_jumble_ignore);
/* total cost (assuming all tuples fetched) */
- Cost total_cost;
+ Cost total_cost pg_node_attr(query_jumble_ignore);
/*
* planner's estimate of result size of this plan step
*/
/* number of rows plan is expected to emit */
- Cardinality plan_rows;
+ Cardinality plan_rows pg_node_attr(query_jumble_ignore);
/* average row width in bytes */
- int plan_width;
+ int plan_width pg_node_attr(query_jumble_ignore);
/*
* information needed for parallel query
@@ -230,10 +230,10 @@ typedef struct Plan
/* implicitly-ANDed qual conditions */
List *qual;
/* input plan tree(s) */
- struct Plan *lefttree;
- struct Plan *righttree;
+ struct Plan *lefttree pg_node_attr(query_jumble_ignore);
+ struct Plan *righttree pg_node_attr(query_jumble_ignore);
/* Init Plan nodes (un-correlated expr subselects) */
- List *initPlan;
+ List *initPlan pg_node_attr(query_jumble_ignore);
/*
* Information for management of parameter-change-driven rescanning
@@ -351,7 +351,7 @@ typedef struct ModifyTable
/* per-target-table RETURNING tlists */
List *returningLists;
/* per-target-table FDW private data lists */
- List *fdwPrivLists;
+ List *fdwPrivLists pg_node_attr(query_jumble_ignore);
/* indices of FDW DM plans */
Bitmapset *fdwDirectModifyPlans;
/* PlanRowMarks (non-locking only) */
@@ -390,7 +390,7 @@ typedef struct Append
Plan plan;
/* RTIs of appendrel(s) formed by this node */
Bitmapset *apprelids;
- List *appendplans;
+ List *appendplans pg_node_attr(query_jumble_ignore);
/* # of asynchronous plans */
int nasyncplans;
@@ -420,7 +420,7 @@ typedef struct MergeAppend
/* RTIs of appendrel(s) formed by this node */
Bitmapset *apprelids;
- List *mergeplans;
+ List *mergeplans pg_node_attr(query_jumble_ignore);
/* these fields are just like the sort-key info in struct Sort: */
@@ -475,7 +475,7 @@ typedef struct RecursiveUnion
Oid *dupCollations pg_node_attr(array_size(numCols));
/* estimated number of groups in input */
- Cardinality numGroups;
+ long numGroups pg_node_attr(query_jumble_ignore);
} RecursiveUnion;
/* ----------------
@@ -489,7 +489,7 @@ typedef struct RecursiveUnion
typedef struct BitmapAnd
{
Plan plan;
- List *bitmapplans;
+ List *bitmapplans pg_node_attr(query_jumble_ignore);
} BitmapAnd;
/* ----------------
@@ -504,7 +504,7 @@ typedef struct BitmapOr
{
Plan plan;
bool isshared;
- List *bitmapplans;
+ List *bitmapplans pg_node_attr(query_jumble_ignore);
} BitmapOr;
/*
@@ -752,7 +752,7 @@ typedef enum SubqueryScanStatus
typedef struct SubqueryScan
{
Scan scan;
- Plan *subplan;
+ Plan *subplan pg_node_attr(query_jumble_ignore);
SubqueryScanStatus scanstatus;
} SubqueryScan;
@@ -880,7 +880,7 @@ typedef struct ForeignScan
/* expressions that FDW may evaluate */
List *fdw_exprs;
/* private data for FDW */
- List *fdw_private;
+ List *fdw_private pg_node_attr(query_jumble_ignore);
/* optional tlist describing scan tuple */
List *fdw_scan_tlist;
/* original quals not in scan.plan.qual */
@@ -918,7 +918,7 @@ typedef struct CustomScan
/* expressions that custom code may evaluate */
List *custom_exprs;
/* private data for custom code */
- List *custom_private;
+ List *custom_private pg_node_attr(query_jumble_ignore);
/* optional tlist describing scan tuple */
List *custom_scan_tlist;
/* RTIs generated by this scan */
@@ -929,7 +929,7 @@ typedef struct CustomScan
* static table of callback functions. So we don't copy the table itself,
* just reference the original one.
*/
- const struct CustomScanMethods *methods;
+ const struct CustomScanMethods *methods pg_node_attr(query_jumble_ignore);
} CustomScan;
/*
@@ -991,7 +991,7 @@ typedef struct NestLoop
typedef struct NestLoopParam
{
- pg_node_attr(no_equal, no_query_jumble)
+ pg_node_attr(no_equal)
NodeTag type;
/* number of the PARAM_EXEC Param to set */
@@ -1099,7 +1099,7 @@ typedef struct Memoize
* The maximum number of entries that the planner expects will fit in the
* cache, or 0 if unknown
*/
- uint32 est_entries;
+ uint32 est_entries pg_node_attr(query_jumble_ignore);
/* paramids from param_exprs */
Bitmapset *keyparamids;
@@ -1206,7 +1206,7 @@ typedef struct Agg
Oid *grpCollations pg_node_attr(array_size(numCols));
/* estimated number of groups in input */
- Cardinality numGroups;
+ long numGroups pg_node_attr(query_jumble_ignore);
/* for pass-by-ref transition data */
uint64 transitionSpace;
@@ -1415,7 +1415,7 @@ typedef struct Hash
bool skewInherit;
/* all other info is in the parent HashJoin node */
/* estimate total rows if parallel_aware */
- Cardinality rows_total;
+ Cardinality rows_total pg_node_attr(query_jumble_ignore);
} Hash;
/* ----------------
@@ -1446,7 +1446,7 @@ typedef struct SetOp
bool *cmpNullsFirst pg_node_attr(array_size(numCols));
/* estimated number of groups in left input */
- Cardinality numGroups;
+ long numGroups pg_node_attr(query_jumble_ignore);
} SetOp;
/* ----------------
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
index 1b4436f2ff6..4ccba6bcd03 100644
--- a/src/include/nodes/primnodes.h
+++ b/src/include/nodes/primnodes.h
@@ -1123,8 +1123,8 @@ typedef struct SubPlan
List *parParam; /* indices of input Params from parent plan */
List *args; /* exprs to pass as parParam values */
/* Estimated execution costs: */
- Cost startup_cost; /* one-time setup cost */
- Cost per_call_cost; /* cost for each subplan evaluation */
+ Cost startup_cost pg_node_attr(query_jumble_ignore); /* one-time setup cost */
+ Cost per_call_cost pg_node_attr(query_jumble_ignore); /* cost for each subplan evaluation */
} SubPlan;
/*
diff --git a/src/include/nodes/queryjumble.h b/src/include/nodes/queryjumble.h
index dcb36dcb44f..aeb20aa0587 100644
--- a/src/include/nodes/queryjumble.h
+++ b/src/include/nodes/queryjumble.h
@@ -15,6 +15,7 @@
#define QUERYJUMBLE_H
#include "nodes/parsenodes.h"
+#include "nodes/pathnodes.h"
/*
* Struct for tracking locations/lengths of constants during normalization
@@ -86,15 +87,26 @@ enum ComputeQueryIdType
COMPUTE_QUERY_ID_REGRESS,
};
+/* Values for the compute_plan_id GUC */
+enum ComputePlanIdType
+{
+ COMPUTE_PLAN_ID_OFF,
+ COMPUTE_PLAN_ID_ON,
+ COMPUTE_PLAN_ID_AUTO,
+ COMPUTE_PLAN_ID_REGRESS,
+};
+
/* GUC parameters */
extern PGDLLIMPORT int compute_query_id;
-
+extern PGDLLIMPORT int compute_plan_id;
extern const char *CleanQuerytext(const char *query, int *location, int *len);
extern JumbleState *JumbleQuery(Query *query);
extern void EnableQueryId(void);
+extern void EnablePlanId(void);
extern PGDLLIMPORT bool query_id_enabled;
+extern PGDLLIMPORT bool plan_id_enabled;
/*
* Returns whether query identifier computation has been enabled, either
@@ -110,4 +122,25 @@ IsQueryIdEnabled(void)
return query_id_enabled;
}
+/*
+ * Returns whether plan identifier computation has been enabled, either
+ * directly in the GUC or by a module when the setting is 'auto'.
+ */
+static inline bool
+IsPlanIdEnabled(void)
+{
+ if (compute_plan_id == COMPUTE_PLAN_ID_OFF)
+ return false;
+ if (compute_plan_id == COMPUTE_PLAN_ID_ON)
+ return true;
+ return plan_id_enabled;
+}
+
+/* Functions called for plan jumbling or extensions doing their own jumbling */
+extern JumbleState *InitializeJumbleState(bool record_clocations);
+//extern void AppendJumble(JumbleState *jstate, const unsigned char *item, Size size);
+extern void JumbleRangeTable(JumbleState *jstate, List *rtable);
+extern void JumbleNode(JumbleState *jstate, Node *node);
+extern uint64 HashJumbleState(JumbleState *jstate);
+
#endif /* QUERYJUMBLE_H */
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index 3016501ac05..45f00eb93a8 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -172,7 +172,7 @@ typedef struct PgBackendStatus
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
- /* plan identifier, optionally computed using planner_hook */
+ /* plan identifier, optionally computed after planning */
int64 st_plan_id;
} PgBackendStatus;
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index 7c1f26b182c..7ac025cca84 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -743,6 +743,17 @@ select explain_filter('explain (verbose) create table test_ctas as select 1');
Query Identifier: N
(3 rows)
+-- Test compute_plan_id
+set compute_plan_id = on;
+select explain_filter('explain (verbose) select * from int8_tbl i8');
+ explain_filter
+----------------------------------------------------------------
+ Seq Scan on public.int8_tbl i8 (cost=N.N..N.N rows=N width=N)
+ Output: q1, q2
+ Query Identifier: N
+ Plan Identifier: N
+(4 rows)
+
-- Test SERIALIZE option
select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
explain_filter
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 4286c266e17..29af9bd2de9 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1794,9 +1794,10 @@ pg_stat_activity| SELECT s.datid,
s.backend_xid,
s.backend_xmin,
s.query_id,
+ s.plan_id,
s.query,
s.backend_type
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
LEFT JOIN pg_database d ON ((s.datid = d.oid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_all_indexes| SELECT c.oid AS relid,
@@ -1926,7 +1927,7 @@ pg_stat_gssapi| SELECT pid,
gss_princ AS principal,
gss_enc AS encrypted,
gss_delegation AS credentials_delegated
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
WHERE (client_port IS NOT NULL);
pg_stat_io| SELECT backend_type,
object,
@@ -2156,7 +2157,7 @@ pg_stat_replication| SELECT s.pid,
w.sync_priority,
w.sync_state,
w.reply_time
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
JOIN pg_stat_get_wal_senders() w(pid, state, sent_lsn, write_lsn, flush_lsn, replay_lsn, write_lag, flush_lag, replay_lag, sync_priority, sync_state, reply_time) ON ((s.pid = w.pid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_replication_slots| SELECT s.slot_name,
@@ -2193,7 +2194,7 @@ pg_stat_ssl| SELECT pid,
ssl_client_dn AS client_dn,
ssl_client_serial AS client_serial,
ssl_issuer_dn AS issuer_dn
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
WHERE (client_port IS NOT NULL);
pg_stat_subscription| SELECT su.oid AS subid,
su.subname,
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index ebdab42604b..0449f05c1e1 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -173,6 +173,10 @@ select explain_filter('explain (verbose) select * from int8_tbl i8');
select explain_filter('explain (verbose) declare test_cur cursor for select * from int8_tbl');
select explain_filter('explain (verbose) create table test_ctas as select 1');
+-- Test compute_plan_id
+set compute_plan_id = on;
+select explain_filter('explain (verbose) select * from int8_tbl i8');
+
-- Test SERIALIZE option
select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
select explain_filter('explain (analyze,serialize text,buffers,timing off) select * from int8_tbl i8');
base-commit: b39013b7b1b116b5d9be51f0919b472b58b3a28d
--
2.43.0
Attachments:
v6-0001-Add-plan_id-support-to-the-core.patchtext/x-diff; name=v6-0001-Add-plan_id-support-to-the-core.patchDownload
From 3d40912f2cba7ba2c2edae06a4a3d016607a1513 Mon Sep 17 00:00:00 2001
From: Andrey Kazachkov <andrey.kazachkov@tantorlabs.ru>
Date: Tue, 16 Dec 2025 19:52:39 +0300
Subject: [PATCH v6] Add plan_id support to the core.
---
doc/src/sgml/config.sgml | 34 ++++
doc/src/sgml/monitoring.sgml | 16 ++
src/backend/catalog/system_views.sql | 1 +
src/backend/commands/explain.c | 17 ++
src/backend/executor/execMain.c | 10 +-
src/backend/nodes/gen_node_support.pl | 34 +++-
src/backend/nodes/queryjumblefuncs.c | 183 +++++++++++-------
src/backend/optimizer/plan/planner.c | 19 ++
src/backend/optimizer/plan/setrefs.c | 9 +
src/backend/postmaster/launch_backend.c | 3 +
src/backend/utils/adt/pgstatfuncs.c | 7 +-
src/backend/utils/misc/guc_parameters.dat | 7 +
src/backend/utils/misc/guc_tables.c | 18 ++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/catalog/pg_proc.dat | 6 +-
src/include/nodes/pathnodes.h | 3 +
src/include/nodes/plannodes.h | 50 ++---
src/include/nodes/primnodes.h | 4 +-
src/include/nodes/queryjumble.h | 35 +++-
src/include/utils/backend_status.h | 2 +-
src/test/regress/expected/explain.out | 11 ++
src/test/regress/expected/rules.out | 9 +-
src/test/regress/sql/explain.sql | 4 +
23 files changed, 374 insertions(+), 109 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 405c9689bd0..4cdaf9c49ba 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -8864,6 +8864,40 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-compute-plan-id" xreflabel="compute_plan_id">
+ <term><varname>compute_plan_id</varname> (<type>enum</type>)
+ <indexterm>
+ <primary><varname>compute_plan_id</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Enables in-core computation of a plan identifier.
+ Plan identifiers can be displayed in the <link
+ linkend="monitoring-pg-stat-activity-view"><structname>pg_stat_activity</structname></link>
+ view or using <command>EXPLAIN</command>.
+ Note that an external module can alternatively be used if the
+ in-core plan identifier computation method is not acceptable.
+ In this case, in-core computation must be always disabled.
+ Valid values are <literal>off</literal> (always disabled),
+ <literal>on</literal> (always enabled), <literal>auto</literal>,
+ which lets modules that utilize plan identifiers enable
+ it automatically, and <literal>regress</literal> which
+ has the same effect as <literal>on</literal>, except that the
+ query identifier is not shown in the <literal>EXPLAIN</literal> output
+ in order to facilitate automated regression testing.
+ The default is <literal>auto</literal>.
+ </para>
+ <note>
+ <para>
+ To ensure that only one plan identifier is calculated and
+ displayed, extensions that calculate plan identifiers should
+ throw an error if a plan identifier has already been computed.
+ </para>
+ </note>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-log-statement-stats">
<term><varname>log_statement_stats</varname> (<type>boolean</type>)
<indexterm>
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 817fd9f4ca7..9e98fd2c198 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -983,6 +983,22 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>plan_id</structfield> <type>bigint</type>
+ </para>
+ <para>
+ Identifier of this backend's most recent query plan. If
+ <structfield>state</structfield> is <literal>active</literal> this
+ field shows the identifier of the currently executing query plan. In
+ all other states, it shows the identifier of last query plan that
+ was executed. Plan identifiers are not computed by default so this
+ field will be null unless <xref linkend="guc-compute-plan-id"/>
+ parameter is enabled or a third-party module that computes plan
+ identifiers is configured.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>query</structfield> <type>text</type>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 0a0f95f6bb9..35744a82f0b 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -919,6 +919,7 @@ CREATE VIEW pg_stat_activity AS
S.backend_xid,
S.backend_xmin,
S.query_id,
+ S.plan_id,
S.query,
S.backend_type
FROM pg_stat_get_activity(NULL) AS S
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 5a6390631eb..b21cd142111 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -818,6 +818,23 @@ ExplainPrintPlan(ExplainState *es, QueryDesc *queryDesc)
ExplainPropertyInteger("Query Identifier", NULL,
queryDesc->plannedstmt->queryId, es);
}
+
+ /*
+ * COMPUTE_PLAN_ID_REGRESS means COMPUTE_PLAN_ID_YES, but we don't show
+ * the queryid in any of the EXPLAIN plans to keep stable the results
+ * generated by regression test suites.
+ */
+ if (es->verbose && queryDesc->plannedstmt->planId != UINT64CONST(0) &&
+ compute_plan_id != COMPUTE_PLAN_ID_REGRESS)
+ {
+ /*
+ * Output the queryid as an int64 rather than a uint64 so we match
+ * what would be seen in the BIGINT pg_stat_activity.plan_id column.
+ */
+ ExplainPropertyInteger("Plan Identifier", NULL,
+ queryDesc->plannedstmt->planId, es);
+ }
+
}
/*
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 797d8b1ca1c..14f107d45b1 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -123,13 +123,15 @@ ExecutorStart(QueryDesc *queryDesc, int eflags)
{
/*
* In some cases (e.g. an EXECUTE statement or an execute message with the
- * extended query protocol) the query_id won't be reported, so do it now.
+ * extended query protocol) the query_id and plan_id won't be reported, so
+ * do it now.
*
- * Note that it's harmless to report the query_id multiple times, as the
- * call will be ignored if the top level query_id has already been
- * reported.
+ * Note that it's harmless to report the identifiers multiple times, as
+ * the call will be ignored if the top level query_id or plan_id has
+ * already been reported.
*/
pgstat_report_query_id(queryDesc->plannedstmt->queryId, false);
+ pgstat_report_plan_id(queryDesc->plannedstmt->planId, false);
if (ExecutorStart_hook)
(*ExecutorStart_hook) (queryDesc, eflags);
diff --git a/src/backend/nodes/gen_node_support.pl b/src/backend/nodes/gen_node_support.pl
index 9ecddb14231..eab144bd962 100644
--- a/src/backend/nodes/gen_node_support.pl
+++ b/src/backend/nodes/gen_node_support.pl
@@ -1291,6 +1291,7 @@ _jumble${n}(JumbleState *jstate, Node *node)
{
my $t = $node_type_info{$n}->{field_types}{$f};
my @a = @{ $node_type_info{$n}->{field_attrs}{$f} };
+ my $array_size_field;
my $query_jumble_ignore = $struct_no_query_jumble;
my $query_jumble_custom = 0;
my $query_jumble_location = 0;
@@ -1303,7 +1304,11 @@ _jumble${n}(JumbleState *jstate, Node *node)
{
$query_jumble_custom = 1;
}
- if ($a eq 'query_jumble_ignore')
+ elsif ($a =~ /^array_size\(([\w.]+)\)$/)
+ {
+ $array_size_field = $1;
+ }
+ elsif ($a eq 'query_jumble_ignore')
{
$query_jumble_ignore = 1;
}
@@ -1317,12 +1322,19 @@ _jumble${n}(JumbleState *jstate, Node *node)
}
}
+ next if $query_jumble_ignore;
+
if ($query_jumble_custom)
{
# Custom function that applies to one field of a node.
print $jff "\tJUMBLE_CUSTOM($n, $f);\n"
unless $query_jumble_ignore;
}
+ elsif ($t eq 'Bitmapset*')
+ {
+ print $jff "\tJUMBLE_BITMAPSET($f);\n"
+ unless $query_jumble_ignore;
+ }
elsif (($t =~ /^(\w+)\*$/ or $t =~ /^struct\s+(\w+)\*$/)
and elem $1, @node_types)
{
@@ -1352,6 +1364,26 @@ _jumble${n}(JumbleState *jstate, Node *node)
print $jff "\tJUMBLE_STRING($f);\n"
unless $query_jumble_ignore;
}
+ elsif ($t =~ /^(\w+)(\*|\[\w+\])$/ and elem $1, @scalar_types)
+ {
+ if (!defined $array_size_field)
+ {
+ die "no array size defined for $n.$f of type $t\n";
+ }
+ if ($node_type_info{$n}->{field_types}{$array_size_field} eq
+ 'List*')
+ {
+ print $jff
+ "\tJUMBLE_ARRAY($f, list_length(expr->$array_size_field));\n"
+ unless $query_jumble_ignore;
+ }
+ else
+ {
+ print $jff
+ "\tJUMBLE_ARRAY($f, expr->$array_size_field);\n"
+ unless $query_jumble_ignore;
+ }
+ }
else
{
print $jff "\tJUMBLE_FIELD($f);\n"
diff --git a/src/backend/nodes/queryjumblefuncs.c b/src/backend/nodes/queryjumblefuncs.c
index ffc230af427..a0d28827269 100644
--- a/src/backend/nodes/queryjumblefuncs.c
+++ b/src/backend/nodes/queryjumblefuncs.c
@@ -3,32 +3,31 @@
* queryjumblefuncs.c
* Query normalization and fingerprinting.
*
- * Normalization is a process whereby similar queries, typically differing only
- * in their constants (though the exact rules are somewhat more subtle than
- * that) are recognized as equivalent, and are tracked as a single entry. This
- * is particularly useful for non-prepared queries.
+ * Fingerprinting selectively serializes key fields within a tree structure,
+ * such as a Query or Plan tree, to create a unique identifier while ignoring
+ * extraneous details. These essential fields are concatenated into a jumble,
+ * from which a 64-bit hash is computed. Unlike regular serialization, this
+ * approach excludes irrelevant information.
+ *
+ * Use Cases:
*
- * Normalization is implemented by fingerprinting queries, selectively
- * serializing those fields of each query tree's nodes that are judged to be
- * essential to the query. This is referred to as a query jumble. This is
- * distinct from a regular serialization in that various extraneous
- * information is ignored as irrelevant or not essential to the query, such
- * as the collations of Vars and, most notably, the values of constants.
- *
- * This jumble is acquired at the end of parse analysis of each query, and
- * a 64-bit hash of it is stored into the query's Query.queryId field.
- * The server then copies this value around, making it available in plan
- * tree(s) generated from the query. The executor can then use this value
- * to blame query costs on the proper queryId.
- *
- * Arrays of two or more constants and PARAM_EXTERN parameters are "squashed"
- * and contribute only once to the jumble. This has the effect that queries
- * that differ only on the length of such lists have the same queryId.
- *
- *
- * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
- * Portions Copyright (c) 1994, Regents of the University of California
+ * 1. In-Core Query Normalization & Identification
*
+ * Fingerprinting is used to normalize query trees by generating a hash stored
+ * in the Query.queryId field. This ID is propagated to plan tree(s), allowing
+ * the executor to attribute query costs on the proper queryId. The process
+ * excludes information like typmod, collation, and most notably, the values
+ * of constants.
+ *
+ * Example: The following queries produce the same queryId:
+ *
+ * SELECT t.* FROM s1.t WHERE c1 = 1;
+ * SELECT t.* FROM s1.t WHERE c1 = 2;
+ *
+ * 2. Modified jumbling logic for extensions
+ *
+ * Extensions can modify the fingerprinting logic for queryId, or fingerprint
+ * other types of trees, such as a plan tree, to compute a plan identifier.
*
* IDENTIFICATION
* src/backend/nodes/queryjumblefuncs.c
@@ -50,6 +49,7 @@
/* GUC parameters */
int compute_query_id = COMPUTE_QUERY_ID_AUTO;
+int compute_plan_id = COMPUTE_PLAN_ID_AUTO;
/*
* True when compute_query_id is ON or AUTO, and a module requests them.
@@ -60,15 +60,21 @@ int compute_query_id = COMPUTE_QUERY_ID_AUTO;
*/
bool query_id_enabled = false;
-static JumbleState *InitJumble(void);
-static int64 DoJumble(JumbleState *jstate, Node *node);
+/*
+ * True when compute_plan_id is ON or AUTO, and a module requests them.
+ *
+ * Note that IsPlanIdEnabled() should be used instead of checking
+ * plan_id_enabled or compute_plan_id directly when we want to know
+ * whether query identifiers are computed in the core or not.
+ */
+bool plan_id_enabled = false;
+
static void AppendJumble(JumbleState *jstate,
const unsigned char *value, Size size);
static void FlushPendingNulls(JumbleState *jstate);
static void RecordConstLocation(JumbleState *jstate,
bool extern_param,
int location, int len);
-static void _jumbleNode(JumbleState *jstate, Node *node);
static void _jumbleList(JumbleState *jstate, Node *node);
static void _jumbleElements(JumbleState *jstate, List *elements, Node *node);
static void _jumbleParam(JumbleState *jstate, Node *node);
@@ -136,13 +142,13 @@ CleanQuerytext(const char *query, int *location, int *len)
JumbleState *
JumbleQuery(Query *query)
{
- JumbleState *jstate;
+ JumbleState *jstate = InitializeJumbleState(true);;
Assert(IsQueryIdEnabled());
- jstate = InitJumble();
-
- query->queryId = DoJumble(jstate, (Node *) query);
+ /* Compute query ID and mark the Query node with it */
+ JumbleNode(jstate, (Node *) query);
+ query->queryId = HashJumbleState(jstate);
/*
* If we are unlucky enough to get a hash of zero, use 1 instead for
@@ -173,44 +179,44 @@ EnableQueryId(void)
}
/*
- * InitJumble
- * Allocate a JumbleState object and make it ready to jumble.
+ * Enables plan identifier computation.
+ *
+ * Third-party plugins can use this function to inform core that they require
+ * a query identifier to be computed.
*/
-static JumbleState *
-InitJumble(void)
+void
+EnablePlanId(void)
{
- JumbleState *jstate;
+ if (compute_plan_id != COMPUTE_PLAN_ID_OFF)
+ plan_id_enabled = true;
+}
- jstate = palloc_object(JumbleState);
+/*
+ * InitializeJumbleState
+ * Allocate a JumbleState object and make it ready to jumble.
+ */
+JumbleState *
+InitializeJumbleState(bool record_clocations)
+{
+ JumbleState *jstate = (JumbleState *) palloc0(sizeof(JumbleState));
/* Set up workspace for query jumbling */
jstate->jumble = (unsigned char *) palloc(JUMBLE_SIZE);
jstate->jumble_len = 0;
- jstate->clocations_buf_size = 32;
- jstate->clocations = (LocationLen *) palloc(jstate->clocations_buf_size *
- sizeof(LocationLen));
- jstate->clocations_count = 0;
- jstate->highest_extern_param_id = 0;
- jstate->pending_nulls = 0;
- jstate->has_squashed_lists = false;
-#ifdef USE_ASSERT_CHECKING
- jstate->total_jumble_len = 0;
-#endif
+
+ if (record_clocations)
+ {
+ jstate->clocations_buf_size = 32;
+ jstate->clocations = (LocationLen *)
+ palloc(jstate->clocations_buf_size * sizeof(LocationLen));
+ }
return jstate;
}
-/*
- * DoJumble
- * Jumble the given Node using the given JumbleState and return the resulting
- * jumble hash.
- */
-static int64
-DoJumble(JumbleState *jstate, Node *node)
+uint64
+HashJumbleState(JumbleState *jstate)
{
- /* Jumble the given node */
- _jumbleNode(jstate, node);
-
/* Flush any pending NULLs before doing the final hash */
if (jstate->pending_nulls > 0)
FlushPendingNulls(jstate);
@@ -219,10 +225,9 @@ DoJumble(JumbleState *jstate, Node *node)
if (jstate->has_squashed_lists)
jstate->highest_extern_param_id = 0;
- /* Process the jumble buffer and produce the hash value */
- return DatumGetInt64(hash_any_extended(jstate->jumble,
- jstate->jumble_len,
- 0));
+ return DatumGetUInt64(hash_any_extended(jstate->jumble,
+ jstate->jumble_len,
+ 0));
}
/*
@@ -398,7 +403,7 @@ static void
RecordConstLocation(JumbleState *jstate, bool extern_param, int location, int len)
{
/* -1 indicates unknown or undefined location */
- if (location >= 0)
+ if (location >= 0 && jstate->clocations_buf_size > 0)
{
/* enlarge array if needed */
if (jstate->clocations_count >= jstate->clocations_buf_size)
@@ -526,7 +531,7 @@ IsSquashableConstantList(List *elements)
}
#define JUMBLE_NODE(item) \
- _jumbleNode(jstate, (Node *) expr->item)
+ JumbleNode(jstate, (Node *) expr->item)
#define JUMBLE_ELEMENTS(list, node) \
_jumbleElements(jstate, (List *) expr->list, node)
#define JUMBLE_LOCATION(location) \
@@ -544,6 +549,16 @@ do { \
else \
AppendJumble(jstate, (const unsigned char *) &(expr->item), sizeof(expr->item)); \
} while (0)
+#define JUMBLE_BITMAPSET(item) \
+do { \
+ if (expr->item) \
+ AppendJumble(jstate, (const unsigned char *) expr->item->words, sizeof(bitmapword) * expr->item->nwords); \
+} while(0)
+#define JUMBLE_ARRAY(item, len) \
+ do { \
+ if (len) \
+ AppendJumble(jstate, (const unsigned char *) expr->item, sizeof(*(expr->item)) * len); \
+ } while (0)
#define JUMBLE_STRING(str) \
do { \
if (expr->str) \
@@ -557,8 +572,8 @@ do { \
#include "queryjumblefuncs.funcs.c"
-static void
-_jumbleNode(JumbleState *jstate, Node *node)
+void
+JumbleNode(JumbleState *jstate, Node *node)
{
Node *expr = node;
#ifdef USE_ASSERT_CHECKING
@@ -612,7 +627,7 @@ _jumbleList(JumbleState *jstate, Node *node)
{
case T_List:
foreach(l, expr)
- _jumbleNode(jstate, lfirst(l));
+ JumbleNode(jstate, lfirst(l));
break;
case T_IntList:
foreach(l, expr)
@@ -668,7 +683,7 @@ _jumbleElements(JumbleState *jstate, List *elements, Node *node)
if (!normalize_list)
{
- _jumbleNode(jstate, (Node *) elements);
+ JumbleNode(jstate, (Node *) elements);
}
}
@@ -758,6 +773,40 @@ _jumbleVariableSetStmt(JumbleState *jstate, Node *node)
JUMBLE_LOCATION(location);
}
+/*
+ * Jumble the entries in the rangle table to map RT indexes to relations
+ *
+ * This ensures jumbled RT indexes (e.g. in a Scan or Modify node), are
+ * distinguished by the target of the RT entry, even if the index is the same.
+ */
+void
+JumbleRangeTable(JumbleState *jstate, List *rtable)
+{
+ ListCell *lc;
+
+ foreach(lc, rtable)
+ {
+ RangeTblEntry *expr = lfirst_node(RangeTblEntry, lc);
+
+ switch (expr->rtekind)
+ {
+ case RTE_RELATION:
+ JUMBLE_FIELD(relid);
+ break;
+ case RTE_CTE:
+ JUMBLE_STRING(ctename);
+ break;
+ default:
+
+ /*
+ * Ignore other targets, the jumble includes something identifying
+ * about them already
+ */
+ break;
+ }
+ }
+}
+
/*
* Custom query jumble function for RangeTblEntry.eref.
*/
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 8b22c30559b..0309566f602 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -37,6 +37,7 @@
#ifdef OPTIMIZER_DEBUG
#include "nodes/print.h"
#endif
+#include "nodes/queryjumble.h"
#include "nodes/supportnodes.h"
#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
@@ -578,6 +579,16 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions,
Assert(glob->finalrowmarks == NIL);
Assert(glob->resultRelations == NIL);
Assert(glob->appendRelations == NIL);
+
+ /*
+ * Initialize plan identifier jumble if needed
+ *
+ * Note the actual jumbling is done in the tree walk in
+ * set_plan_references
+ */
+ if (IsPlanIdEnabled())
+ glob->plan_jumble_state = InitializeJumbleState(false);
+
top_plan = set_plan_references(root, top_plan);
/* ... and the subplans (both regular subplans and initplans) */
Assert(list_length(glob->subplans) == list_length(glob->subroots));
@@ -645,6 +656,14 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions,
result->jitFlags |= PGJIT_DEFORM;
}
+ if (IsPlanIdEnabled())
+ {
+ JumbleRangeTable(glob->plan_jumble_state, glob->finalrtable);
+ result->planId = HashJumbleState(glob->plan_jumble_state);
+ pfree(glob->plan_jumble_state->jumble);
+ pfree(glob->plan_jumble_state);
+ }
+
/* Allow plugins to take control before we discard "glob" */
if (planner_shutdown_hook)
(*planner_shutdown_hook) (glob, parse, query_string, result);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index cd7ea1e6b58..947bd980c66 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -19,6 +19,7 @@
#include "catalog/pg_type.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
+#include "nodes/queryjumble.h"
#include "optimizer/optimizer.h"
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
@@ -1338,6 +1339,14 @@ set_plan_refs(PlannerInfo *root, Plan *plan, int rtoffset)
plan->lefttree = set_plan_refs(root, plan->lefttree, rtoffset);
plan->righttree = set_plan_refs(root, plan->righttree, rtoffset);
+ /*
+ * If enabled, append significant information to the plan identifier
+ * jumble (we do this here since we're already walking the tree in a
+ * near-final state)
+ */
+ if (IsPlanIdEnabled())
+ JumbleNode(root->glob->plan_jumble_state, (Node *) plan);
+
return plan;
}
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 98f7c4848c9..b64c88db200 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -118,6 +118,7 @@ typedef struct
bool redirection_done;
bool IsBinaryUpgrade;
bool query_id_enabled;
+ bool plan_id_enabled;
int max_safe_fds;
int MaxBackends;
int num_pmchild_slots;
@@ -757,6 +758,7 @@ save_backend_variables(BackendParameters *param,
param->redirection_done = redirection_done;
param->IsBinaryUpgrade = IsBinaryUpgrade;
param->query_id_enabled = query_id_enabled;
+ param->plan_id_enabled = plan_id_enabled;
param->max_safe_fds = max_safe_fds;
param->MaxBackends = MaxBackends;
@@ -1019,6 +1021,7 @@ restore_backend_variables(BackendParameters *param)
redirection_done = param->redirection_done;
IsBinaryUpgrade = param->IsBinaryUpgrade;
query_id_enabled = param->query_id_enabled;
+ plan_id_enabled = param->plan_id_enabled;
max_safe_fds = param->max_safe_fds;
MaxBackends = param->MaxBackends;
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index ef6fffe60b9..854c9f8bffd 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -351,7 +351,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum
pg_stat_get_activity(PG_FUNCTION_ARGS)
{
-#define PG_STAT_GET_ACTIVITY_COLS 31
+#define PG_STAT_GET_ACTIVITY_COLS 32
int num_backends = pgstat_fetch_stat_numbackends();
int curr_backend;
int pid = PG_ARGISNULL(0) ? -1 : PG_GETARG_INT32(0);
@@ -665,6 +665,10 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
nulls[30] = true;
else
values[30] = Int64GetDatum(beentry->st_query_id);
+ if (beentry->st_plan_id == INT64CONST(0))
+ nulls[31] = true;
+ else
+ values[31] = UInt64GetDatum(beentry->st_plan_id);
}
else
{
@@ -694,6 +698,7 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
nulls[28] = true;
nulls[29] = true;
nulls[30] = true;
+ nulls[31] = true;
}
tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index 3b9d8349078..d9f5cf31c34 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -462,6 +462,13 @@
check_hook => 'check_commit_ts_buffers',
},
+{ name => 'compute_plan_id', type => 'enum', context => 'PGC_SUSET', group => 'STATS_MONITORING',
+ short_desc => 'Enables in-core computation of plan identifiers.',
+ variable => 'compute_plan_id',
+ boot_val => 'COMPUTE_PLAN_ID_AUTO',
+ options => 'compute_plan_id_options',
+},
+
{ name => 'compute_query_id', type => 'enum', context => 'PGC_SUSET', group => 'STATS_MONITORING',
short_desc => 'Enables in-core computation of query identifiers.',
variable => 'compute_query_id',
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index f87b558c2c6..3a5d7ee32ab 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -318,6 +318,24 @@ static const struct config_enum_entry compute_query_id_options[] = {
{NULL, 0, false}
};
+/*
+ * Although only "on" and "off" are documented, we accept
+ * all the likely variants of "on" and "off".
+ */
+static const struct config_enum_entry compute_plan_id_options[] = {
+ {"auto", COMPUTE_PLAN_ID_AUTO, false},
+ {"regress", COMPUTE_PLAN_ID_REGRESS, false},
+ {"on", COMPUTE_PLAN_ID_ON, false},
+ {"off", COMPUTE_PLAN_ID_OFF, false},
+ {"true", COMPUTE_PLAN_ID_ON, true},
+ {"false", COMPUTE_PLAN_ID_OFF, true},
+ {"yes", COMPUTE_PLAN_ID_ON, true},
+ {"no", COMPUTE_PLAN_ID_OFF, true},
+ {"1", COMPUTE_PLAN_ID_ON, true},
+ {"0", COMPUTE_PLAN_ID_OFF, true},
+ {NULL, 0, false}
+};
+
/*
* Although only "on", "off", and "partition" are documented, we
* accept all the likely variants of "on" and "off".
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index dc9e2255f8a..3d0e42f083b 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -674,6 +674,7 @@
# - Monitoring -
#compute_query_id = auto
+#compute_plan_id = auto
#log_statement_stats = off
#log_parser_stats = off
#log_planner_stats = off
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index fd9448ec7b9..2bf284e2732 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5653,9 +5653,9 @@
proname => 'pg_stat_get_activity', prorows => '100', proisstrict => 'f',
proretset => 't', provolatile => 's', proparallel => 'r',
prorettype => 'record', proargtypes => 'int4',
- proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,bool,int4,int8}',
- proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
- proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,gss_delegation,leader_pid,query_id}',
+ proallargtypes => '{int4,oid,int4,oid,text,text,text,text,text,timestamptz,timestamptz,timestamptz,timestamptz,inet,text,int4,xid,xid,text,bool,text,text,int4,text,numeric,text,bool,text,bool,bool,int4,int8,int8}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{pid,datid,pid,usesysid,application_name,state,query,wait_event_type,wait_event,xact_start,query_start,backend_start,state_change,client_addr,client_hostname,client_port,backend_xid,backend_xmin,backend_type,ssl,sslversion,sslcipher,sslbits,ssl_client_dn,ssl_client_serial,ssl_issuer_dn,gss_auth,gss_princ,gss_enc,gss_delegation,leader_pid,query_id,plan_id}',
prosrc => 'pg_stat_get_activity' },
{ oid => '6318', descr => 'describe wait events',
proname => 'pg_get_wait_events', procost => '10', prorows => '250',
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index b5ff456ef7f..c98f7b97283 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -189,6 +189,9 @@ typedef struct PlannerGlobal
/* extension state */
void **extension_state pg_node_attr(read_write_ignore);
int extension_state_allocated;
+
+ /* optional jumble state for plan identifier calculation */
+ struct JumbleState *plan_jumble_state pg_node_attr(read_write_ignore);
} PlannerGlobal;
/* macro for fetching the Plan associated with a SubPlan node */
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index c4393a94321..a5a2ccea1f7 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -70,7 +70,7 @@ typedef struct PlannedStmt
/* query identifier (copied from Query) */
int64 queryId;
- /* plan identifier (can be set by plugins) */
+ /* plan identifier (set when compute_plan_id is enabled or by plugins) */
int64 planId;
/* origin of plan */
@@ -184,7 +184,7 @@ typedef struct PlannedStmt
*/
typedef struct Plan
{
- pg_node_attr(abstract, no_equal, no_query_jumble)
+ pg_node_attr(abstract, no_equal)
NodeTag type;
@@ -192,19 +192,19 @@ typedef struct Plan
* estimated execution costs for plan (see costsize.c for more info)
*/
/* count of disabled nodes */
- int disabled_nodes;
+ int disabled_nodes pg_node_attr(query_jumble_ignore);
/* cost expended before fetching any tuples */
- Cost startup_cost;
+ Cost startup_cost pg_node_attr(query_jumble_ignore);
/* total cost (assuming all tuples fetched) */
- Cost total_cost;
+ Cost total_cost pg_node_attr(query_jumble_ignore);
/*
* planner's estimate of result size of this plan step
*/
/* number of rows plan is expected to emit */
- Cardinality plan_rows;
+ Cardinality plan_rows pg_node_attr(query_jumble_ignore);
/* average row width in bytes */
- int plan_width;
+ int plan_width pg_node_attr(query_jumble_ignore);
/*
* information needed for parallel query
@@ -230,10 +230,10 @@ typedef struct Plan
/* implicitly-ANDed qual conditions */
List *qual;
/* input plan tree(s) */
- struct Plan *lefttree;
- struct Plan *righttree;
+ struct Plan *lefttree pg_node_attr(query_jumble_ignore);
+ struct Plan *righttree pg_node_attr(query_jumble_ignore);
/* Init Plan nodes (un-correlated expr subselects) */
- List *initPlan;
+ List *initPlan pg_node_attr(query_jumble_ignore);
/*
* Information for management of parameter-change-driven rescanning
@@ -351,7 +351,7 @@ typedef struct ModifyTable
/* per-target-table RETURNING tlists */
List *returningLists;
/* per-target-table FDW private data lists */
- List *fdwPrivLists;
+ List *fdwPrivLists pg_node_attr(query_jumble_ignore);
/* indices of FDW DM plans */
Bitmapset *fdwDirectModifyPlans;
/* PlanRowMarks (non-locking only) */
@@ -390,7 +390,7 @@ typedef struct Append
Plan plan;
/* RTIs of appendrel(s) formed by this node */
Bitmapset *apprelids;
- List *appendplans;
+ List *appendplans pg_node_attr(query_jumble_ignore);
/* # of asynchronous plans */
int nasyncplans;
@@ -420,7 +420,7 @@ typedef struct MergeAppend
/* RTIs of appendrel(s) formed by this node */
Bitmapset *apprelids;
- List *mergeplans;
+ List *mergeplans pg_node_attr(query_jumble_ignore);
/* these fields are just like the sort-key info in struct Sort: */
@@ -475,7 +475,7 @@ typedef struct RecursiveUnion
Oid *dupCollations pg_node_attr(array_size(numCols));
/* estimated number of groups in input */
- Cardinality numGroups;
+ long numGroups pg_node_attr(query_jumble_ignore);
} RecursiveUnion;
/* ----------------
@@ -489,7 +489,7 @@ typedef struct RecursiveUnion
typedef struct BitmapAnd
{
Plan plan;
- List *bitmapplans;
+ List *bitmapplans pg_node_attr(query_jumble_ignore);
} BitmapAnd;
/* ----------------
@@ -504,7 +504,7 @@ typedef struct BitmapOr
{
Plan plan;
bool isshared;
- List *bitmapplans;
+ List *bitmapplans pg_node_attr(query_jumble_ignore);
} BitmapOr;
/*
@@ -752,7 +752,7 @@ typedef enum SubqueryScanStatus
typedef struct SubqueryScan
{
Scan scan;
- Plan *subplan;
+ Plan *subplan pg_node_attr(query_jumble_ignore);
SubqueryScanStatus scanstatus;
} SubqueryScan;
@@ -880,7 +880,7 @@ typedef struct ForeignScan
/* expressions that FDW may evaluate */
List *fdw_exprs;
/* private data for FDW */
- List *fdw_private;
+ List *fdw_private pg_node_attr(query_jumble_ignore);
/* optional tlist describing scan tuple */
List *fdw_scan_tlist;
/* original quals not in scan.plan.qual */
@@ -918,7 +918,7 @@ typedef struct CustomScan
/* expressions that custom code may evaluate */
List *custom_exprs;
/* private data for custom code */
- List *custom_private;
+ List *custom_private pg_node_attr(query_jumble_ignore);
/* optional tlist describing scan tuple */
List *custom_scan_tlist;
/* RTIs generated by this scan */
@@ -929,7 +929,7 @@ typedef struct CustomScan
* static table of callback functions. So we don't copy the table itself,
* just reference the original one.
*/
- const struct CustomScanMethods *methods;
+ const struct CustomScanMethods *methods pg_node_attr(query_jumble_ignore);
} CustomScan;
/*
@@ -991,7 +991,7 @@ typedef struct NestLoop
typedef struct NestLoopParam
{
- pg_node_attr(no_equal, no_query_jumble)
+ pg_node_attr(no_equal)
NodeTag type;
/* number of the PARAM_EXEC Param to set */
@@ -1099,7 +1099,7 @@ typedef struct Memoize
* The maximum number of entries that the planner expects will fit in the
* cache, or 0 if unknown
*/
- uint32 est_entries;
+ uint32 est_entries pg_node_attr(query_jumble_ignore);
/* paramids from param_exprs */
Bitmapset *keyparamids;
@@ -1206,7 +1206,7 @@ typedef struct Agg
Oid *grpCollations pg_node_attr(array_size(numCols));
/* estimated number of groups in input */
- Cardinality numGroups;
+ long numGroups pg_node_attr(query_jumble_ignore);
/* for pass-by-ref transition data */
uint64 transitionSpace;
@@ -1415,7 +1415,7 @@ typedef struct Hash
bool skewInherit;
/* all other info is in the parent HashJoin node */
/* estimate total rows if parallel_aware */
- Cardinality rows_total;
+ Cardinality rows_total pg_node_attr(query_jumble_ignore);
} Hash;
/* ----------------
@@ -1446,7 +1446,7 @@ typedef struct SetOp
bool *cmpNullsFirst pg_node_attr(array_size(numCols));
/* estimated number of groups in left input */
- Cardinality numGroups;
+ long numGroups pg_node_attr(query_jumble_ignore);
} SetOp;
/* ----------------
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
index 1b4436f2ff6..4ccba6bcd03 100644
--- a/src/include/nodes/primnodes.h
+++ b/src/include/nodes/primnodes.h
@@ -1123,8 +1123,8 @@ typedef struct SubPlan
List *parParam; /* indices of input Params from parent plan */
List *args; /* exprs to pass as parParam values */
/* Estimated execution costs: */
- Cost startup_cost; /* one-time setup cost */
- Cost per_call_cost; /* cost for each subplan evaluation */
+ Cost startup_cost pg_node_attr(query_jumble_ignore); /* one-time setup cost */
+ Cost per_call_cost pg_node_attr(query_jumble_ignore); /* cost for each subplan evaluation */
} SubPlan;
/*
diff --git a/src/include/nodes/queryjumble.h b/src/include/nodes/queryjumble.h
index dcb36dcb44f..aeb20aa0587 100644
--- a/src/include/nodes/queryjumble.h
+++ b/src/include/nodes/queryjumble.h
@@ -15,6 +15,7 @@
#define QUERYJUMBLE_H
#include "nodes/parsenodes.h"
+#include "nodes/pathnodes.h"
/*
* Struct for tracking locations/lengths of constants during normalization
@@ -86,15 +87,26 @@ enum ComputeQueryIdType
COMPUTE_QUERY_ID_REGRESS,
};
+/* Values for the compute_plan_id GUC */
+enum ComputePlanIdType
+{
+ COMPUTE_PLAN_ID_OFF,
+ COMPUTE_PLAN_ID_ON,
+ COMPUTE_PLAN_ID_AUTO,
+ COMPUTE_PLAN_ID_REGRESS,
+};
+
/* GUC parameters */
extern PGDLLIMPORT int compute_query_id;
-
+extern PGDLLIMPORT int compute_plan_id;
extern const char *CleanQuerytext(const char *query, int *location, int *len);
extern JumbleState *JumbleQuery(Query *query);
extern void EnableQueryId(void);
+extern void EnablePlanId(void);
extern PGDLLIMPORT bool query_id_enabled;
+extern PGDLLIMPORT bool plan_id_enabled;
/*
* Returns whether query identifier computation has been enabled, either
@@ -110,4 +122,25 @@ IsQueryIdEnabled(void)
return query_id_enabled;
}
+/*
+ * Returns whether plan identifier computation has been enabled, either
+ * directly in the GUC or by a module when the setting is 'auto'.
+ */
+static inline bool
+IsPlanIdEnabled(void)
+{
+ if (compute_plan_id == COMPUTE_PLAN_ID_OFF)
+ return false;
+ if (compute_plan_id == COMPUTE_PLAN_ID_ON)
+ return true;
+ return plan_id_enabled;
+}
+
+/* Functions called for plan jumbling or extensions doing their own jumbling */
+extern JumbleState *InitializeJumbleState(bool record_clocations);
+//extern void AppendJumble(JumbleState *jstate, const unsigned char *item, Size size);
+extern void JumbleRangeTable(JumbleState *jstate, List *rtable);
+extern void JumbleNode(JumbleState *jstate, Node *node);
+extern uint64 HashJumbleState(JumbleState *jstate);
+
#endif /* QUERYJUMBLE_H */
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index 3016501ac05..45f00eb93a8 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -172,7 +172,7 @@ typedef struct PgBackendStatus
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
- /* plan identifier, optionally computed using planner_hook */
+ /* plan identifier, optionally computed after planning */
int64 st_plan_id;
} PgBackendStatus;
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index 7c1f26b182c..7ac025cca84 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -743,6 +743,17 @@ select explain_filter('explain (verbose) create table test_ctas as select 1');
Query Identifier: N
(3 rows)
+-- Test compute_plan_id
+set compute_plan_id = on;
+select explain_filter('explain (verbose) select * from int8_tbl i8');
+ explain_filter
+----------------------------------------------------------------
+ Seq Scan on public.int8_tbl i8 (cost=N.N..N.N rows=N width=N)
+ Output: q1, q2
+ Query Identifier: N
+ Plan Identifier: N
+(4 rows)
+
-- Test SERIALIZE option
select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
explain_filter
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 4286c266e17..29af9bd2de9 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1794,9 +1794,10 @@ pg_stat_activity| SELECT s.datid,
s.backend_xid,
s.backend_xmin,
s.query_id,
+ s.plan_id,
s.query,
s.backend_type
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
LEFT JOIN pg_database d ON ((s.datid = d.oid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_all_indexes| SELECT c.oid AS relid,
@@ -1926,7 +1927,7 @@ pg_stat_gssapi| SELECT pid,
gss_princ AS principal,
gss_enc AS encrypted,
gss_delegation AS credentials_delegated
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
WHERE (client_port IS NOT NULL);
pg_stat_io| SELECT backend_type,
object,
@@ -2156,7 +2157,7 @@ pg_stat_replication| SELECT s.pid,
w.sync_priority,
w.sync_state,
w.reply_time
- FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM ((pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
JOIN pg_stat_get_wal_senders() w(pid, state, sent_lsn, write_lsn, flush_lsn, replay_lsn, write_lag, flush_lag, replay_lag, sync_priority, sync_state, reply_time) ON ((s.pid = w.pid)))
LEFT JOIN pg_authid u ON ((s.usesysid = u.oid)));
pg_stat_replication_slots| SELECT s.slot_name,
@@ -2193,7 +2194,7 @@ pg_stat_ssl| SELECT pid,
ssl_client_dn AS client_dn,
ssl_client_serial AS client_serial,
ssl_issuer_dn AS issuer_dn
- FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id)
+ FROM pg_stat_get_activity(NULL::integer) s(datid, pid, usesysid, application_name, state, query, wait_event_type, wait_event, xact_start, query_start, backend_start, state_change, client_addr, client_hostname, client_port, backend_xid, backend_xmin, backend_type, ssl, sslversion, sslcipher, sslbits, ssl_client_dn, ssl_client_serial, ssl_issuer_dn, gss_auth, gss_princ, gss_enc, gss_delegation, leader_pid, query_id, plan_id)
WHERE (client_port IS NOT NULL);
pg_stat_subscription| SELECT su.oid AS subid,
su.subname,
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index ebdab42604b..0449f05c1e1 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -173,6 +173,10 @@ select explain_filter('explain (verbose) select * from int8_tbl i8');
select explain_filter('explain (verbose) declare test_cur cursor for select * from int8_tbl');
select explain_filter('explain (verbose) create table test_ctas as select 1');
+-- Test compute_plan_id
+set compute_plan_id = on;
+select explain_filter('explain (verbose) select * from int8_tbl i8');
+
-- Test SERIALIZE option
select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
select explain_filter('explain (analyze,serialize text,buffers,timing off) select * from int8_tbl i8');
base-commit: b39013b7b1b116b5d9be51f0919b472b58b3a28d
--
2.43.0
On Thu, Dec 25, 2025 at 05:33:11PM +0300, Андрей Казачков wrote:
I’ve been testing the proposed v5 plan id work and found out instability of
computing the plan identifier after feeding different query texts that
produces the same physical plan trees but with different plan ids. The main
pattern regards with fields of Plan node structures that depend on positions of
RTEs in a RTE list.
FWIW, I don't think that we have a clear agreement about what would be
a good enough ID for plan trees, as it may be also a per-vendor
computation that fills specific user requirements.
+ /*
+ * COMPUTE_PLAN_ID_REGRESS means COMPUTE_PLAN_ID_YES, but we don't show
+ * the queryid in any of the EXPLAIN plans to keep stable the results
+ * generated by regression test suites.
+ */
+ if (es->verbose && queryDesc->plannedstmt->planId != UINT64CONST(0) &&
+ compute_plan_id != COMPUTE_PLAN_ID_REGRESS)
+ {
+ /*
+ * Output the queryid as an int64 rather than a uint64 so we match
+ * what would be seen in the BIGINT pg_stat_activity.plan_id column.
+ */
+ ExplainPropertyInteger("Plan Identifier", NULL,
+ queryDesc->plannedstmt->planId, es);
+ }
Now, looking at this block of code, I am wondering if you don't have a
point here even without compute_plan_id.. Could there be merit in
showing this information for an EXPLAIN if this field is not zero?
With EXPLAIN being pluggable in a hook, I doubt that it matters much,
but I am wondering if providing this information could make the work
of some extensions easier.
--
Michael