Make EXPLAIN generate a generic plan for a parameterized query

Started by Laurenz Albeover 3 years ago28 messages
#1Laurenz Albe
laurenz.albe@cybertec.at
1 attachment(s)

Today you get

test=> EXPLAIN SELECT * FROM tab WHERE col = $1;
ERROR: there is no parameter $1

which makes sense. Nonetheless, it would be great to get a generic plan
for such a query. Sometimes you don't have the parameters (if you grab
the statement from "pg_stat_statements", or if it is from an error message
in the log, and you didn't enable "log_parameter_max_length_on_error").
Sometimes it is just very painful to substitute the 25 parameters from
the detail message.

With the attached patch you can get the following:

test=> SET plan_cache_mode = force_generic_plan;
SET
test=> EXPLAIN (COSTS OFF) SELECT * FROM pg_proc WHERE oid = $1;
QUERY PLAN
═══════════════════════════════════════════════
Index Scan using pg_proc_oid_index on pg_proc
Index Cond: (oid = $1)
(2 rows)

That's not the same as a full-fledged EXPLAIN (ANALYZE, BUFFERS),
but it can definitely be helpful.

I tied that behavior to the setting of "plan_cache_mode" where you
are guaranteed to get a generic plan; I couldn't think of a better way.

Yours,
Laurenz Albe

Attachments:

0001-Add-EXPLAIN-support-for-parameterized-statements.patchtext/x-patch; charset=UTF-8; name=0001-Add-EXPLAIN-support-for-parameterized-statements.patchDownload
From 2bc91581acd478d4648176b58745cadb835d5fbc Mon Sep 17 00:00:00 2001
From: Laurenz Albe <laurenz.albe@cybertec.at>
Date: Tue, 11 Oct 2022 13:05:31 +0200
Subject: [PATCH] Add EXPLAIN support for parameterized statements

If "plan_cache_mode = force_generic_plan", allow EXPLAIN to
generate generic plans for parameterized statements (that
have parameter placeholders like $1 in the statement text).

This repurposes hooks used by PL/pgSQL, so we better not try
to do that inside PL/pgSQL.
---
 doc/src/sgml/ref/explain.sgml         | 10 +++++
 src/backend/parser/analyze.c          | 53 +++++++++++++++++++++++++++
 src/test/regress/expected/explain.out | 28 ++++++++++++++
 src/test/regress/sql/explain.sql      | 13 +++++++
 4 files changed, 104 insertions(+)

diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
index d4895b9d7d..928d67b9b4 100644
--- a/doc/src/sgml/ref/explain.sgml
+++ b/doc/src/sgml/ref/explain.sgml
@@ -321,6 +321,16 @@ ROLLBACK;
    execution, and on machines that have relatively slow operating
    system calls for obtaining the time of day.
   </para>
+
+  <para>
+   If <xref linkend="guc-plan-cache_mode"/> is set to
+   <literal>force_generic_plan</literal>, you can use <command>EXPLAIN</command>
+   to generate generic plans for statements that contain placeholders like
+   <literal>$1</literal> without knowing the actual parameter type or value.
+   Note that expressions like <literal>$1 + $2</literal> are ambiguous if you
+   don't specify the parameter data types, so you may have to add explicit type
+   casts in such cases.
+  </para>
  </refsect1>
 
  <refsect1>
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 6688c2a865..c481d45376 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -52,6 +52,7 @@
 #include "utils/guc.h"
 #include "utils/queryjumble.h"
 #include "utils/rel.h"
+#include "utils/plancache.h"
 #include "utils/syscache.h"
 
 
@@ -86,6 +87,10 @@ static Query *transformCallStmt(ParseState *pstate,
 								CallStmt *stmt);
 static void transformLockingClause(ParseState *pstate, Query *qry,
 								   LockingClause *lc, bool pushedDown);
+static Node * fakeUnknownParam(ParseState *pstate, ParamRef *pref);
+static Node * coerceUnknownParam(ParseState *pstate, Param *param,
+								 Oid targetTypeId, int32 targetTypeMod,
+								 int location);
 #ifdef RAW_EXPRESSION_COVERAGE_TEST
 static bool test_raw_expression_coverage(Node *node, void *context);
 #endif
@@ -2895,6 +2900,22 @@ transformExplainStmt(ParseState *pstate, ExplainStmt *stmt)
 {
 	Query	   *result;
 
+	/*
+	 * If we EXPLAIN a statement and are certain to generate a generic plan,
+	 * we can tolerate undefined parameters.  For that purpose, supply
+	 * parameters of type "unknown" and coerce them to the appropriate type
+	 * as needed.
+	 * If we are called from PL/pgSQL, the hooks are already set for the
+	 * purpose of resolving variables, and we don't want to disturb that.
+	 */
+	if (plan_cache_mode == PLAN_CACHE_MODE_FORCE_GENERIC_PLAN &&
+		pstate->p_paramref_hook == NULL &&
+		pstate->p_coerce_param_hook == NULL)
+	{
+		pstate->p_paramref_hook = fakeUnknownParam;
+		pstate->p_coerce_param_hook = coerceUnknownParam;
+	}
+
 	/* transform contained query, allowing SELECT INTO */
 	stmt->query = (Node *) transformOptionalSelectInto(pstate, stmt->query);
 
@@ -3466,6 +3487,38 @@ applyLockingClause(Query *qry, Index rtindex,
 	qry->rowMarks = lappend(qry->rowMarks, rc);
 }
 
+/*
+ * Return an "unknown" parameter for use with EXPLAIN of a parameterized
+ * statement.
+ */
+Node *
+fakeUnknownParam(ParseState *pstate, ParamRef *pref)
+{
+	Param  *param;
+
+	param = makeNode(Param);
+	param->paramkind = PARAM_EXTERN;
+	param->paramid = pref->number;
+	param->paramtype = UNKNOWNOID;
+	param->paramtypmod = -1;
+	param->paramcollid = InvalidOid;
+	param->location = pref->location;
+
+	return (Node *)param;
+}
+
+/*
+ * Set the parameter's type from "unknown" to the target type.
+ */
+Node *
+coerceUnknownParam(ParseState *pstate, Param *param, Oid targetTypeId,
+				   int32 targetTypeMod, int location)
+{
+	param->paramtype = targetTypeId;
+
+	return (Node *)param;
+}
+
 /*
  * Coverage testing for raw_expression_tree_walker().
  *
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index 48620edbc2..4f6755b172 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -517,3 +517,31 @@ select explain_filter('explain (verbose) select * from int8_tbl i8');
  Query Identifier: N
 (3 rows)
 
+reset compute_query_id;
+-- Test parameterized plans
+-- it will fail unless plan_cache_mode = force_generic_plan
+explain (costs off) select unique1 from tenk1 where thousand = $1;
+ERROR:  there is no parameter $1
+LINE 1: ...n (costs off) select unique1 from tenk1 where thousand = $1;
+                                                                    ^
+set plan_cache_mode=force_generic_plan;
+explain (costs off) select unique1 from tenk1 where thousand = $1;
+                   QUERY PLAN                    
+-------------------------------------------------
+ Bitmap Heap Scan on tenk1
+   Recheck Cond: (thousand = $1)
+   ->  Bitmap Index Scan on tenk1_thous_tenthous
+         Index Cond: (thousand = $1)
+(4 rows)
+
+-- it should also fail from PL/pgSQL
+do $_$declare
+  x text;
+begin
+  explain (costs off) select unique1 from tenk1 where thousand = $1 into x;
+end;$_$;
+ERROR:  there is no parameter $1
+LINE 1: ...in (costs off) select unique1 from tenk1 where thousand = $1
+                                                                     ^
+QUERY:  explain (costs off) select unique1 from tenk1 where thousand = $1
+CONTEXT:  PL/pgSQL function inline_code_block line 4 at SQL statement
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index ae3f7a308d..9d4827d299 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -128,3 +128,16 @@ select explain_filter('explain (verbose) select * from t1 where pg_temp.mysin(f1
 -- Test compute_query_id
 set compute_query_id = on;
 select explain_filter('explain (verbose) select * from int8_tbl i8');
+reset compute_query_id;
+
+-- Test parameterized plans
+-- it will fail unless plan_cache_mode = force_generic_plan
+explain (costs off) select unique1 from tenk1 where thousand = $1;
+set plan_cache_mode=force_generic_plan;
+explain (costs off) select unique1 from tenk1 where thousand = $1;
+-- it should also fail from PL/pgSQL
+do $_$declare
+  x text;
+begin
+  explain (costs off) select unique1 from tenk1 where thousand = $1 into x;
+end;$_$;
-- 
2.37.3

#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: Laurenz Albe (#1)
Re: Make EXPLAIN generate a generic plan for a parameterized query

Laurenz Albe <laurenz.albe@cybertec.at> writes:

Today you get
test=> EXPLAIN SELECT * FROM tab WHERE col = $1;
ERROR: there is no parameter $1
which makes sense. Nonetheless, it would be great to get a generic plan
for such a query.

I can see the point, but it also seems like it risks masking stupid
mistakes.

I tied that behavior to the setting of "plan_cache_mode" where you
are guaranteed to get a generic plan; I couldn't think of a better way.

I think it might be better to drive it off an explicit EXPLAIN option,
perhaps

EXPLAIN (GENERIC_PLAN) SELECT * FROM tab WHERE col = $1;

This option (bikeshedding on the name welcome) would have the effect
both of allowing unanchored Param symbols and of temporarily forcing
generic-plan mode, so that you don't need additional commands to
set and reset plan_cache_mode. We could also trivially add logic
to disallow the combination of ANALYZE and GENERIC_PLAN, which
would otherwise be a bit messy to prevent.

For context, it does already work to do this when you want to
investigate parameterized plans:

regression=# prepare foo as select * from tenk1 where unique1 = $1;
PREPARE
regression=# explain execute foo(42);
QUERY PLAN
-----------------------------------------------------------------------------
Index Scan using tenk1_unique1 on tenk1 (cost=0.29..8.30 rows=1 width=244)
Index Cond: (unique1 = 42)
(2 rows)

If you're trying to investigate custom-plan behavior, then you
need to supply concrete parameter values somewhere, so I think
this approach is fine for that case. (Shoehorning parameter
values into EXPLAIN options seems like it'd be a bit much.)
However, investigating generic-plan behavior this way is tedious,
since you have to invent irrelevant parameter values, plus mess
with plan_cache_mode or else run the explain half a dozen times.
So I can get behind having a more convenient way for that.

regards, tom lane

#3Julien Rouhaud
rjuju123@gmail.com
In reply to: Tom Lane (#2)
Re: Make EXPLAIN generate a generic plan for a parameterized query

On Tue, Oct 11, 2022 at 09:49:14AM -0400, Tom Lane wrote:

If you're trying to investigate custom-plan behavior, then you
need to supply concrete parameter values somewhere, so I think
this approach is fine for that case. (Shoehorning parameter
values into EXPLAIN options seems like it'd be a bit much.)
However, investigating generic-plan behavior this way is tedious,
since you have to invent irrelevant parameter values, plus mess
with plan_cache_mode or else run the explain half a dozen times.
So I can get behind having a more convenient way for that.

One common use case is tools identifying a slow query using pg_stat_statements,
identifying some missing indexes and then wanting to check whether the index
should be useful using some hypothetical index.

FTR I'm working on such a project and for now we have to go to great lengths
trying to "unjumble" such queries, so having a way to easily get the answer for
a generic plan would be great.

#4Laurenz Albe
laurenz.albe@cybertec.at
In reply to: Julien Rouhaud (#3)
1 attachment(s)
Re: Make EXPLAIN generate a generic plan for a parameterized query

On Wed, 2022-10-12 at 00:03 +0800, Julien Rouhaud wrote:

On Tue, Oct 11, 2022 at 09:49:14AM -0400, Tom Lane wrote:

I think it might be better to drive it off an explicit EXPLAIN option,
perhaps

EXPLAIN (GENERIC_PLAN) SELECT * FROM tab WHERE col = $1;

If you're trying to investigate custom-plan behavior, then you
need to supply concrete parameter values somewhere, so I think
this approach is fine for that case.  (Shoehorning parameter
values into EXPLAIN options seems like it'd be a bit much.)
However, investigating generic-plan behavior this way is tedious,
since you have to invent irrelevant parameter values, plus mess
with plan_cache_mode or else run the explain half a dozen times.
So I can get behind having a more convenient way for that.

One common use case is tools identifying a slow query using pg_stat_statements,
identifying some missing indexes and then wanting to check whether the index
should be useful using some hypothetical index.

FTR I'm working on such a project and for now we have to go to great lengths
trying to "unjumble" such queries, so having a way to easily get the answer for
a generic plan would be great.

Thanks for the suggestions and the encouragement. Here is a patch that
implements it with an EXPLAIN option named GENERIC_PLAN.

Yours,
Laurenz Albe

Attachments:

0001-Add-EXPLAIN-option-GENERIC_PLAN.patchtext/x-patch; charset=UTF-8; name=0001-Add-EXPLAIN-option-GENERIC_PLAN.patchDownload
From 85991f35f0de6e4e0a0b5843373e2ba3d5976c85 Mon Sep 17 00:00:00 2001
From: Laurenz Albe <laurenz.albe@cybertec.at>
Date: Tue, 25 Oct 2022 11:01:53 +0200
Subject: [PATCH] Add EXPLAIN option GENERIC_PLAN

This allows EXPLAIN to generate generic plans for parameterized
statements (that have parameter placeholders like $1 in the
statement text).

Author: Laurenz Albe
Discussion: https://postgr.es/m/0a29b954b10b57f0d135fe12aa0909bd41883eb0.camel%40cybertec.at
---
 doc/src/sgml/ref/explain.sgml         | 15 ++++++++++++++
 src/backend/commands/explain.c        |  9 +++++++++
 src/backend/parser/analyze.c          | 13 +++++++++++++
 src/backend/parser/parse_coerce.c     | 15 ++++++++++++++
 src/backend/parser/parse_expr.c       | 16 +++++++++++++++
 src/include/commands/explain.h        |  1 +
 src/include/parser/parse_node.h       |  2 ++
 src/test/regress/expected/explain.out | 28 +++++++++++++++++++++++++++
 src/test/regress/sql/explain.sql      | 16 +++++++++++++++
 9 files changed, 115 insertions(+)

diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
index d4895b9d7d..659d5c51b6 100644
--- a/doc/src/sgml/ref/explain.sgml
+++ b/doc/src/sgml/ref/explain.sgml
@@ -40,6 +40,7 @@ EXPLAIN [ ANALYZE ] [ VERBOSE ] <replaceable class="parameter">statement</replac
     VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
     COSTS [ <replaceable class="parameter">boolean</replaceable> ]
     SETTINGS [ <replaceable class="parameter">boolean</replaceable> ]
+    GENERIC_PLAN [ <replaceable class="parameter">boolean</replaceable> ]
     BUFFERS [ <replaceable class="parameter">boolean</replaceable> ]
     WAL [ <replaceable class="parameter">boolean</replaceable> ]
     TIMING [ <replaceable class="parameter">boolean</replaceable> ]
@@ -167,6 +168,20 @@ ROLLBACK;
     </listitem>
    </varlistentry>
 
+   <varlistentry>
+    <term><literal>GENERIC_PLAN</literal></term>
+    <listitem>
+     <para>
+      Generate a generic plan for the statement (see <xref linkend="sql-prepare"/>
+      for details about generic plans).  The statement can contain parameter
+      placeholders like <literal>$1</literal> and must be a statement that can
+      use parameters.  This option cannot be used together with
+      <likeral>ANALYZE</literal>, since a statement with unknown parameters
+      cannot be executed.
+     </para>
+    </listitem>
+   </varlistentry>
+
    <varlistentry>
     <term><literal>BUFFERS</literal></term>
     <listitem>
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index f86983c660..7b7ca3f90a 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -190,6 +190,8 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
 			es->wal = defGetBoolean(opt);
 		else if (strcmp(opt->defname, "settings") == 0)
 			es->settings = defGetBoolean(opt);
+		else if (strcmp(opt->defname, "generic_plan") == 0)
+			es->generic = defGetBoolean(opt);
 		else if (strcmp(opt->defname, "timing") == 0)
 		{
 			timing_set = true;
@@ -227,6 +229,13 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
 					 parser_errposition(pstate, opt->location)));
 	}
 
+	/* check that GENERIC_PLAN is not used with EXPLAIN ANALYZE */
+	if (es->generic && es->analyze)
+		ereport(ERROR,
+				(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+				 errmsg("EXPLAIN ANALYZE cannot be used with GENERIC_PLAN")));
+
+	/* check that WAL is used with EXPLAIN ANALYZE */
 	if (es->wal && !es->analyze)
 		ereport(ERROR,
 				(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 6688c2a865..c849765151 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -27,6 +27,7 @@
 #include "access/sysattr.h"
 #include "catalog/pg_proc.h"
 #include "catalog/pg_type.h"
+#include "commands/defrem.h"
 #include "miscadmin.h"
 #include "nodes/makefuncs.h"
 #include "nodes/nodeFuncs.h"
@@ -2894,6 +2895,18 @@ static Query *
 transformExplainStmt(ParseState *pstate, ExplainStmt *stmt)
 {
 	Query	   *result;
+	ListCell   *lc;
+	bool		generic_plan;
+
+	foreach(lc, stmt->options)
+	{
+		DefElem    *opt = (DefElem *) lfirst(lc);
+
+		if (strcmp(opt->defname, "generic_plan") == 0)
+			generic_plan = defGetBoolean(opt);
+		/* don't "break", as we want the last value */
+	}
+	pstate->p_generic_explain = generic_plan;
 
 	/* transform contained query, allowing SELECT INTO */
 	stmt->query = (Node *) transformOptionalSelectInto(pstate, stmt->query);
diff --git a/src/backend/parser/parse_coerce.c b/src/backend/parser/parse_coerce.c
index c4e958e4aa..171d8c60a8 100644
--- a/src/backend/parser/parse_coerce.c
+++ b/src/backend/parser/parse_coerce.c
@@ -369,6 +369,21 @@ coerce_type(ParseState *pstate, Node *node,
 
 		return result;
 	}
+	/*
+	 * If we are to generate a generic plan for EXPLAIN, simply let the
+	 * parameter be of the desired type.
+	 */
+	if (IsA(node, Param) &&
+		pstate != NULL && pstate->p_generic_explain)
+	{
+		Param *param = (Param *) node;
+
+		param->paramtype = targetTypeId;
+		param->paramtypmod = targetTypeMod;
+		param->location = location;
+
+		return node;
+	}
 	if (IsA(node, Param) &&
 		pstate != NULL && pstate->p_coerce_param_hook != NULL)
 	{
diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c
index 7aaf1c673f..75a24ed9b2 100644
--- a/src/backend/parser/parse_expr.c
+++ b/src/backend/parser/parse_expr.c
@@ -815,6 +815,22 @@ transformParamRef(ParseState *pstate, ParamRef *pref)
 {
 	Node	   *result;
 
+	/* for "generic_plan" EXPLAIN, supply an unknown parameter */
+	if (pstate->p_generic_explain)
+	{
+		Param  *param;
+
+		param = makeNode(Param);
+		param->paramkind = PARAM_EXTERN;
+		param->paramid = pref->number;
+		param->paramtype = UNKNOWNOID;
+		param->paramtypmod = -1;
+		param->paramcollid = InvalidOid;
+		param->location = pref->location;
+
+		return (Node *)param;
+	}
+
 	/*
 	 * The core parser knows nothing about Params.  If a hook is supplied,
 	 * call it.  If not, or if the hook returns NULL, throw a generic error.
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 9ebde089ae..524d355961 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -46,6 +46,7 @@ typedef struct ExplainState
 	bool		timing;			/* print detailed node timing */
 	bool		summary;		/* print total planning and execution timing */
 	bool		settings;		/* print modified settings */
+	bool		generic;		/* generate a generic plan */
 	ExplainFormat format;		/* output format */
 	/* state for output formatting --- not reset for each new plan tree */
 	int			indent;			/* current indentation level */
diff --git a/src/include/parser/parse_node.h b/src/include/parser/parse_node.h
index 962ebf65de..0095fdb347 100644
--- a/src/include/parser/parse_node.h
+++ b/src/include/parser/parse_node.h
@@ -202,6 +202,8 @@ struct ParseState
 										 * with FOR UPDATE/FOR SHARE */
 	bool		p_resolve_unknowns; /* resolve unknown-type SELECT outputs as
 									 * type text */
+	bool		p_generic_explain;	/* accept undefined parameter
+									 * placeholders */
 
 	QueryEnvironment *p_queryEnv;	/* curr env, incl refs to enclosing env */
 
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index 48620edbc2..53bb024813 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -517,3 +517,31 @@ select explain_filter('explain (verbose) select * from int8_tbl i8');
  Query Identifier: N
 (3 rows)
 
+-- Test EXPLAIN (GENERIC_PLAN)
+EXPLAIN (COSTS OFF, GENERIC_PLAN) SELECT unique1 FROM tenk1 WHERE thousand = $1;
+                   QUERY PLAN                    
+-------------------------------------------------
+ Bitmap Heap Scan on tenk1
+   Recheck Cond: (thousand = $1)
+   ->  Bitmap Index Scan on tenk1_thous_tenthous
+         Index Cond: (thousand = $1)
+(4 rows)
+
+-- should fail
+EXPLAIN (COSTS OFF, GENERIC_PLAN, ANALYZE) SELECT unique1 FROM tenk1 WHERE thousand = $1;
+ERROR:  EXPLAIN ANALYZE cannot be used with GENERIC_PLAN
+-- should also work in PL/pgSQL
+DO
+$$DECLARE
+    t text;
+BEGIN
+    FOR t IN
+        EXPLAIN (COSTS OFF, GENERIC_PLAN) SELECT unique1 FROM tenk1 WHERE thousand = $1
+    LOOP
+        RAISE NOTICE '%', t;
+    END LOOP;
+END;$$;
+NOTICE:  Bitmap Heap Scan on tenk1
+NOTICE:    Recheck Cond: (thousand = $1)
+NOTICE:    ->  Bitmap Index Scan on tenk1_thous_tenthous
+NOTICE:          Index Cond: (thousand = $1)
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index ae3f7a308d..c2438efdcc 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -128,3 +128,19 @@ select explain_filter('explain (verbose) select * from t1 where pg_temp.mysin(f1
 -- Test compute_query_id
 set compute_query_id = on;
 select explain_filter('explain (verbose) select * from int8_tbl i8');
+
+-- Test EXPLAIN (GENERIC_PLAN)
+EXPLAIN (COSTS OFF, GENERIC_PLAN) SELECT unique1 FROM tenk1 WHERE thousand = $1;
+-- should fail
+EXPLAIN (COSTS OFF, GENERIC_PLAN, ANALYZE) SELECT unique1 FROM tenk1 WHERE thousand = $1;
+-- should also work in PL/pgSQL
+DO
+$$DECLARE
+    t text;
+BEGIN
+    FOR t IN
+        EXPLAIN (COSTS OFF, GENERIC_PLAN) SELECT unique1 FROM tenk1 WHERE thousand = $1
+    LOOP
+        RAISE NOTICE '%', t;
+    END LOOP;
+END;$$;
-- 
2.37.3

#5Julien Rouhaud
rjuju123@gmail.com
In reply to: Laurenz Albe (#1)
Re: Make EXPLAIN generate a generic plan for a parameterized query

Hi,

On Tue, Oct 25, 2022 at 11:08:27AM +0200, Laurenz Albe wrote:

Here is a patch that
implements it with an EXPLAIN option named GENERIC_PLAN.

I only have a quick look at the patch for now. Any reason why you don't rely
on the existing explain_filter() function for emitting stable output (without
having to remove the costs)? It would also take care of checking that it works
in plpgsql.

#6Laurenz Albe
laurenz.albe@cybertec.at
In reply to: Julien Rouhaud (#5)
1 attachment(s)
Re: Make EXPLAIN generate a generic plan for a parameterized query

On Tue, 2022-10-25 at 19:03 +0800, Julien Rouhaud wrote:

On Tue, Oct 25, 2022 at 11:08:27AM +0200, Laurenz Albe wrote:

Here is a patch that
implements it with an EXPLAIN option named GENERIC_PLAN.

I only have a quick look at the patch for now.  Any reason why you don't rely
on the existing explain_filter() function for emitting stable output (without
having to remove the costs)?  It would also take care of checking that it works
in plpgsql.

No, there is no principled reason I did it like that. Version 2 does it like
you suggest.

Yours,
Laurenz Albe

Attachments:

0001-Add-EXPLAIN-option-GENERIC_PLAN.v2.patchtext/x-patch; charset=UTF-8; name=0001-Add-EXPLAIN-option-GENERIC_PLAN.v2.patchDownload
From 8704d51f5810619be152ae68faa5743dcf26c7a9 Mon Sep 17 00:00:00 2001
From: Laurenz Albe <laurenz.albe@cybertec.at>
Date: Fri, 28 Oct 2022 20:58:59 +0200
Subject: [PATCH] Add EXPLAIN option GENERIC_PLAN

This allows EXPLAIN to generate generic plans for parameterized
statements (that have parameter placeholders like $1 in the
statement text).

Author: Laurenz Albe
Discussion: https://postgr.es/m/0a29b954b10b57f0d135fe12aa0909bd41883eb0.camel%40cybertec.at
---
 doc/src/sgml/ref/explain.sgml         | 15 +++++++++++++++
 src/backend/commands/explain.c        |  9 +++++++++
 src/backend/parser/analyze.c          | 13 +++++++++++++
 src/backend/parser/parse_coerce.c     | 15 +++++++++++++++
 src/backend/parser/parse_expr.c       | 16 ++++++++++++++++
 src/include/commands/explain.h        |  1 +
 src/include/parser/parse_node.h       |  2 ++
 src/test/regress/expected/explain.out | 14 ++++++++++++++
 src/test/regress/sql/explain.sql      |  5 +++++
 9 files changed, 90 insertions(+)

diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
index d4895b9d7d..659d5c51b6 100644
--- a/doc/src/sgml/ref/explain.sgml
+++ b/doc/src/sgml/ref/explain.sgml
@@ -40,6 +40,7 @@ EXPLAIN [ ANALYZE ] [ VERBOSE ] <replaceable class="parameter">statement</replac
     VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
     COSTS [ <replaceable class="parameter">boolean</replaceable> ]
     SETTINGS [ <replaceable class="parameter">boolean</replaceable> ]
+    GENERIC_PLAN [ <replaceable class="parameter">boolean</replaceable> ]
     BUFFERS [ <replaceable class="parameter">boolean</replaceable> ]
     WAL [ <replaceable class="parameter">boolean</replaceable> ]
     TIMING [ <replaceable class="parameter">boolean</replaceable> ]
@@ -167,6 +168,20 @@ ROLLBACK;
     </listitem>
    </varlistentry>
 
+   <varlistentry>
+    <term><literal>GENERIC_PLAN</literal></term>
+    <listitem>
+     <para>
+      Generate a generic plan for the statement (see <xref linkend="sql-prepare"/>
+      for details about generic plans).  The statement can contain parameter
+      placeholders like <literal>$1</literal> and must be a statement that can
+      use parameters.  This option cannot be used together with
+      <likeral>ANALYZE</literal>, since a statement with unknown parameters
+      cannot be executed.
+     </para>
+    </listitem>
+   </varlistentry>
+
    <varlistentry>
     <term><literal>BUFFERS</literal></term>
     <listitem>
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index f86983c660..7b7ca3f90a 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -190,6 +190,8 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
 			es->wal = defGetBoolean(opt);
 		else if (strcmp(opt->defname, "settings") == 0)
 			es->settings = defGetBoolean(opt);
+		else if (strcmp(opt->defname, "generic_plan") == 0)
+			es->generic = defGetBoolean(opt);
 		else if (strcmp(opt->defname, "timing") == 0)
 		{
 			timing_set = true;
@@ -227,6 +229,13 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
 					 parser_errposition(pstate, opt->location)));
 	}
 
+	/* check that GENERIC_PLAN is not used with EXPLAIN ANALYZE */
+	if (es->generic && es->analyze)
+		ereport(ERROR,
+				(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+				 errmsg("EXPLAIN ANALYZE cannot be used with GENERIC_PLAN")));
+
+	/* check that WAL is used with EXPLAIN ANALYZE */
 	if (es->wal && !es->analyze)
 		ereport(ERROR,
 				(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 6688c2a865..c849765151 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -27,6 +27,7 @@
 #include "access/sysattr.h"
 #include "catalog/pg_proc.h"
 #include "catalog/pg_type.h"
+#include "commands/defrem.h"
 #include "miscadmin.h"
 #include "nodes/makefuncs.h"
 #include "nodes/nodeFuncs.h"
@@ -2894,6 +2895,18 @@ static Query *
 transformExplainStmt(ParseState *pstate, ExplainStmt *stmt)
 {
 	Query	   *result;
+	ListCell   *lc;
+	bool		generic_plan;
+
+	foreach(lc, stmt->options)
+	{
+		DefElem    *opt = (DefElem *) lfirst(lc);
+
+		if (strcmp(opt->defname, "generic_plan") == 0)
+			generic_plan = defGetBoolean(opt);
+		/* don't "break", as we want the last value */
+	}
+	pstate->p_generic_explain = generic_plan;
 
 	/* transform contained query, allowing SELECT INTO */
 	stmt->query = (Node *) transformOptionalSelectInto(pstate, stmt->query);
diff --git a/src/backend/parser/parse_coerce.c b/src/backend/parser/parse_coerce.c
index c4e958e4aa..171d8c60a8 100644
--- a/src/backend/parser/parse_coerce.c
+++ b/src/backend/parser/parse_coerce.c
@@ -369,6 +369,21 @@ coerce_type(ParseState *pstate, Node *node,
 
 		return result;
 	}
+	/*
+	 * If we are to generate a generic plan for EXPLAIN, simply let the
+	 * parameter be of the desired type.
+	 */
+	if (IsA(node, Param) &&
+		pstate != NULL && pstate->p_generic_explain)
+	{
+		Param *param = (Param *) node;
+
+		param->paramtype = targetTypeId;
+		param->paramtypmod = targetTypeMod;
+		param->location = location;
+
+		return node;
+	}
 	if (IsA(node, Param) &&
 		pstate != NULL && pstate->p_coerce_param_hook != NULL)
 	{
diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c
index 7aaf1c673f..75a24ed9b2 100644
--- a/src/backend/parser/parse_expr.c
+++ b/src/backend/parser/parse_expr.c
@@ -815,6 +815,22 @@ transformParamRef(ParseState *pstate, ParamRef *pref)
 {
 	Node	   *result;
 
+	/* for "generic_plan" EXPLAIN, supply an unknown parameter */
+	if (pstate->p_generic_explain)
+	{
+		Param  *param;
+
+		param = makeNode(Param);
+		param->paramkind = PARAM_EXTERN;
+		param->paramid = pref->number;
+		param->paramtype = UNKNOWNOID;
+		param->paramtypmod = -1;
+		param->paramcollid = InvalidOid;
+		param->location = pref->location;
+
+		return (Node *)param;
+	}
+
 	/*
 	 * The core parser knows nothing about Params.  If a hook is supplied,
 	 * call it.  If not, or if the hook returns NULL, throw a generic error.
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 9ebde089ae..524d355961 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -46,6 +46,7 @@ typedef struct ExplainState
 	bool		timing;			/* print detailed node timing */
 	bool		summary;		/* print total planning and execution timing */
 	bool		settings;		/* print modified settings */
+	bool		generic;		/* generate a generic plan */
 	ExplainFormat format;		/* output format */
 	/* state for output formatting --- not reset for each new plan tree */
 	int			indent;			/* current indentation level */
diff --git a/src/include/parser/parse_node.h b/src/include/parser/parse_node.h
index 962ebf65de..0095fdb347 100644
--- a/src/include/parser/parse_node.h
+++ b/src/include/parser/parse_node.h
@@ -202,6 +202,8 @@ struct ParseState
 										 * with FOR UPDATE/FOR SHARE */
 	bool		p_resolve_unknowns; /* resolve unknown-type SELECT outputs as
 									 * type text */
+	bool		p_generic_explain;	/* accept undefined parameter
+									 * placeholders */
 
 	QueryEnvironment *p_queryEnv;	/* curr env, incl refs to enclosing env */
 
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index 48620edbc2..18f7ac93c4 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -517,3 +517,17 @@ select explain_filter('explain (verbose) select * from int8_tbl i8');
  Query Identifier: N
 (3 rows)
 
+-- Test EXPLAIN (GENERIC_PLAN)
+select explain_filter('explain (generic_plan) select unique1 from tenk1 where thousand = $1');
+                                 explain_filter                                  
+---------------------------------------------------------------------------------
+ Bitmap Heap Scan on tenk1  (cost=N.N..N.N rows=N width=N)
+   Recheck Cond: (thousand = $N)
+   ->  Bitmap Index Scan on tenk1_thous_tenthous  (cost=N.N..N.N rows=N width=N)
+         Index Cond: (thousand = $N)
+(4 rows)
+
+-- should fail
+select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1 where thousand = $1');
+ERROR:  EXPLAIN ANALYZE cannot be used with GENERIC_PLAN
+CONTEXT:  PL/pgSQL function explain_filter(text) line 5 at FOR over EXECUTE statement
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index ae3f7a308d..fce031775a 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -128,3 +128,8 @@ select explain_filter('explain (verbose) select * from t1 where pg_temp.mysin(f1
 -- Test compute_query_id
 set compute_query_id = on;
 select explain_filter('explain (verbose) select * from int8_tbl i8');
+
+-- Test EXPLAIN (GENERIC_PLAN)
+select explain_filter('explain (generic_plan) select unique1 from tenk1 where thousand = $1');
+-- should fail
+select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1 where thousand = $1');
-- 
2.37.3

#7Andres Freund
andres@anarazel.de
In reply to: Laurenz Albe (#6)
Re: Make EXPLAIN generate a generic plan for a parameterized query

Hi,

On 2022-10-29 10:35:26 +0200, Laurenz Albe wrote:

On Tue, 2022-10-25 at 19:03 +0800, Julien Rouhaud wrote:

On Tue, Oct 25, 2022 at 11:08:27AM +0200, Laurenz Albe wrote:

Here is a patch that
implements it with an EXPLAIN option named GENERIC_PLAN.

I only have a quick look at the patch for now.� Any reason why you don't rely
on the existing explain_filter() function for emitting stable output (without
having to remove the costs)?� It would also take care of checking that it works
in plpgsql.

No, there is no principled reason I did it like that. Version 2 does it like
you suggest.

This fails to build the docs:

https://cirrus-ci.com/task/5609301511766016

[17:47:01.064] ref/explain.sgml:179: parser error : Opening and ending tag mismatch: likeral line 179 and literal
[17:47:01.064] <likeral>ANALYZE</literal>, since a statement with unknown parameters
[17:47:01.064] ^

Greetings,

Andres Freund

#8Laurenz Albe
laurenz.albe@cybertec.at
In reply to: Andres Freund (#7)
1 attachment(s)
Re: Make EXPLAIN generate a generic plan for a parameterized query

On Tue, 2022-12-06 at 10:17 -0800, Andres Freund wrote:

On 2022-10-29 10:35:26 +0200, Laurenz Albe wrote:

Here is a patch that
implements it with an EXPLAIN option named GENERIC_PLAN.

This fails to build the docs:

https://cirrus-ci.com/task/5609301511766016

[17:47:01.064] ref/explain.sgml:179: parser error : Opening and ending tag mismatch: likeral line 179 and literal
[17:47:01.064]       <likeral>ANALYZE</literal>, since a statement with unknown parameters
[17:47:01.064]                                 ^

*blush* Here is a fixed version.

Yours,
Laurenz Albe

Attachments:

0001-Add-EXPLAIN-option-GENERIC_PLAN.v3.patchtext/x-patch; charset=UTF-8; name=0001-Add-EXPLAIN-option-GENERIC_PLAN.v3.patchDownload
From 8704d51f5810619be152ae68faa5743dcf26c7a9 Mon Sep 17 00:00:00 2001
From: Laurenz Albe <laurenz.albe@cybertec.at>
Date: Fri, 28 Oct 2022 20:58:59 +0200
Subject: [PATCH] Add EXPLAIN option GENERIC_PLAN

This allows EXPLAIN to generate generic plans for parameterized
statements (that have parameter placeholders like $1 in the
statement text).

Author: Laurenz Albe
Discussion: https://postgr.es/m/0a29b954b10b57f0d135fe12aa0909bd41883eb0.camel%40cybertec.at
---
 doc/src/sgml/ref/explain.sgml         | 15 +++++++++++++++
 src/backend/commands/explain.c        |  9 +++++++++
 src/backend/parser/analyze.c          | 13 +++++++++++++
 src/backend/parser/parse_coerce.c     | 15 +++++++++++++++
 src/backend/parser/parse_expr.c       | 16 ++++++++++++++++
 src/include/commands/explain.h        |  1 +
 src/include/parser/parse_node.h       |  2 ++
 src/test/regress/expected/explain.out | 14 ++++++++++++++
 src/test/regress/sql/explain.sql      |  5 +++++
 9 files changed, 90 insertions(+)

diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
index d4895b9d7d..659d5c51b6 100644
--- a/doc/src/sgml/ref/explain.sgml
+++ b/doc/src/sgml/ref/explain.sgml
@@ -40,6 +40,7 @@ EXPLAIN [ ANALYZE ] [ VERBOSE ] <replaceable class="parameter">statement</replac
     VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
     COSTS [ <replaceable class="parameter">boolean</replaceable> ]
     SETTINGS [ <replaceable class="parameter">boolean</replaceable> ]
+    GENERIC_PLAN [ <replaceable class="parameter">boolean</replaceable> ]
     BUFFERS [ <replaceable class="parameter">boolean</replaceable> ]
     WAL [ <replaceable class="parameter">boolean</replaceable> ]
     TIMING [ <replaceable class="parameter">boolean</replaceable> ]
@@ -167,6 +168,20 @@ ROLLBACK;
     </listitem>
    </varlistentry>
 
+   <varlistentry>
+    <term><literal>GENERIC_PLAN</literal></term>
+    <listitem>
+     <para>
+      Generate a generic plan for the statement (see <xref linkend="sql-prepare"/>
+      for details about generic plans).  The statement can contain parameter
+      placeholders like <literal>$1</literal> and must be a statement that can
+      use parameters.  This option cannot be used together with
+      <literal>ANALYZE</literal>, since a statement with unknown parameters
+      cannot be executed.
+     </para>
+    </listitem>
+   </varlistentry>
+
    <varlistentry>
     <term><literal>BUFFERS</literal></term>
     <listitem>
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index f86983c660..7b7ca3f90a 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -190,6 +190,8 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
 			es->wal = defGetBoolean(opt);
 		else if (strcmp(opt->defname, "settings") == 0)
 			es->settings = defGetBoolean(opt);
+		else if (strcmp(opt->defname, "generic_plan") == 0)
+			es->generic = defGetBoolean(opt);
 		else if (strcmp(opt->defname, "timing") == 0)
 		{
 			timing_set = true;
@@ -227,6 +229,13 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
 					 parser_errposition(pstate, opt->location)));
 	}
 
+	/* check that GENERIC_PLAN is not used with EXPLAIN ANALYZE */
+	if (es->generic && es->analyze)
+		ereport(ERROR,
+				(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+				 errmsg("EXPLAIN ANALYZE cannot be used with GENERIC_PLAN")));
+
+	/* check that WAL is used with EXPLAIN ANALYZE */
 	if (es->wal && !es->analyze)
 		ereport(ERROR,
 				(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 6688c2a865..c849765151 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -27,6 +27,7 @@
 #include "access/sysattr.h"
 #include "catalog/pg_proc.h"
 #include "catalog/pg_type.h"
+#include "commands/defrem.h"
 #include "miscadmin.h"
 #include "nodes/makefuncs.h"
 #include "nodes/nodeFuncs.h"
@@ -2894,6 +2895,18 @@ static Query *
 transformExplainStmt(ParseState *pstate, ExplainStmt *stmt)
 {
 	Query	   *result;
+	ListCell   *lc;
+	bool		generic_plan;
+
+	foreach(lc, stmt->options)
+	{
+		DefElem    *opt = (DefElem *) lfirst(lc);
+
+		if (strcmp(opt->defname, "generic_plan") == 0)
+			generic_plan = defGetBoolean(opt);
+		/* don't "break", as we want the last value */
+	}
+	pstate->p_generic_explain = generic_plan;
 
 	/* transform contained query, allowing SELECT INTO */
 	stmt->query = (Node *) transformOptionalSelectInto(pstate, stmt->query);
diff --git a/src/backend/parser/parse_coerce.c b/src/backend/parser/parse_coerce.c
index c4e958e4aa..171d8c60a8 100644
--- a/src/backend/parser/parse_coerce.c
+++ b/src/backend/parser/parse_coerce.c
@@ -369,6 +369,21 @@ coerce_type(ParseState *pstate, Node *node,
 
 		return result;
 	}
+	/*
+	 * If we are to generate a generic plan for EXPLAIN, simply let the
+	 * parameter be of the desired type.
+	 */
+	if (IsA(node, Param) &&
+		pstate != NULL && pstate->p_generic_explain)
+	{
+		Param *param = (Param *) node;
+
+		param->paramtype = targetTypeId;
+		param->paramtypmod = targetTypeMod;
+		param->location = location;
+
+		return node;
+	}
 	if (IsA(node, Param) &&
 		pstate != NULL && pstate->p_coerce_param_hook != NULL)
 	{
diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c
index 7aaf1c673f..75a24ed9b2 100644
--- a/src/backend/parser/parse_expr.c
+++ b/src/backend/parser/parse_expr.c
@@ -815,6 +815,22 @@ transformParamRef(ParseState *pstate, ParamRef *pref)
 {
 	Node	   *result;
 
+	/* for "generic_plan" EXPLAIN, supply an unknown parameter */
+	if (pstate->p_generic_explain)
+	{
+		Param  *param;
+
+		param = makeNode(Param);
+		param->paramkind = PARAM_EXTERN;
+		param->paramid = pref->number;
+		param->paramtype = UNKNOWNOID;
+		param->paramtypmod = -1;
+		param->paramcollid = InvalidOid;
+		param->location = pref->location;
+
+		return (Node *)param;
+	}
+
 	/*
 	 * The core parser knows nothing about Params.  If a hook is supplied,
 	 * call it.  If not, or if the hook returns NULL, throw a generic error.
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 9ebde089ae..524d355961 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -46,6 +46,7 @@ typedef struct ExplainState
 	bool		timing;			/* print detailed node timing */
 	bool		summary;		/* print total planning and execution timing */
 	bool		settings;		/* print modified settings */
+	bool		generic;		/* generate a generic plan */
 	ExplainFormat format;		/* output format */
 	/* state for output formatting --- not reset for each new plan tree */
 	int			indent;			/* current indentation level */
diff --git a/src/include/parser/parse_node.h b/src/include/parser/parse_node.h
index 962ebf65de..0095fdb347 100644
--- a/src/include/parser/parse_node.h
+++ b/src/include/parser/parse_node.h
@@ -202,6 +202,8 @@ struct ParseState
 										 * with FOR UPDATE/FOR SHARE */
 	bool		p_resolve_unknowns; /* resolve unknown-type SELECT outputs as
 									 * type text */
+	bool		p_generic_explain;	/* accept undefined parameter
+									 * placeholders */
 
 	QueryEnvironment *p_queryEnv;	/* curr env, incl refs to enclosing env */
 
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index 48620edbc2..18f7ac93c4 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -517,3 +517,17 @@ select explain_filter('explain (verbose) select * from int8_tbl i8');
  Query Identifier: N
 (3 rows)
 
+-- Test EXPLAIN (GENERIC_PLAN)
+select explain_filter('explain (generic_plan) select unique1 from tenk1 where thousand = $1');
+                                 explain_filter                                  
+---------------------------------------------------------------------------------
+ Bitmap Heap Scan on tenk1  (cost=N.N..N.N rows=N width=N)
+   Recheck Cond: (thousand = $N)
+   ->  Bitmap Index Scan on tenk1_thous_tenthous  (cost=N.N..N.N rows=N width=N)
+         Index Cond: (thousand = $N)
+(4 rows)
+
+-- should fail
+select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1 where thousand = $1');
+ERROR:  EXPLAIN ANALYZE cannot be used with GENERIC_PLAN
+CONTEXT:  PL/pgSQL function explain_filter(text) line 5 at FOR over EXECUTE statement
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index ae3f7a308d..fce031775a 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -128,3 +128,8 @@ select explain_filter('explain (verbose) select * from t1 where pg_temp.mysin(f1
 -- Test compute_query_id
 set compute_query_id = on;
 select explain_filter('explain (verbose) select * from int8_tbl i8');
+
+-- Test EXPLAIN (GENERIC_PLAN)
+select explain_filter('explain (generic_plan) select unique1 from tenk1 where thousand = $1');
+-- should fail
+select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1 where thousand = $1');
-- 
2.37.3

#9Michel Pelletier
pelletier.michel@gmail.com
In reply to: Laurenz Albe (#8)
Re: Make EXPLAIN generate a generic plan for a parameterized query

On Wed, Dec 7, 2022 at 3:23 AM Laurenz Albe <laurenz.albe@cybertec.at>
wrote:

On Tue, 2022-12-06 at 10:17 -0800, Andres Freund wrote:

On 2022-10-29 10:35:26 +0200, Laurenz Albe wrote:

Here is a patch that
implements it with an EXPLAIN option named GENERIC_PLAN.

This fails to build the docs:

https://cirrus-ci.com/task/5609301511766016

[17:47:01.064] ref/explain.sgml:179: parser error : Opening and ending

tag mismatch: likeral line 179 and literal

[17:47:01.064] <likeral>ANALYZE</literal>, since a statement with

unknown parameters

[17:47:01.064] ^

*blush* Here is a fixed version.

I built and tested this patch for review and it works well, although I got
the following warning when building:

analyze.c: In function 'transformStmt':
analyze.c:2919:35: warning: 'generic_plan' may be used uninitialized in
this function [-Wmaybe-uninitialized]
2919 | pstate->p_generic_explain = generic_plan;
| ~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~
analyze.c:2909:25: note: 'generic_plan' was declared here
2909 | bool generic_plan;
| ^~~~~~~~~~~~

#10Laurenz Albe
laurenz.albe@cybertec.at
In reply to: Michel Pelletier (#9)
1 attachment(s)
Re: Make EXPLAIN generate a generic plan for a parameterized query

On Tue, 2022-12-27 at 14:37 -0800, Michel Pelletier wrote:

I built and tested this patch for review and it works well, although I got the following warning when building:

analyze.c: In function 'transformStmt':
analyze.c:2919:35: warning: 'generic_plan' may be used uninitialized in this function [-Wmaybe-uninitialized]
 2919 |         pstate->p_generic_explain = generic_plan;
      |         ~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~
analyze.c:2909:25: note: 'generic_plan' was declared here
 2909 |         bool            generic_plan;
      |                         ^~~~~~~~~~~~

Thanks for checking. The variable should indeed be initialized, although
my compiler didn't complain.

Attached is a fixed version.

Yours,
Laurenz Albe

Attachments:

0001-Add-EXPLAIN-option-GENERIC_PLAN.v4.patchtext/x-patch; charset=UTF-8; name=0001-Add-EXPLAIN-option-GENERIC_PLAN.v4.patchDownload
From baf60d9480d8022866d1ed77b00c7b8506f97f70 Mon Sep 17 00:00:00 2001
From: Laurenz Albe <laurenz.albe@cybertec.at>
Date: Mon, 9 Jan 2023 17:37:40 +0100
Subject: [PATCH] Add EXPLAIN option GENERIC_PLAN

This allows EXPLAIN to generate generic plans for parameterized
statements (that have parameter placeholders like $1 in the
statement text).

Author: Laurenz Albe
Discussion: https://postgr.es/m/0a29b954b10b57f0d135fe12aa0909bd41883eb0.camel%40cybertec.at
---
 doc/src/sgml/ref/explain.sgml         | 15 +++++++++++++++
 src/backend/commands/explain.c        |  9 +++++++++
 src/backend/parser/analyze.c          | 13 +++++++++++++
 src/backend/parser/parse_coerce.c     | 15 +++++++++++++++
 src/backend/parser/parse_expr.c       | 16 ++++++++++++++++
 src/include/commands/explain.h        |  1 +
 src/include/parser/parse_node.h       |  2 ++
 src/test/regress/expected/explain.out | 14 ++++++++++++++
 src/test/regress/sql/explain.sql      |  5 +++++
 9 files changed, 90 insertions(+)

diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
index d4895b9d7d..221f905a59 100644
--- a/doc/src/sgml/ref/explain.sgml
+++ b/doc/src/sgml/ref/explain.sgml
@@ -40,6 +40,7 @@ EXPLAIN [ ANALYZE ] [ VERBOSE ] <replaceable class="parameter">statement</replac
     VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
     COSTS [ <replaceable class="parameter">boolean</replaceable> ]
     SETTINGS [ <replaceable class="parameter">boolean</replaceable> ]
+    GENERIC_PLAN [ <replaceable class="parameter">boolean</replaceable> ]
     BUFFERS [ <replaceable class="parameter">boolean</replaceable> ]
     WAL [ <replaceable class="parameter">boolean</replaceable> ]
     TIMING [ <replaceable class="parameter">boolean</replaceable> ]
@@ -167,6 +168,20 @@ ROLLBACK;
     </listitem>
    </varlistentry>
 
+   <varlistentry>
+    <term><literal>GENERIC_PLAN</literal></term>
+    <listitem>
+     <para>
+      Generate a generic plan for the statement (see <xref linkend="sql-prepare"/>
+      for details about generic plans).  The statement can contain parameter
+      placeholders like <literal>$1</literal> and must be a statement that can
+      use parameters.  This option cannot be used together with
+      <literal>ANALYZE</literal>, since a statement with unknown parameters
+      cannot be executed.
+     </para>
+    </listitem>
+   </varlistentry>
+
    <varlistentry>
     <term><literal>BUFFERS</literal></term>
     <listitem>
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index e4621ef8d6..7ee3d24da2 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -190,6 +190,8 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
 			es->wal = defGetBoolean(opt);
 		else if (strcmp(opt->defname, "settings") == 0)
 			es->settings = defGetBoolean(opt);
+		else if (strcmp(opt->defname, "generic_plan") == 0)
+			es->generic = defGetBoolean(opt);
 		else if (strcmp(opt->defname, "timing") == 0)
 		{
 			timing_set = true;
@@ -227,6 +229,13 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
 					 parser_errposition(pstate, opt->location)));
 	}
 
+	/* check that GENERIC_PLAN is not used with EXPLAIN ANALYZE */
+	if (es->generic && es->analyze)
+		ereport(ERROR,
+				(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+				 errmsg("EXPLAIN ANALYZE cannot be used with GENERIC_PLAN")));
+
+	/* check that WAL is used with EXPLAIN ANALYZE */
 	if (es->wal && !es->analyze)
 		ereport(ERROR,
 				(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 5b90974e83..8b56eadf7e 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -27,6 +27,7 @@
 #include "access/sysattr.h"
 #include "catalog/pg_proc.h"
 #include "catalog/pg_type.h"
+#include "commands/defrem.h"
 #include "miscadmin.h"
 #include "nodes/makefuncs.h"
 #include "nodes/nodeFuncs.h"
@@ -2905,6 +2906,18 @@ static Query *
 transformExplainStmt(ParseState *pstate, ExplainStmt *stmt)
 {
 	Query	   *result;
+	ListCell   *lc;
+	bool		generic_plan = false;
+
+	foreach(lc, stmt->options)
+	{
+		DefElem    *opt = (DefElem *) lfirst(lc);
+
+		if (strcmp(opt->defname, "generic_plan") == 0)
+			generic_plan = defGetBoolean(opt);
+		/* don't "break", as we want the last value */
+	}
+	pstate->p_generic_explain = generic_plan;
 
 	/* transform contained query, allowing SELECT INTO */
 	stmt->query = (Node *) transformOptionalSelectInto(pstate, stmt->query);
diff --git a/src/backend/parser/parse_coerce.c b/src/backend/parser/parse_coerce.c
index 34757da0fa..b9c8dc1a25 100644
--- a/src/backend/parser/parse_coerce.c
+++ b/src/backend/parser/parse_coerce.c
@@ -369,6 +369,21 @@ coerce_type(ParseState *pstate, Node *node,
 
 		return result;
 	}
+	/*
+	 * If we are to generate a generic plan for EXPLAIN, simply let the
+	 * parameter be of the desired type.
+	 */
+	if (IsA(node, Param) &&
+		pstate != NULL && pstate->p_generic_explain)
+	{
+		Param *param = (Param *) node;
+
+		param->paramtype = targetTypeId;
+		param->paramtypmod = targetTypeMod;
+		param->location = location;
+
+		return node;
+	}
 	if (IsA(node, Param) &&
 		pstate != NULL && pstate->p_coerce_param_hook != NULL)
 	{
diff --git a/src/backend/parser/parse_expr.c b/src/backend/parser/parse_expr.c
index 53e904ca6d..26f42bcc38 100644
--- a/src/backend/parser/parse_expr.c
+++ b/src/backend/parser/parse_expr.c
@@ -808,6 +808,22 @@ transformParamRef(ParseState *pstate, ParamRef *pref)
 {
 	Node	   *result;
 
+	/* for "generic_plan" EXPLAIN, supply an unknown parameter */
+	if (pstate->p_generic_explain)
+	{
+		Param  *param;
+
+		param = makeNode(Param);
+		param->paramkind = PARAM_EXTERN;
+		param->paramid = pref->number;
+		param->paramtype = UNKNOWNOID;
+		param->paramtypmod = -1;
+		param->paramcollid = InvalidOid;
+		param->location = pref->location;
+
+		return (Node *)param;
+	}
+
 	/*
 	 * The core parser knows nothing about Params.  If a hook is supplied,
 	 * call it.  If not, or if the hook returns NULL, throw a generic error.
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 7c1071ddd1..3d3e632a0c 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -46,6 +46,7 @@ typedef struct ExplainState
 	bool		timing;			/* print detailed node timing */
 	bool		summary;		/* print total planning and execution timing */
 	bool		settings;		/* print modified settings */
+	bool		generic;		/* generate a generic plan */
 	ExplainFormat format;		/* output format */
 	/* state for output formatting --- not reset for each new plan tree */
 	int			indent;			/* current indentation level */
diff --git a/src/include/parser/parse_node.h b/src/include/parser/parse_node.h
index 1a3792236a..90be0bd5ed 100644
--- a/src/include/parser/parse_node.h
+++ b/src/include/parser/parse_node.h
@@ -207,6 +207,8 @@ struct ParseState
 										 * with FOR UPDATE/FOR SHARE */
 	bool		p_resolve_unknowns; /* resolve unknown-type SELECT outputs as
 									 * type text */
+	bool		p_generic_explain;	/* accept undefined parameter
+									 * placeholders */
 
 	QueryEnvironment *p_queryEnv;	/* curr env, incl refs to enclosing env */
 
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index 48620edbc2..18f7ac93c4 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -517,3 +517,17 @@ select explain_filter('explain (verbose) select * from int8_tbl i8');
  Query Identifier: N
 (3 rows)
 
+-- Test EXPLAIN (GENERIC_PLAN)
+select explain_filter('explain (generic_plan) select unique1 from tenk1 where thousand = $1');
+                                 explain_filter                                  
+---------------------------------------------------------------------------------
+ Bitmap Heap Scan on tenk1  (cost=N.N..N.N rows=N width=N)
+   Recheck Cond: (thousand = $N)
+   ->  Bitmap Index Scan on tenk1_thous_tenthous  (cost=N.N..N.N rows=N width=N)
+         Index Cond: (thousand = $N)
+(4 rows)
+
+-- should fail
+select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1 where thousand = $1');
+ERROR:  EXPLAIN ANALYZE cannot be used with GENERIC_PLAN
+CONTEXT:  PL/pgSQL function explain_filter(text) line 5 at FOR over EXECUTE statement
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index ae3f7a308d..fce031775a 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -128,3 +128,8 @@ select explain_filter('explain (verbose) select * from t1 where pg_temp.mysin(f1
 -- Test compute_query_id
 set compute_query_id = on;
 select explain_filter('explain (verbose) select * from int8_tbl i8');
+
+-- Test EXPLAIN (GENERIC_PLAN)
+select explain_filter('explain (generic_plan) select unique1 from tenk1 where thousand = $1');
+-- should fail
+select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1 where thousand = $1');
-- 
2.39.0

#11Jim Jones
jim.jones@uni-muenster.de
In reply to: Laurenz Albe (#10)
Re: Make EXPLAIN generate a generic plan for a parameterized query

Hi Laurenz,

I'm testing your patch and the GENERIC_PLAN parameter seems to work just
OK ..

db=# CREATE TABLE t (col numeric);
CREATE TABLE
db=# CREATE INDEX t_col_idx ON t (col);
CREATE INDEX
db=# INSERT INTO t SELECT random() FROM generate_series(1,100000) ;
INSERT 0 100000
db=# EXPLAIN (GENERIC_PLAN) SELECT * FROM t WHERE col = $1;
                                QUERY PLAN
---------------------------------------------------------------------------
 Bitmap Heap Scan on t  (cost=15.27..531.67 rows=368 width=32)
   Recheck Cond: (col = $1)
   ->  Bitmap Index Scan on t_col_idx  (cost=0.00..15.18 rows=368 width=0)
         Index Cond: (col = $1)
(4 rows)

.. the error message when combining GENERIC_PLAN with ANALYSE also works
as expected

db=# EXPLAIN (ANALYSE, GENERIC_PLAN) SELECT * FROM t WHERE col = $1;
ERROR:  EXPLAIN ANALYZE cannot be used with GENERIC_PLAN

.. and the system also does not throw an error when it's used along
other parameters, e.g. VERBOSE, WAL, SUMMARY, etc.

However, when GENERIC_PLAN is used combined with BUFFERS, the 'Buffers'
node is shown the first time the query executed in a session:

psql (16devel)
Type "help" for help.

postgres=# \c db
You are now connected to database "db" as user "postgres".
db=# EXPLAIN (BUFFERS, GENERIC_PLAN) SELECT * FROM t WHERE col = $1;
                               QUERY PLAN
-------------------------------------------------------------------------
 Index Only Scan using t_col_idx on t  (cost=0.42..4.44 rows=1 width=11)
   Index Cond: (col = $1)
 Planning:
   Buffers: shared hit=62
(4 rows)

db=# EXPLAIN (BUFFERS, GENERIC_PLAN) SELECT * FROM t WHERE col = $1;
                               QUERY PLAN
-------------------------------------------------------------------------
 Index Only Scan using t_col_idx on t  (cost=0.42..4.44 rows=1 width=11)
   Index Cond: (col = $1)
(2 rows)

db=# EXPLAIN (BUFFERS, GENERIC_PLAN) SELECT * FROM t WHERE col = $1;
                               QUERY PLAN
-------------------------------------------------------------------------
 Index Only Scan using t_col_idx on t  (cost=0.42..4.44 rows=1 width=11)
   Index Cond: (col = $1)
(2 rows)

Is it the expected behaviour?

Also, this new parameter seems only to work between parenthesis
`(GENERIC_PLAN)`:

db=# EXPLAIN GENERIC_PLAN SELECT * FROM t WHERE col = $1;
ERROR:  syntax error at or near "GENERIC_PLAN"
LINE 1: EXPLAIN GENERIC_PLAN SELECT * FROM t WHERE col = $1;

If it's intended to be consistent with the other "single parameters",
perhaps it should work also without parenthesis? e.g.

db=# EXPLAIN ANALYSE SELECT * FROM t WHERE col < 0.42;
                                                          QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------
 Index Only Scan using t_col_idx on t  (cost=0.42..1637.25 rows=41876
width=11) (actual time=0.103..6.293 rows=41932 loops=1)
   Index Cond: (col < 0.42)
   Heap Fetches: 0
 Planning Time: 0.071 ms
 Execution Time: 7.316 ms
(5 rows)

db=# EXPLAIN VERBOSE SELECT * FROM t WHERE col < 0.42;
                                      QUERY PLAN
---------------------------------------------------------------------------------------
 Index Only Scan using t_col_idx on public.t (cost=0.42..1637.25
rows=41876 width=11)
   Output: col
   Index Cond: (t.col < 0.42)
(3 rows)

On a very personal note: wouldn't just GENERIC (without _PLAN) suffice?
Don't bother with it if you disagree :-)

Cheers
Jim

Show quoted text

On 09.01.23 17:40, Laurenz Albe wrote:

On Tue, 2022-12-27 at 14:37 -0800, Michel Pelletier wrote:

I built and tested this patch for review and it works well, although I got the following warning when building:

analyze.c: In function 'transformStmt':
analyze.c:2919:35: warning: 'generic_plan' may be used uninitialized in this function [-Wmaybe-uninitialized]
 2919 |         pstate->p_generic_explain = generic_plan;
      |         ~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~
analyze.c:2909:25: note: 'generic_plan' was declared here
 2909 |         bool            generic_plan;
      |                         ^~~~~~~~~~~~

Thanks for checking. The variable should indeed be initialized, although
my compiler didn't complain.

Attached is a fixed version.

Yours,
Laurenz Albe

#12Tom Lane
tgl@sss.pgh.pa.us
In reply to: Jim Jones (#11)
Re: Make EXPLAIN generate a generic plan for a parameterized query

Jim Jones <jim.jones@uni-muenster.de> writes:

However, when GENERIC_PLAN is used combined with BUFFERS, the 'Buffers'
node is shown the first time the query executed in a session:

psql (16devel)
Type "help" for help.

postgres=# \c db
You are now connected to database "db" as user "postgres".
db=# EXPLAIN (BUFFERS, GENERIC_PLAN) SELECT * FROM t WHERE col = $1;
                               QUERY PLAN
-------------------------------------------------------------------------
 Index Only Scan using t_col_idx on t  (cost=0.42..4.44 rows=1 width=11)
   Index Cond: (col = $1)
 Planning:
   Buffers: shared hit=62
(4 rows)

db=# EXPLAIN (BUFFERS, GENERIC_PLAN) SELECT * FROM t WHERE col = $1;
                               QUERY PLAN
-------------------------------------------------------------------------
 Index Only Scan using t_col_idx on t  (cost=0.42..4.44 rows=1 width=11)
   Index Cond: (col = $1)
(2 rows)

That's not new to this patch, the same thing happens without it.
It's reflecting catalog accesses involved in loading per-session
caches (which, therefore, needn't be repeated the second time).

Also, this new parameter seems only to work between parenthesis
`(GENERIC_PLAN)`:

db=# EXPLAIN GENERIC_PLAN SELECT * FROM t WHERE col = $1;
ERROR:  syntax error at or near "GENERIC_PLAN"
LINE 1: EXPLAIN GENERIC_PLAN SELECT * FROM t WHERE col = $1;

That's true of all but the oldest EXPLAIN options. We don't do that
anymore because new options would have to become grammar keywords
to support that.

On a very personal note: wouldn't just GENERIC (without _PLAN) suffice?
Don't bother with it if you disagree :-)

FWIW, "GENERIC" would be too generic for my taste ;-). But I agree
it's a judgement call.

regards, tom lane

#13Tom Lane
tgl@sss.pgh.pa.us
In reply to: Laurenz Albe (#10)
1 attachment(s)
Re: Make EXPLAIN generate a generic plan for a parameterized query

Laurenz Albe <laurenz.albe@cybertec.at> writes:

[ 0001-Add-EXPLAIN-option-GENERIC_PLAN.v4.patch ]

I took a closer look at this patch, and didn't like the implementation
much. You're not matching the behavior of PREPARE at all: for example,
this patch is content to let $1 be resolved with different types in
different places. We should be using the existing infrastructure that
parse_analyze_varparams uses.

Also, I believe that in contexts such as plpgsql, it is possible that
there's an external source of $N definitions, which we should probably
continue to honor even with GENERIC_PLAN.

So that leads me to think the code should be more like this. I'm not
sure if it's worth spending documentation and testing effort on the
case where we don't override an existing p_paramref_hook.

regards, tom lane

Attachments:

0001-Add-EXPLAIN-option-GENERIC_PLAN.v5.patchtext/x-diff; charset=us-ascii; name=0001-Add-EXPLAIN-option-GENERIC_PLAN.v5.patchDownload
diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
index d4895b9d7d..58350624e7 100644
--- a/doc/src/sgml/ref/explain.sgml
+++ b/doc/src/sgml/ref/explain.sgml
@@ -40,6 +40,7 @@ EXPLAIN [ ANALYZE ] [ VERBOSE ] <replaceable class="parameter">statement</replac
     VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
     COSTS [ <replaceable class="parameter">boolean</replaceable> ]
     SETTINGS [ <replaceable class="parameter">boolean</replaceable> ]
+    GENERIC_PLAN [ <replaceable class="parameter">boolean</replaceable> ]
     BUFFERS [ <replaceable class="parameter">boolean</replaceable> ]
     WAL [ <replaceable class="parameter">boolean</replaceable> ]
     TIMING [ <replaceable class="parameter">boolean</replaceable> ]
@@ -167,6 +168,20 @@ ROLLBACK;
     </listitem>
    </varlistentry>
 
+   <varlistentry>
+    <term><literal>GENERIC_PLAN</literal></term>
+    <listitem>
+     <para>
+      Generate a generic plan for the statement (see <xref linkend="sql-prepare"/>
+      for details about generic plans).  The statement can contain parameter
+      placeholders like <literal>$1</literal>, if it is a statement that can
+      use parameters.  This option cannot be used together with
+      <literal>ANALYZE</literal>, since a statement with unknown parameters
+      cannot be executed.
+     </para>
+    </listitem>
+   </varlistentry>
+
    <varlistentry>
     <term><literal>BUFFERS</literal></term>
     <listitem>
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 35c23bd27d..ab21a67862 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -190,6 +190,8 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
 			es->wal = defGetBoolean(opt);
 		else if (strcmp(opt->defname, "settings") == 0)
 			es->settings = defGetBoolean(opt);
+		else if (strcmp(opt->defname, "generic_plan") == 0)
+			es->generic = defGetBoolean(opt);
 		else if (strcmp(opt->defname, "timing") == 0)
 		{
 			timing_set = true;
@@ -227,6 +229,13 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
 					 parser_errposition(pstate, opt->location)));
 	}
 
+	/* check that GENERIC_PLAN is not used with EXPLAIN ANALYZE */
+	if (es->generic && es->analyze)
+		ereport(ERROR,
+				(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+				 errmsg("EXPLAIN ANALYZE cannot be used with GENERIC_PLAN")));
+
+	/* check that WAL is used with EXPLAIN ANALYZE */
 	if (es->wal && !es->analyze)
 		ereport(ERROR,
 				(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index e892df9819..9143964e78 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -27,6 +27,7 @@
 #include "access/sysattr.h"
 #include "catalog/pg_proc.h"
 #include "catalog/pg_type.h"
+#include "commands/defrem.h"
 #include "miscadmin.h"
 #include "nodes/makefuncs.h"
 #include "nodes/nodeFuncs.h"
@@ -2906,10 +2907,38 @@ static Query *
 transformExplainStmt(ParseState *pstate, ExplainStmt *stmt)
 {
 	Query	   *result;
+	bool		generic_plan = false;
+	Oid		   *paramTypes = NULL;
+	int			numParams = 0;
+
+	/*
+	 * If we have no external source of parameter definitions, and the
+	 * GENERIC_PLAN option is specified, then accept variable parameter
+	 * definitions (as occurs in PREPARE, for example).
+	 */
+	if (pstate->p_paramref_hook == NULL)
+	{
+		ListCell   *lc;
+
+		foreach(lc, stmt->options)
+		{
+			DefElem    *opt = (DefElem *) lfirst(lc);
+
+			if (strcmp(opt->defname, "generic_plan") == 0)
+				generic_plan = defGetBoolean(opt);
+			/* don't "break", as we want the last value */
+		}
+		if (generic_plan)
+			setup_parse_variable_parameters(pstate, &paramTypes, &numParams);
+	}
 
 	/* transform contained query, allowing SELECT INTO */
 	stmt->query = (Node *) transformOptionalSelectInto(pstate, stmt->query);
 
+	/* make sure all is well with parameter types */
+	if (generic_plan)
+		check_variable_parameters(pstate, (Query *) stmt->query);
+
 	/* represent the command as a utility Query */
 	result = makeNode(Query);
 	result->commandType = CMD_UTILITY;
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 7c1071ddd1..3d3e632a0c 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -46,6 +46,7 @@ typedef struct ExplainState
 	bool		timing;			/* print detailed node timing */
 	bool		summary;		/* print total planning and execution timing */
 	bool		settings;		/* print modified settings */
+	bool		generic;		/* generate a generic plan */
 	ExplainFormat format;		/* output format */
 	/* state for output formatting --- not reset for each new plan tree */
 	int			indent;			/* current indentation level */
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index 48620edbc2..18f7ac93c4 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -517,3 +517,17 @@ select explain_filter('explain (verbose) select * from int8_tbl i8');
  Query Identifier: N
 (3 rows)
 
+-- Test EXPLAIN (GENERIC_PLAN)
+select explain_filter('explain (generic_plan) select unique1 from tenk1 where thousand = $1');
+                                 explain_filter                                  
+---------------------------------------------------------------------------------
+ Bitmap Heap Scan on tenk1  (cost=N.N..N.N rows=N width=N)
+   Recheck Cond: (thousand = $N)
+   ->  Bitmap Index Scan on tenk1_thous_tenthous  (cost=N.N..N.N rows=N width=N)
+         Index Cond: (thousand = $N)
+(4 rows)
+
+-- should fail
+select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1 where thousand = $1');
+ERROR:  EXPLAIN ANALYZE cannot be used with GENERIC_PLAN
+CONTEXT:  PL/pgSQL function explain_filter(text) line 5 at FOR over EXECUTE statement
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index ae3f7a308d..fce031775a 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -128,3 +128,8 @@ select explain_filter('explain (verbose) select * from t1 where pg_temp.mysin(f1
 -- Test compute_query_id
 set compute_query_id = on;
 select explain_filter('explain (verbose) select * from int8_tbl i8');
+
+-- Test EXPLAIN (GENERIC_PLAN)
+select explain_filter('explain (generic_plan) select unique1 from tenk1 where thousand = $1');
+-- should fail
+select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1 where thousand = $1');
#14Laurenz Albe
laurenz.albe@cybertec.at
In reply to: Tom Lane (#13)
Re: Make EXPLAIN generate a generic plan for a parameterized query

On Tue, 2023-01-31 at 13:49 -0500, Tom Lane wrote:

Laurenz Albe <laurenz.albe@cybertec.at> writes:

[ 0001-Add-EXPLAIN-option-GENERIC_PLAN.v4.patch ]

I took a closer look at this patch, and didn't like the implementation
much.  You're not matching the behavior of PREPARE at all: for example,
this patch is content to let $1 be resolved with different types in
different places.  We should be using the existing infrastructure that
parse_analyze_varparams uses.

Also, I believe that in contexts such as plpgsql, it is possible that
there's an external source of $N definitions, which we should probably
continue to honor even with GENERIC_PLAN.

So that leads me to think the code should be more like this.  I'm not
sure if it's worth spending documentation and testing effort on the
case where we don't override an existing p_paramref_hook.

Thanks, that looks way cleaner.

I played around with it, and I ran into a problem with partitions that
are foreign tables:

CREATE TABLE loc1 (id integer NOT NULL, key integer NOT NULL CHECK (key = 1), value text);

CREATE TABLE loc2 (id integer NOT NULL, key integer NOT NULL CHECK (key = 2), value text);

CREATE TABLE looppart (id integer GENERATED ALWAYS AS IDENTITY, key integer NOT NULL, value text) PARTITION BY LIST (key);

CREATE FOREIGN TABLE looppart1 PARTITION OF looppart FOR VALUES IN (1) SERVER loopback OPTIONS (table_name 'loc1');

CREATE FOREIGN TABLE looppart2 PARTITION OF looppart FOR VALUES IN (2) SERVER loopback OPTIONS (table_name 'loc2');

EXPLAIN (GENERIC_PLAN) SELECT * FROM looppart WHERE key = $1;
ERROR: no value found for parameter 1

The solution could be to set up a dynamic parameter hook in the
ExprContext in ecxt_param_list_info->paramFetch so that
ExecEvalParamExtern doesn't complain about the missing parameter.

Does that make sense? How do I best hook into the executor to set that up?

Yours,
Laurenz Albe

#15Tom Lane
tgl@sss.pgh.pa.us
In reply to: Laurenz Albe (#14)
Re: Make EXPLAIN generate a generic plan for a parameterized query

Laurenz Albe <laurenz.albe@cybertec.at> writes:

I played around with it, and I ran into a problem with partitions that
are foreign tables:
...
EXPLAIN (GENERIC_PLAN) SELECT * FROM looppart WHERE key = $1;
ERROR: no value found for parameter 1

Hmm, offhand I'd say that something is doing something it has no
business doing when EXEC_FLAG_EXPLAIN_ONLY is set (that is, premature
evaluation of an expression). I wonder whether this failure is
reachable without this patch.

regards, tom lane

#16Laurenz Albe
laurenz.albe@cybertec.at
In reply to: Tom Lane (#15)
1 attachment(s)
Re: Make EXPLAIN generate a generic plan for a parameterized query

On Fri, 2023-02-03 at 09:44 -0500, Tom Lane wrote:

Laurenz Albe <laurenz.albe@cybertec.at> writes:

I played around with it, and I ran into a problem with partitions that
are foreign tables:
...
  EXPLAIN (GENERIC_PLAN) SELECT * FROM looppart WHERE key = $1;
  ERROR:  no value found for parameter 1

Hmm, offhand I'd say that something is doing something it has no
business doing when EXEC_FLAG_EXPLAIN_ONLY is set (that is, premature
evaluation of an expression).  I wonder whether this failure is
reachable without this patch.

Thanks for the pointer. Perhaps something like the attached?
The changes in "CreatePartitionPruneState" make my test case work,
but they cause a difference in the regression tests, which would be
intended if we have no partition pruning with plain EXPLAIN.

Yours,
Laurenz Albe

Attachments:

0001-Add-EXPLAIN-option-GENERIC_PLAN.v6.patchtext/x-patch; charset=UTF-8; name=0001-Add-EXPLAIN-option-GENERIC_PLAN.v6.patchDownload
From ff16316aab8e5f5de84ae925e965a210d4368b75 Mon Sep 17 00:00:00 2001
From: Laurenz Albe <laurenz.albe@cybertec.at>
Date: Fri, 3 Feb 2023 17:08:53 +0100
Subject: [PATCH] Add EXPLAIN option GENERIC_PLAN

This allows EXPLAIN to generate generic plans for parameterized
statements (that have parameter placeholders like $1 in the
statement text).

Author: Laurenz Albe
Discussion: https://postgr.es/m/0a29b954b10b57f0d135fe12aa0909bd41883eb0.camel%40cybertec.at
---
 doc/src/sgml/ref/explain.sgml         | 15 ++++++++++++++
 src/backend/commands/explain.c        |  9 +++++++++
 src/backend/executor/execPartition.c  |  6 ++++--
 src/backend/parser/analyze.c          | 29 +++++++++++++++++++++++++++
 src/include/commands/explain.h        |  1 +
 src/test/regress/expected/explain.out | 14 +++++++++++++
 src/test/regress/sql/explain.sql      |  5 +++++
 7 files changed, 77 insertions(+), 2 deletions(-)

diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
index d4895b9d7d..58350624e7 100644
--- a/doc/src/sgml/ref/explain.sgml
+++ b/doc/src/sgml/ref/explain.sgml
@@ -40,6 +40,7 @@ EXPLAIN [ ANALYZE ] [ VERBOSE ] <replaceable class="parameter">statement</replac
     VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
     COSTS [ <replaceable class="parameter">boolean</replaceable> ]
     SETTINGS [ <replaceable class="parameter">boolean</replaceable> ]
+    GENERIC_PLAN [ <replaceable class="parameter">boolean</replaceable> ]
     BUFFERS [ <replaceable class="parameter">boolean</replaceable> ]
     WAL [ <replaceable class="parameter">boolean</replaceable> ]
     TIMING [ <replaceable class="parameter">boolean</replaceable> ]
@@ -167,6 +168,20 @@ ROLLBACK;
     </listitem>
    </varlistentry>
 
+   <varlistentry>
+    <term><literal>GENERIC_PLAN</literal></term>
+    <listitem>
+     <para>
+      Generate a generic plan for the statement (see <xref linkend="sql-prepare"/>
+      for details about generic plans).  The statement can contain parameter
+      placeholders like <literal>$1</literal>, if it is a statement that can
+      use parameters.  This option cannot be used together with
+      <literal>ANALYZE</literal>, since a statement with unknown parameters
+      cannot be executed.
+     </para>
+    </listitem>
+   </varlistentry>
+
    <varlistentry>
     <term><literal>BUFFERS</literal></term>
     <listitem>
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 35c23bd27d..ab21a67862 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -190,6 +190,8 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
 			es->wal = defGetBoolean(opt);
 		else if (strcmp(opt->defname, "settings") == 0)
 			es->settings = defGetBoolean(opt);
+		else if (strcmp(opt->defname, "generic_plan") == 0)
+			es->generic = defGetBoolean(opt);
 		else if (strcmp(opt->defname, "timing") == 0)
 		{
 			timing_set = true;
@@ -227,6 +229,13 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
 					 parser_errposition(pstate, opt->location)));
 	}
 
+	/* check that GENERIC_PLAN is not used with EXPLAIN ANALYZE */
+	if (es->generic && es->analyze)
+		ereport(ERROR,
+				(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+				 errmsg("EXPLAIN ANALYZE cannot be used with GENERIC_PLAN")));
+
+	/* check that WAL is used with EXPLAIN ANALYZE */
 	if (es->wal && !es->analyze)
 		ereport(ERROR,
 				(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index 651ad24fc1..38d14433a6 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -2043,7 +2043,8 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
 			 * Initialize pruning contexts as needed.
 			 */
 			pprune->initial_pruning_steps = pinfo->initial_pruning_steps;
-			if (pinfo->initial_pruning_steps)
+			if (pinfo->initial_pruning_steps &&
+				!(econtext->ecxt_estate->es_top_eflags & EXEC_FLAG_EXPLAIN_ONLY))
 			{
 				InitPartitionPruneContext(&pprune->initial_context,
 										  pinfo->initial_pruning_steps,
@@ -2053,7 +2054,8 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
 				prunestate->do_initial_prune = true;
 			}
 			pprune->exec_pruning_steps = pinfo->exec_pruning_steps;
-			if (pinfo->exec_pruning_steps)
+			if (pinfo->exec_pruning_steps &&
+				!(econtext->ecxt_estate->es_top_eflags & EXEC_FLAG_EXPLAIN_ONLY))
 			{
 				InitPartitionPruneContext(&pprune->exec_context,
 										  pinfo->exec_pruning_steps,
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index e892df9819..9143964e78 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -27,6 +27,7 @@
 #include "access/sysattr.h"
 #include "catalog/pg_proc.h"
 #include "catalog/pg_type.h"
+#include "commands/defrem.h"
 #include "miscadmin.h"
 #include "nodes/makefuncs.h"
 #include "nodes/nodeFuncs.h"
@@ -2906,10 +2907,38 @@ static Query *
 transformExplainStmt(ParseState *pstate, ExplainStmt *stmt)
 {
 	Query	   *result;
+	bool		generic_plan = false;
+	Oid		   *paramTypes = NULL;
+	int			numParams = 0;
+
+	/*
+	 * If we have no external source of parameter definitions, and the
+	 * GENERIC_PLAN option is specified, then accept variable parameter
+	 * definitions (as occurs in PREPARE, for example).
+	 */
+	if (pstate->p_paramref_hook == NULL)
+	{
+		ListCell   *lc;
+
+		foreach(lc, stmt->options)
+		{
+			DefElem    *opt = (DefElem *) lfirst(lc);
+
+			if (strcmp(opt->defname, "generic_plan") == 0)
+				generic_plan = defGetBoolean(opt);
+			/* don't "break", as we want the last value */
+		}
+		if (generic_plan)
+			setup_parse_variable_parameters(pstate, &paramTypes, &numParams);
+	}
 
 	/* transform contained query, allowing SELECT INTO */
 	stmt->query = (Node *) transformOptionalSelectInto(pstate, stmt->query);
 
+	/* make sure all is well with parameter types */
+	if (generic_plan)
+		check_variable_parameters(pstate, (Query *) stmt->query);
+
 	/* represent the command as a utility Query */
 	result = makeNode(Query);
 	result->commandType = CMD_UTILITY;
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 7c1071ddd1..3d3e632a0c 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -46,6 +46,7 @@ typedef struct ExplainState
 	bool		timing;			/* print detailed node timing */
 	bool		summary;		/* print total planning and execution timing */
 	bool		settings;		/* print modified settings */
+	bool		generic;		/* generate a generic plan */
 	ExplainFormat format;		/* output format */
 	/* state for output formatting --- not reset for each new plan tree */
 	int			indent;			/* current indentation level */
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index 48620edbc2..18f7ac93c4 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -517,3 +517,17 @@ select explain_filter('explain (verbose) select * from int8_tbl i8');
  Query Identifier: N
 (3 rows)
 
+-- Test EXPLAIN (GENERIC_PLAN)
+select explain_filter('explain (generic_plan) select unique1 from tenk1 where thousand = $1');
+                                 explain_filter                                  
+---------------------------------------------------------------------------------
+ Bitmap Heap Scan on tenk1  (cost=N.N..N.N rows=N width=N)
+   Recheck Cond: (thousand = $N)
+   ->  Bitmap Index Scan on tenk1_thous_tenthous  (cost=N.N..N.N rows=N width=N)
+         Index Cond: (thousand = $N)
+(4 rows)
+
+-- should fail
+select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1 where thousand = $1');
+ERROR:  EXPLAIN ANALYZE cannot be used with GENERIC_PLAN
+CONTEXT:  PL/pgSQL function explain_filter(text) line 5 at FOR over EXECUTE statement
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index ae3f7a308d..fce031775a 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -128,3 +128,8 @@ select explain_filter('explain (verbose) select * from t1 where pg_temp.mysin(f1
 -- Test compute_query_id
 set compute_query_id = on;
 select explain_filter('explain (verbose) select * from int8_tbl i8');
+
+-- Test EXPLAIN (GENERIC_PLAN)
+select explain_filter('explain (generic_plan) select unique1 from tenk1 where thousand = $1');
+-- should fail
+select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1 where thousand = $1');
-- 
2.39.1

#17Tom Lane
tgl@sss.pgh.pa.us
In reply to: Laurenz Albe (#16)
Re: Make EXPLAIN generate a generic plan for a parameterized query

Laurenz Albe <laurenz.albe@cybertec.at> writes:

Thanks for the pointer. Perhaps something like the attached?
The changes in "CreatePartitionPruneState" make my test case work,
but they cause a difference in the regression tests, which would be
intended if we have no partition pruning with plain EXPLAIN.

Hmm, so I see (and the cfbot entry for this patch is now all red,
because you didn't include that diff in the patch).

I'm not sure if we can get away with that behavioral change.
People are probably expecting the current behavior for existing
cases.

I can think of a couple of possible ways forward:

* Fix things so that the generic parameters appear to have NULL
values when inspected during executor startup. I'm not sure
how useful that'd be though. In partition-pruning cases that'd
lead to EXPLAIN (GENERIC_PLAN) showing the plan with all
partitions pruned, other than the one for NULL values if there
is one. Doesn't sound too helpful.

* Invent another executor flag that's a "stronger" version of
EXEC_FLAG_EXPLAIN_ONLY, and pass that when any generic parameters
exist, and check it in CreatePartitionPruneState to decide whether
to do startup pruning. This avoids changing behavior for existing
cases, but I'm not sure how much it has to recommend it otherwise.
Skipping the startup prune step seems like it'd produce misleading
results in another way, ie showing too many partitions not too few.

This whole business of partition pruning dependent on the
generic parameters is making me uncomfortable. It seems like
we *can't* show a plan that is either representative of real
execution or comparable to what you get from more-traditional
EXPLAIN usage. Maybe we need to step back and think more.

regards, tom lane

#18Laurenz Albe
laurenz.albe@cybertec.at
In reply to: Tom Lane (#17)
1 attachment(s)
Re: Make EXPLAIN generate a generic plan for a parameterized query

On Fri, 2023-02-03 at 15:11 -0500, Tom Lane wrote:

I can think of a couple of possible ways forward:

* Fix things so that the generic parameters appear to have NULL
values when inspected during executor startup.  I'm not sure
how useful that'd be though.  In partition-pruning cases that'd
lead to EXPLAIN (GENERIC_PLAN) showing the plan with all
partitions pruned, other than the one for NULL values if there
is one.  Doesn't sound too helpful.

* Invent another executor flag that's a "stronger" version of
EXEC_FLAG_EXPLAIN_ONLY, and pass that when any generic parameters
exist, and check it in CreatePartitionPruneState to decide whether
to do startup pruning.  This avoids changing behavior for existing
cases, but I'm not sure how much it has to recommend it otherwise.
Skipping the startup prune step seems like it'd produce misleading
results in another way, ie showing too many partitions not too few.

This whole business of partition pruning dependent on the
generic parameters is making me uncomfortable.  It seems like
we *can't* show a plan that is either representative of real
execution or comparable to what you get from more-traditional
EXPLAIN usage.  Maybe we need to step back and think more.

I slept over it, and the second idea now looks like the the right
approach to me. My idea of seeing a generic plan is that plan-time
partition pruning happens, but not execution-time pruning, so that
I get no "subplans removed".
Are there any weird side effects of skipping the startup prune step?

Anyway, attached is v7 that tries to do it that way. It feels fairly
good to me. I invented a new executor flag EXEC_FLAG_EXPLAIN_GENERIC.
To avoid having to change all the places that check EXEC_FLAG_EXPLAIN_ONLY
to also check for the new flag, I decided that the new flag can only be
used as "add-on" to EXEC_FLAG_EXPLAIN_ONLY.

Yours,
Laurenz Albe

Attachments:

0001-Add-EXPLAIN-option-GENERIC_PLAN.v7.patchtext/x-patch; charset=UTF-8; name=0001-Add-EXPLAIN-option-GENERIC_PLAN.v7.patchDownload
From cd0b5a1a4f301bb7fad9088d5763989f5dde4636 Mon Sep 17 00:00:00 2001
From: Laurenz Albe <laurenz.albe@cybertec.at>
Date: Sun, 5 Feb 2023 18:11:57 +0100
Subject: [PATCH] Add EXPLAIN option GENERIC_PLAN

This allows EXPLAIN to generate generic plans for parameterized statements
(that have parameter placeholders like $1 in the statement text).
Invent a new executor flag EXEC_FLAG_EXPLAIN_GENERIC that disables runtime
partition pruning for such plans: that would fail if the non-existing
parameters are involved, and we don't want to remove subplans anyway.

Author: Laurenz Albe
Discussion: https://postgr.es/m/0a29b954b10b57f0d135fe12aa0909bd41883eb0.camel%40cybertec.at
---
 doc/src/sgml/ref/explain.sgml         | 15 +++++++++
 src/backend/commands/explain.c        | 11 +++++++
 src/backend/executor/execMain.c       |  3 ++
 src/backend/executor/execPartition.c  |  9 ++++--
 src/backend/parser/analyze.c          | 29 +++++++++++++++++
 src/include/commands/explain.h        |  1 +
 src/include/executor/executor.h       | 18 +++++++----
 src/test/regress/expected/explain.out | 46 +++++++++++++++++++++++++++
 src/test/regress/sql/explain.sql      | 29 +++++++++++++++++
 9 files changed, 152 insertions(+), 9 deletions(-)

diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
index d4895b9d7d..a1fdd31bc7 100644
--- a/doc/src/sgml/ref/explain.sgml
+++ b/doc/src/sgml/ref/explain.sgml
@@ -40,6 +40,7 @@ EXPLAIN [ ANALYZE ] [ VERBOSE ] <replaceable class="parameter">statement</replac
     VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
     COSTS [ <replaceable class="parameter">boolean</replaceable> ]
     SETTINGS [ <replaceable class="parameter">boolean</replaceable> ]
+    GENERIC_PLAN [ <replaceable class="parameter">boolean</replaceable> ]
     BUFFERS [ <replaceable class="parameter">boolean</replaceable> ]
     WAL [ <replaceable class="parameter">boolean</replaceable> ]
     TIMING [ <replaceable class="parameter">boolean</replaceable> ]
@@ -167,6 +168,20 @@ ROLLBACK;
     </listitem>
    </varlistentry>
 
+   <varlistentry>
+    <term><literal>GENERIC_PLAN</literal></term>
+    <listitem>
+     <para>
+      Generate a generic plan for the statement (see <xref linkend="sql-prepare"/>
+      for details about generic plans).  The statement can contain parameter
+      placeholders like <literal>$1</literal> (but then it has to be a statement
+      that supports parameters).  This option cannot be used together with
+      <literal>ANALYZE</literal>, since a statement with unknown parameters
+      cannot be executed.
+     </para>
+    </listitem>
+   </varlistentry>
+
    <varlistentry>
     <term><literal>BUFFERS</literal></term>
     <listitem>
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 35c23bd27d..37ea4e5035 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -190,6 +190,8 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
 			es->wal = defGetBoolean(opt);
 		else if (strcmp(opt->defname, "settings") == 0)
 			es->settings = defGetBoolean(opt);
+		else if (strcmp(opt->defname, "generic_plan") == 0)
+			es->generic = defGetBoolean(opt);
 		else if (strcmp(opt->defname, "timing") == 0)
 		{
 			timing_set = true;
@@ -227,6 +229,13 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
 					 parser_errposition(pstate, opt->location)));
 	}
 
+	/* check that GENERIC_PLAN is not used with EXPLAIN ANALYZE */
+	if (es->generic && es->analyze)
+		ereport(ERROR,
+				(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+				 errmsg("EXPLAIN ANALYZE cannot be used with GENERIC_PLAN")));
+
+	/* check that WAL is used with EXPLAIN ANALYZE */
 	if (es->wal && !es->analyze)
 		ereport(ERROR,
 				(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
@@ -574,6 +583,8 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
 		eflags = EXEC_FLAG_EXPLAIN_ONLY;
 	if (into)
 		eflags |= GetIntoRelEFlags(into);
+	if (es->generic)
+		eflags |= EXEC_FLAG_EXPLAIN_GENERIC;
 
 	/* call ExecutorStart to prepare the plan for execution */
 	ExecutorStart(queryDesc, eflags);
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index a5115b9c1f..fea4314033 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -131,6 +131,9 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree);
 void
 ExecutorStart(QueryDesc *queryDesc, int eflags)
 {
+	/* EXEC_FLAG_EXPLAIN_GENERIC can only occur with EXEC_FLAG_EXPLAIN_ONLY */
+	Assert((eflags & EXEC_FLAG_EXPLAIN_ONLY) ||
+		   !(eflags & EXEC_FLAG_EXPLAIN_GENERIC));
 	/*
 	 * In some cases (e.g. an EXECUTE statement) a query execution will skip
 	 * parse analysis, which means that the query_id won't be reported.  Note
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index 651ad24fc1..7fc2f0d1ab 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -2040,10 +2040,12 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
 			pprune->present_parts = bms_copy(pinfo->present_parts);
 
 			/*
-			 * Initialize pruning contexts as needed.
+			 * Initialize pruning contexts as needed.  Specifically, we want to
+			 * skip execution-time partition pruning for EXPLAIN (GENERIC_PLAN).
 			 */
 			pprune->initial_pruning_steps = pinfo->initial_pruning_steps;
-			if (pinfo->initial_pruning_steps)
+			if (pinfo->initial_pruning_steps &&
+				!(econtext->ecxt_estate->es_top_eflags & EXEC_FLAG_EXPLAIN_GENERIC))
 			{
 				InitPartitionPruneContext(&pprune->initial_context,
 										  pinfo->initial_pruning_steps,
@@ -2053,7 +2055,8 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
 				prunestate->do_initial_prune = true;
 			}
 			pprune->exec_pruning_steps = pinfo->exec_pruning_steps;
-			if (pinfo->exec_pruning_steps)
+			if (pinfo->exec_pruning_steps &&
+				!(econtext->ecxt_estate->es_top_eflags & EXEC_FLAG_EXPLAIN_GENERIC))
 			{
 				InitPartitionPruneContext(&pprune->exec_context,
 										  pinfo->exec_pruning_steps,
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index e892df9819..9143964e78 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -27,6 +27,7 @@
 #include "access/sysattr.h"
 #include "catalog/pg_proc.h"
 #include "catalog/pg_type.h"
+#include "commands/defrem.h"
 #include "miscadmin.h"
 #include "nodes/makefuncs.h"
 #include "nodes/nodeFuncs.h"
@@ -2906,10 +2907,38 @@ static Query *
 transformExplainStmt(ParseState *pstate, ExplainStmt *stmt)
 {
 	Query	   *result;
+	bool		generic_plan = false;
+	Oid		   *paramTypes = NULL;
+	int			numParams = 0;
+
+	/*
+	 * If we have no external source of parameter definitions, and the
+	 * GENERIC_PLAN option is specified, then accept variable parameter
+	 * definitions (as occurs in PREPARE, for example).
+	 */
+	if (pstate->p_paramref_hook == NULL)
+	{
+		ListCell   *lc;
+
+		foreach(lc, stmt->options)
+		{
+			DefElem    *opt = (DefElem *) lfirst(lc);
+
+			if (strcmp(opt->defname, "generic_plan") == 0)
+				generic_plan = defGetBoolean(opt);
+			/* don't "break", as we want the last value */
+		}
+		if (generic_plan)
+			setup_parse_variable_parameters(pstate, &paramTypes, &numParams);
+	}
 
 	/* transform contained query, allowing SELECT INTO */
 	stmt->query = (Node *) transformOptionalSelectInto(pstate, stmt->query);
 
+	/* make sure all is well with parameter types */
+	if (generic_plan)
+		check_variable_parameters(pstate, (Query *) stmt->query);
+
 	/* represent the command as a utility Query */
 	result = makeNode(Query);
 	result->commandType = CMD_UTILITY;
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 7c1071ddd1..3d3e632a0c 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -46,6 +46,7 @@ typedef struct ExplainState
 	bool		timing;			/* print detailed node timing */
 	bool		summary;		/* print total planning and execution timing */
 	bool		settings;		/* print modified settings */
+	bool		generic;		/* generate a generic plan */
 	ExplainFormat format;		/* output format */
 	/* state for output formatting --- not reset for each new plan tree */
 	int			indent;			/* current indentation level */
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index e7e25c057e..a6b6ad943c 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -36,6 +36,11 @@
  * of startup should occur.  However, error checks (such as permission checks)
  * should be performed.
  *
+ * EXPLAIN_GENERIC can only be used together with EXPLAIN_ONLY.  It indicates
+ * that a generic plan is being calculated using EXPLAIN (GENERIC_PLAN), which
+ * means that missing parameters must be tolerated.  Currently, the only effect
+ * is to suppress execution-time partition pruning.
+ *
  * REWIND indicates that the plan node should try to efficiently support
  * rescans without parameter changes.  (Nodes must support ExecReScan calls
  * in any case, but if this flag was not given, they are at liberty to do it
@@ -53,12 +58,13 @@
  * mean that the plan can't queue any AFTER triggers; just that the caller
  * is responsible for there being a trigger context for them to be queued in.
  */
-#define EXEC_FLAG_EXPLAIN_ONLY	0x0001	/* EXPLAIN, no ANALYZE */
-#define EXEC_FLAG_REWIND		0x0002	/* need efficient rescan */
-#define EXEC_FLAG_BACKWARD		0x0004	/* need backward scan */
-#define EXEC_FLAG_MARK			0x0008	/* need mark/restore */
-#define EXEC_FLAG_SKIP_TRIGGERS 0x0010	/* skip AfterTrigger calls */
-#define EXEC_FLAG_WITH_NO_DATA	0x0020	/* rel scannability doesn't matter */
+#define EXEC_FLAG_EXPLAIN_ONLY		0x0001	/* EXPLAIN, no ANALYZE */
+#define EXEC_FLAG_REWIND			0x0002	/* need efficient rescan */
+#define EXEC_FLAG_BACKWARD			0x0004	/* need backward scan */
+#define EXEC_FLAG_MARK				0x0008	/* need mark/restore */
+#define EXEC_FLAG_SKIP_TRIGGERS		0x0010	/* skip AfterTrigger calls */
+#define EXEC_FLAG_WITH_NO_DATA		0x0020	/* rel scannability doesn't matter */
+#define EXEC_FLAG_EXPLAIN_GENERIC	0x0040	/* EXPLAIN (GENERIC_PLAN) */
 
 
 /* Hook for plugins to get control in ExecutorStart() */
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index 48620edbc2..73922cbe9d 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -517,3 +517,49 @@ select explain_filter('explain (verbose) select * from int8_tbl i8');
  Query Identifier: N
 (3 rows)
 
+-- Test EXPLAIN (GENERIC_PLAN)
+select explain_filter('explain (generic_plan) select unique1 from tenk1 where thousand = $1');
+                                 explain_filter                                  
+---------------------------------------------------------------------------------
+ Bitmap Heap Scan on tenk1  (cost=N.N..N.N rows=N width=N)
+   Recheck Cond: (thousand = $N)
+   ->  Bitmap Index Scan on tenk1_thous_tenthous  (cost=N.N..N.N rows=N width=N)
+         Index Cond: (thousand = $N)
+(4 rows)
+
+-- should fail
+select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1 where thousand = $1');
+ERROR:  EXPLAIN ANALYZE cannot be used with GENERIC_PLAN
+CONTEXT:  PL/pgSQL function explain_filter(text) line 5 at FOR over EXECUTE statement
+-- Test EXPLAIN (GENERIC_PLAN) with partition pruning
+-- should prune at plan time, but not at execution time
+create extension if not exists postgres_fdw;
+create server loop42 foreign data wrapper postgres_fdw;
+create user mapping for current_role server loop42 options (password_required 'false');
+create table gen_part (
+  key1 integer not null,
+  key2 integer not null
+) partition by list (key1);
+create table gen_part_1
+  partition of gen_part for values in (1)
+  partition by range (key2);
+create foreign table gen_part_1_1
+  partition of gen_part_1 for values from (1) to (2)
+  server loop42 options (table_name 'whatever_1_1');
+create foreign table gen_part_1_2
+  partition of gen_part_1 for values from (2) to (3)
+  server loop42 options (table_name 'whatever_1_2');
+create foreign table gen_part_2
+  partition of gen_part for values in (2)
+  server loop42 options (table_name 'whatever_2');
+select explain_filter('explain (generic_plan) select key1, key2 from gen_part where key1 = 1 and key2 = $1');
+                                explain_filter                                 
+-------------------------------------------------------------------------------
+ Append  (cost=N.N..N.N rows=N width=N)
+   ->  Foreign Scan on gen_part_1_1 gen_part_1  (cost=N.N..N.N rows=N width=N)
+   ->  Foreign Scan on gen_part_1_2 gen_part_2  (cost=N.N..N.N rows=N width=N)
+(3 rows)
+
+drop table gen_part;
+drop server loop42 cascade;
+NOTICE:  drop cascades to user mapping for postgres on server loop42
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index ae3f7a308d..c39a7afaee 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -128,3 +128,32 @@ select explain_filter('explain (verbose) select * from t1 where pg_temp.mysin(f1
 -- Test compute_query_id
 set compute_query_id = on;
 select explain_filter('explain (verbose) select * from int8_tbl i8');
+
+-- Test EXPLAIN (GENERIC_PLAN)
+select explain_filter('explain (generic_plan) select unique1 from tenk1 where thousand = $1');
+-- should fail
+select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1 where thousand = $1');
+-- Test EXPLAIN (GENERIC_PLAN) with partition pruning
+-- should prune at plan time, but not at execution time
+create extension if not exists postgres_fdw;
+create server loop42 foreign data wrapper postgres_fdw;
+create user mapping for current_role server loop42 options (password_required 'false');
+create table gen_part (
+  key1 integer not null,
+  key2 integer not null
+) partition by list (key1);
+create table gen_part_1
+  partition of gen_part for values in (1)
+  partition by range (key2);
+create foreign table gen_part_1_1
+  partition of gen_part_1 for values from (1) to (2)
+  server loop42 options (table_name 'whatever_1_1');
+create foreign table gen_part_1_2
+  partition of gen_part_1 for values from (2) to (3)
+  server loop42 options (table_name 'whatever_1_2');
+create foreign table gen_part_2
+  partition of gen_part for values in (2)
+  server loop42 options (table_name 'whatever_2');
+select explain_filter('explain (generic_plan) select key1, key2 from gen_part where key1 = 1 and key2 = $1');
+drop table gen_part;
+drop server loop42 cascade;
-- 
2.39.1

#19Andres Freund
andres@anarazel.de
In reply to: Laurenz Albe (#18)
Re: Make EXPLAIN generate a generic plan for a parameterized query

Hi,

On 2023-02-05 18:24:03 +0100, Laurenz Albe wrote:

Anyway, attached is v7 that tries to do it that way.

This consistently fails on CI:
https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest%2F42%2F3962

https://api.cirrus-ci.com/v1/artifact/task/5156723929907200/testrun/build/testrun/regress/regress/regression.diffs

Greetings,

Andres Freund

#20Laurenz Albe
laurenz.albe@cybertec.at
In reply to: Andres Freund (#19)
1 attachment(s)
Re: Make EXPLAIN generate a generic plan for a parameterized query

On Mon, 2023-02-13 at 16:33 -0800, Andres Freund wrote:

On 2023-02-05 18:24:03 +0100, Laurenz Albe wrote:

Anyway, attached is v7 that tries to do it that way.

This consistently fails on CI:
https://cirrus-ci.com/github/postgresql-cfbot/postgresql/commitfest%2F42%2F3962

https://api.cirrus-ci.com/v1/artifact/task/5156723929907200/testrun/build/testrun/regress/regress/regression.diffs

Thanks for pointing out.

Here is an improved version.

Yours,
Laurenz Albe

Attachments:

0001-Add-EXPLAIN-option-GENERIC_PLAN.v8.patchtext/x-patch; charset=UTF-8; name=0001-Add-EXPLAIN-option-GENERIC_PLAN.v8.patchDownload
From f55d14a50ab2773a450f5535aebbd4011c129ba1 Mon Sep 17 00:00:00 2001
From: Laurenz Albe <laurenz.albe@cybertec.at>
Date: Tue, 14 Feb 2023 13:42:37 +0100
Subject: [PATCH] Add EXPLAIN option GENERIC_PLAN

This allows EXPLAIN to generate generic plans for parameterized statements
(that have parameter placeholders like $1 in the statement text).
Invent a new executor flag EXEC_FLAG_EXPLAIN_GENERIC that disables runtime
partition pruning for such plans: that would fail if the non-existing
parameters are involved, and we don't want to remove subplans anyway.

Author: Laurenz Albe
Discussion: https://postgr.es/m/0a29b954b10b57f0d135fe12aa0909bd41883eb0.camel%40cybertec.at
---
 doc/src/sgml/ref/explain.sgml         | 15 +++++++++
 src/backend/commands/explain.c        | 11 +++++++
 src/backend/executor/execMain.c       |  3 ++
 src/backend/executor/execPartition.c  |  9 ++++--
 src/backend/parser/analyze.c          | 29 +++++++++++++++++
 src/include/commands/explain.h        |  1 +
 src/include/executor/executor.h       | 18 +++++++----
 src/test/regress/expected/explain.out | 46 +++++++++++++++++++++++++++
 src/test/regress/sql/explain.sql      | 30 +++++++++++++++++
 9 files changed, 153 insertions(+), 9 deletions(-)

diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
index d4895b9d7d..a1fdd31bc7 100644
--- a/doc/src/sgml/ref/explain.sgml
+++ b/doc/src/sgml/ref/explain.sgml
@@ -40,6 +40,7 @@ EXPLAIN [ ANALYZE ] [ VERBOSE ] <replaceable class="parameter">statement</replac
     VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
     COSTS [ <replaceable class="parameter">boolean</replaceable> ]
     SETTINGS [ <replaceable class="parameter">boolean</replaceable> ]
+    GENERIC_PLAN [ <replaceable class="parameter">boolean</replaceable> ]
     BUFFERS [ <replaceable class="parameter">boolean</replaceable> ]
     WAL [ <replaceable class="parameter">boolean</replaceable> ]
     TIMING [ <replaceable class="parameter">boolean</replaceable> ]
@@ -167,6 +168,20 @@ ROLLBACK;
     </listitem>
    </varlistentry>
 
+   <varlistentry>
+    <term><literal>GENERIC_PLAN</literal></term>
+    <listitem>
+     <para>
+      Generate a generic plan for the statement (see <xref linkend="sql-prepare"/>
+      for details about generic plans).  The statement can contain parameter
+      placeholders like <literal>$1</literal> (but then it has to be a statement
+      that supports parameters).  This option cannot be used together with
+      <literal>ANALYZE</literal>, since a statement with unknown parameters
+      cannot be executed.
+     </para>
+    </listitem>
+   </varlistentry>
+
    <varlistentry>
     <term><literal>BUFFERS</literal></term>
     <listitem>
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index fbbf28cf06..1f3055c7af 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -190,6 +190,8 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
 			es->wal = defGetBoolean(opt);
 		else if (strcmp(opt->defname, "settings") == 0)
 			es->settings = defGetBoolean(opt);
+		else if (strcmp(opt->defname, "generic_plan") == 0)
+			es->generic = defGetBoolean(opt);
 		else if (strcmp(opt->defname, "timing") == 0)
 		{
 			timing_set = true;
@@ -227,6 +229,13 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
 					 parser_errposition(pstate, opt->location)));
 	}
 
+	/* check that GENERIC_PLAN is not used with EXPLAIN ANALYZE */
+	if (es->generic && es->analyze)
+		ereport(ERROR,
+				(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+				 errmsg("EXPLAIN ANALYZE cannot be used with GENERIC_PLAN")));
+
+	/* check that WAL is used with EXPLAIN ANALYZE */
 	if (es->wal && !es->analyze)
 		ereport(ERROR,
 				(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
@@ -574,6 +583,8 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
 		eflags = EXEC_FLAG_EXPLAIN_ONLY;
 	if (into)
 		eflags |= GetIntoRelEFlags(into);
+	if (es->generic)
+		eflags |= EXEC_FLAG_EXPLAIN_GENERIC;
 
 	/* call ExecutorStart to prepare the plan for execution */
 	ExecutorStart(queryDesc, eflags);
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 39bfb48dc2..6e91b56294 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -131,6 +131,9 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree);
 void
 ExecutorStart(QueryDesc *queryDesc, int eflags)
 {
+	/* EXEC_FLAG_EXPLAIN_GENERIC can only occur with EXEC_FLAG_EXPLAIN_ONLY */
+	Assert((eflags & EXEC_FLAG_EXPLAIN_ONLY) ||
+		   !(eflags & EXEC_FLAG_EXPLAIN_GENERIC));
 	/*
 	 * In some cases (e.g. an EXECUTE statement) a query execution will skip
 	 * parse analysis, which means that the query_id won't be reported.  Note
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index 651ad24fc1..7fc2f0d1ab 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -2040,10 +2040,12 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
 			pprune->present_parts = bms_copy(pinfo->present_parts);
 
 			/*
-			 * Initialize pruning contexts as needed.
+			 * Initialize pruning contexts as needed.  Specifically, we want to
+			 * skip execution-time partition pruning for EXPLAIN (GENERIC_PLAN).
 			 */
 			pprune->initial_pruning_steps = pinfo->initial_pruning_steps;
-			if (pinfo->initial_pruning_steps)
+			if (pinfo->initial_pruning_steps &&
+				!(econtext->ecxt_estate->es_top_eflags & EXEC_FLAG_EXPLAIN_GENERIC))
 			{
 				InitPartitionPruneContext(&pprune->initial_context,
 										  pinfo->initial_pruning_steps,
@@ -2053,7 +2055,8 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
 				prunestate->do_initial_prune = true;
 			}
 			pprune->exec_pruning_steps = pinfo->exec_pruning_steps;
-			if (pinfo->exec_pruning_steps)
+			if (pinfo->exec_pruning_steps &&
+				!(econtext->ecxt_estate->es_top_eflags & EXEC_FLAG_EXPLAIN_GENERIC))
 			{
 				InitPartitionPruneContext(&pprune->exec_context,
 										  pinfo->exec_pruning_steps,
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index e892df9819..9143964e78 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -27,6 +27,7 @@
 #include "access/sysattr.h"
 #include "catalog/pg_proc.h"
 #include "catalog/pg_type.h"
+#include "commands/defrem.h"
 #include "miscadmin.h"
 #include "nodes/makefuncs.h"
 #include "nodes/nodeFuncs.h"
@@ -2906,10 +2907,38 @@ static Query *
 transformExplainStmt(ParseState *pstate, ExplainStmt *stmt)
 {
 	Query	   *result;
+	bool		generic_plan = false;
+	Oid		   *paramTypes = NULL;
+	int			numParams = 0;
+
+	/*
+	 * If we have no external source of parameter definitions, and the
+	 * GENERIC_PLAN option is specified, then accept variable parameter
+	 * definitions (as occurs in PREPARE, for example).
+	 */
+	if (pstate->p_paramref_hook == NULL)
+	{
+		ListCell   *lc;
+
+		foreach(lc, stmt->options)
+		{
+			DefElem    *opt = (DefElem *) lfirst(lc);
+
+			if (strcmp(opt->defname, "generic_plan") == 0)
+				generic_plan = defGetBoolean(opt);
+			/* don't "break", as we want the last value */
+		}
+		if (generic_plan)
+			setup_parse_variable_parameters(pstate, &paramTypes, &numParams);
+	}
 
 	/* transform contained query, allowing SELECT INTO */
 	stmt->query = (Node *) transformOptionalSelectInto(pstate, stmt->query);
 
+	/* make sure all is well with parameter types */
+	if (generic_plan)
+		check_variable_parameters(pstate, (Query *) stmt->query);
+
 	/* represent the command as a utility Query */
 	result = makeNode(Query);
 	result->commandType = CMD_UTILITY;
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 7c1071ddd1..3d3e632a0c 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -46,6 +46,7 @@ typedef struct ExplainState
 	bool		timing;			/* print detailed node timing */
 	bool		summary;		/* print total planning and execution timing */
 	bool		settings;		/* print modified settings */
+	bool		generic;		/* generate a generic plan */
 	ExplainFormat format;		/* output format */
 	/* state for output formatting --- not reset for each new plan tree */
 	int			indent;			/* current indentation level */
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index e7e25c057e..a6b6ad943c 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -36,6 +36,11 @@
  * of startup should occur.  However, error checks (such as permission checks)
  * should be performed.
  *
+ * EXPLAIN_GENERIC can only be used together with EXPLAIN_ONLY.  It indicates
+ * that a generic plan is being calculated using EXPLAIN (GENERIC_PLAN), which
+ * means that missing parameters must be tolerated.  Currently, the only effect
+ * is to suppress execution-time partition pruning.
+ *
  * REWIND indicates that the plan node should try to efficiently support
  * rescans without parameter changes.  (Nodes must support ExecReScan calls
  * in any case, but if this flag was not given, they are at liberty to do it
@@ -53,12 +58,13 @@
  * mean that the plan can't queue any AFTER triggers; just that the caller
  * is responsible for there being a trigger context for them to be queued in.
  */
-#define EXEC_FLAG_EXPLAIN_ONLY	0x0001	/* EXPLAIN, no ANALYZE */
-#define EXEC_FLAG_REWIND		0x0002	/* need efficient rescan */
-#define EXEC_FLAG_BACKWARD		0x0004	/* need backward scan */
-#define EXEC_FLAG_MARK			0x0008	/* need mark/restore */
-#define EXEC_FLAG_SKIP_TRIGGERS 0x0010	/* skip AfterTrigger calls */
-#define EXEC_FLAG_WITH_NO_DATA	0x0020	/* rel scannability doesn't matter */
+#define EXEC_FLAG_EXPLAIN_ONLY		0x0001	/* EXPLAIN, no ANALYZE */
+#define EXEC_FLAG_REWIND			0x0002	/* need efficient rescan */
+#define EXEC_FLAG_BACKWARD			0x0004	/* need backward scan */
+#define EXEC_FLAG_MARK				0x0008	/* need mark/restore */
+#define EXEC_FLAG_SKIP_TRIGGERS		0x0010	/* skip AfterTrigger calls */
+#define EXEC_FLAG_WITH_NO_DATA		0x0020	/* rel scannability doesn't matter */
+#define EXEC_FLAG_EXPLAIN_GENERIC	0x0040	/* EXPLAIN (GENERIC_PLAN) */
 
 
 /* Hook for plugins to get control in ExecutorStart() */
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index 48620edbc2..4f222594e3 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -517,3 +517,49 @@ select explain_filter('explain (verbose) select * from int8_tbl i8');
  Query Identifier: N
 (3 rows)
 
+-- Test EXPLAIN (GENERIC_PLAN)
+select explain_filter('explain (generic_plan) select unique1 from tenk1 where thousand = $1');
+                                 explain_filter                                  
+---------------------------------------------------------------------------------
+ Bitmap Heap Scan on tenk1  (cost=N.N..N.N rows=N width=N)
+   Recheck Cond: (thousand = $N)
+   ->  Bitmap Index Scan on tenk1_thous_tenthous  (cost=N.N..N.N rows=N width=N)
+         Index Cond: (thousand = $N)
+(4 rows)
+
+-- should fail
+select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1 where thousand = $1');
+ERROR:  EXPLAIN ANALYZE cannot be used with GENERIC_PLAN
+CONTEXT:  PL/pgSQL function explain_filter(text) line 5 at FOR over EXECUTE statement
+-- Test EXPLAIN (GENERIC_PLAN) with partition pruning
+-- should prune at plan time, but not at execution time
+create extension if not exists postgres_fdw;
+create server loop42 foreign data wrapper postgres_fdw;
+create user mapping for current_role server loop42 options (password_required 'false');
+create table gen_part (
+  key1 integer not null,
+  key2 integer not null
+) partition by list (key1);
+create table gen_part_1
+  partition of gen_part for values in (1)
+  partition by range (key2);
+create foreign table gen_part_1_1
+  partition of gen_part_1 for values from (1) to (2)
+  server loop42 options (table_name 'whatever_1_1');
+create foreign table gen_part_1_2
+  partition of gen_part_1 for values from (2) to (3)
+  server loop42 options (table_name 'whatever_1_2');
+create foreign table gen_part_2
+  partition of gen_part for values in (2)
+  server loop42 options (table_name 'whatever_2');
+select explain_filter('explain (generic_plan) select key1, key2 from gen_part where key1 = 1 and key2 = $1');
+                                explain_filter                                 
+-------------------------------------------------------------------------------
+ Append  (cost=N.N..N.N rows=N width=N)
+   ->  Foreign Scan on gen_part_1_1 gen_part_1  (cost=N.N..N.N rows=N width=N)
+   ->  Foreign Scan on gen_part_1_2 gen_part_2  (cost=N.N..N.N rows=N width=N)
+(3 rows)
+
+drop table gen_part;
+drop user mapping for current_role server loop42;
+drop server loop42 cascade;
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index ae3f7a308d..8ec33ff47b 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -128,3 +128,33 @@ select explain_filter('explain (verbose) select * from t1 where pg_temp.mysin(f1
 -- Test compute_query_id
 set compute_query_id = on;
 select explain_filter('explain (verbose) select * from int8_tbl i8');
+
+-- Test EXPLAIN (GENERIC_PLAN)
+select explain_filter('explain (generic_plan) select unique1 from tenk1 where thousand = $1');
+-- should fail
+select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1 where thousand = $1');
+-- Test EXPLAIN (GENERIC_PLAN) with partition pruning
+-- should prune at plan time, but not at execution time
+create extension if not exists postgres_fdw;
+create server loop42 foreign data wrapper postgres_fdw;
+create user mapping for current_role server loop42 options (password_required 'false');
+create table gen_part (
+  key1 integer not null,
+  key2 integer not null
+) partition by list (key1);
+create table gen_part_1
+  partition of gen_part for values in (1)
+  partition by range (key2);
+create foreign table gen_part_1_1
+  partition of gen_part_1 for values from (1) to (2)
+  server loop42 options (table_name 'whatever_1_1');
+create foreign table gen_part_1_2
+  partition of gen_part_1 for values from (2) to (3)
+  server loop42 options (table_name 'whatever_1_2');
+create foreign table gen_part_2
+  partition of gen_part for values in (2)
+  server loop42 options (table_name 'whatever_2');
+select explain_filter('explain (generic_plan) select key1, key2 from gen_part where key1 = 1 and key2 = $1');
+drop table gen_part;
+drop user mapping for current_role server loop42;
+drop server loop42 cascade;
-- 
2.39.1

#21Christoph Berg
myon@debian.org
In reply to: Laurenz Albe (#20)
Re: Make EXPLAIN generate a generic plan for a parameterized query

Hi,

I reviewed the patch and find the idea of allowing $placeholders with
EXPLAIN very useful, it will solve relevant real-world use-cases.
(Queries from pg_stat_statements, found in the log, or in application
code.)

I have some comments:

This allows EXPLAIN to generate generic plans for parameterized statements
(that have parameter placeholders like $1 in the statement text).

+   <varlistentry>
+    <term><literal>GENERIC_PLAN</literal></term>
+    <listitem>
+     <para>
+      Generate a generic plan for the statement (see <xref linkend="sql-prepare"/>
+      for details about generic plans).  The statement can contain parameter
+      placeholders like <literal>$1</literal> (but then it has to be a statement
+      that supports parameters).  This option cannot be used together with
+      <literal>ANALYZE</literal>, since a statement with unknown parameters
+      cannot be executed.

Like in the commit message quoted above, I would put more emphasis on
"parameterized query" here:

Allow the statement to contain parameter placeholders like
<literal>$1</literal> and generate a generic plan for it.
This option cannot be used together with <literal>ANALYZE</literal>.

+	/* check that GENERIC_PLAN is not used with EXPLAIN ANALYZE */
+	if (es->generic && es->analyze)
+		ereport(ERROR,
+				(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+				 errmsg("EXPLAIN ANALYZE cannot be used with GENERIC_PLAN")));

To put that in line with the other error messages in that context, I'd
inject an extra "option":

errmsg("EXPLAIN option ANALYZE cannot be used with GENERIC_PLAN")));

--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -128,3 +128,33 @@ select explain_filter('explain (verbose) select * from t1 where pg_temp.mysin(f1
-- Test compute_query_id
set compute_query_id = on;
select explain_filter('explain (verbose) select * from int8_tbl i8');
+
+-- Test EXPLAIN (GENERIC_PLAN)
+select explain_filter('explain (generic_plan) select unique1 from tenk1 where thousand = $1');
+-- should fail
+select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1 where thousand = $1');
+-- Test EXPLAIN (GENERIC_PLAN) with partition pruning
+-- should prune at plan time, but not at execution time
+create extension if not exists postgres_fdw;

"create extension postgres_fdw" cannot be used from src/test/regress/
since contrib/ might not have been built.

+create server loop42 foreign data wrapper postgres_fdw;
+create user mapping for current_role server loop42 options (password_required 'false');
+create table gen_part (
+  key1 integer not null,
+  key2 integer not null
+) partition by list (key1);
+create table gen_part_1
+  partition of gen_part for values in (1)
+  partition by range (key2);
+create foreign table gen_part_1_1
+  partition of gen_part_1 for values from (1) to (2)
+  server loop42 options (table_name 'whatever_1_1');
+create foreign table gen_part_1_2
+  partition of gen_part_1 for values from (2) to (3)
+  server loop42 options (table_name 'whatever_1_2');
+create foreign table gen_part_2
+  partition of gen_part for values in (2)
+  server loop42 options (table_name 'whatever_2');
+select explain_filter('explain (generic_plan) select key1, key2 from gen_part where key1 = 1 and key2 = $1');

I suggest leaving this test in place here, but with local tables (to
show that plan time pruning using the one provided parameter works),
and add a comment here explaining that is being tested:

-- create a partition hierarchy to show that plan time pruning removes
-- the key1=2 table but generates a generic plan for key2=$1

The test involving postgres_fdw is still necessary to exercise the new
EXEC_FLAG_EXPLAIN_GENERIC code path, but needs to be moved elsewhere,
probably src/test/modules/.

In the new location, likewise add a comment why this test needs to
look this way.

Christoph

#22Laurenz Albe
laurenz.albe@cybertec.at
In reply to: Christoph Berg (#21)
1 attachment(s)
Re: Make EXPLAIN generate a generic plan for a parameterized query

Thanks for the review!

On Tue, 2023-03-21 at 16:32 +0100, Christoph Berg wrote:

I have some comments:

This allows EXPLAIN to generate generic plans for parameterized statements
(that have parameter placeholders like $1 in the statement text).

+   <varlistentry>
+    <term><literal>GENERIC_PLAN</literal></term>
+    <listitem>
+     <para>
+      Generate a generic plan for the statement (see <xref linkend="sql-prepare"/>
+      for details about generic plans).  The statement can contain parameter
+      placeholders like <literal>$1</literal> (but then it has to be a statement
+      that supports parameters).  This option cannot be used together with
+      <literal>ANALYZE</literal>, since a statement with unknown parameters
+      cannot be executed.

Like in the commit message quoted above, I would put more emphasis on
"parameterized query" here:

  Allow the statement to contain parameter placeholders like
  <literal>$1</literal> and generate a generic plan for it.
  This option cannot be used together with <literal>ANALYZE</literal>.

I went with

Allow the statement to contain parameter placeholders like
<literal>$1</literal> and generate a generic plan for it.
See <xref linkend="sql-prepare"/> for details about generic plans
and the statements that support parameters.
This option cannot be used together with <literal>ANALYZE</literal>.

+       /* check that GENERIC_PLAN is not used with EXPLAIN ANALYZE */
+       if (es->generic && es->analyze)
+               ereport(ERROR,
+                               (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+                                errmsg("EXPLAIN ANALYZE cannot be used with GENERIC_PLAN")));

To put that in line with the other error messages in that context, I'd
inject an extra "option":

  errmsg("EXPLAIN option ANALYZE cannot be used with GENERIC_PLAN")));

Done.

--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
[...]
+create extension if not exists postgres_fdw;

"create extension postgres_fdw" cannot be used from src/test/regress/
since contrib/ might not have been built.

Ouch. Good catch.

I suggest leaving this test in place here, but with local tables (to
show that plan time pruning using the one provided parameter works),
and add a comment here explaining that is being tested:

-- create a partition hierarchy to show that plan time pruning removes
-- the key1=2 table but generates a generic plan for key2=$1

I did that, with a different comment.

The test involving postgres_fdw is still necessary to exercise the new
EXEC_FLAG_EXPLAIN_GENERIC code path, but needs to be moved elsewhere,
probably src/test/modules/.

Tests for postgres_fdw are in contrib/postgres_fdw/sql/postgres_fdw.sql,
so I added the test there.

Version 9 of the patch is attached.

Yours,
Laurenz Albe

Attachments:

0001-Add-EXPLAIN-option-GENERIC_PLAN.v9.patchtext/x-patch; charset=UTF-8; name=0001-Add-EXPLAIN-option-GENERIC_PLAN.v9.patchDownload
From 85aa88280069ca2befe7f4308d7e6f724cdb143a Mon Sep 17 00:00:00 2001
From: Laurenz Albe <laurenz.albe@cybertec.at>
Date: Wed, 22 Mar 2023 14:08:49 +0100
Subject: [PATCH] Add EXPLAIN option GENERIC_PLAN

This allows EXPLAIN to generate generic plans for parameterized statements
(that have parameter placeholders like $1 in the statement text).
Invent a new executor flag EXEC_FLAG_EXPLAIN_GENERIC that disables runtime
partition pruning for such plans: that would fail if the non-existing
parameters are involved, and we want to show all subplans anyway.

Author: Laurenz Albe
Reviewed-by: Julien Rouhaud, Christoph Berg, Michel Pelletier, Jim Jones
Discussion: https://postgr.es/m/0a29b954b10b57f0d135fe12aa0909bd41883eb0.camel%40cybertec.at
---
 .../postgres_fdw/expected/postgres_fdw.out    | 30 +++++++++++++
 contrib/postgres_fdw/sql/postgres_fdw.sql     | 25 +++++++++++
 doc/src/sgml/ref/explain.sgml                 | 14 +++++++
 src/backend/commands/explain.c                | 11 +++++
 src/backend/executor/execMain.c               |  3 ++
 src/backend/executor/execPartition.c          |  9 ++--
 src/backend/parser/analyze.c                  | 29 +++++++++++++
 src/include/commands/explain.h                |  1 +
 src/include/executor/executor.h               | 18 +++++---
 src/test/regress/expected/explain.out         | 42 +++++++++++++++++++
 src/test/regress/sql/explain.sql              | 24 +++++++++++
 11 files changed, 197 insertions(+), 9 deletions(-)

diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index 04a3ef450c..25b91ab2e1 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -11783,3 +11783,33 @@ ANALYZE analyze_table;
 -- cleanup
 DROP FOREIGN TABLE analyze_ftable;
 DROP TABLE analyze_table;
+-- ===================================================
+-- test EXPLAIN (GENERIC_PLAN) with foreign partitions
+-- ===================================================
+-- this is needed to exercise the EXEC_FLAG_EXPLAIN_GENERIC flag
+CREATE TABLE gen_part (
+    key1 integer NOT NULL,
+    key2 integer NOT NULL
+) PARTITION BY LIST (key1);
+CREATE TABLE gen_part_1
+    PARTITION OF gen_part FOR VALUES IN (1)
+    PARTITION BY RANGE (key2);
+CREATE FOREIGN TABLE gen_part_1_1
+    PARTITION OF gen_part_1 FOR VALUES FROM (1) TO (2)
+    SERVER testserver1 OPTIONS (table_name 'whatever_1_1');
+CREATE FOREIGN TABLE gen_part_1_2
+    PARTITION OF gen_part_1 FOR VALUES FROM (2) TO (3)
+    SERVER testserver1 OPTIONS (table_name 'whatever_1_2');
+CREATE FOREIGN TABLE gen_part_2
+    PARTITION OF gen_part FOR VALUES IN (2)
+    SERVER testserver1 OPTIONS (table_name 'whatever_2');
+-- this should only scan "gen_part_1_1" and "gen_part_1_2", but not "gen_part_2"
+EXPLAIN (GENERIC_PLAN, COSTS OFF) SELECT key1, key2 FROM gen_part WHERE key1 = 1 AND key2 = $1;
+                  QUERY PLAN                   
+-----------------------------------------------
+ Append
+   ->  Foreign Scan on gen_part_1_1 gen_part_1
+   ->  Foreign Scan on gen_part_1_2 gen_part_2
+(3 rows)
+
+DROP TABLE gen_part;
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index 4f3088c03e..6adc3f2c78 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -3979,3 +3979,28 @@ ANALYZE analyze_table;
 -- cleanup
 DROP FOREIGN TABLE analyze_ftable;
 DROP TABLE analyze_table;
+
+-- ===================================================
+-- test EXPLAIN (GENERIC_PLAN) with foreign partitions
+-- ===================================================
+
+-- this is needed to exercise the EXEC_FLAG_EXPLAIN_GENERIC flag
+CREATE TABLE gen_part (
+    key1 integer NOT NULL,
+    key2 integer NOT NULL
+) PARTITION BY LIST (key1);
+CREATE TABLE gen_part_1
+    PARTITION OF gen_part FOR VALUES IN (1)
+    PARTITION BY RANGE (key2);
+CREATE FOREIGN TABLE gen_part_1_1
+    PARTITION OF gen_part_1 FOR VALUES FROM (1) TO (2)
+    SERVER testserver1 OPTIONS (table_name 'whatever_1_1');
+CREATE FOREIGN TABLE gen_part_1_2
+    PARTITION OF gen_part_1 FOR VALUES FROM (2) TO (3)
+    SERVER testserver1 OPTIONS (table_name 'whatever_1_2');
+CREATE FOREIGN TABLE gen_part_2
+    PARTITION OF gen_part FOR VALUES IN (2)
+    SERVER testserver1 OPTIONS (table_name 'whatever_2');
+-- this should only scan "gen_part_1_1" and "gen_part_1_2", but not "gen_part_2"
+EXPLAIN (GENERIC_PLAN, COSTS OFF) SELECT key1, key2 FROM gen_part WHERE key1 = 1 AND key2 = $1;
+DROP TABLE gen_part;
diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
index 0fce622423..4985545c78 100644
--- a/doc/src/sgml/ref/explain.sgml
+++ b/doc/src/sgml/ref/explain.sgml
@@ -40,6 +40,7 @@ EXPLAIN [ ANALYZE ] [ VERBOSE ] <replaceable class="parameter">statement</replac
     VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
     COSTS [ <replaceable class="parameter">boolean</replaceable> ]
     SETTINGS [ <replaceable class="parameter">boolean</replaceable> ]
+    GENERIC_PLAN [ <replaceable class="parameter">boolean</replaceable> ]
     BUFFERS [ <replaceable class="parameter">boolean</replaceable> ]
     WAL [ <replaceable class="parameter">boolean</replaceable> ]
     TIMING [ <replaceable class="parameter">boolean</replaceable> ]
@@ -168,6 +169,19 @@ ROLLBACK;
     </listitem>
    </varlistentry>
 
+   <varlistentry>
+    <term><literal>GENERIC_PLAN</literal></term>
+    <listitem>
+     <para>
+      Allow the statement to contain parameter placeholders like
+      <literal>$1</literal> and generate a generic plan for it.
+      See <xref linkend="sql-prepare"/> for details about generic plans
+      and the statements that support parameters.
+      This option cannot be used together with <literal>ANALYZE</literal>.
+     </para>
+    </listitem>
+   </varlistentry>
+
    <varlistentry>
     <term><literal>BUFFERS</literal></term>
     <listitem>
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index e57bda7b62..aaa9783d73 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -190,6 +190,8 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
 			es->wal = defGetBoolean(opt);
 		else if (strcmp(opt->defname, "settings") == 0)
 			es->settings = defGetBoolean(opt);
+		else if (strcmp(opt->defname, "generic_plan") == 0)
+			es->generic = defGetBoolean(opt);
 		else if (strcmp(opt->defname, "timing") == 0)
 		{
 			timing_set = true;
@@ -227,6 +229,13 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
 					 parser_errposition(pstate, opt->location)));
 	}
 
+	/* check that GENERIC_PLAN is not used with EXPLAIN ANALYZE */
+	if (es->generic && es->analyze)
+		ereport(ERROR,
+				(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+				 errmsg("EXPLAIN option ANALYZE cannot be used with GENERIC_PLAN")));
+
+	/* check that WAL is used with EXPLAIN ANALYZE */
 	if (es->wal && !es->analyze)
 		ereport(ERROR,
 				(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
@@ -574,6 +583,8 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
 		eflags = EXEC_FLAG_EXPLAIN_ONLY;
 	if (into)
 		eflags |= GetIntoRelEFlags(into);
+	if (es->generic)
+		eflags |= EXEC_FLAG_EXPLAIN_GENERIC;
 
 	/* call ExecutorStart to prepare the plan for execution */
 	ExecutorStart(queryDesc, eflags);
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index b32f419176..23ffcbf1aa 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -131,6 +131,9 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree);
 void
 ExecutorStart(QueryDesc *queryDesc, int eflags)
 {
+	/* EXEC_FLAG_EXPLAIN_GENERIC can only occur with EXEC_FLAG_EXPLAIN_ONLY */
+	Assert((eflags & EXEC_FLAG_EXPLAIN_ONLY) ||
+		   !(eflags & EXEC_FLAG_EXPLAIN_GENERIC));
 	/*
 	 * In some cases (e.g. an EXECUTE statement) a query execution will skip
 	 * parse analysis, which means that the query_id won't be reported.  Note
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index fd6ca8a5d9..6333822ff9 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -2044,10 +2044,12 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
 			pprune->present_parts = bms_copy(pinfo->present_parts);
 
 			/*
-			 * Initialize pruning contexts as needed.
+			 * Initialize pruning contexts as needed.  Specifically, we want to
+			 * skip execution-time partition pruning for EXPLAIN (GENERIC_PLAN).
 			 */
 			pprune->initial_pruning_steps = pinfo->initial_pruning_steps;
-			if (pinfo->initial_pruning_steps)
+			if (pinfo->initial_pruning_steps &&
+				!(econtext->ecxt_estate->es_top_eflags & EXEC_FLAG_EXPLAIN_GENERIC))
 			{
 				InitPartitionPruneContext(&pprune->initial_context,
 										  pinfo->initial_pruning_steps,
@@ -2057,7 +2059,8 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
 				prunestate->do_initial_prune = true;
 			}
 			pprune->exec_pruning_steps = pinfo->exec_pruning_steps;
-			if (pinfo->exec_pruning_steps)
+			if (pinfo->exec_pruning_steps &&
+				!(econtext->ecxt_estate->es_top_eflags & EXEC_FLAG_EXPLAIN_GENERIC))
 			{
 				InitPartitionPruneContext(&pprune->exec_context,
 										  pinfo->exec_pruning_steps,
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index e892df9819..9143964e78 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -27,6 +27,7 @@
 #include "access/sysattr.h"
 #include "catalog/pg_proc.h"
 #include "catalog/pg_type.h"
+#include "commands/defrem.h"
 #include "miscadmin.h"
 #include "nodes/makefuncs.h"
 #include "nodes/nodeFuncs.h"
@@ -2906,10 +2907,38 @@ static Query *
 transformExplainStmt(ParseState *pstate, ExplainStmt *stmt)
 {
 	Query	   *result;
+	bool		generic_plan = false;
+	Oid		   *paramTypes = NULL;
+	int			numParams = 0;
+
+	/*
+	 * If we have no external source of parameter definitions, and the
+	 * GENERIC_PLAN option is specified, then accept variable parameter
+	 * definitions (as occurs in PREPARE, for example).
+	 */
+	if (pstate->p_paramref_hook == NULL)
+	{
+		ListCell   *lc;
+
+		foreach(lc, stmt->options)
+		{
+			DefElem    *opt = (DefElem *) lfirst(lc);
+
+			if (strcmp(opt->defname, "generic_plan") == 0)
+				generic_plan = defGetBoolean(opt);
+			/* don't "break", as we want the last value */
+		}
+		if (generic_plan)
+			setup_parse_variable_parameters(pstate, &paramTypes, &numParams);
+	}
 
 	/* transform contained query, allowing SELECT INTO */
 	stmt->query = (Node *) transformOptionalSelectInto(pstate, stmt->query);
 
+	/* make sure all is well with parameter types */
+	if (generic_plan)
+		check_variable_parameters(pstate, (Query *) stmt->query);
+
 	/* represent the command as a utility Query */
 	result = makeNode(Query);
 	result->commandType = CMD_UTILITY;
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 7c1071ddd1..3d3e632a0c 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -46,6 +46,7 @@ typedef struct ExplainState
 	bool		timing;			/* print detailed node timing */
 	bool		summary;		/* print total planning and execution timing */
 	bool		settings;		/* print modified settings */
+	bool		generic;		/* generate a generic plan */
 	ExplainFormat format;		/* output format */
 	/* state for output formatting --- not reset for each new plan tree */
 	int			indent;			/* current indentation level */
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index dbd77050c7..7b4c1834ef 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -36,6 +36,11 @@
  * of startup should occur.  However, error checks (such as permission checks)
  * should be performed.
  *
+ * EXPLAIN_GENERIC can only be used together with EXPLAIN_ONLY.  It indicates
+ * that a generic plan is being calculated using EXPLAIN (GENERIC_PLAN), which
+ * means that missing parameters must be tolerated.  Currently, the only effect
+ * is to suppress execution-time partition pruning.
+ *
  * REWIND indicates that the plan node should try to efficiently support
  * rescans without parameter changes.  (Nodes must support ExecReScan calls
  * in any case, but if this flag was not given, they are at liberty to do it
@@ -53,12 +58,13 @@
  * mean that the plan can't queue any AFTER triggers; just that the caller
  * is responsible for there being a trigger context for them to be queued in.
  */
-#define EXEC_FLAG_EXPLAIN_ONLY	0x0001	/* EXPLAIN, no ANALYZE */
-#define EXEC_FLAG_REWIND		0x0002	/* need efficient rescan */
-#define EXEC_FLAG_BACKWARD		0x0004	/* need backward scan */
-#define EXEC_FLAG_MARK			0x0008	/* need mark/restore */
-#define EXEC_FLAG_SKIP_TRIGGERS 0x0010	/* skip AfterTrigger calls */
-#define EXEC_FLAG_WITH_NO_DATA	0x0020	/* rel scannability doesn't matter */
+#define EXEC_FLAG_EXPLAIN_ONLY		0x0001	/* EXPLAIN, no ANALYZE */
+#define EXEC_FLAG_REWIND			0x0002	/* need efficient rescan */
+#define EXEC_FLAG_BACKWARD			0x0004	/* need backward scan */
+#define EXEC_FLAG_MARK				0x0008	/* need mark/restore */
+#define EXEC_FLAG_SKIP_TRIGGERS		0x0010	/* skip AfterTrigger calls */
+#define EXEC_FLAG_WITH_NO_DATA		0x0020	/* rel scannability doesn't matter */
+#define EXEC_FLAG_EXPLAIN_GENERIC	0x0040	/* EXPLAIN (GENERIC_PLAN) */
 
 
 /* Hook for plugins to get control in ExecutorStart() */
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index 48620edbc2..253b818c77 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -517,3 +517,45 @@ select explain_filter('explain (verbose) select * from int8_tbl i8');
  Query Identifier: N
 (3 rows)
 
+-- Test EXPLAIN (GENERIC_PLAN)
+select explain_filter('explain (generic_plan) select unique1 from tenk1 where thousand = $1');
+                                 explain_filter                                  
+---------------------------------------------------------------------------------
+ Bitmap Heap Scan on tenk1  (cost=N.N..N.N rows=N width=N)
+   Recheck Cond: (thousand = $N)
+   ->  Bitmap Index Scan on tenk1_thous_tenthous  (cost=N.N..N.N rows=N width=N)
+         Index Cond: (thousand = $N)
+(4 rows)
+
+-- should fail
+select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1 where thousand = $1');
+ERROR:  EXPLAIN option ANALYZE cannot be used with GENERIC_PLAN
+CONTEXT:  PL/pgSQL function explain_filter(text) line 5 at FOR over EXECUTE statement
+-- Test EXPLAIN (GENERIC_PLAN) with partition pruning
+-- partitions should be pruned at plan time, based on constants,
+-- but there should be no pruning based on parameter placeholders
+create table gen_part (
+  key1 integer not null,
+  key2 integer not null
+) partition by list (key1);
+create table gen_part_1
+  partition of gen_part for values in (1)
+  partition by range (key2);
+create table gen_part_1_1
+  partition of gen_part_1 for values from (1) to (2);
+create table gen_part_1_2
+  partition of gen_part_1 for values from (2) to (3);
+create table gen_part_2
+  partition of gen_part for values in (2);
+-- should only scan gen_part_1_1 and gen_part_1_2, but not gen_part_2
+select explain_filter('explain (generic_plan) select key1, key2 from gen_part where key1 = 1 and key2 = $1');
+                              explain_filter                               
+---------------------------------------------------------------------------
+ Append  (cost=N.N..N.N rows=N width=N)
+   ->  Seq Scan on gen_part_1_1 gen_part_1  (cost=N.N..N.N rows=N width=N)
+         Filter: ((key1 = N) AND (key2 = $N))
+   ->  Seq Scan on gen_part_1_2 gen_part_2  (cost=N.N..N.N rows=N width=N)
+         Filter: ((key1 = N) AND (key2 = $N))
+(5 rows)
+
+drop table gen_part;
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index ae3f7a308d..ff9c51e1d1 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -128,3 +128,27 @@ select explain_filter('explain (verbose) select * from t1 where pg_temp.mysin(f1
 -- Test compute_query_id
 set compute_query_id = on;
 select explain_filter('explain (verbose) select * from int8_tbl i8');
+
+-- Test EXPLAIN (GENERIC_PLAN)
+select explain_filter('explain (generic_plan) select unique1 from tenk1 where thousand = $1');
+-- should fail
+select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1 where thousand = $1');
+-- Test EXPLAIN (GENERIC_PLAN) with partition pruning
+-- partitions should be pruned at plan time, based on constants,
+-- but there should be no pruning based on parameter placeholders
+create table gen_part (
+  key1 integer not null,
+  key2 integer not null
+) partition by list (key1);
+create table gen_part_1
+  partition of gen_part for values in (1)
+  partition by range (key2);
+create table gen_part_1_1
+  partition of gen_part_1 for values from (1) to (2);
+create table gen_part_1_2
+  partition of gen_part_1 for values from (2) to (3);
+create table gen_part_2
+  partition of gen_part for values in (2);
+-- should only scan gen_part_1_1 and gen_part_1_2, but not gen_part_2
+select explain_filter('explain (generic_plan) select key1, key2 from gen_part where key1 = 1 and key2 = $1');
+drop table gen_part;
-- 
2.39.2

#23Laurenz Albe
laurenz.albe@cybertec.at
In reply to: Laurenz Albe (#22)
1 attachment(s)
Re: Make EXPLAIN generate a generic plan for a parameterized query

And here is v10, which includes tab completion for the new option.

Yours,
Laurenz Albe

Attachments:

0001-Add-EXPLAIN-option-GENERIC_PLAN.v10.patchtext/x-patch; charset=UTF-8; name=0001-Add-EXPLAIN-option-GENERIC_PLAN.v10.patchDownload
From dfe6d36d79c74fba7bf70b990fdada166d012ff4 Mon Sep 17 00:00:00 2001
From: Laurenz Albe <laurenz.albe@cybertec.at>
Date: Thu, 23 Mar 2023 19:28:49 +0100
Subject: [PATCH] Add EXPLAIN option GENERIC_PLAN

This allows EXPLAIN to generate generic plans for parameterized statements
(that have parameter placeholders like $1 in the statement text).
Invent a new executor flag EXEC_FLAG_EXPLAIN_GENERIC that disables runtime
partition pruning for such plans: that would fail if the non-existing
parameters are involved, and we want to show all subplans anyway.

Author: Laurenz Albe
Reviewed-by: Julien Rouhaud, Christoph Berg, Michel Pelletier, Jim Jones
Discussion: https://postgr.es/m/0a29b954b10b57f0d135fe12aa0909bd41883eb0.camel%40cybertec.at
---
 .../postgres_fdw/expected/postgres_fdw.out    | 30 +++++++++++++
 contrib/postgres_fdw/sql/postgres_fdw.sql     | 25 +++++++++++
 doc/src/sgml/ref/explain.sgml                 | 14 +++++++
 src/backend/commands/explain.c                | 11 +++++
 src/backend/executor/execMain.c               |  3 ++
 src/backend/executor/execPartition.c          |  9 ++--
 src/backend/parser/analyze.c                  | 29 +++++++++++++
 src/bin/psql/tab-complete.c                   |  4 +-
 src/include/commands/explain.h                |  1 +
 src/include/executor/executor.h               | 18 +++++---
 src/test/regress/expected/explain.out         | 42 +++++++++++++++++++
 src/test/regress/sql/explain.sql              | 24 +++++++++++
 12 files changed, 199 insertions(+), 11 deletions(-)

diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index 04a3ef450c..25b91ab2e1 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -11783,3 +11783,33 @@ ANALYZE analyze_table;
 -- cleanup
 DROP FOREIGN TABLE analyze_ftable;
 DROP TABLE analyze_table;
+-- ===================================================
+-- test EXPLAIN (GENERIC_PLAN) with foreign partitions
+-- ===================================================
+-- this is needed to exercise the EXEC_FLAG_EXPLAIN_GENERIC flag
+CREATE TABLE gen_part (
+    key1 integer NOT NULL,
+    key2 integer NOT NULL
+) PARTITION BY LIST (key1);
+CREATE TABLE gen_part_1
+    PARTITION OF gen_part FOR VALUES IN (1)
+    PARTITION BY RANGE (key2);
+CREATE FOREIGN TABLE gen_part_1_1
+    PARTITION OF gen_part_1 FOR VALUES FROM (1) TO (2)
+    SERVER testserver1 OPTIONS (table_name 'whatever_1_1');
+CREATE FOREIGN TABLE gen_part_1_2
+    PARTITION OF gen_part_1 FOR VALUES FROM (2) TO (3)
+    SERVER testserver1 OPTIONS (table_name 'whatever_1_2');
+CREATE FOREIGN TABLE gen_part_2
+    PARTITION OF gen_part FOR VALUES IN (2)
+    SERVER testserver1 OPTIONS (table_name 'whatever_2');
+-- this should only scan "gen_part_1_1" and "gen_part_1_2", but not "gen_part_2"
+EXPLAIN (GENERIC_PLAN, COSTS OFF) SELECT key1, key2 FROM gen_part WHERE key1 = 1 AND key2 = $1;
+                  QUERY PLAN                   
+-----------------------------------------------
+ Append
+   ->  Foreign Scan on gen_part_1_1 gen_part_1
+   ->  Foreign Scan on gen_part_1_2 gen_part_2
+(3 rows)
+
+DROP TABLE gen_part;
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index 4f3088c03e..6adc3f2c78 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -3979,3 +3979,28 @@ ANALYZE analyze_table;
 -- cleanup
 DROP FOREIGN TABLE analyze_ftable;
 DROP TABLE analyze_table;
+
+-- ===================================================
+-- test EXPLAIN (GENERIC_PLAN) with foreign partitions
+-- ===================================================
+
+-- this is needed to exercise the EXEC_FLAG_EXPLAIN_GENERIC flag
+CREATE TABLE gen_part (
+    key1 integer NOT NULL,
+    key2 integer NOT NULL
+) PARTITION BY LIST (key1);
+CREATE TABLE gen_part_1
+    PARTITION OF gen_part FOR VALUES IN (1)
+    PARTITION BY RANGE (key2);
+CREATE FOREIGN TABLE gen_part_1_1
+    PARTITION OF gen_part_1 FOR VALUES FROM (1) TO (2)
+    SERVER testserver1 OPTIONS (table_name 'whatever_1_1');
+CREATE FOREIGN TABLE gen_part_1_2
+    PARTITION OF gen_part_1 FOR VALUES FROM (2) TO (3)
+    SERVER testserver1 OPTIONS (table_name 'whatever_1_2');
+CREATE FOREIGN TABLE gen_part_2
+    PARTITION OF gen_part FOR VALUES IN (2)
+    SERVER testserver1 OPTIONS (table_name 'whatever_2');
+-- this should only scan "gen_part_1_1" and "gen_part_1_2", but not "gen_part_2"
+EXPLAIN (GENERIC_PLAN, COSTS OFF) SELECT key1, key2 FROM gen_part WHERE key1 = 1 AND key2 = $1;
+DROP TABLE gen_part;
diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
index 0fce622423..4985545c78 100644
--- a/doc/src/sgml/ref/explain.sgml
+++ b/doc/src/sgml/ref/explain.sgml
@@ -40,6 +40,7 @@ EXPLAIN [ ANALYZE ] [ VERBOSE ] <replaceable class="parameter">statement</replac
     VERBOSE [ <replaceable class="parameter">boolean</replaceable> ]
     COSTS [ <replaceable class="parameter">boolean</replaceable> ]
     SETTINGS [ <replaceable class="parameter">boolean</replaceable> ]
+    GENERIC_PLAN [ <replaceable class="parameter">boolean</replaceable> ]
     BUFFERS [ <replaceable class="parameter">boolean</replaceable> ]
     WAL [ <replaceable class="parameter">boolean</replaceable> ]
     TIMING [ <replaceable class="parameter">boolean</replaceable> ]
@@ -168,6 +169,19 @@ ROLLBACK;
     </listitem>
    </varlistentry>
 
+   <varlistentry>
+    <term><literal>GENERIC_PLAN</literal></term>
+    <listitem>
+     <para>
+      Allow the statement to contain parameter placeholders like
+      <literal>$1</literal> and generate a generic plan for it.
+      See <xref linkend="sql-prepare"/> for details about generic plans
+      and the statements that support parameters.
+      This option cannot be used together with <literal>ANALYZE</literal>.
+     </para>
+    </listitem>
+   </varlistentry>
+
    <varlistentry>
     <term><literal>BUFFERS</literal></term>
     <listitem>
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index e57bda7b62..aaa9783d73 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -190,6 +190,8 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
 			es->wal = defGetBoolean(opt);
 		else if (strcmp(opt->defname, "settings") == 0)
 			es->settings = defGetBoolean(opt);
+		else if (strcmp(opt->defname, "generic_plan") == 0)
+			es->generic = defGetBoolean(opt);
 		else if (strcmp(opt->defname, "timing") == 0)
 		{
 			timing_set = true;
@@ -227,6 +229,13 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
 					 parser_errposition(pstate, opt->location)));
 	}
 
+	/* check that GENERIC_PLAN is not used with EXPLAIN ANALYZE */
+	if (es->generic && es->analyze)
+		ereport(ERROR,
+				(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+				 errmsg("EXPLAIN option ANALYZE cannot be used with GENERIC_PLAN")));
+
+	/* check that WAL is used with EXPLAIN ANALYZE */
 	if (es->wal && !es->analyze)
 		ereport(ERROR,
 				(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
@@ -574,6 +583,8 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
 		eflags = EXEC_FLAG_EXPLAIN_ONLY;
 	if (into)
 		eflags |= GetIntoRelEFlags(into);
+	if (es->generic)
+		eflags |= EXEC_FLAG_EXPLAIN_GENERIC;
 
 	/* call ExecutorStart to prepare the plan for execution */
 	ExecutorStart(queryDesc, eflags);
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index b32f419176..23ffcbf1aa 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -131,6 +131,9 @@ static void EvalPlanQualStart(EPQState *epqstate, Plan *planTree);
 void
 ExecutorStart(QueryDesc *queryDesc, int eflags)
 {
+	/* EXEC_FLAG_EXPLAIN_GENERIC can only occur with EXEC_FLAG_EXPLAIN_ONLY */
+	Assert((eflags & EXEC_FLAG_EXPLAIN_ONLY) ||
+		   !(eflags & EXEC_FLAG_EXPLAIN_GENERIC));
 	/*
 	 * In some cases (e.g. an EXECUTE statement) a query execution will skip
 	 * parse analysis, which means that the query_id won't be reported.  Note
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index fd6ca8a5d9..6333822ff9 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -2044,10 +2044,12 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
 			pprune->present_parts = bms_copy(pinfo->present_parts);
 
 			/*
-			 * Initialize pruning contexts as needed.
+			 * Initialize pruning contexts as needed.  Specifically, we want to
+			 * skip execution-time partition pruning for EXPLAIN (GENERIC_PLAN).
 			 */
 			pprune->initial_pruning_steps = pinfo->initial_pruning_steps;
-			if (pinfo->initial_pruning_steps)
+			if (pinfo->initial_pruning_steps &&
+				!(econtext->ecxt_estate->es_top_eflags & EXEC_FLAG_EXPLAIN_GENERIC))
 			{
 				InitPartitionPruneContext(&pprune->initial_context,
 										  pinfo->initial_pruning_steps,
@@ -2057,7 +2059,8 @@ CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
 				prunestate->do_initial_prune = true;
 			}
 			pprune->exec_pruning_steps = pinfo->exec_pruning_steps;
-			if (pinfo->exec_pruning_steps)
+			if (pinfo->exec_pruning_steps &&
+				!(econtext->ecxt_estate->es_top_eflags & EXEC_FLAG_EXPLAIN_GENERIC))
 			{
 				InitPartitionPruneContext(&pprune->exec_context,
 										  pinfo->exec_pruning_steps,
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index e892df9819..9143964e78 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -27,6 +27,7 @@
 #include "access/sysattr.h"
 #include "catalog/pg_proc.h"
 #include "catalog/pg_type.h"
+#include "commands/defrem.h"
 #include "miscadmin.h"
 #include "nodes/makefuncs.h"
 #include "nodes/nodeFuncs.h"
@@ -2906,10 +2907,38 @@ static Query *
 transformExplainStmt(ParseState *pstate, ExplainStmt *stmt)
 {
 	Query	   *result;
+	bool		generic_plan = false;
+	Oid		   *paramTypes = NULL;
+	int			numParams = 0;
+
+	/*
+	 * If we have no external source of parameter definitions, and the
+	 * GENERIC_PLAN option is specified, then accept variable parameter
+	 * definitions (as occurs in PREPARE, for example).
+	 */
+	if (pstate->p_paramref_hook == NULL)
+	{
+		ListCell   *lc;
+
+		foreach(lc, stmt->options)
+		{
+			DefElem    *opt = (DefElem *) lfirst(lc);
+
+			if (strcmp(opt->defname, "generic_plan") == 0)
+				generic_plan = defGetBoolean(opt);
+			/* don't "break", as we want the last value */
+		}
+		if (generic_plan)
+			setup_parse_variable_parameters(pstate, &paramTypes, &numParams);
+	}
 
 	/* transform contained query, allowing SELECT INTO */
 	stmt->query = (Node *) transformOptionalSelectInto(pstate, stmt->query);
 
+	/* make sure all is well with parameter types */
+	if (generic_plan)
+		check_variable_parameters(pstate, (Query *) stmt->query);
+
 	/* represent the command as a utility Query */
 	result = makeNode(Query);
 	result->commandType = CMD_UTILITY;
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 42e87b9e49..e38a49e8bd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -3752,9 +3752,9 @@ psql_completion(const char *text, int start, int end)
 		 * one word, so the above test is correct.
 		 */
 		if (ends_with(prev_wd, '(') || ends_with(prev_wd, ','))
-			COMPLETE_WITH("ANALYZE", "VERBOSE", "COSTS", "SETTINGS",
+			COMPLETE_WITH("ANALYZE", "VERBOSE", "COSTS", "SETTINGS", "GENERIC_PLAN",
 						  "BUFFERS", "WAL", "TIMING", "SUMMARY", "FORMAT");
-		else if (TailMatches("ANALYZE|VERBOSE|COSTS|SETTINGS|BUFFERS|WAL|TIMING|SUMMARY"))
+		else if (TailMatches("ANALYZE|VERBOSE|COSTS|SETTINGS|GENERIC_PLAN|BUFFERS|WAL|TIMING|SUMMARY"))
 			COMPLETE_WITH("ON", "OFF");
 		else if (TailMatches("FORMAT"))
 			COMPLETE_WITH("TEXT", "XML", "JSON", "YAML");
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
index 7c1071ddd1..3d3e632a0c 100644
--- a/src/include/commands/explain.h
+++ b/src/include/commands/explain.h
@@ -46,6 +46,7 @@ typedef struct ExplainState
 	bool		timing;			/* print detailed node timing */
 	bool		summary;		/* print total planning and execution timing */
 	bool		settings;		/* print modified settings */
+	bool		generic;		/* generate a generic plan */
 	ExplainFormat format;		/* output format */
 	/* state for output formatting --- not reset for each new plan tree */
 	int			indent;			/* current indentation level */
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index dbd77050c7..7b4c1834ef 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -36,6 +36,11 @@
  * of startup should occur.  However, error checks (such as permission checks)
  * should be performed.
  *
+ * EXPLAIN_GENERIC can only be used together with EXPLAIN_ONLY.  It indicates
+ * that a generic plan is being calculated using EXPLAIN (GENERIC_PLAN), which
+ * means that missing parameters must be tolerated.  Currently, the only effect
+ * is to suppress execution-time partition pruning.
+ *
  * REWIND indicates that the plan node should try to efficiently support
  * rescans without parameter changes.  (Nodes must support ExecReScan calls
  * in any case, but if this flag was not given, they are at liberty to do it
@@ -53,12 +58,13 @@
  * mean that the plan can't queue any AFTER triggers; just that the caller
  * is responsible for there being a trigger context for them to be queued in.
  */
-#define EXEC_FLAG_EXPLAIN_ONLY	0x0001	/* EXPLAIN, no ANALYZE */
-#define EXEC_FLAG_REWIND		0x0002	/* need efficient rescan */
-#define EXEC_FLAG_BACKWARD		0x0004	/* need backward scan */
-#define EXEC_FLAG_MARK			0x0008	/* need mark/restore */
-#define EXEC_FLAG_SKIP_TRIGGERS 0x0010	/* skip AfterTrigger calls */
-#define EXEC_FLAG_WITH_NO_DATA	0x0020	/* rel scannability doesn't matter */
+#define EXEC_FLAG_EXPLAIN_ONLY		0x0001	/* EXPLAIN, no ANALYZE */
+#define EXEC_FLAG_REWIND			0x0002	/* need efficient rescan */
+#define EXEC_FLAG_BACKWARD			0x0004	/* need backward scan */
+#define EXEC_FLAG_MARK				0x0008	/* need mark/restore */
+#define EXEC_FLAG_SKIP_TRIGGERS		0x0010	/* skip AfterTrigger calls */
+#define EXEC_FLAG_WITH_NO_DATA		0x0020	/* rel scannability doesn't matter */
+#define EXEC_FLAG_EXPLAIN_GENERIC	0x0040	/* EXPLAIN (GENERIC_PLAN) */
 
 
 /* Hook for plugins to get control in ExecutorStart() */
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index 48620edbc2..253b818c77 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -517,3 +517,45 @@ select explain_filter('explain (verbose) select * from int8_tbl i8');
  Query Identifier: N
 (3 rows)
 
+-- Test EXPLAIN (GENERIC_PLAN)
+select explain_filter('explain (generic_plan) select unique1 from tenk1 where thousand = $1');
+                                 explain_filter                                  
+---------------------------------------------------------------------------------
+ Bitmap Heap Scan on tenk1  (cost=N.N..N.N rows=N width=N)
+   Recheck Cond: (thousand = $N)
+   ->  Bitmap Index Scan on tenk1_thous_tenthous  (cost=N.N..N.N rows=N width=N)
+         Index Cond: (thousand = $N)
+(4 rows)
+
+-- should fail
+select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1 where thousand = $1');
+ERROR:  EXPLAIN option ANALYZE cannot be used with GENERIC_PLAN
+CONTEXT:  PL/pgSQL function explain_filter(text) line 5 at FOR over EXECUTE statement
+-- Test EXPLAIN (GENERIC_PLAN) with partition pruning
+-- partitions should be pruned at plan time, based on constants,
+-- but there should be no pruning based on parameter placeholders
+create table gen_part (
+  key1 integer not null,
+  key2 integer not null
+) partition by list (key1);
+create table gen_part_1
+  partition of gen_part for values in (1)
+  partition by range (key2);
+create table gen_part_1_1
+  partition of gen_part_1 for values from (1) to (2);
+create table gen_part_1_2
+  partition of gen_part_1 for values from (2) to (3);
+create table gen_part_2
+  partition of gen_part for values in (2);
+-- should only scan gen_part_1_1 and gen_part_1_2, but not gen_part_2
+select explain_filter('explain (generic_plan) select key1, key2 from gen_part where key1 = 1 and key2 = $1');
+                              explain_filter                               
+---------------------------------------------------------------------------
+ Append  (cost=N.N..N.N rows=N width=N)
+   ->  Seq Scan on gen_part_1_1 gen_part_1  (cost=N.N..N.N rows=N width=N)
+         Filter: ((key1 = N) AND (key2 = $N))
+   ->  Seq Scan on gen_part_1_2 gen_part_2  (cost=N.N..N.N rows=N width=N)
+         Filter: ((key1 = N) AND (key2 = $N))
+(5 rows)
+
+drop table gen_part;
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index ae3f7a308d..ff9c51e1d1 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -128,3 +128,27 @@ select explain_filter('explain (verbose) select * from t1 where pg_temp.mysin(f1
 -- Test compute_query_id
 set compute_query_id = on;
 select explain_filter('explain (verbose) select * from int8_tbl i8');
+
+-- Test EXPLAIN (GENERIC_PLAN)
+select explain_filter('explain (generic_plan) select unique1 from tenk1 where thousand = $1');
+-- should fail
+select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1 where thousand = $1');
+-- Test EXPLAIN (GENERIC_PLAN) with partition pruning
+-- partitions should be pruned at plan time, based on constants,
+-- but there should be no pruning based on parameter placeholders
+create table gen_part (
+  key1 integer not null,
+  key2 integer not null
+) partition by list (key1);
+create table gen_part_1
+  partition of gen_part for values in (1)
+  partition by range (key2);
+create table gen_part_1_1
+  partition of gen_part_1 for values from (1) to (2);
+create table gen_part_1_2
+  partition of gen_part_1 for values from (2) to (3);
+create table gen_part_2
+  partition of gen_part for values in (2);
+-- should only scan gen_part_1_1 and gen_part_1_2, but not gen_part_2
+select explain_filter('explain (generic_plan) select key1, key2 from gen_part where key1 = 1 and key2 = $1');
+drop table gen_part;
-- 
2.39.2

#24Christoph Berg
myon@debian.org
In reply to: Laurenz Albe (#23)
Re: Make EXPLAIN generate a generic plan for a parameterized query

Re: Laurenz Albe

And here is v10, which includes tab completion for the new option.

IMHO everything looks good now. Marking as ready for committer.

Thanks!

Christoph

#25Tom Lane
tgl@sss.pgh.pa.us
In reply to: Laurenz Albe (#22)
Re: Make EXPLAIN generate a generic plan for a parameterized query

Laurenz Albe <laurenz.albe@cybertec.at> writes:

On Tue, 2023-03-21 at 16:32 +0100, Christoph Berg wrote:

The test involving postgres_fdw is still necessary to exercise the new
EXEC_FLAG_EXPLAIN_GENERIC code path, but needs to be moved elsewhere,
probably src/test/modules/.

Tests for postgres_fdw are in contrib/postgres_fdw/sql/postgres_fdw.sql,
so I added the test there.

I don't actually see why a postgres_fdw test case is needed at all?
The tests in explain.sql seem sufficient.

regards, tom lane

#26Christoph Berg
myon@debian.org
In reply to: Tom Lane (#25)
Re: Make EXPLAIN generate a generic plan for a parameterized query

Re: Tom Lane

I don't actually see why a postgres_fdw test case is needed at all?
The tests in explain.sql seem sufficient.

When I asked Laurenz about that he told me that local tables wouldn't
exercise the code specific for EXEC_FLAG_EXPLAIN_GENERIC.

(Admittedly my knowledge of the planner wasn't deep enough to verify
that. Laurenz is currently traveling, so I don't know if he could
answer this himself now.)

Christoph

#27Tom Lane
tgl@sss.pgh.pa.us
In reply to: Christoph Berg (#26)
Re: Make EXPLAIN generate a generic plan for a parameterized query

Christoph Berg <myon@debian.org> writes:

Re: Tom Lane

I don't actually see why a postgres_fdw test case is needed at all?
The tests in explain.sql seem sufficient.

When I asked Laurenz about that he told me that local tables wouldn't
exercise the code specific for EXEC_FLAG_EXPLAIN_GENERIC.

But there isn't any ... or at least, I fail to see what isn't sufficiently
exercised by that new explain.sql test case that's identical to this one
except for being a non-foreign table. Perhaps at some point this patch
modified postgres_fdw code? But it doesn't now.

I don't mind having a postgres_fdw test if there's something for it
to test, but it just looks duplicative to me. Other things being
equal, I'd prefer to test this feature in explain.sql, since (a) it's
a core feature and (b) the core tests are better parallelized than the
contrib tests, so the same test should be cheaper to run.

(Admittedly my knowledge of the planner wasn't deep enough to verify
that. Laurenz is currently traveling, so I don't know if he could
answer this himself now.)

OK, thanks for the status update. What I'll do to get this off my
plate is to push the patch without the postgres_fdw test -- if
Laurenz wants to advocate for that when he returns, we can discuss it
more.

regards, tom lane

#28Laurenz Albe
laurenz.albe@cybertec.at
In reply to: Tom Lane (#27)
Re: Make EXPLAIN generate a generic plan for a parameterized query

On Fri, 2023-03-24 at 16:41 -0400, Tom Lane wrote:

Christoph Berg <myon@debian.org> writes:

Re: Tom Lane

I don't actually see why a postgres_fdw test case is needed at all?
The tests in explain.sql seem sufficient.

When I asked Laurenz about that he told me that local tables wouldn't
exercise the code specific for EXEC_FLAG_EXPLAIN_GENERIC.

But there isn't any ... or at least, I fail to see what isn't sufficiently
exercised by that new explain.sql test case that's identical to this one
except for being a non-foreign table.  Perhaps at some point this patch
modified postgres_fdw code?  But it doesn't now.

I don't mind having a postgres_fdw test if there's something for it
to test, but it just looks duplicative to me.  Other things being
equal, I'd prefer to test this feature in explain.sql, since (a) it's
a core feature and (b) the core tests are better parallelized than the
contrib tests, so the same test should be cheaper to run.

(Admittedly my knowledge of the planner wasn't deep enough to verify
that. Laurenz is currently traveling, so I don't know if he could
answer this himself now.)

OK, thanks for the status update.  What I'll do to get this off my
plate is to push the patch without the postgres_fdw test -- if
Laurenz wants to advocate for that when he returns, we can discuss it
more.

Thanks for committing this.

As Chrisoph mentioned, I found that I could not reproduce the problem
that was addressed by the EXEC_FLAG_EXPLAIN_GENERIC hack using local
partitioned tables. My optimizer knowledge is not deep enough to tell
why, and it might well be a bug lurking in the FDW part of the
optimizer code. It is not FDW specific, since I discovered it with
oracle_fdw and could reproduce it with postgres_fdw.

I was aware that it is awkward to add a test to a contrib module, but
I thought that I should add a test that exercises the new code path.
But I am fine without the postgres_fdw test.

Yours,
Laurenz Albe