Parallelize correlated subqueries that execute within each worker
Started by James Colemanover 4 years ago46 messages
In a bug report back in November [1] a subthread explored why parallel
query is excluded any time we have "Plan nodes which reference a
correlated SubPlan". Amit's understanding was that the reasoning had
to do with inability to easily pass (potentially variable length)
Param values between workers.
However a decent-sized subset of this kind of query doesn't actually
require that we communicate between workers. If the Subplan executes
per-tuple within the worker then there's no reason I can see why it
needs to be marked parallel unsafe. Amit concurred but noted that
identifying that subset of plans is the difficult part (as is usually
the case!)
At the time I'd started work on an approach to handle this case and
hoped to "post about it in a new thread later this week." That didn't
happen, but here we are now, and I finally have this patch cleaned up
enough to share.
The basic idea is that we need to track (both on nodes and relations)
not only whether that node or rel is parallel safe but also whether
it's parallel safe assuming params are rechecked in the using context.
That allows us to delay making a final decision until we have
sufficient context to conclude that a given usage of a Param is
actually parallel safe or unsafe.
The first patch in this series was previously posted in the thread
"Consider parallel for lateral subqueries with limit" [2] and is
required as a precursor for various test cases to work here.
The second patch implements the core of the series. It results in
parallel query being possible for subplans that execute entirely
within the context of a parallel worker for cases where that subplan
is in the target, a LATERAL JOIN, or the WHERE and ORDER BY clauses.
The final patch notes several places where we set e.g.
rel->consider_parallel but setting the corresponding new value
rel->consider_parallel_recheckng_params wasn't yet necessary. It shows
opportunity either for further improvement or concluding certain cases
can't benefit and should be left unchanged.
James
1: /messages/by-id/CAAaqYe_vihKjc+8LuQa49EHW4+Kfefb3wHqPYFnCuUqozo+LFg@mail.gmail.com
2: /messages/by-id/CAAaqYe_HEkmLwf_1iEHxXwQOWiRyiFd=uOu6kwj3sWYdVd1-zA@mail.gmail.com
Attachments:
v1-0003-Other-places-to-consider-for-completeness.patchapplication/octet-stream; name=v1-0003-Other-places-to-consider-for-completeness.patchDownload
From ac93b6513dfd3fd066cbbe51eced2bc00146f0c0 Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Fri, 7 May 2021 15:37:22 +0000
Subject: [PATCH v1 3/3] Other places to consider for completeness
---
src/backend/optimizer/path/allpaths.c | 4 ++++
src/backend/optimizer/plan/planmain.c | 2 +-
src/backend/optimizer/plan/planner.c | 3 +++
src/backend/optimizer/plan/subselect.c | 3 +++
src/backend/optimizer/prep/prepunion.c | 2 ++
5 files changed, 13 insertions(+), 1 deletion(-)
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index ae744da135..5696a54b18 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -1142,6 +1142,8 @@ set_append_rel_size(PlannerInfo *root, RelOptInfo *rel,
*/
if (!childrel->consider_parallel)
rel->consider_parallel = false;
+ if (!childrel->consider_parallel_rechecking_params)
+ rel->consider_parallel_rechecking_params = false;
/*
* Accumulate size information from each live child.
@@ -1263,6 +1265,8 @@ set_append_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
*/
if (!rel->consider_parallel)
childrel->consider_parallel = false;
+ if (!childrel->consider_parallel_rechecking_params)
+ rel->consider_parallel_rechecking_params = false;
/*
* Compute the child's access paths.
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index bdbce2b87d..5135f9ee97 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -119,7 +119,7 @@ query_planner(PlannerInfo *root,
if (root->glob->parallelModeOK &&
force_parallel_mode != FORCE_PARALLEL_OFF)
final_rel->consider_parallel =
- is_parallel_safe(root, parse->jointree->quals, NULL);
+ is_parallel_safe(root, parse->jointree->quals, &final_rel->consider_parallel_rechecking_params);
/*
* The only path for it is a trivial Result path. We cheat a
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 28b8f7d8b5..fb35cd78d1 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -4088,6 +4088,7 @@ create_window_paths(PlannerInfo *root,
if (input_rel->consider_parallel && output_target_parallel_safe &&
is_parallel_safe(root, (Node *) activeWindows, NULL))
window_rel->consider_parallel = true;
+ /* consider_parallel_rechecking_params */
/*
* If the input rel belongs to a single FDW, so does the window rel.
@@ -4292,6 +4293,7 @@ create_distinct_paths(PlannerInfo *root,
* expressions are parallel-safe.
*/
distinct_rel->consider_parallel = input_rel->consider_parallel;
+ distinct_rel->consider_parallel_rechecking_params = input_rel->consider_parallel_rechecking_params;
/*
* If the input rel belongs to a single FDW, so does the distinct_rel.
@@ -4493,6 +4495,7 @@ create_ordered_paths(PlannerInfo *root,
*/
if (input_rel->consider_parallel && target_parallel_safe)
ordered_rel->consider_parallel = true;
+ /* consider_parallel_rechecking_params */
/*
* If the input rel belongs to a single FDW, so does the ordered_rel.
diff --git a/src/backend/optimizer/plan/subselect.c b/src/backend/optimizer/plan/subselect.c
index fd1cfe6c73..0c6fbf16a2 100644
--- a/src/backend/optimizer/plan/subselect.c
+++ b/src/backend/optimizer/plan/subselect.c
@@ -1022,6 +1022,7 @@ SS_process_ctes(PlannerInfo *root)
* parallel-safe.
*/
splan->parallel_safe = false;
+ splan->parallel_safe_ignoring_params = false;
splan->setParam = NIL;
splan->parParam = NIL;
splan->args = NIL;
@@ -2178,6 +2179,7 @@ SS_charge_for_initplans(PlannerInfo *root, RelOptInfo *final_rel)
path->startup_cost += initplan_cost;
path->total_cost += initplan_cost;
path->parallel_safe = false;
+ path->parallel_safe_ignoring_params = false;
}
/*
@@ -2186,6 +2188,7 @@ SS_charge_for_initplans(PlannerInfo *root, RelOptInfo *final_rel)
*/
final_rel->partial_pathlist = NIL;
final_rel->consider_parallel = false;
+ final_rel->consider_parallel_rechecking_params = false;
/* We needn't do set_cheapest() here, caller will do it */
}
diff --git a/src/backend/optimizer/prep/prepunion.c b/src/backend/optimizer/prep/prepunion.c
index a57bc1c2a8..c535131144 100644
--- a/src/backend/optimizer/prep/prepunion.c
+++ b/src/backend/optimizer/prep/prepunion.c
@@ -273,6 +273,7 @@ recurse_set_operations(Node *setOp, PlannerInfo *root,
*/
final_rel = fetch_upper_rel(subroot, UPPERREL_FINAL, NULL);
rel->consider_parallel = final_rel->consider_parallel;
+ rel->consider_parallel_rechecking_params = final_rel->consider_parallel_rechecking_params;
/*
* For the moment, we consider only a single Path for the subquery.
@@ -617,6 +618,7 @@ generate_union_paths(SetOperationStmt *op, PlannerInfo *root,
result_rel = fetch_upper_rel(root, UPPERREL_SETOP, relids);
result_rel->reltarget = create_pathtarget(root, tlist);
result_rel->consider_parallel = consider_parallel;
+ /* consider_parallel_rechecking_params */
/*
* Append the child results together.
--
2.20.1
v1-0002-Parallel-query-support-for-basic-correlated-subqu.patchapplication/octet-stream; name=v1-0002-Parallel-query-support-for-basic-correlated-subqu.patchDownload
From e3df76aa8823ff05d311fd3aa4505a7f7d58b015 Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Fri, 27 Nov 2020 18:44:30 -0500
Subject: [PATCH v1 2/3] Parallel query support for basic correlated subqueries
Not all Params are inherently parallel-unsafe, but we can't know whether
they're parallel-safe up-front: we need contextual information for a
given path shape. Here we delay the final determination of whether or
not a Param is parallel-safe by initially verifying that it is minimally
parallel-safe for things that are inherent (e.g., no parallel-unsafe
functions or relations involved) and later re-checking that a given
usage is contextually safe (e.g., the Param is for correlation that can
happen entirely within a parallel worker (as opposed to needing to pass
values between workers).
---
doc/src/sgml/parallel.sgml | 3 +-
src/backend/nodes/copyfuncs.c | 2 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/outfuncs.c | 3 +
src/backend/nodes/readfuncs.c | 2 +
src/backend/optimizer/path/allpaths.c | 61 +++++--
src/backend/optimizer/path/equivclass.c | 4 +-
src/backend/optimizer/path/indxpath.c | 20 ++-
src/backend/optimizer/path/joinpath.c | 21 ++-
src/backend/optimizer/plan/createplan.c | 2 +
src/backend/optimizer/plan/planmain.c | 2 +-
src/backend/optimizer/plan/planner.c | 155 +++++++++++++++---
src/backend/optimizer/plan/subselect.c | 21 ++-
src/backend/optimizer/prep/prepunion.c | 1 +
src/backend/optimizer/util/clauses.c | 113 ++++++++++---
src/backend/optimizer/util/pathnode.c | 36 +++-
src/backend/optimizer/util/relnode.c | 30 +++-
src/backend/utils/misc/guc.c | 11 ++
src/include/nodes/pathnodes.h | 5 +-
src/include/nodes/plannodes.h | 1 +
src/include/nodes/primnodes.h | 1 +
src/include/optimizer/clauses.h | 4 +-
.../regress/expected/incremental_sort.out | 28 ++--
src/test/regress/expected/select_parallel.out | 137 ++++++++++++++++
src/test/regress/expected/sysviews.out | 3 +-
src/test/regress/sql/select_parallel.sql | 37 +++++
26 files changed, 604 insertions(+), 100 deletions(-)
diff --git a/doc/src/sgml/parallel.sgml b/doc/src/sgml/parallel.sgml
index 479e24a1dc..25a689f690 100644
--- a/doc/src/sgml/parallel.sgml
+++ b/doc/src/sgml/parallel.sgml
@@ -517,7 +517,8 @@ EXPLAIN SELECT * FROM pgbench_accounts WHERE filler LIKE '%x%';
<listitem>
<para>
- Plan nodes which reference a correlated <literal>SubPlan</literal>.
+ Plan nodes which reference a correlated <literal>SubPlan</literal> where
+ the result is shared between workers.
</para>
</listitem>
</itemizedlist>
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 632cc31a04..240895ac83 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -121,6 +121,7 @@ CopyPlanFields(const Plan *from, Plan *newnode)
COPY_SCALAR_FIELD(parallel_aware);
COPY_SCALAR_FIELD(parallel_safe);
COPY_SCALAR_FIELD(async_capable);
+ COPY_SCALAR_FIELD(parallel_safe_ignoring_params);
COPY_SCALAR_FIELD(plan_node_id);
COPY_NODE_FIELD(targetlist);
COPY_NODE_FIELD(qual);
@@ -1777,6 +1778,7 @@ _copySubPlan(const SubPlan *from)
COPY_SCALAR_FIELD(useHashTable);
COPY_SCALAR_FIELD(unknownEqFalse);
COPY_SCALAR_FIELD(parallel_safe);
+ COPY_SCALAR_FIELD(parallel_safe_ignoring_params);
COPY_NODE_FIELD(setParam);
COPY_NODE_FIELD(parParam);
COPY_NODE_FIELD(args);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index a410a29a17..011befabf2 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -459,6 +459,7 @@ _equalSubPlan(const SubPlan *a, const SubPlan *b)
COMPARE_SCALAR_FIELD(useHashTable);
COMPARE_SCALAR_FIELD(unknownEqFalse);
COMPARE_SCALAR_FIELD(parallel_safe);
+ COMPARE_SCALAR_FIELD(parallel_safe_ignoring_params);
COMPARE_NODE_FIELD(setParam);
COMPARE_NODE_FIELD(parParam);
COMPARE_NODE_FIELD(args);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index c723f6d635..84156d6465 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -334,6 +334,7 @@ _outPlanInfo(StringInfo str, const Plan *node)
WRITE_BOOL_FIELD(parallel_aware);
WRITE_BOOL_FIELD(parallel_safe);
WRITE_BOOL_FIELD(async_capable);
+ WRITE_BOOL_FIELD(parallel_safe_ignoring_params);
WRITE_INT_FIELD(plan_node_id);
WRITE_NODE_FIELD(targetlist);
WRITE_NODE_FIELD(qual);
@@ -1372,6 +1373,7 @@ _outSubPlan(StringInfo str, const SubPlan *node)
WRITE_BOOL_FIELD(useHashTable);
WRITE_BOOL_FIELD(unknownEqFalse);
WRITE_BOOL_FIELD(parallel_safe);
+ WRITE_BOOL_FIELD(parallel_safe_ignoring_params);
WRITE_NODE_FIELD(setParam);
WRITE_NODE_FIELD(parParam);
WRITE_NODE_FIELD(args);
@@ -1770,6 +1772,7 @@ _outPathInfo(StringInfo str, const Path *node)
outBitmapset(str, NULL);
WRITE_BOOL_FIELD(parallel_aware);
WRITE_BOOL_FIELD(parallel_safe);
+ WRITE_BOOL_FIELD(parallel_safe_ignoring_params);
WRITE_INT_FIELD(parallel_workers);
WRITE_FLOAT_FIELD(rows, "%.0f");
WRITE_FLOAT_FIELD(startup_cost, "%.2f");
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 3746668f52..2b22978616 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -1620,6 +1620,7 @@ ReadCommonPlan(Plan *local_node)
READ_BOOL_FIELD(parallel_aware);
READ_BOOL_FIELD(parallel_safe);
READ_BOOL_FIELD(async_capable);
+ READ_BOOL_FIELD(parallel_safe_ignoring_params);
READ_INT_FIELD(plan_node_id);
READ_NODE_FIELD(targetlist);
READ_NODE_FIELD(qual);
@@ -2613,6 +2614,7 @@ _readSubPlan(void)
READ_BOOL_FIELD(useHashTable);
READ_BOOL_FIELD(unknownEqFalse);
READ_BOOL_FIELD(parallel_safe);
+ READ_BOOL_FIELD(parallel_safe_ignoring_params);
READ_NODE_FIELD(setParam);
READ_NODE_FIELD(parParam);
READ_NODE_FIELD(args);
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 45e37590d8..ae744da135 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -556,7 +556,8 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
* (see grouping_planner).
*/
if (rel->reloptkind == RELOPT_BASEREL &&
- bms_membership(root->all_baserels) != BMS_SINGLETON)
+ bms_membership(root->all_baserels) != BMS_SINGLETON
+ && (rel->subplan_params == NIL || rte->rtekind != RTE_SUBQUERY))
generate_useful_gather_paths(root, rel, false);
/* Now find the cheapest of the paths for this rel */
@@ -592,6 +593,9 @@ static void
set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte)
{
+ bool parallel_safe;
+ bool parallel_safe_except_in_params;
+
/*
* The flag has previously been initialized to false, so we can just
* return if it becomes clear that we can't safely set it.
@@ -632,7 +636,7 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
if (proparallel != PROPARALLEL_SAFE)
return;
- if (!is_parallel_safe(root, (Node *) rte->tablesample->args))
+ if (!is_parallel_safe(root, (Node *) rte->tablesample->args, NULL))
return;
}
@@ -700,7 +704,7 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
case RTE_FUNCTION:
/* Check for parallel-restricted functions. */
- if (!is_parallel_safe(root, (Node *) rte->functions))
+ if (!is_parallel_safe(root, (Node *) rte->functions, NULL))
return;
break;
@@ -710,7 +714,7 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
case RTE_VALUES:
/* Check for parallel-restricted functions. */
- if (!is_parallel_safe(root, (Node *) rte->values_lists))
+ if (!is_parallel_safe(root, (Node *) rte->values_lists, NULL))
return;
break;
@@ -747,18 +751,28 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
* outer join clauses work correctly. It would likely break equivalence
* classes, too.
*/
- if (!is_parallel_safe(root, (Node *) rel->baserestrictinfo))
- return;
+ parallel_safe = is_parallel_safe(root, (Node *) rel->baserestrictinfo,
+ ¶llel_safe_except_in_params);
/*
* Likewise, if the relation's outputs are not parallel-safe, give up.
* (Usually, they're just Vars, but sometimes they're not.)
*/
- if (!is_parallel_safe(root, (Node *) rel->reltarget->exprs))
- return;
+ if (parallel_safe || parallel_safe_except_in_params)
+ {
+ bool target_parallel_safe;
+ bool target_parallel_safe_ignoring_params = false;
+
+ target_parallel_safe = is_parallel_safe(root,
+ (Node *) rel->reltarget->exprs,
+ &target_parallel_safe_ignoring_params);
+ parallel_safe = parallel_safe && target_parallel_safe;
+ parallel_safe_except_in_params = parallel_safe_except_in_params
+ && target_parallel_safe_ignoring_params;
+ }
- /* We have a winner. */
- rel->consider_parallel = true;
+ rel->consider_parallel = parallel_safe;
+ rel->consider_parallel_rechecking_params = parallel_safe_except_in_params;
}
/*
@@ -2277,9 +2291,21 @@ set_subquery_pathlist(PlannerInfo *root, RelOptInfo *rel,
pathkeys, required_outer));
}
+ /*
+ * XXX: As far as I can tell, the only time partial paths exist here
+ * is when we're going to execute multiple partial parths in parallel
+ * under a gather node (instead of executing paths serially under
+ * an append node). That means that the subquery scan path here
+ * is self-contained at this point -- so by definition it can't be
+ * reliant on lateral relids, which means we'll never have to consider
+ * rechecking params here.
+ */
+ Assert(!(rel->consider_parallel_rechecking_params && rel->partial_pathlist && !rel->consider_parallel));
+
/* If outer rel allows parallelism, do same for partial paths. */
if (rel->consider_parallel && bms_is_empty(required_outer))
{
+
/* If consider_parallel is false, there should be no partial paths. */
Assert(sub_final_rel->consider_parallel ||
sub_final_rel->partial_pathlist == NIL);
@@ -2633,7 +2659,7 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
cheapest_partial_path->rows * cheapest_partial_path->parallel_workers;
simple_gather_path = (Path *)
create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
- NULL, rowsp);
+ rel->lateral_relids, rowsp);
add_path(rel, simple_gather_path);
/*
@@ -2650,7 +2676,7 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
rows = subpath->rows * subpath->parallel_workers;
path = create_gather_merge_path(root, rel, subpath, rel->reltarget,
- subpath->pathkeys, NULL, rowsp);
+ subpath->pathkeys, rel->lateral_relids, rowsp);
add_path(rel, &path->path);
}
}
@@ -2752,11 +2778,15 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
double *rowsp = NULL;
List *useful_pathkeys_list = NIL;
Path *cheapest_partial_path = NULL;
+ Relids required_outer = rel->lateral_relids;
/* If there are no partial paths, there's nothing to do here. */
if (rel->partial_pathlist == NIL)
return;
+ if (!bms_is_subset(required_outer, rel->relids))
+ return;
+
/* Should we override the rel's rowcount estimate? */
if (override_rows)
rowsp = &rows;
@@ -2828,7 +2858,7 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
tmp,
rel->reltarget,
tmp->pathkeys,
- NULL,
+ required_outer,
rowsp);
add_path(rel, &path->path);
@@ -2862,7 +2892,7 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
tmp,
rel->reltarget,
tmp->pathkeys,
- NULL,
+ required_outer,
rowsp);
add_path(rel, &path->path);
@@ -3041,7 +3071,8 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
/*
* Except for the topmost scan/join rel, consider gathering
* partial paths. We'll do the same for the topmost scan/join rel
- * once we know the final targetlist (see grouping_planner).
+ * once we know the final targetlist (see
+ * apply_scanjoin_target_to_paths).
*/
if (lev < levels_needed)
generate_useful_gather_paths(root, rel, false);
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 6f1abbe47d..4c176d2d49 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -897,7 +897,7 @@ find_computable_ec_member(PlannerInfo *root,
* check this last because it's a rather expensive test.
*/
if (require_parallel_safe &&
- !is_parallel_safe(root, (Node *) em->em_expr))
+ !is_parallel_safe(root, (Node *) em->em_expr, NULL))
continue;
return em; /* found usable expression */
@@ -1012,7 +1012,7 @@ relation_can_be_sorted_early(PlannerInfo *root, RelOptInfo *rel,
* check this last because it's a rather expensive test.
*/
if (require_parallel_safe &&
- !is_parallel_safe(root, (Node *) em->em_expr))
+ !is_parallel_safe(root, (Node *) em->em_expr, NULL))
continue;
return true;
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
index 0e4e00eaf0..262c129ff5 100644
--- a/src/backend/optimizer/path/indxpath.c
+++ b/src/backend/optimizer/path/indxpath.c
@@ -1047,10 +1047,21 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
/*
* If appropriate, consider parallel index scan. We don't allow
* parallel index scan for bitmap index scans.
+ *
+ * XXX: Checking rel->consider_parallel_rechecking_params here resulted
+ * in some odd behavior on:
+ * select (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1) from tenk1 t;
+ * where the total cost on the chosen plan went *up* considering
+ * the extra path.
+ *
+ * Current working theory is that this method is about base relation
+ * scans, and we only want parameterized paths to be parallelized as
+ * companions to existing parallel plans and so don't really care to
+ * consider a separate parallel index scan here.
*/
if (index->amcanparallel &&
- rel->consider_parallel && outer_relids == NULL &&
- scantype != ST_BITMAPSCAN)
+ rel->consider_parallel && outer_relids == NULL &&
+ scantype != ST_BITMAPSCAN)
{
ipath = create_index_path(root, index,
index_clauses,
@@ -1100,9 +1111,10 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
result = lappend(result, ipath);
/* If appropriate, consider parallel index scan */
+ /* XXX: As above here for rel->consider_parallel_rechecking_params? */
if (index->amcanparallel &&
- rel->consider_parallel && outer_relids == NULL &&
- scantype != ST_BITMAPSCAN)
+ rel->consider_parallel && outer_relids == NULL &&
+ scantype != ST_BITMAPSCAN)
{
ipath = create_index_path(root, index,
index_clauses,
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index 4c30c65564..c6af979714 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -690,6 +690,7 @@ try_partial_nestloop_path(PlannerInfo *root,
else
outerrelids = outerrel->relids;
+ /* TODO: recheck parallel safety here? */
if (!bms_is_subset(inner_paramrels, outerrelids))
return;
}
@@ -1714,16 +1715,24 @@ match_unsorted_outer(PlannerInfo *root,
* partial path and the joinrel is parallel-safe. However, we can't
* handle JOIN_UNIQUE_OUTER, because the outer path will be partial, and
* therefore we won't be able to properly guarantee uniqueness. Nor can
- * we handle joins needing lateral rels, since partial paths must not be
- * parameterized. Similarly, we can't handle JOIN_FULL and JOIN_RIGHT,
- * because they can produce false null extended rows.
+ * we handle JOIN_FULL and JOIN_RIGHT, because they can produce false null
+ * extended rows.
+ *
+ * Partial paths may only have parameters in limited cases
+ * where the parameterization is fully satisfied without sharing state
+ * between workers, so we only allow lateral rels on inputs to the join
+ * if the resulting join contains no lateral rels, the inner rel's laterals
+ * are fully satisfied by the outer rel, and the outer rel doesn't depend
+ * on the inner rel to produce any laterals.
*/
if (joinrel->consider_parallel &&
save_jointype != JOIN_UNIQUE_OUTER &&
save_jointype != JOIN_FULL &&
save_jointype != JOIN_RIGHT &&
outerrel->partial_pathlist != NIL &&
- bms_is_empty(joinrel->lateral_relids))
+ bms_is_empty(joinrel->lateral_relids) &&
+ bms_is_subset(innerrel->lateral_relids, outerrel->relids) &&
+ (bms_is_empty(outerrel->lateral_relids) || !bms_is_subset(outerrel->lateral_relids, innerrel->relids)))
{
if (nestjoinOK)
consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
@@ -1838,7 +1847,9 @@ consider_parallel_nestloop(PlannerInfo *root,
Path *rcpath;
/* Can't join to an inner path that is not parallel-safe */
- if (!innerpath->parallel_safe)
+ /* TODO: recheck parallel safety of params here? */
+ if (!innerpath->parallel_safe &&
+ !(innerpath->parallel_safe_ignoring_params))
continue;
/*
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index a9aff24831..817ce04596 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -5274,6 +5274,7 @@ copy_generic_path_info(Plan *dest, Path *src)
dest->plan_width = src->pathtarget->width;
dest->parallel_aware = src->parallel_aware;
dest->parallel_safe = src->parallel_safe;
+ dest->parallel_safe_ignoring_params = src->parallel_safe_ignoring_params;
}
/*
@@ -5291,6 +5292,7 @@ copy_plan_costsize(Plan *dest, Plan *src)
dest->parallel_aware = false;
/* Assume the inserted node is parallel-safe, if child plan is. */
dest->parallel_safe = src->parallel_safe;
+ dest->parallel_safe_ignoring_params = src->parallel_safe_ignoring_params;
}
/*
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index 273ac0acf7..bdbce2b87d 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -119,7 +119,7 @@ query_planner(PlannerInfo *root,
if (root->glob->parallelModeOK &&
force_parallel_mode != FORCE_PARALLEL_OFF)
final_rel->consider_parallel =
- is_parallel_safe(root, parse->jointree->quals);
+ is_parallel_safe(root, parse->jointree->quals, NULL);
/*
* The only path for it is a trivial Result path. We cheat a
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 1868c4eff4..28b8f7d8b5 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -150,6 +150,7 @@ static RelOptInfo *create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
PathTarget *target,
bool target_parallel_safe,
+ bool target_parallel_safe_ignoring_params,
grouping_sets_data *gd);
static bool is_degenerate_grouping(PlannerInfo *root);
static void create_degenerate_grouping_paths(PlannerInfo *root,
@@ -157,6 +158,7 @@ static void create_degenerate_grouping_paths(PlannerInfo *root,
RelOptInfo *grouped_rel);
static RelOptInfo *make_grouping_rel(PlannerInfo *root, RelOptInfo *input_rel,
PathTarget *target, bool target_parallel_safe,
+ bool target_parallel_safe_ignoring_params,
Node *havingQual);
static void create_ordinary_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
@@ -231,6 +233,7 @@ static void apply_scanjoin_target_to_paths(PlannerInfo *root,
List *scanjoin_targets,
List *scanjoin_targets_contain_srfs,
bool scanjoin_target_parallel_safe,
+ bool scanjoin_target_parallel_safe_ignoring_params,
bool tlist_same_exprs);
static void create_partitionwise_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
@@ -1235,6 +1238,7 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
List *final_targets;
List *final_targets_contain_srfs;
bool final_target_parallel_safe;
+ bool final_target_parallel_safe_ignoring_params = false;
RelOptInfo *current_rel;
RelOptInfo *final_rel;
FinalPathExtraData extra;
@@ -1297,7 +1301,7 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
/* And check whether it's parallel safe */
final_target_parallel_safe =
- is_parallel_safe(root, (Node *) final_target->exprs);
+ is_parallel_safe(root, (Node *) final_target->exprs, &final_target_parallel_safe_ignoring_params);
/* The setop result tlist couldn't contain any SRFs */
Assert(!parse->hasTargetSRFs);
@@ -1331,14 +1335,17 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
List *sort_input_targets;
List *sort_input_targets_contain_srfs;
bool sort_input_target_parallel_safe;
+ bool sort_input_target_parallel_safe_ignoring_params = false;
PathTarget *grouping_target;
List *grouping_targets;
List *grouping_targets_contain_srfs;
bool grouping_target_parallel_safe;
+ bool grouping_target_parallel_safe_ignoring_params = false;
PathTarget *scanjoin_target;
List *scanjoin_targets;
List *scanjoin_targets_contain_srfs;
bool scanjoin_target_parallel_safe;
+ bool scanjoin_target_parallel_safe_ignoring_params = false;
bool scanjoin_target_same_exprs;
bool have_grouping;
WindowFuncLists *wflists = NULL;
@@ -1451,7 +1458,7 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
*/
final_target = create_pathtarget(root, root->processed_tlist);
final_target_parallel_safe =
- is_parallel_safe(root, (Node *) final_target->exprs);
+ is_parallel_safe(root, (Node *) final_target->exprs, &final_target_parallel_safe_ignoring_params);
/*
* If ORDER BY was given, consider whether we should use a post-sort
@@ -1464,12 +1471,13 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
final_target,
&have_postponed_srfs);
sort_input_target_parallel_safe =
- is_parallel_safe(root, (Node *) sort_input_target->exprs);
+ is_parallel_safe(root, (Node *) sort_input_target->exprs, &sort_input_target_parallel_safe_ignoring_params);
}
else
{
sort_input_target = final_target;
sort_input_target_parallel_safe = final_target_parallel_safe;
+ sort_input_target_parallel_safe_ignoring_params = final_target_parallel_safe_ignoring_params;
}
/*
@@ -1483,12 +1491,13 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
final_target,
activeWindows);
grouping_target_parallel_safe =
- is_parallel_safe(root, (Node *) grouping_target->exprs);
+ is_parallel_safe(root, (Node *) grouping_target->exprs, &grouping_target_parallel_safe_ignoring_params);
}
else
{
grouping_target = sort_input_target;
grouping_target_parallel_safe = sort_input_target_parallel_safe;
+ grouping_target_parallel_safe_ignoring_params = sort_input_target_parallel_safe_ignoring_params;
}
/*
@@ -1502,12 +1511,13 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
{
scanjoin_target = make_group_input_target(root, final_target);
scanjoin_target_parallel_safe =
- is_parallel_safe(root, (Node *) scanjoin_target->exprs);
+ is_parallel_safe(root, (Node *) scanjoin_target->exprs, &scanjoin_target_parallel_safe_ignoring_params);
}
else
{
scanjoin_target = grouping_target;
scanjoin_target_parallel_safe = grouping_target_parallel_safe;
+ scanjoin_target_parallel_safe_ignoring_params = grouping_target_parallel_safe_ignoring_params;
}
/*
@@ -1559,6 +1569,7 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
apply_scanjoin_target_to_paths(root, current_rel, scanjoin_targets,
scanjoin_targets_contain_srfs,
scanjoin_target_parallel_safe,
+ scanjoin_target_parallel_safe_ignoring_params,
scanjoin_target_same_exprs);
/*
@@ -1585,6 +1596,7 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
current_rel,
grouping_target,
grouping_target_parallel_safe,
+ grouping_target_parallel_safe_ignoring_params,
gset_data);
/* Fix things up if grouping_target contains SRFs */
if (parse->hasTargetSRFs)
@@ -1658,10 +1670,25 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
* not a SELECT, consider_parallel will be false for every relation in the
* query.
*/
- if (current_rel->consider_parallel &&
- is_parallel_safe(root, parse->limitOffset) &&
- is_parallel_safe(root, parse->limitCount))
- final_rel->consider_parallel = true;
+ if (current_rel->consider_parallel || current_rel->consider_parallel_rechecking_params)
+ {
+ bool limit_count_parallel_safe;
+ bool limit_offset_parallel_safe;
+ bool limit_count_parallel_safe_ignoring_params = false;
+ bool limit_offset_parallel_safe_ignoring_params = false;
+
+ limit_count_parallel_safe = is_parallel_safe(root, parse->limitCount, &limit_count_parallel_safe_ignoring_params);
+ limit_offset_parallel_safe = is_parallel_safe(root, parse->limitOffset, &limit_offset_parallel_safe_ignoring_params);
+
+ if (current_rel->consider_parallel &&
+ limit_count_parallel_safe &&
+ limit_offset_parallel_safe)
+ final_rel->consider_parallel = true;
+ if (current_rel->consider_parallel_rechecking_params &&
+ limit_count_parallel_safe_ignoring_params &&
+ limit_offset_parallel_safe_ignoring_params)
+ final_rel->consider_parallel_rechecking_params = true;
+ }
/*
* If the current_rel belongs to a single FDW, so does the final_rel.
@@ -1862,8 +1889,8 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
* Generate partial paths for final_rel, too, if outer query levels might
* be able to make use of them.
*/
- if (final_rel->consider_parallel && root->query_level > 1 &&
- !limit_needed(parse))
+ if ((final_rel->consider_parallel || final_rel->consider_parallel_rechecking_params) &&
+ root->query_level > 1 && !limit_needed(parse))
{
Assert(!parse->rowMarks && parse->commandType == CMD_SELECT);
foreach(lc, current_rel->partial_pathlist)
@@ -3275,6 +3302,7 @@ create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
PathTarget *target,
bool target_parallel_safe,
+ bool target_parallel_safe_ignoring_params,
grouping_sets_data *gd)
{
Query *parse = root->parse;
@@ -3290,7 +3318,9 @@ create_grouping_paths(PlannerInfo *root,
* aggregation paths.
*/
grouped_rel = make_grouping_rel(root, input_rel, target,
- target_parallel_safe, parse->havingQual);
+ target_parallel_safe,
+ target_parallel_safe_ignoring_params,
+ parse->havingQual);
/*
* Create either paths for a degenerate grouping or paths for ordinary
@@ -3351,6 +3381,7 @@ create_grouping_paths(PlannerInfo *root,
extra.flags = flags;
extra.target_parallel_safe = target_parallel_safe;
+ extra.target_parallel_safe_ignoring_params = target_parallel_safe_ignoring_params;
extra.havingQual = parse->havingQual;
extra.targetList = parse->targetList;
extra.partial_costs_set = false;
@@ -3386,7 +3417,7 @@ create_grouping_paths(PlannerInfo *root,
static RelOptInfo *
make_grouping_rel(PlannerInfo *root, RelOptInfo *input_rel,
PathTarget *target, bool target_parallel_safe,
- Node *havingQual)
+ bool target_parallel_safe_ignoring_params, Node *havingQual)
{
RelOptInfo *grouped_rel;
@@ -3414,9 +3445,21 @@ make_grouping_rel(PlannerInfo *root, RelOptInfo *input_rel,
* can't be parallel-safe, either. Otherwise, it's parallel-safe if the
* target list and HAVING quals are parallel-safe.
*/
- if (input_rel->consider_parallel && target_parallel_safe &&
- is_parallel_safe(root, (Node *) havingQual))
- grouped_rel->consider_parallel = true;
+ if ((input_rel->consider_parallel || input_rel->consider_parallel_rechecking_params)
+ && (target_parallel_safe || target_parallel_safe_ignoring_params))
+ {
+ bool having_qual_parallel_safe;
+ bool having_qual_parallel_safe_ignoring_params = false;
+
+ having_qual_parallel_safe = is_parallel_safe(root, (Node *) havingQual,
+ &having_qual_parallel_safe_ignoring_params);
+
+ grouped_rel->consider_parallel = input_rel->consider_parallel &&
+ having_qual_parallel_safe && target_parallel_safe;
+ grouped_rel->consider_parallel_rechecking_params =
+ input_rel->consider_parallel_rechecking_params &&
+ having_qual_parallel_safe_ignoring_params && target_parallel_safe_ignoring_params;
+ }
/*
* If the input rel belongs to a single FDW, so does the grouped rel.
@@ -4043,7 +4086,7 @@ create_window_paths(PlannerInfo *root,
* target list and active windows for non-parallel-safe constructs.
*/
if (input_rel->consider_parallel && output_target_parallel_safe &&
- is_parallel_safe(root, (Node *) activeWindows))
+ is_parallel_safe(root, (Node *) activeWindows, NULL))
window_rel->consider_parallel = true;
/*
@@ -5602,6 +5645,7 @@ adjust_paths_for_srfs(PlannerInfo *root, RelOptInfo *rel,
PathTarget *thistarget = lfirst_node(PathTarget, lc1);
bool contains_srfs = (bool) lfirst_int(lc2);
+ /* TODO: How do we know the new target is parallel safe? */
/* If this level doesn't contain SRFs, do regular projection */
if (contains_srfs)
newpath = (Path *) create_set_projection_path(root,
@@ -5918,8 +5962,8 @@ plan_create_index_workers(Oid tableOid, Oid indexOid)
* safe.
*/
if (heap->rd_rel->relpersistence == RELPERSISTENCE_TEMP ||
- !is_parallel_safe(root, (Node *) RelationGetIndexExpressions(index)) ||
- !is_parallel_safe(root, (Node *) RelationGetIndexPredicate(index)))
+ !is_parallel_safe(root, (Node *) RelationGetIndexExpressions(index), NULL) ||
+ !is_parallel_safe(root, (Node *) RelationGetIndexPredicate(index), NULL))
{
parallel_workers = 0;
goto done;
@@ -6382,6 +6426,8 @@ create_partial_grouping_paths(PlannerInfo *root,
grouped_rel->relids);
partially_grouped_rel->consider_parallel =
grouped_rel->consider_parallel;
+ partially_grouped_rel->consider_parallel_rechecking_params =
+ grouped_rel->consider_parallel_rechecking_params;
partially_grouped_rel->reloptkind = grouped_rel->reloptkind;
partially_grouped_rel->serverid = grouped_rel->serverid;
partially_grouped_rel->userid = grouped_rel->userid;
@@ -6845,6 +6891,7 @@ apply_scanjoin_target_to_paths(PlannerInfo *root,
List *scanjoin_targets,
List *scanjoin_targets_contain_srfs,
bool scanjoin_target_parallel_safe,
+ bool scanjoin_target_parallel_safe_ignoring_params,
bool tlist_same_exprs)
{
bool rel_is_partitioned = IS_PARTITIONED_REL(rel);
@@ -6854,6 +6901,11 @@ apply_scanjoin_target_to_paths(PlannerInfo *root,
/* This recurses, so be paranoid. */
check_stack_depth();
+ /*
+ * TOOD: when/how do we want to generate gather paths if
+ * scanjoin_target_parallel_safe_ignoring_params = true
+ */
+
/*
* If the rel is partitioned, we want to drop its existing paths and
* generate new ones. This function would still be correct if we kept the
@@ -6894,6 +6946,64 @@ apply_scanjoin_target_to_paths(PlannerInfo *root,
generate_useful_gather_paths(root, rel, false);
/* Can't use parallel query above this level. */
+
+ /*
+ * There are cases where:
+ * (rel->consider_parallel &&
+ * !scanjoin_target_parallel_safe &&
+ * scanjoin_target_parallel_safe_ignoring_params)
+ * is true at this point. See longer commment below.
+ */
+ if (!(rel->consider_parallel_rechecking_params && scanjoin_target_parallel_safe_ignoring_params))
+ {
+ /*
+ * TODO: if we limit this to this condition, we're pushing off the
+ * checks as to whether or not a given param usage is safe in the
+ * context of a given path (in the context of a given rel?). That
+ * almost certainly means we'd have to add other checks later (maybe
+ * just on lateral/relids and not parallel safety overall), because
+ * at the end of grouping_planner() we copy partial paths to the
+ * final_rel, and while that path may be acceptable in some contexts
+ * it may not be in all contexts.
+ *
+ * OTOH if we're only dependent on PARAM_EXEC params, and we already
+ * know that subpath->param_info == NULL holds (and that seems like
+ * it must since we were going to replace the path target anyway...
+ * though the one caveat is from the original form of this function
+ * we'd only ever actually assert that for paths not partial paths)
+ * then if a param shows up in the target why would it not be parallel
+ * safe.
+ *
+ * Adding to the mystery even with the original form of this function
+ * we still end up with parallel paths where I'd expect this to
+ * disallow them. For example:
+ *
+ * SELECT '' AS six, f1 AS "Correlated Field", f3 AS "Second Field"
+ * FROM SUBSELECT_TBL upper
+ * WHERE f3 IN (
+ * SELECT upper.f1 + f2
+ * FROM SUBSELECT_TBL
+ * WHERE f2 = CAST(f3 AS integer)
+ * );
+ *
+ * ends up with the correlated query underneath parallel plan despite
+ * its target containing a param, and therefore this function marking
+ * the rel as consider_parallel=false and removing the partial paths.
+ *
+ * But the plan as a whole is parallel safe, and so the subplan is also
+ * parallel safe, which means we can incorporate it into a full parallel
+ * plan. In other words, this is a parallel safe, but not parallel aware
+ * subplan (and regular, not parallel, seq scan inside that subplan).
+ * It's not a partial path; it'a a full path that is executed as a subquery.
+ *
+ * Current conclusion: it's fine for subplans, which is the case we're
+ * currently targeting anyway. And it might even be the only case that
+ * matters at all.
+ */
+ /* rel->consider_parallel_rechecking_params = false; */
+ /* rel->partial_pathlist = NIL; */
+ }
+ rel->consider_parallel_rechecking_params = false;
rel->partial_pathlist = NIL;
rel->consider_parallel = false;
}
@@ -7026,6 +7136,7 @@ apply_scanjoin_target_to_paths(PlannerInfo *root,
child_scanjoin_targets,
scanjoin_targets_contain_srfs,
scanjoin_target_parallel_safe,
+ scanjoin_target_parallel_safe_ignoring_params,
tlist_same_exprs);
/* Save non-dummy children for Append paths. */
@@ -7043,6 +7154,11 @@ apply_scanjoin_target_to_paths(PlannerInfo *root,
* avoid creating multiple Gather nodes within the same plan. We must do
* this after all paths have been generated and before set_cheapest, since
* one of the generated paths may turn out to be the cheapest one.
+ *
+ * TODO: This is the same problem as earlier in this function: when allowing
+ * "parallel safe ignoring params" paths here we don't actually know we are
+ * safe in any possible context just possibly safe in the context of the
+ * right rel.
*/
if (rel->consider_parallel && !IS_OTHER_REL(rel))
generate_useful_gather_paths(root, rel, false);
@@ -7146,6 +7262,7 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
child_grouped_rel = make_grouping_rel(root, child_input_rel,
child_target,
extra->target_parallel_safe,
+ extra->target_parallel_safe_ignoring_params,
child_extra.havingQual);
/* Create grouping paths for this child relation. */
diff --git a/src/backend/optimizer/plan/subselect.c b/src/backend/optimizer/plan/subselect.c
index 0881a208ac..fd1cfe6c73 100644
--- a/src/backend/optimizer/plan/subselect.c
+++ b/src/backend/optimizer/plan/subselect.c
@@ -342,6 +342,7 @@ build_subplan(PlannerInfo *root, Plan *plan, PlannerInfo *subroot,
splan->useHashTable = false;
splan->unknownEqFalse = unknownEqFalse;
splan->parallel_safe = plan->parallel_safe;
+ splan->parallel_safe_ignoring_params = plan->parallel_safe_ignoring_params;
splan->setParam = NIL;
splan->parParam = NIL;
splan->args = NIL;
@@ -1939,6 +1940,7 @@ process_sublinks_mutator(Node *node, process_sublinks_context *context)
{
SubLink *sublink = (SubLink *) node;
Node *testexpr;
+ Node *result;
/*
* First, recursively process the lefthand-side expressions, if any.
@@ -1950,12 +1952,29 @@ process_sublinks_mutator(Node *node, process_sublinks_context *context)
/*
* Now build the SubPlan node and make the expr to return.
*/
- return make_subplan(context->root,
+ result = make_subplan(context->root,
(Query *) sublink->subselect,
sublink->subLinkType,
sublink->subLinkId,
testexpr,
context->isTopQual);
+
+ /*
+ * If planning determined that a subpath was parallel safe as long
+ * as required params are provided by each individual worker then we
+ * can mark the resulting subplan actually parallel safe since we now
+ * know for certain how that path will be used.
+ */
+ if (IsA(result, SubPlan) && !((SubPlan*)result)->parallel_safe
+ && ((SubPlan*)result)->parallel_safe_ignoring_params
+ && enable_parallel_params_recheck)
+ {
+ Plan *subplan = planner_subplan_get_plan(context->root, (SubPlan*)result);
+ ((SubPlan*)result)->parallel_safe = is_parallel_safe(context->root, testexpr, NULL);
+ subplan->parallel_safe = ((SubPlan*)result)->parallel_safe;
+ }
+
+ return result;
}
/*
diff --git a/src/backend/optimizer/prep/prepunion.c b/src/backend/optimizer/prep/prepunion.c
index 037dfaacfd..a57bc1c2a8 100644
--- a/src/backend/optimizer/prep/prepunion.c
+++ b/src/backend/optimizer/prep/prepunion.c
@@ -409,6 +409,7 @@ recurse_set_operations(Node *setOp, PlannerInfo *root,
Assert(subpath->param_info == NULL);
/* avoid apply_projection_to_path, in case of multiple refs */
+ /* TODO: how to we know the target is parallel safe? */
path = (Path *) create_projection_path(root, subpath->parent,
subpath, target);
lfirst(lc) = path;
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index d9ad4efc5e..4c188b85a4 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -54,6 +54,15 @@
#include "utils/syscache.h"
#include "utils/typcache.h"
+bool enable_parallel_params_recheck = true;
+
+typedef struct
+{
+ PlannerInfo *root;
+ AggSplit aggsplit;
+ AggClauseCosts *costs;
+} get_agg_clause_costs_context;
+
typedef struct
{
ParamListInfo boundParams;
@@ -87,6 +96,8 @@ typedef struct
{
char max_hazard; /* worst proparallel hazard found so far */
char max_interesting; /* worst proparallel hazard of interest */
+ char max_hazard_ignoring_params;
+ bool check_params_independently;
List *safe_param_ids; /* PARAM_EXEC Param IDs to treat as safe */
} max_parallel_hazard_context;
@@ -620,19 +631,27 @@ max_parallel_hazard(Query *parse)
context.max_hazard = PROPARALLEL_SAFE;
context.max_interesting = PROPARALLEL_UNSAFE;
context.safe_param_ids = NIL;
+ context.check_params_independently = false;
(void) max_parallel_hazard_walker((Node *) parse, &context);
return context.max_hazard;
}
/*
* is_parallel_safe
- * Detect whether the given expr contains only parallel-safe functions
+ * Detect whether the given expr contains only parallel safe funcions
+ * XXX: It does more than functions?
+ *
+ * If provided, safe_ignoring_params will be set the result of running the same
+ * parallel safety checks with the exception that params will be allowed. This
+ * value is useful since params are not inherently parallel unsafe, but rather
+ * their usage context (whether or not the worker is able to provide the value)
+ * determines parallel safety.
*
* root->glob->maxParallelHazard must previously have been set to the
- * result of max_parallel_hazard() on the whole query.
+ * result of max_parallel_hazard() on the whole query
*/
bool
-is_parallel_safe(PlannerInfo *root, Node *node)
+is_parallel_safe(PlannerInfo *root, Node *node, bool *safe_ignoring_params)
{
max_parallel_hazard_context context;
PlannerInfo *proot;
@@ -649,8 +668,10 @@ is_parallel_safe(PlannerInfo *root, Node *node)
return true;
/* Else use max_parallel_hazard's search logic, but stop on RESTRICTED */
context.max_hazard = PROPARALLEL_SAFE;
+ context.max_hazard_ignoring_params = PROPARALLEL_SAFE;
context.max_interesting = PROPARALLEL_RESTRICTED;
context.safe_param_ids = NIL;
+ context.check_params_independently = safe_ignoring_params != NULL;
/*
* The params that refer to the same or parent query level are considered
@@ -668,12 +689,17 @@ is_parallel_safe(PlannerInfo *root, Node *node)
}
}
- return !max_parallel_hazard_walker(node, &context);
+ (void) max_parallel_hazard_walker(node, &context);
+
+ if (safe_ignoring_params)
+ *safe_ignoring_params = context.max_hazard_ignoring_params == PROPARALLEL_SAFE;
+
+ return context.max_hazard == PROPARALLEL_SAFE;
}
/* core logic for all parallel-hazard checks */
static bool
-max_parallel_hazard_test(char proparallel, max_parallel_hazard_context *context)
+max_parallel_hazard_test(char proparallel, max_parallel_hazard_context *context, bool from_param)
{
switch (proparallel)
{
@@ -684,12 +710,16 @@ max_parallel_hazard_test(char proparallel, max_parallel_hazard_context *context)
/* increase max_hazard to RESTRICTED */
Assert(context->max_hazard != PROPARALLEL_UNSAFE);
context->max_hazard = proparallel;
+ if (!from_param)
+ context->max_hazard_ignoring_params = proparallel;
/* done if we are not expecting any unsafe functions */
if (context->max_interesting == proparallel)
return true;
break;
case PROPARALLEL_UNSAFE:
context->max_hazard = proparallel;
+ if (!from_param)
+ context->max_hazard_ignoring_params = proparallel;
/* we're always done at the first unsafe construct */
return true;
default:
@@ -704,7 +734,41 @@ static bool
max_parallel_hazard_checker(Oid func_id, void *context)
{
return max_parallel_hazard_test(func_parallel(func_id),
- (max_parallel_hazard_context *) context);
+ (max_parallel_hazard_context *) context, false);
+}
+
+static bool
+max_parallel_hazard_walker_can_short_circuit(max_parallel_hazard_context *context)
+{
+ if (!context->check_params_independently)
+ return true;
+
+ switch (context->max_hazard)
+ {
+ case PROPARALLEL_SAFE:
+ /* nothing to see here, move along */
+ break;
+ case PROPARALLEL_RESTRICTED:
+ if (context->max_interesting == PROPARALLEL_RESTRICTED)
+ return context->max_hazard_ignoring_params != PROPARALLEL_SAFE;
+
+ /*
+ * We haven't even met our max interesting yet, so
+ * we certainly can't short-circuit.
+ */
+ break;
+ case PROPARALLEL_UNSAFE:
+ if (context->max_interesting == PROPARALLEL_RESTRICTED)
+ return context->max_hazard_ignoring_params != PROPARALLEL_SAFE;
+ else if (context->max_interesting == PROPARALLEL_UNSAFE)
+ return context->max_hazard_ignoring_params == PROPARALLEL_UNSAFE;
+
+ break;
+ default:
+ elog(ERROR, "unrecognized proparallel value \"%c\"", context->max_hazard);
+ break;
+ }
+ return false;
}
static bool
@@ -716,7 +780,7 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
/* Check for hazardous functions in node itself */
if (check_functions_in_node(node, max_parallel_hazard_checker,
context))
- return true;
+ return max_parallel_hazard_walker_can_short_circuit(context);
/*
* It should be OK to treat MinMaxExpr as parallel-safe, since btree
@@ -731,14 +795,14 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
*/
if (IsA(node, CoerceToDomain))
{
- if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
- return true;
+ if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context, false))
+ return max_parallel_hazard_walker_can_short_circuit(context);
}
else if (IsA(node, NextValueExpr))
{
- if (max_parallel_hazard_test(PROPARALLEL_UNSAFE, context))
- return true;
+ if (max_parallel_hazard_test(PROPARALLEL_UNSAFE, context, false))
+ return max_parallel_hazard_walker_can_short_circuit(context);
}
/*
@@ -751,8 +815,8 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
*/
else if (IsA(node, WindowFunc))
{
- if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
- return true;
+ if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context, false))
+ return max_parallel_hazard_walker_can_short_circuit(context);
}
/*
@@ -771,8 +835,8 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
*/
else if (IsA(node, SubLink))
{
- if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
- return true;
+ if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context, false))
+ return max_parallel_hazard_walker_can_short_circuit(context);
}
/*
@@ -787,18 +851,23 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
List *save_safe_param_ids;
if (!subplan->parallel_safe &&
- max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
+ (!enable_parallel_params_recheck || !subplan->parallel_safe_ignoring_params) &&
+ max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context, false) &&
+ max_parallel_hazard_walker_can_short_circuit(context))
return true;
save_safe_param_ids = context->safe_param_ids;
context->safe_param_ids = list_concat_copy(context->safe_param_ids,
subplan->paramIds);
- if (max_parallel_hazard_walker(subplan->testexpr, context))
- return true; /* no need to restore safe_param_ids */
+ if (max_parallel_hazard_walker(subplan->testexpr, context) &&
+ max_parallel_hazard_walker_can_short_circuit(context))
+ /* no need to restore safe_param_ids */
+ return true;
+
list_free(context->safe_param_ids);
context->safe_param_ids = save_safe_param_ids;
/* we must also check args, but no special Param treatment there */
if (max_parallel_hazard_walker((Node *) subplan->args, context))
- return true;
+ return max_parallel_hazard_walker_can_short_circuit(context);
/* don't want to recurse normally, so we're done */
return false;
}
@@ -820,8 +889,8 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
if (param->paramkind != PARAM_EXEC ||
!list_member_int(context->safe_param_ids, param->paramid))
{
- if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
- return true;
+ if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context, true))
+ return max_parallel_hazard_walker_can_short_circuit(context);
}
return false; /* nothing to recurse to */
}
@@ -839,7 +908,7 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
if (query->rowMarks != NULL)
{
context->max_hazard = PROPARALLEL_UNSAFE;
- return true;
+ return max_parallel_hazard_walker_can_short_circuit(context);
}
/* Recurse into subselects */
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index b248b038e0..93956f3828 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -756,10 +756,10 @@ add_partial_path(RelOptInfo *parent_rel, Path *new_path)
CHECK_FOR_INTERRUPTS();
/* Path to be added must be parallel safe. */
- Assert(new_path->parallel_safe);
+ Assert(new_path->parallel_safe || new_path->parallel_safe_ignoring_params);
/* Relation should be OK for parallelism, too. */
- Assert(parent_rel->consider_parallel);
+ Assert(parent_rel->consider_parallel || parent_rel->consider_parallel_rechecking_params);
/*
* As in add_path, throw out any paths which are dominated by the new
@@ -938,6 +938,7 @@ create_seqscan_path(PlannerInfo *root, RelOptInfo *rel,
required_outer);
pathnode->parallel_aware = parallel_workers > 0 ? true : false;
pathnode->parallel_safe = rel->consider_parallel;
+ pathnode->parallel_safe_ignoring_params = rel->consider_parallel_rechecking_params;
pathnode->parallel_workers = parallel_workers;
pathnode->pathkeys = NIL; /* seqscan has unordered result */
@@ -1016,6 +1017,7 @@ create_index_path(PlannerInfo *root,
required_outer);
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel;
+ pathnode->path.parallel_safe_ignoring_params = rel->consider_parallel_rechecking_params;
pathnode->path.parallel_workers = 0;
pathnode->path.pathkeys = pathkeys;
@@ -1868,7 +1870,7 @@ create_gather_merge_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
Cost input_startup_cost = 0;
Cost input_total_cost = 0;
- Assert(subpath->parallel_safe);
+ Assert(subpath->parallel_safe || subpath->parallel_safe_ignoring_params);
Assert(pathkeys);
pathnode->path.pathtype = T_GatherMerge;
@@ -1956,7 +1958,7 @@ create_gather_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
{
GatherPath *pathnode = makeNode(GatherPath);
- Assert(subpath->parallel_safe);
+ Assert(subpath->parallel_safe || subpath->parallel_safe_ignoring_params);
pathnode->path.pathtype = T_Gather;
pathnode->path.parent = rel;
@@ -2003,6 +2005,8 @@ create_subqueryscan_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe;
+ pathnode->path.parallel_safe_ignoring_params = rel->consider_parallel_rechecking_params &&
+ subpath->parallel_safe_ignoring_params;
pathnode->path.parallel_workers = subpath->parallel_workers;
pathnode->path.pathkeys = pathkeys;
pathnode->subpath = subpath;
@@ -2420,6 +2424,8 @@ create_nestloop_path(PlannerInfo *root,
NestPath *pathnode = makeNode(NestPath);
Relids inner_req_outer = PATH_REQ_OUTER(inner_path);
+ /* TODO: Assert lateral relids subset safety? */
+
/*
* If the inner path is parameterized by the outer, we must drop any
* restrict_clauses that are due to be moved into the inner path. We have
@@ -2460,6 +2466,8 @@ create_nestloop_path(PlannerInfo *root,
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = joinrel->consider_parallel &&
outer_path->parallel_safe && inner_path->parallel_safe;
+ pathnode->path.parallel_safe_ignoring_params = joinrel->consider_parallel_rechecking_params &&
+ outer_path->parallel_safe_ignoring_params && inner_path->parallel_safe_ignoring_params;
/* This is a foolish way to estimate parallel_workers, but for now... */
pathnode->path.parallel_workers = outer_path->parallel_workers;
pathnode->path.pathkeys = pathkeys;
@@ -2633,6 +2641,8 @@ create_projection_path(PlannerInfo *root,
{
ProjectionPath *pathnode = makeNode(ProjectionPath);
PathTarget *oldtarget = subpath->pathtarget;
+ bool target_parallel_safe;
+ bool target_parallel_safe_ignoring_params = false;
pathnode->path.pathtype = T_Result;
pathnode->path.parent = rel;
@@ -2640,9 +2650,12 @@ create_projection_path(PlannerInfo *root,
/* For now, assume we are above any joins, so no parameterization */
pathnode->path.param_info = NULL;
pathnode->path.parallel_aware = false;
+ target_parallel_safe = is_parallel_safe(root, (Node *) target->exprs,
+ &target_parallel_safe_ignoring_params);
pathnode->path.parallel_safe = rel->consider_parallel &&
- subpath->parallel_safe &&
- is_parallel_safe(root, (Node *) target->exprs);
+ subpath->parallel_safe && target_parallel_safe;
+ pathnode->path.parallel_safe_ignoring_params = rel->consider_parallel_rechecking_params &&
+ subpath->parallel_safe_ignoring_params && target_parallel_safe_ignoring_params;
pathnode->path.parallel_workers = subpath->parallel_workers;
/* Projection does not change the sort order */
pathnode->path.pathkeys = subpath->pathkeys;
@@ -2749,7 +2762,7 @@ apply_projection_to_path(PlannerInfo *root,
* parallel-safe in the target expressions, then we can't.
*/
if ((IsA(path, GatherPath) || IsA(path, GatherMergePath)) &&
- is_parallel_safe(root, (Node *) target->exprs))
+ is_parallel_safe(root, (Node *) target->exprs, NULL))
{
/*
* We always use create_projection_path here, even if the subpath is
@@ -2783,7 +2796,7 @@ apply_projection_to_path(PlannerInfo *root,
}
}
else if (path->parallel_safe &&
- !is_parallel_safe(root, (Node *) target->exprs))
+ !is_parallel_safe(root, (Node *) target->exprs, NULL))
{
/*
* We're inserting a parallel-restricted target list into a path
@@ -2791,6 +2804,7 @@ apply_projection_to_path(PlannerInfo *root,
* safe.
*/
path->parallel_safe = false;
+ path->parallel_safe_ignoring_params = false; /* TODO */
}
return path;
@@ -2823,7 +2837,7 @@ create_set_projection_path(PlannerInfo *root,
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe &&
- is_parallel_safe(root, (Node *) target->exprs);
+ is_parallel_safe(root, (Node *) target->exprs, NULL);
pathnode->path.parallel_workers = subpath->parallel_workers;
/* Projection does not change the sort order XXX? */
pathnode->path.pathkeys = subpath->pathkeys;
@@ -3100,6 +3114,8 @@ create_agg_path(PlannerInfo *root,
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe;
+ pathnode->path.parallel_safe_ignoring_params = rel->consider_parallel_rechecking_params &&
+ subpath->parallel_safe_ignoring_params;
pathnode->path.parallel_workers = subpath->parallel_workers;
if (aggstrategy == AGG_SORTED)
pathnode->path.pathkeys = subpath->pathkeys; /* preserves order */
@@ -3721,6 +3737,8 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe;
+ pathnode->path.parallel_safe_ignoring_params = rel->consider_parallel_rechecking_params &&
+ subpath->parallel_safe_ignoring_params;
pathnode->path.parallel_workers = subpath->parallel_workers;
pathnode->path.rows = subpath->rows;
pathnode->path.startup_cost = subpath->startup_cost;
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index e105a4d5f1..d026db8759 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -213,6 +213,7 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
rel->consider_startup = (root->tuple_fraction > 0);
rel->consider_param_startup = false; /* might get changed later */
rel->consider_parallel = false; /* might get changed later */
+ rel->consider_parallel_rechecking_params = false; /* might get changed later */
rel->reltarget = create_empty_pathtarget();
rel->pathlist = NIL;
rel->ppilist = NIL;
@@ -616,6 +617,7 @@ build_join_rel(PlannerInfo *root,
joinrel->consider_startup = (root->tuple_fraction > 0);
joinrel->consider_param_startup = false;
joinrel->consider_parallel = false;
+ joinrel->consider_parallel_rechecking_params = false;
joinrel->reltarget = create_empty_pathtarget();
joinrel->pathlist = NIL;
joinrel->ppilist = NIL;
@@ -741,10 +743,27 @@ build_join_rel(PlannerInfo *root,
* take; therefore, we should make the same decision here however we get
* here.
*/
- if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
- is_parallel_safe(root, (Node *) restrictlist) &&
- is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
- joinrel->consider_parallel = true;
+ if ((inner_rel->consider_parallel || inner_rel->consider_parallel_rechecking_params)
+ && (outer_rel->consider_parallel || outer_rel->consider_parallel_rechecking_params))
+ {
+ bool restrictlist_parallel_safe;
+ bool restrictlist_parallel_safe_ignoring_params = false;
+ bool target_parallel_safe;
+ bool target_parallel_safe_ignoring_params = false;
+
+ restrictlist_parallel_safe = is_parallel_safe(root, (Node *) restrictlist, &restrictlist_parallel_safe_ignoring_params);
+ target_parallel_safe = is_parallel_safe(root, (Node *) joinrel->reltarget->exprs, &target_parallel_safe_ignoring_params);
+
+ if (inner_rel->consider_parallel && outer_rel->consider_parallel
+ && restrictlist_parallel_safe && target_parallel_safe)
+ joinrel->consider_parallel = true;
+
+ if (inner_rel->consider_parallel_rechecking_params
+ && outer_rel->consider_parallel_rechecking_params
+ && restrictlist_parallel_safe_ignoring_params
+ && target_parallel_safe_ignoring_params)
+ joinrel->consider_parallel_rechecking_params = true;
+ }
/* Add the joinrel to the PlannerInfo. */
add_join_rel(root, joinrel);
@@ -803,6 +822,7 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->consider_startup = (root->tuple_fraction > 0);
joinrel->consider_param_startup = false;
joinrel->consider_parallel = false;
+ joinrel->consider_parallel_rechecking_params = false;
joinrel->reltarget = create_empty_pathtarget();
joinrel->pathlist = NIL;
joinrel->ppilist = NIL;
@@ -889,6 +909,7 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
/* Child joinrel is parallel safe if parent is parallel safe. */
joinrel->consider_parallel = parent_joinrel->consider_parallel;
+ joinrel->consider_parallel_rechecking_params = parent_joinrel->consider_parallel_rechecking_params;
/* Set estimates of the child-joinrel's size. */
set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
@@ -1233,6 +1254,7 @@ fetch_upper_rel(PlannerInfo *root, UpperRelationKind kind, Relids relids)
upperrel->consider_startup = (root->tuple_fraction > 0);
upperrel->consider_param_startup = false;
upperrel->consider_parallel = false; /* might get changed later */
+ upperrel->consider_parallel_rechecking_params = false; /* might get changed later */
upperrel->reltarget = create_empty_pathtarget();
upperrel->pathlist = NIL;
upperrel->cheapest_startup_path = NULL;
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index b130874bdc..1fbab55e2c 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -58,6 +58,7 @@
#include "libpq/libpq.h"
#include "libpq/pqformat.h"
#include "miscadmin.h"
+#include "optimizer/clauses.h"
#include "optimizer/cost.h"
#include "optimizer/geqo.h"
#include "optimizer/optimizer.h"
@@ -955,6 +956,16 @@ static const unit_conversion time_unit_conversion_table[] =
static struct config_bool ConfigureNamesBool[] =
{
+ {
+ {"enable_parallel_params_recheck", PGC_USERSET, QUERY_TUNING_METHOD,
+ gettext_noop("Enables the planner's rechecking of parallel safety in the presence of PARAM_EXEC params (for correlated subqueries)."),
+ NULL,
+ GUC_EXPLAIN
+ },
+ &enable_parallel_params_recheck,
+ true,
+ NULL, NULL, NULL
+ },
{
{"enable_seqscan", PGC_USERSET, QUERY_TUNING_METHOD,
gettext_noop("Enables the planner's use of sequential-scan plans."),
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index a65bda7e3c..2ecbb016a2 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -682,6 +682,7 @@ typedef struct RelOptInfo
bool consider_startup; /* keep cheap-startup-cost paths? */
bool consider_param_startup; /* ditto, for parameterized paths? */
bool consider_parallel; /* consider parallel paths? */
+ bool consider_parallel_rechecking_params; /* consider parallel paths? */
/* default result targetlist for Paths scanning this relation */
struct PathTarget *reltarget; /* list of Vars/Exprs, cost, width */
@@ -1178,6 +1179,7 @@ typedef struct Path
bool parallel_aware; /* engage parallel-aware logic? */
bool parallel_safe; /* OK to use as part of parallel plan? */
+ bool parallel_safe_ignoring_params; /* OK to use as part of parallel plan if worker context provides params? */
int parallel_workers; /* desired # of workers; 0 = not parallel */
/* estimated size/costs for path (see costsize.c for more info) */
@@ -2464,7 +2466,7 @@ typedef struct MinMaxAggInfo
* for conflicting purposes.
*
* In addition, PARAM_EXEC slots are assigned for Params representing outputs
- * from subplans (values that are setParam items for those subplans). These
+ * from subplans (values that are setParam items for those subplans). [TODO: is this true, or only for init plans?] These
* IDs need not be tracked via PlannerParamItems, since we do not need any
* duplicate-elimination nor later processing of the represented expressions.
* Instead, we just record the assignment of the slot number by appending to
@@ -2583,6 +2585,7 @@ typedef struct
/* Data which may differ across partitions. */
bool target_parallel_safe;
+ bool target_parallel_safe_ignoring_params;
Node *havingQual;
List *targetList;
PartitionwiseAggregateType patype;
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index d671328dfd..b36def462b 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -128,6 +128,7 @@ typedef struct Plan
*/
bool parallel_aware; /* engage parallel-aware logic? */
bool parallel_safe; /* OK to use as part of parallel plan? */
+ bool parallel_safe_ignoring_params; /* OK to use as part of parallel plan if worker context provides params? */
/*
* information needed for asynchronous execution
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
index 9ae851d847..279a8e106b 100644
--- a/src/include/nodes/primnodes.h
+++ b/src/include/nodes/primnodes.h
@@ -754,6 +754,7 @@ typedef struct SubPlan
* spec result is UNKNOWN; this allows much
* simpler handling of null values */
bool parallel_safe; /* is the subplan parallel-safe? */
+ bool parallel_safe_ignoring_params; /* is the subplan parallel-safe when params are provided by the worker context? */
/* Note: parallel_safe does not consider contents of testexpr or args */
/* Information for passing params into and out of the subselect: */
/* setParam and parParam are lists of integers (param IDs) */
diff --git a/src/include/optimizer/clauses.h b/src/include/optimizer/clauses.h
index 0673887a85..df01be2c61 100644
--- a/src/include/optimizer/clauses.h
+++ b/src/include/optimizer/clauses.h
@@ -16,6 +16,8 @@
#include "nodes/pathnodes.h"
+extern PGDLLIMPORT bool enable_parallel_params_recheck;
+
typedef struct
{
int numWindowFuncs; /* total number of WindowFuncs found */
@@ -33,7 +35,7 @@ extern double expression_returns_set_rows(PlannerInfo *root, Node *clause);
extern bool contain_subplans(Node *clause);
extern char max_parallel_hazard(Query *parse);
-extern bool is_parallel_safe(PlannerInfo *root, Node *node);
+extern bool is_parallel_safe(PlannerInfo *root, Node *node, bool *safe_ignoring_params);
extern bool contain_nonstrict_functions(Node *clause);
extern bool contain_exec_param(Node *clause, List *param_ids);
extern bool contain_leaked_vars(Node *clause);
diff --git a/src/test/regress/expected/incremental_sort.out b/src/test/regress/expected/incremental_sort.out
index 545e301e48..8f9ca05e60 100644
--- a/src/test/regress/expected/incremental_sort.out
+++ b/src/test/regress/expected/incremental_sort.out
@@ -1614,16 +1614,16 @@ from tenk1 t, generate_series(1, 1000);
QUERY PLAN
---------------------------------------------------------------------------------
Unique
- -> Sort
- Sort Key: t.unique1, ((SubPlan 1))
- -> Gather
- Workers Planned: 2
+ -> Gather Merge
+ Workers Planned: 2
+ -> Sort
+ Sort Key: t.unique1, ((SubPlan 1))
-> Nested Loop
-> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
-> Function Scan on generate_series
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on tenk1
- Index Cond: (unique1 = t.unique1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on tenk1
+ Index Cond: (unique1 = t.unique1)
(11 rows)
explain (costs off) select
@@ -1633,16 +1633,16 @@ from tenk1 t, generate_series(1, 1000)
order by 1, 2;
QUERY PLAN
---------------------------------------------------------------------------
- Sort
- Sort Key: t.unique1, ((SubPlan 1))
- -> Gather
- Workers Planned: 2
+ Gather Merge
+ Workers Planned: 2
+ -> Sort
+ Sort Key: t.unique1, ((SubPlan 1))
-> Nested Loop
-> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
-> Function Scan on generate_series
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on tenk1
- Index Cond: (unique1 = t.unique1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on tenk1
+ Index Cond: (unique1 = t.unique1)
(10 rows)
-- Parallel sort but with expression not available until the upper rel.
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 224a1da2af..e6f2453a1f 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -311,6 +311,131 @@ select count(*) from tenk1 where (two, four) not in
10000
(1 row)
+-- test parallel plans for queries containing correlated subplans
+-- where the subplan only needs params available from the current
+-- worker's scan.
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t, generate_series(1, 10);
+ QUERY PLAN
+----------------------------------------------------------------------------
+ Gather
+ Output: ((SubPlan 1))
+ Workers Planned: 4
+ -> Nested Loop
+ Output: (SubPlan 1)
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ -> Function Scan on pg_catalog.generate_series
+ Output: generate_series.generate_series
+ Function Call: generate_series(1, 10)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(14 rows)
+
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t;
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: ((SubPlan 1))
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(9 rows)
+
+explain (analyze, costs off, summary off, verbose, timing off) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t
+ limit 1;
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------
+ Limit (actual rows=1 loops=1)
+ Output: ((SubPlan 1))
+ -> Gather (actual rows=1 loops=1)
+ Output: ((SubPlan 1))
+ Workers Planned: 4
+ Workers Launched: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t (actual rows=1 loops=5)
+ Output: (SubPlan 1)
+ Heap Fetches: 0
+ Worker 0: actual rows=1 loops=1
+ Worker 1: actual rows=1 loops=1
+ Worker 2: actual rows=1 loops=1
+ Worker 3: actual rows=1 loops=1
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1 (actual rows=1 loops=5)
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+ Heap Fetches: 0
+ Worker 0: actual rows=1 loops=1
+ Worker 1: actual rows=1 loops=1
+ Worker 2: actual rows=1 loops=1
+ Worker 3: actual rows=1 loops=1
+(22 rows)
+
+explain (costs off, verbose) select t.unique1
+ from tenk1 t
+ where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: t.unique1
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ Filter: (t.unique1 = (SubPlan 1))
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(10 rows)
+
+explain (costs off, verbose) select *
+ from tenk1 t
+ order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Gather Merge
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
+ Workers Planned: 4
+ -> Sort
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
+ Sort Key: ((SubPlan 1))
+ -> Parallel Seq Scan on public.tenk1 t
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(12 rows)
+
+-- test subplan in join/lateral join
+explain (costs off, verbose, timing off) select t.unique1, l.*
+ from tenk1 t
+ join lateral (
+ select (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1 offset 0)
+ ) l on true;
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: t.unique1, ((SubPlan 1))
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1, (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(9 rows)
+
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
explain (costs off)
select * from tenk1 where (unique1 + random())::integer not in
@@ -1169,6 +1294,18 @@ EXECUTE pstmt('1', make_some_array(1,2));
DEALLOCATE pstmt;
-- test interaction between subquery and partial_paths
+-- this plan changes to using a non-parallel index only
+-- scan on tenk1_unique1 (the parallel version of the subquery scan
+-- is cheaper, but only by ~30, and cost comparison treats them as equal
+-- since the costs are so large) because set_rel_consider_parallel
+-- called from make_one_rel sees the subplan as parallel safe now
+-- (in context it now knows the params are actually parallel safe).
+-- Because of that the non-parallel index path is now parallel_safe=true,
+-- therefore it wins the COSTS_EQUAL comparison in add_path.
+-- Perhaps any is_parallel_safe calls made for the purpose of determining
+-- consider_parallel should disable that behavior? It's not clear which is
+-- correct.
+set enable_parallel_params_recheck = off;
CREATE VIEW tenk1_vw_sec WITH (security_barrier) AS SELECT * FROM tenk1;
EXPLAIN (COSTS OFF)
SELECT 1 FROM tenk1_vw_sec
diff --git a/src/test/regress/expected/sysviews.out b/src/test/regress/expected/sysviews.out
index 0bb558d93c..22fee51455 100644
--- a/src/test/regress/expected/sysviews.out
+++ b/src/test/regress/expected/sysviews.out
@@ -108,6 +108,7 @@ select name, setting from pg_settings where name like 'enable%';
enable_nestloop | on
enable_parallel_append | on
enable_parallel_hash | on
+ enable_parallel_params_recheck | on
enable_partition_pruning | on
enable_partitionwise_aggregate | off
enable_partitionwise_join | off
@@ -115,7 +116,7 @@ select name, setting from pg_settings where name like 'enable%';
enable_seqscan | on
enable_sort | on
enable_tidscan | on
-(20 rows)
+(21 rows)
-- Test that the pg_timezone_names and pg_timezone_abbrevs views are
-- more-or-less working. We can't test their contents in any great detail
diff --git a/src/test/regress/sql/select_parallel.sql b/src/test/regress/sql/select_parallel.sql
index f656764fa9..55a3bd9025 100644
--- a/src/test/regress/sql/select_parallel.sql
+++ b/src/test/regress/sql/select_parallel.sql
@@ -111,6 +111,31 @@ explain (costs off)
(select hundred, thousand from tenk2 where thousand > 100);
select count(*) from tenk1 where (two, four) not in
(select hundred, thousand from tenk2 where thousand > 100);
+-- test parallel plans for queries containing correlated subplans
+-- where the subplan only needs params available from the current
+-- worker's scan.
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t, generate_series(1, 10);
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t;
+explain (analyze, costs off, summary off, verbose, timing off) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t
+ limit 1;
+explain (costs off, verbose) select t.unique1
+ from tenk1 t
+ where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+explain (costs off, verbose) select *
+ from tenk1 t
+ order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+-- test subplan in join/lateral join
+explain (costs off, verbose, timing off) select t.unique1, l.*
+ from tenk1 t
+ join lateral (
+ select (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1 offset 0)
+ ) l on true;
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
explain (costs off)
select * from tenk1 where (unique1 + random())::integer not in
@@ -452,6 +477,18 @@ EXECUTE pstmt('1', make_some_array(1,2));
DEALLOCATE pstmt;
-- test interaction between subquery and partial_paths
+-- this plan changes to using a non-parallel index only
+-- scan on tenk1_unique1 (the parallel version of the subquery scan
+-- is cheaper, but only by ~30, and cost comparison treats them as equal
+-- since the costs are so large) because set_rel_consider_parallel
+-- called from make_one_rel sees the subplan as parallel safe now
+-- (in context it now knows the params are actually parallel safe).
+-- Because of that the non-parallel index path is now parallel_safe=true,
+-- therefore it wins the COSTS_EQUAL comparison in add_path.
+-- Perhaps any is_parallel_safe calls made for the purpose of determining
+-- consider_parallel should disable that behavior? It's not clear which is
+-- correct.
+set enable_parallel_params_recheck = off;
CREATE VIEW tenk1_vw_sec WITH (security_barrier) AS SELECT * FROM tenk1;
EXPLAIN (COSTS OFF)
SELECT 1 FROM tenk1_vw_sec
--
2.20.1
v1-0001-Allow-parallel-LATERAL-subqueries-with-LIMIT-OFFS.patchapplication/octet-stream; name=v1-0001-Allow-parallel-LATERAL-subqueries-with-LIMIT-OFFS.patchDownload
From 4f51cf83caaffcbe2500898d7d939c885c0c352c Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Mon, 30 Nov 2020 11:36:35 -0500
Subject: [PATCH v1 1/3] Allow parallel LATERAL subqueries with LIMIT/OFFSET
The code that determined whether or not a rel should be considered for
parallel query excluded subqueries with LIMIT/OFFSET. That's correct in
the general case: as the comment notes that'd mean we have to guarantee
ordering (and claims it's not worth checking that) for results to be
consistent across workers. However there's a simpler case that hasn't
been considered: LATERAL subqueries with LIMIT/OFFSET don't fall under
the same reasoning since they're executed (when not converted to a JOIN)
per tuple anyway, so consistency of results across workers isn't a
factor.
---
src/backend/optimizer/path/allpaths.c | 4 +++-
src/test/regress/expected/select_parallel.out | 15 +++++++++++++++
src/test/regress/sql/select_parallel.sql | 6 ++++++
3 files changed, 24 insertions(+), 1 deletion(-)
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 353454b183..45e37590d8 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -682,11 +682,13 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
* inconsistent results at the top-level. (In some cases, where
* the result is ordered, we could relax this restriction. But it
* doesn't currently seem worth expending extra effort to do so.)
+ * LATERAL is an exception: LIMIT/OFFSET is safe to execute within
+ * workers since the sub-select is executed per tuple
*/
{
Query *subquery = castNode(Query, rte->subquery);
- if (limit_needed(subquery))
+ if (!rte->lateral && limit_needed(subquery))
return;
}
break;
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 05ebcb284a..224a1da2af 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -1040,6 +1040,21 @@ explain (costs off)
Filter: (stringu1 ~~ '%AAAA'::text)
(11 rows)
+-- ...unless it's LATERAL
+savepoint settings;
+set parallel_tuple_cost=0;
+explain (costs off) select t.unique1 from tenk1 t
+join lateral (select t.unique1 from tenk1 offset 0) l on true;
+ QUERY PLAN
+---------------------------------------------------------------------
+ Gather
+ Workers Planned: 4
+ -> Nested Loop
+ -> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
+ -> Index Only Scan using tenk1_hundred on tenk1
+(5 rows)
+
+rollback to savepoint settings;
-- to increase the parallel query test coverage
SAVEPOINT settings;
SET LOCAL force_parallel_mode = 1;
diff --git a/src/test/regress/sql/select_parallel.sql b/src/test/regress/sql/select_parallel.sql
index d31e290ec2..f656764fa9 100644
--- a/src/test/regress/sql/select_parallel.sql
+++ b/src/test/regress/sql/select_parallel.sql
@@ -390,6 +390,12 @@ explain (costs off, verbose)
explain (costs off)
select * from tenk1 a where two in
(select two from tenk1 b where stringu1 like '%AAAA' limit 3);
+-- ...unless it's LATERAL
+savepoint settings;
+set parallel_tuple_cost=0;
+explain (costs off) select t.unique1 from tenk1 t
+join lateral (select t.unique1 from tenk1 offset 0) l on true;
+rollback to savepoint settings;
-- to increase the parallel query test coverage
SAVEPOINT settings;
--
2.20.1
Re: Parallelize correlated subqueries that execute within each worker
On 7 May 2021, at 18:30, James Coleman <jtc331@gmail.com> wrote:
..here we are now, and I finally have this patch cleaned up
enough to share.
This patch no longer applies to HEAD, can you please submit a rebased version?
--
Daniel Gustafsson https://vmware.com/
Re: Parallelize correlated subqueries that execute within each worker
On Wed, Sep 1, 2021 at 7:06 AM Daniel Gustafsson <daniel@yesql.se> wrote:
On 7 May 2021, at 18:30, James Coleman <jtc331@gmail.com> wrote:
..here we are now, and I finally have this patch cleaned up
enough to share.This patch no longer applies to HEAD, can you please submit a rebased version?
See attached.
Thanks,
James
Attachments:
v2-0001-Allow-parallel-LATERAL-subqueries-with-LIMIT-OFFS.patchapplication/octet-stream; name=v2-0001-Allow-parallel-LATERAL-subqueries-with-LIMIT-OFFS.patchDownload
From 7ba602847a13fbf7c87aab3651b4304ffba5e9e6 Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Mon, 30 Nov 2020 11:36:35 -0500
Subject: [PATCH v2 1/3] Allow parallel LATERAL subqueries with LIMIT/OFFSET
The code that determined whether or not a rel should be considered for
parallel query excluded subqueries with LIMIT/OFFSET. That's correct in
the general case: as the comment notes that'd mean we have to guarantee
ordering (and claims it's not worth checking that) for results to be
consistent across workers. However there's a simpler case that hasn't
been considered: LATERAL subqueries with LIMIT/OFFSET don't fall under
the same reasoning since they're executed (when not converted to a JOIN)
per tuple anyway, so consistency of results across workers isn't a
factor.
---
src/backend/optimizer/path/allpaths.c | 4 +++-
src/test/regress/expected/select_parallel.out | 15 +++++++++++++++
src/test/regress/sql/select_parallel.sql | 6 ++++++
3 files changed, 24 insertions(+), 1 deletion(-)
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 296dd75c1b..1363f1bc6c 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -682,11 +682,13 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
* inconsistent results at the top-level. (In some cases, where
* the result is ordered, we could relax this restriction. But it
* doesn't currently seem worth expending extra effort to do so.)
+ * LATERAL is an exception: LIMIT/OFFSET is safe to execute within
+ * workers since the sub-select is executed per tuple
*/
{
Query *subquery = castNode(Query, rte->subquery);
- if (limit_needed(subquery))
+ if (!rte->lateral && limit_needed(subquery))
return;
}
break;
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 4ea1aa7dfd..2303f70d6e 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -1040,6 +1040,21 @@ explain (costs off)
Filter: (stringu1 ~~ '%AAAA'::text)
(11 rows)
+-- ...unless it's LATERAL
+savepoint settings;
+set parallel_tuple_cost=0;
+explain (costs off) select t.unique1 from tenk1 t
+join lateral (select t.unique1 from tenk1 offset 0) l on true;
+ QUERY PLAN
+---------------------------------------------------------------------
+ Gather
+ Workers Planned: 4
+ -> Nested Loop
+ -> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
+ -> Index Only Scan using tenk1_hundred on tenk1
+(5 rows)
+
+rollback to savepoint settings;
-- to increase the parallel query test coverage
SAVEPOINT settings;
SET LOCAL force_parallel_mode = 1;
diff --git a/src/test/regress/sql/select_parallel.sql b/src/test/regress/sql/select_parallel.sql
index f924731248..019e17e751 100644
--- a/src/test/regress/sql/select_parallel.sql
+++ b/src/test/regress/sql/select_parallel.sql
@@ -390,6 +390,12 @@ explain (costs off, verbose)
explain (costs off)
select * from tenk1 a where two in
(select two from tenk1 b where stringu1 like '%AAAA' limit 3);
+-- ...unless it's LATERAL
+savepoint settings;
+set parallel_tuple_cost=0;
+explain (costs off) select t.unique1 from tenk1 t
+join lateral (select t.unique1 from tenk1 offset 0) l on true;
+rollback to savepoint settings;
-- to increase the parallel query test coverage
SAVEPOINT settings;
--
2.20.1
v2-0002-Parallel-query-support-for-basic-correlated-subqu.patchapplication/octet-stream; name=v2-0002-Parallel-query-support-for-basic-correlated-subqu.patchDownload
From 41ce0d022fb0dcccd12fa713a07c139452d9c763 Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Fri, 27 Nov 2020 18:44:30 -0500
Subject: [PATCH v2 2/3] Parallel query support for basic correlated subqueries
Not all Params are inherently parallel-unsafe, but we can't know whether
they're parallel-safe up-front: we need contextual information for a
given path shape. Here we delay the final determination of whether or
not a Param is parallel-safe by initially verifying that it is minimally
parallel-safe for things that are inherent (e.g., no parallel-unsafe
functions or relations involved) and later re-checking that a given
usage is contextually safe (e.g., the Param is for correlation that can
happen entirely within a parallel worker (as opposed to needing to pass
values between workers).
---
doc/src/sgml/parallel.sgml | 3 +-
src/backend/nodes/copyfuncs.c | 2 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/outfuncs.c | 3 +
src/backend/nodes/readfuncs.c | 2 +
src/backend/optimizer/path/allpaths.c | 61 +++++--
src/backend/optimizer/path/equivclass.c | 4 +-
src/backend/optimizer/path/indxpath.c | 20 ++-
src/backend/optimizer/path/joinpath.c | 21 ++-
src/backend/optimizer/plan/createplan.c | 2 +
src/backend/optimizer/plan/planmain.c | 2 +-
src/backend/optimizer/plan/planner.c | 155 +++++++++++++++---
src/backend/optimizer/plan/subselect.c | 21 ++-
src/backend/optimizer/prep/prepunion.c | 1 +
src/backend/optimizer/util/clauses.c | 113 ++++++++++---
src/backend/optimizer/util/pathnode.c | 36 +++-
src/backend/optimizer/util/relnode.c | 30 +++-
src/backend/utils/misc/guc.c | 11 ++
src/include/nodes/pathnodes.h | 5 +-
src/include/nodes/plannodes.h | 1 +
src/include/nodes/primnodes.h | 1 +
src/include/optimizer/clauses.h | 4 +-
.../regress/expected/incremental_sort.out | 28 ++--
src/test/regress/expected/select_parallel.out | 137 ++++++++++++++++
src/test/regress/expected/sysviews.out | 3 +-
src/test/regress/sql/select_parallel.sql | 37 +++++
26 files changed, 604 insertions(+), 100 deletions(-)
diff --git a/doc/src/sgml/parallel.sgml b/doc/src/sgml/parallel.sgml
index 13479d7e5e..a651a9fe73 100644
--- a/doc/src/sgml/parallel.sgml
+++ b/doc/src/sgml/parallel.sgml
@@ -517,7 +517,8 @@ EXPLAIN SELECT * FROM pgbench_accounts WHERE filler LIKE '%x%';
<listitem>
<para>
- Plan nodes that reference a correlated <literal>SubPlan</literal>.
+ Plan nodes thatreference a correlated <literal>SubPlan</literal> where
+ the result is shared between workers.
</para>
</listitem>
</itemizedlist>
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 38251c2b8e..4042cef2ea 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -124,6 +124,7 @@ CopyPlanFields(const Plan *from, Plan *newnode)
COPY_SCALAR_FIELD(parallel_aware);
COPY_SCALAR_FIELD(parallel_safe);
COPY_SCALAR_FIELD(async_capable);
+ COPY_SCALAR_FIELD(parallel_safe_ignoring_params);
COPY_SCALAR_FIELD(plan_node_id);
COPY_NODE_FIELD(targetlist);
COPY_NODE_FIELD(qual);
@@ -1767,6 +1768,7 @@ _copySubPlan(const SubPlan *from)
COPY_SCALAR_FIELD(useHashTable);
COPY_SCALAR_FIELD(unknownEqFalse);
COPY_SCALAR_FIELD(parallel_safe);
+ COPY_SCALAR_FIELD(parallel_safe_ignoring_params);
COPY_NODE_FIELD(setParam);
COPY_NODE_FIELD(parParam);
COPY_NODE_FIELD(args);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 8a1762000c..482063c066 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -465,6 +465,7 @@ _equalSubPlan(const SubPlan *a, const SubPlan *b)
COMPARE_SCALAR_FIELD(useHashTable);
COMPARE_SCALAR_FIELD(unknownEqFalse);
COMPARE_SCALAR_FIELD(parallel_safe);
+ COMPARE_SCALAR_FIELD(parallel_safe_ignoring_params);
COMPARE_NODE_FIELD(setParam);
COMPARE_NODE_FIELD(parParam);
COMPARE_NODE_FIELD(args);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 87561cbb6f..42dbb05db5 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -334,6 +334,7 @@ _outPlanInfo(StringInfo str, const Plan *node)
WRITE_BOOL_FIELD(parallel_aware);
WRITE_BOOL_FIELD(parallel_safe);
WRITE_BOOL_FIELD(async_capable);
+ WRITE_BOOL_FIELD(parallel_safe_ignoring_params);
WRITE_INT_FIELD(plan_node_id);
WRITE_NODE_FIELD(targetlist);
WRITE_NODE_FIELD(qual);
@@ -1374,6 +1375,7 @@ _outSubPlan(StringInfo str, const SubPlan *node)
WRITE_BOOL_FIELD(useHashTable);
WRITE_BOOL_FIELD(unknownEqFalse);
WRITE_BOOL_FIELD(parallel_safe);
+ WRITE_BOOL_FIELD(parallel_safe_ignoring_params);
WRITE_NODE_FIELD(setParam);
WRITE_NODE_FIELD(parParam);
WRITE_NODE_FIELD(args);
@@ -1772,6 +1774,7 @@ _outPathInfo(StringInfo str, const Path *node)
outBitmapset(str, NULL);
WRITE_BOOL_FIELD(parallel_aware);
WRITE_BOOL_FIELD(parallel_safe);
+ WRITE_BOOL_FIELD(parallel_safe_ignoring_params);
WRITE_INT_FIELD(parallel_workers);
WRITE_FLOAT_FIELD(rows, "%.0f");
WRITE_FLOAT_FIELD(startup_cost, "%.2f");
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 0dd1ad7dfc..b6598726bb 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -1621,6 +1621,7 @@ ReadCommonPlan(Plan *local_node)
READ_BOOL_FIELD(parallel_aware);
READ_BOOL_FIELD(parallel_safe);
READ_BOOL_FIELD(async_capable);
+ READ_BOOL_FIELD(parallel_safe_ignoring_params);
READ_INT_FIELD(plan_node_id);
READ_NODE_FIELD(targetlist);
READ_NODE_FIELD(qual);
@@ -2615,6 +2616,7 @@ _readSubPlan(void)
READ_BOOL_FIELD(useHashTable);
READ_BOOL_FIELD(unknownEqFalse);
READ_BOOL_FIELD(parallel_safe);
+ READ_BOOL_FIELD(parallel_safe_ignoring_params);
READ_NODE_FIELD(setParam);
READ_NODE_FIELD(parParam);
READ_NODE_FIELD(args);
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 1363f1bc6c..5ab6d0e23f 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -556,7 +556,8 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
* (see grouping_planner).
*/
if (rel->reloptkind == RELOPT_BASEREL &&
- bms_membership(root->all_baserels) != BMS_SINGLETON)
+ bms_membership(root->all_baserels) != BMS_SINGLETON
+ && (rel->subplan_params == NIL || rte->rtekind != RTE_SUBQUERY))
generate_useful_gather_paths(root, rel, false);
/* Now find the cheapest of the paths for this rel */
@@ -592,6 +593,9 @@ static void
set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte)
{
+ bool parallel_safe;
+ bool parallel_safe_except_in_params;
+
/*
* The flag has previously been initialized to false, so we can just
* return if it becomes clear that we can't safely set it.
@@ -632,7 +636,7 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
if (proparallel != PROPARALLEL_SAFE)
return;
- if (!is_parallel_safe(root, (Node *) rte->tablesample->args))
+ if (!is_parallel_safe(root, (Node *) rte->tablesample->args, NULL))
return;
}
@@ -700,7 +704,7 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
case RTE_FUNCTION:
/* Check for parallel-restricted functions. */
- if (!is_parallel_safe(root, (Node *) rte->functions))
+ if (!is_parallel_safe(root, (Node *) rte->functions, NULL))
return;
break;
@@ -710,7 +714,7 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
case RTE_VALUES:
/* Check for parallel-restricted functions. */
- if (!is_parallel_safe(root, (Node *) rte->values_lists))
+ if (!is_parallel_safe(root, (Node *) rte->values_lists, NULL))
return;
break;
@@ -747,18 +751,28 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
* outer join clauses work correctly. It would likely break equivalence
* classes, too.
*/
- if (!is_parallel_safe(root, (Node *) rel->baserestrictinfo))
- return;
+ parallel_safe = is_parallel_safe(root, (Node *) rel->baserestrictinfo,
+ ¶llel_safe_except_in_params);
/*
* Likewise, if the relation's outputs are not parallel-safe, give up.
* (Usually, they're just Vars, but sometimes they're not.)
*/
- if (!is_parallel_safe(root, (Node *) rel->reltarget->exprs))
- return;
+ if (parallel_safe || parallel_safe_except_in_params)
+ {
+ bool target_parallel_safe;
+ bool target_parallel_safe_ignoring_params = false;
+
+ target_parallel_safe = is_parallel_safe(root,
+ (Node *) rel->reltarget->exprs,
+ &target_parallel_safe_ignoring_params);
+ parallel_safe = parallel_safe && target_parallel_safe;
+ parallel_safe_except_in_params = parallel_safe_except_in_params
+ && target_parallel_safe_ignoring_params;
+ }
- /* We have a winner. */
- rel->consider_parallel = true;
+ rel->consider_parallel = parallel_safe;
+ rel->consider_parallel_rechecking_params = parallel_safe_except_in_params;
}
/*
@@ -2277,9 +2291,21 @@ set_subquery_pathlist(PlannerInfo *root, RelOptInfo *rel,
pathkeys, required_outer));
}
+ /*
+ * XXX: As far as I can tell, the only time partial paths exist here
+ * is when we're going to execute multiple partial parths in parallel
+ * under a gather node (instead of executing paths serially under
+ * an append node). That means that the subquery scan path here
+ * is self-contained at this point -- so by definition it can't be
+ * reliant on lateral relids, which means we'll never have to consider
+ * rechecking params here.
+ */
+ Assert(!(rel->consider_parallel_rechecking_params && rel->partial_pathlist && !rel->consider_parallel));
+
/* If outer rel allows parallelism, do same for partial paths. */
if (rel->consider_parallel && bms_is_empty(required_outer))
{
+
/* If consider_parallel is false, there should be no partial paths. */
Assert(sub_final_rel->consider_parallel ||
sub_final_rel->partial_pathlist == NIL);
@@ -2633,7 +2659,7 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
cheapest_partial_path->rows * cheapest_partial_path->parallel_workers;
simple_gather_path = (Path *)
create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
- NULL, rowsp);
+ rel->lateral_relids, rowsp);
add_path(rel, simple_gather_path);
/*
@@ -2650,7 +2676,7 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
rows = subpath->rows * subpath->parallel_workers;
path = create_gather_merge_path(root, rel, subpath, rel->reltarget,
- subpath->pathkeys, NULL, rowsp);
+ subpath->pathkeys, rel->lateral_relids, rowsp);
add_path(rel, &path->path);
}
}
@@ -2752,11 +2778,15 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
double *rowsp = NULL;
List *useful_pathkeys_list = NIL;
Path *cheapest_partial_path = NULL;
+ Relids required_outer = rel->lateral_relids;
/* If there are no partial paths, there's nothing to do here. */
if (rel->partial_pathlist == NIL)
return;
+ if (!bms_is_subset(required_outer, rel->relids))
+ return;
+
/* Should we override the rel's rowcount estimate? */
if (override_rows)
rowsp = &rows;
@@ -2828,7 +2858,7 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
tmp,
rel->reltarget,
tmp->pathkeys,
- NULL,
+ required_outer,
rowsp);
add_path(rel, &path->path);
@@ -2862,7 +2892,7 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
tmp,
rel->reltarget,
tmp->pathkeys,
- NULL,
+ required_outer,
rowsp);
add_path(rel, &path->path);
@@ -3041,7 +3071,8 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
/*
* Except for the topmost scan/join rel, consider gathering
* partial paths. We'll do the same for the topmost scan/join rel
- * once we know the final targetlist (see grouping_planner).
+ * once we know the final targetlist (see
+ * apply_scanjoin_target_to_paths).
*/
if (lev < levels_needed)
generate_useful_gather_paths(root, rel, false);
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 6f1abbe47d..4c176d2d49 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -897,7 +897,7 @@ find_computable_ec_member(PlannerInfo *root,
* check this last because it's a rather expensive test.
*/
if (require_parallel_safe &&
- !is_parallel_safe(root, (Node *) em->em_expr))
+ !is_parallel_safe(root, (Node *) em->em_expr, NULL))
continue;
return em; /* found usable expression */
@@ -1012,7 +1012,7 @@ relation_can_be_sorted_early(PlannerInfo *root, RelOptInfo *rel,
* check this last because it's a rather expensive test.
*/
if (require_parallel_safe &&
- !is_parallel_safe(root, (Node *) em->em_expr))
+ !is_parallel_safe(root, (Node *) em->em_expr, NULL))
continue;
return true;
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
index 0e4e00eaf0..262c129ff5 100644
--- a/src/backend/optimizer/path/indxpath.c
+++ b/src/backend/optimizer/path/indxpath.c
@@ -1047,10 +1047,21 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
/*
* If appropriate, consider parallel index scan. We don't allow
* parallel index scan for bitmap index scans.
+ *
+ * XXX: Checking rel->consider_parallel_rechecking_params here resulted
+ * in some odd behavior on:
+ * select (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1) from tenk1 t;
+ * where the total cost on the chosen plan went *up* considering
+ * the extra path.
+ *
+ * Current working theory is that this method is about base relation
+ * scans, and we only want parameterized paths to be parallelized as
+ * companions to existing parallel plans and so don't really care to
+ * consider a separate parallel index scan here.
*/
if (index->amcanparallel &&
- rel->consider_parallel && outer_relids == NULL &&
- scantype != ST_BITMAPSCAN)
+ rel->consider_parallel && outer_relids == NULL &&
+ scantype != ST_BITMAPSCAN)
{
ipath = create_index_path(root, index,
index_clauses,
@@ -1100,9 +1111,10 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
result = lappend(result, ipath);
/* If appropriate, consider parallel index scan */
+ /* XXX: As above here for rel->consider_parallel_rechecking_params? */
if (index->amcanparallel &&
- rel->consider_parallel && outer_relids == NULL &&
- scantype != ST_BITMAPSCAN)
+ rel->consider_parallel && outer_relids == NULL &&
+ scantype != ST_BITMAPSCAN)
{
ipath = create_index_path(root, index,
index_clauses,
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index 6407ede12a..f8daa7b265 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -722,6 +722,7 @@ try_partial_nestloop_path(PlannerInfo *root,
else
outerrelids = outerrel->relids;
+ /* TODO: recheck parallel safety here? */
if (!bms_is_subset(inner_paramrels, outerrelids))
return;
}
@@ -1746,16 +1747,24 @@ match_unsorted_outer(PlannerInfo *root,
* partial path and the joinrel is parallel-safe. However, we can't
* handle JOIN_UNIQUE_OUTER, because the outer path will be partial, and
* therefore we won't be able to properly guarantee uniqueness. Nor can
- * we handle joins needing lateral rels, since partial paths must not be
- * parameterized. Similarly, we can't handle JOIN_FULL and JOIN_RIGHT,
- * because they can produce false null extended rows.
+ * we handle JOIN_FULL and JOIN_RIGHT, because they can produce false null
+ * extended rows.
+ *
+ * Partial paths may only have parameters in limited cases
+ * where the parameterization is fully satisfied without sharing state
+ * between workers, so we only allow lateral rels on inputs to the join
+ * if the resulting join contains no lateral rels, the inner rel's laterals
+ * are fully satisfied by the outer rel, and the outer rel doesn't depend
+ * on the inner rel to produce any laterals.
*/
if (joinrel->consider_parallel &&
save_jointype != JOIN_UNIQUE_OUTER &&
save_jointype != JOIN_FULL &&
save_jointype != JOIN_RIGHT &&
outerrel->partial_pathlist != NIL &&
- bms_is_empty(joinrel->lateral_relids))
+ bms_is_empty(joinrel->lateral_relids) &&
+ bms_is_subset(innerrel->lateral_relids, outerrel->relids) &&
+ (bms_is_empty(outerrel->lateral_relids) || !bms_is_subset(outerrel->lateral_relids, innerrel->relids)))
{
if (nestjoinOK)
consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
@@ -1870,7 +1879,9 @@ consider_parallel_nestloop(PlannerInfo *root,
Path *mpath;
/* Can't join to an inner path that is not parallel-safe */
- if (!innerpath->parallel_safe)
+ /* TODO: recheck parallel safety of params here? */
+ if (!innerpath->parallel_safe &&
+ !(innerpath->parallel_safe_ignoring_params))
continue;
/*
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index a5f6d678cc..e84834427b 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -5273,6 +5273,7 @@ copy_generic_path_info(Plan *dest, Path *src)
dest->plan_width = src->pathtarget->width;
dest->parallel_aware = src->parallel_aware;
dest->parallel_safe = src->parallel_safe;
+ dest->parallel_safe_ignoring_params = src->parallel_safe_ignoring_params;
}
/*
@@ -5290,6 +5291,7 @@ copy_plan_costsize(Plan *dest, Plan *src)
dest->parallel_aware = false;
/* Assume the inserted node is parallel-safe, if child plan is. */
dest->parallel_safe = src->parallel_safe;
+ dest->parallel_safe_ignoring_params = src->parallel_safe_ignoring_params;
}
/*
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index 273ac0acf7..bdbce2b87d 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -119,7 +119,7 @@ query_planner(PlannerInfo *root,
if (root->glob->parallelModeOK &&
force_parallel_mode != FORCE_PARALLEL_OFF)
final_rel->consider_parallel =
- is_parallel_safe(root, parse->jointree->quals);
+ is_parallel_safe(root, parse->jointree->quals, NULL);
/*
* The only path for it is a trivial Result path. We cheat a
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 1e42d75465..25ed2b2788 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -150,6 +150,7 @@ static RelOptInfo *create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
PathTarget *target,
bool target_parallel_safe,
+ bool target_parallel_safe_ignoring_params,
grouping_sets_data *gd);
static bool is_degenerate_grouping(PlannerInfo *root);
static void create_degenerate_grouping_paths(PlannerInfo *root,
@@ -157,6 +158,7 @@ static void create_degenerate_grouping_paths(PlannerInfo *root,
RelOptInfo *grouped_rel);
static RelOptInfo *make_grouping_rel(PlannerInfo *root, RelOptInfo *input_rel,
PathTarget *target, bool target_parallel_safe,
+ bool target_parallel_safe_ignoring_params,
Node *havingQual);
static void create_ordinary_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
@@ -237,6 +239,7 @@ static void apply_scanjoin_target_to_paths(PlannerInfo *root,
List *scanjoin_targets,
List *scanjoin_targets_contain_srfs,
bool scanjoin_target_parallel_safe,
+ bool scanjoin_target_parallel_safe_ignoring_params,
bool tlist_same_exprs);
static void create_partitionwise_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
@@ -1241,6 +1244,7 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
List *final_targets;
List *final_targets_contain_srfs;
bool final_target_parallel_safe;
+ bool final_target_parallel_safe_ignoring_params = false;
RelOptInfo *current_rel;
RelOptInfo *final_rel;
FinalPathExtraData extra;
@@ -1303,7 +1307,7 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
/* And check whether it's parallel safe */
final_target_parallel_safe =
- is_parallel_safe(root, (Node *) final_target->exprs);
+ is_parallel_safe(root, (Node *) final_target->exprs, &final_target_parallel_safe_ignoring_params);
/* The setop result tlist couldn't contain any SRFs */
Assert(!parse->hasTargetSRFs);
@@ -1337,14 +1341,17 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
List *sort_input_targets;
List *sort_input_targets_contain_srfs;
bool sort_input_target_parallel_safe;
+ bool sort_input_target_parallel_safe_ignoring_params = false;
PathTarget *grouping_target;
List *grouping_targets;
List *grouping_targets_contain_srfs;
bool grouping_target_parallel_safe;
+ bool grouping_target_parallel_safe_ignoring_params = false;
PathTarget *scanjoin_target;
List *scanjoin_targets;
List *scanjoin_targets_contain_srfs;
bool scanjoin_target_parallel_safe;
+ bool scanjoin_target_parallel_safe_ignoring_params = false;
bool scanjoin_target_same_exprs;
bool have_grouping;
WindowFuncLists *wflists = NULL;
@@ -1457,7 +1464,7 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
*/
final_target = create_pathtarget(root, root->processed_tlist);
final_target_parallel_safe =
- is_parallel_safe(root, (Node *) final_target->exprs);
+ is_parallel_safe(root, (Node *) final_target->exprs, &final_target_parallel_safe_ignoring_params);
/*
* If ORDER BY was given, consider whether we should use a post-sort
@@ -1470,12 +1477,13 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
final_target,
&have_postponed_srfs);
sort_input_target_parallel_safe =
- is_parallel_safe(root, (Node *) sort_input_target->exprs);
+ is_parallel_safe(root, (Node *) sort_input_target->exprs, &sort_input_target_parallel_safe_ignoring_params);
}
else
{
sort_input_target = final_target;
sort_input_target_parallel_safe = final_target_parallel_safe;
+ sort_input_target_parallel_safe_ignoring_params = final_target_parallel_safe_ignoring_params;
}
/*
@@ -1489,12 +1497,13 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
final_target,
activeWindows);
grouping_target_parallel_safe =
- is_parallel_safe(root, (Node *) grouping_target->exprs);
+ is_parallel_safe(root, (Node *) grouping_target->exprs, &grouping_target_parallel_safe_ignoring_params);
}
else
{
grouping_target = sort_input_target;
grouping_target_parallel_safe = sort_input_target_parallel_safe;
+ grouping_target_parallel_safe_ignoring_params = sort_input_target_parallel_safe_ignoring_params;
}
/*
@@ -1508,12 +1517,13 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
{
scanjoin_target = make_group_input_target(root, final_target);
scanjoin_target_parallel_safe =
- is_parallel_safe(root, (Node *) scanjoin_target->exprs);
+ is_parallel_safe(root, (Node *) scanjoin_target->exprs, &scanjoin_target_parallel_safe_ignoring_params);
}
else
{
scanjoin_target = grouping_target;
scanjoin_target_parallel_safe = grouping_target_parallel_safe;
+ scanjoin_target_parallel_safe_ignoring_params = grouping_target_parallel_safe_ignoring_params;
}
/*
@@ -1565,6 +1575,7 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
apply_scanjoin_target_to_paths(root, current_rel, scanjoin_targets,
scanjoin_targets_contain_srfs,
scanjoin_target_parallel_safe,
+ scanjoin_target_parallel_safe_ignoring_params,
scanjoin_target_same_exprs);
/*
@@ -1592,6 +1603,7 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
current_rel,
grouping_target,
grouping_target_parallel_safe,
+ grouping_target_parallel_safe_ignoring_params,
gset_data);
/* Fix things up if grouping_target contains SRFs */
if (parse->hasTargetSRFs)
@@ -1665,10 +1677,25 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
* not a SELECT, consider_parallel will be false for every relation in the
* query.
*/
- if (current_rel->consider_parallel &&
- is_parallel_safe(root, parse->limitOffset) &&
- is_parallel_safe(root, parse->limitCount))
- final_rel->consider_parallel = true;
+ if (current_rel->consider_parallel || current_rel->consider_parallel_rechecking_params)
+ {
+ bool limit_count_parallel_safe;
+ bool limit_offset_parallel_safe;
+ bool limit_count_parallel_safe_ignoring_params = false;
+ bool limit_offset_parallel_safe_ignoring_params = false;
+
+ limit_count_parallel_safe = is_parallel_safe(root, parse->limitCount, &limit_count_parallel_safe_ignoring_params);
+ limit_offset_parallel_safe = is_parallel_safe(root, parse->limitOffset, &limit_offset_parallel_safe_ignoring_params);
+
+ if (current_rel->consider_parallel &&
+ limit_count_parallel_safe &&
+ limit_offset_parallel_safe)
+ final_rel->consider_parallel = true;
+ if (current_rel->consider_parallel_rechecking_params &&
+ limit_count_parallel_safe_ignoring_params &&
+ limit_offset_parallel_safe_ignoring_params)
+ final_rel->consider_parallel_rechecking_params = true;
+ }
/*
* If the current_rel belongs to a single FDW, so does the final_rel.
@@ -1869,8 +1896,8 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
* Generate partial paths for final_rel, too, if outer query levels might
* be able to make use of them.
*/
- if (final_rel->consider_parallel && root->query_level > 1 &&
- !limit_needed(parse))
+ if ((final_rel->consider_parallel || final_rel->consider_parallel_rechecking_params) &&
+ root->query_level > 1 && !limit_needed(parse))
{
Assert(!parse->rowMarks && parse->commandType == CMD_SELECT);
foreach(lc, current_rel->partial_pathlist)
@@ -3282,6 +3309,7 @@ create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
PathTarget *target,
bool target_parallel_safe,
+ bool target_parallel_safe_ignoring_params,
grouping_sets_data *gd)
{
Query *parse = root->parse;
@@ -3297,7 +3325,9 @@ create_grouping_paths(PlannerInfo *root,
* aggregation paths.
*/
grouped_rel = make_grouping_rel(root, input_rel, target,
- target_parallel_safe, parse->havingQual);
+ target_parallel_safe,
+ target_parallel_safe_ignoring_params,
+ parse->havingQual);
/*
* Create either paths for a degenerate grouping or paths for ordinary
@@ -3358,6 +3388,7 @@ create_grouping_paths(PlannerInfo *root,
extra.flags = flags;
extra.target_parallel_safe = target_parallel_safe;
+ extra.target_parallel_safe_ignoring_params = target_parallel_safe_ignoring_params;
extra.havingQual = parse->havingQual;
extra.targetList = parse->targetList;
extra.partial_costs_set = false;
@@ -3393,7 +3424,7 @@ create_grouping_paths(PlannerInfo *root,
static RelOptInfo *
make_grouping_rel(PlannerInfo *root, RelOptInfo *input_rel,
PathTarget *target, bool target_parallel_safe,
- Node *havingQual)
+ bool target_parallel_safe_ignoring_params, Node *havingQual)
{
RelOptInfo *grouped_rel;
@@ -3421,9 +3452,21 @@ make_grouping_rel(PlannerInfo *root, RelOptInfo *input_rel,
* can't be parallel-safe, either. Otherwise, it's parallel-safe if the
* target list and HAVING quals are parallel-safe.
*/
- if (input_rel->consider_parallel && target_parallel_safe &&
- is_parallel_safe(root, (Node *) havingQual))
- grouped_rel->consider_parallel = true;
+ if ((input_rel->consider_parallel || input_rel->consider_parallel_rechecking_params)
+ && (target_parallel_safe || target_parallel_safe_ignoring_params))
+ {
+ bool having_qual_parallel_safe;
+ bool having_qual_parallel_safe_ignoring_params = false;
+
+ having_qual_parallel_safe = is_parallel_safe(root, (Node *) havingQual,
+ &having_qual_parallel_safe_ignoring_params);
+
+ grouped_rel->consider_parallel = input_rel->consider_parallel &&
+ having_qual_parallel_safe && target_parallel_safe;
+ grouped_rel->consider_parallel_rechecking_params =
+ input_rel->consider_parallel_rechecking_params &&
+ having_qual_parallel_safe_ignoring_params && target_parallel_safe_ignoring_params;
+ }
/*
* If the input rel belongs to a single FDW, so does the grouped rel.
@@ -4050,7 +4093,7 @@ create_window_paths(PlannerInfo *root,
* target list and active windows for non-parallel-safe constructs.
*/
if (input_rel->consider_parallel && output_target_parallel_safe &&
- is_parallel_safe(root, (Node *) activeWindows))
+ is_parallel_safe(root, (Node *) activeWindows, NULL))
window_rel->consider_parallel = true;
/*
@@ -5755,6 +5798,7 @@ adjust_paths_for_srfs(PlannerInfo *root, RelOptInfo *rel,
PathTarget *thistarget = lfirst_node(PathTarget, lc1);
bool contains_srfs = (bool) lfirst_int(lc2);
+ /* TODO: How do we know the new target is parallel safe? */
/* If this level doesn't contain SRFs, do regular projection */
if (contains_srfs)
newpath = (Path *) create_set_projection_path(root,
@@ -6071,8 +6115,8 @@ plan_create_index_workers(Oid tableOid, Oid indexOid)
* safe.
*/
if (heap->rd_rel->relpersistence == RELPERSISTENCE_TEMP ||
- !is_parallel_safe(root, (Node *) RelationGetIndexExpressions(index)) ||
- !is_parallel_safe(root, (Node *) RelationGetIndexPredicate(index)))
+ !is_parallel_safe(root, (Node *) RelationGetIndexExpressions(index), NULL) ||
+ !is_parallel_safe(root, (Node *) RelationGetIndexPredicate(index), NULL))
{
parallel_workers = 0;
goto done;
@@ -6535,6 +6579,8 @@ create_partial_grouping_paths(PlannerInfo *root,
grouped_rel->relids);
partially_grouped_rel->consider_parallel =
grouped_rel->consider_parallel;
+ partially_grouped_rel->consider_parallel_rechecking_params =
+ grouped_rel->consider_parallel_rechecking_params;
partially_grouped_rel->reloptkind = grouped_rel->reloptkind;
partially_grouped_rel->serverid = grouped_rel->serverid;
partially_grouped_rel->userid = grouped_rel->userid;
@@ -6998,6 +7044,7 @@ apply_scanjoin_target_to_paths(PlannerInfo *root,
List *scanjoin_targets,
List *scanjoin_targets_contain_srfs,
bool scanjoin_target_parallel_safe,
+ bool scanjoin_target_parallel_safe_ignoring_params,
bool tlist_same_exprs)
{
bool rel_is_partitioned = IS_PARTITIONED_REL(rel);
@@ -7007,6 +7054,11 @@ apply_scanjoin_target_to_paths(PlannerInfo *root,
/* This recurses, so be paranoid. */
check_stack_depth();
+ /*
+ * TOOD: when/how do we want to generate gather paths if
+ * scanjoin_target_parallel_safe_ignoring_params = true
+ */
+
/*
* If the rel is partitioned, we want to drop its existing paths and
* generate new ones. This function would still be correct if we kept the
@@ -7047,6 +7099,64 @@ apply_scanjoin_target_to_paths(PlannerInfo *root,
generate_useful_gather_paths(root, rel, false);
/* Can't use parallel query above this level. */
+
+ /*
+ * There are cases where:
+ * (rel->consider_parallel &&
+ * !scanjoin_target_parallel_safe &&
+ * scanjoin_target_parallel_safe_ignoring_params)
+ * is true at this point. See longer commment below.
+ */
+ if (!(rel->consider_parallel_rechecking_params && scanjoin_target_parallel_safe_ignoring_params))
+ {
+ /*
+ * TODO: if we limit this to this condition, we're pushing off the
+ * checks as to whether or not a given param usage is safe in the
+ * context of a given path (in the context of a given rel?). That
+ * almost certainly means we'd have to add other checks later (maybe
+ * just on lateral/relids and not parallel safety overall), because
+ * at the end of grouping_planner() we copy partial paths to the
+ * final_rel, and while that path may be acceptable in some contexts
+ * it may not be in all contexts.
+ *
+ * OTOH if we're only dependent on PARAM_EXEC params, and we already
+ * know that subpath->param_info == NULL holds (and that seems like
+ * it must since we were going to replace the path target anyway...
+ * though the one caveat is from the original form of this function
+ * we'd only ever actually assert that for paths not partial paths)
+ * then if a param shows up in the target why would it not be parallel
+ * safe.
+ *
+ * Adding to the mystery even with the original form of this function
+ * we still end up with parallel paths where I'd expect this to
+ * disallow them. For example:
+ *
+ * SELECT '' AS six, f1 AS "Correlated Field", f3 AS "Second Field"
+ * FROM SUBSELECT_TBL upper
+ * WHERE f3 IN (
+ * SELECT upper.f1 + f2
+ * FROM SUBSELECT_TBL
+ * WHERE f2 = CAST(f3 AS integer)
+ * );
+ *
+ * ends up with the correlated query underneath parallel plan despite
+ * its target containing a param, and therefore this function marking
+ * the rel as consider_parallel=false and removing the partial paths.
+ *
+ * But the plan as a whole is parallel safe, and so the subplan is also
+ * parallel safe, which means we can incorporate it into a full parallel
+ * plan. In other words, this is a parallel safe, but not parallel aware
+ * subplan (and regular, not parallel, seq scan inside that subplan).
+ * It's not a partial path; it'a a full path that is executed as a subquery.
+ *
+ * Current conclusion: it's fine for subplans, which is the case we're
+ * currently targeting anyway. And it might even be the only case that
+ * matters at all.
+ */
+ /* rel->consider_parallel_rechecking_params = false; */
+ /* rel->partial_pathlist = NIL; */
+ }
+ rel->consider_parallel_rechecking_params = false;
rel->partial_pathlist = NIL;
rel->consider_parallel = false;
}
@@ -7182,6 +7292,7 @@ apply_scanjoin_target_to_paths(PlannerInfo *root,
child_scanjoin_targets,
scanjoin_targets_contain_srfs,
scanjoin_target_parallel_safe,
+ scanjoin_target_parallel_safe_ignoring_params,
tlist_same_exprs);
/* Save non-dummy children for Append paths. */
@@ -7199,6 +7310,11 @@ apply_scanjoin_target_to_paths(PlannerInfo *root,
* avoid creating multiple Gather nodes within the same plan. We must do
* this after all paths have been generated and before set_cheapest, since
* one of the generated paths may turn out to be the cheapest one.
+ *
+ * TODO: This is the same problem as earlier in this function: when allowing
+ * "parallel safe ignoring params" paths here we don't actually know we are
+ * safe in any possible context just possibly safe in the context of the
+ * right rel.
*/
if (rel->consider_parallel && !IS_OTHER_REL(rel))
generate_useful_gather_paths(root, rel, false);
@@ -7306,6 +7422,7 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
child_grouped_rel = make_grouping_rel(root, child_input_rel,
child_target,
extra->target_parallel_safe,
+ extra->target_parallel_safe_ignoring_params,
child_extra.havingQual);
/* Create grouping paths for this child relation. */
diff --git a/src/backend/optimizer/plan/subselect.c b/src/backend/optimizer/plan/subselect.c
index c9f7a09d10..393db3b42d 100644
--- a/src/backend/optimizer/plan/subselect.c
+++ b/src/backend/optimizer/plan/subselect.c
@@ -342,6 +342,7 @@ build_subplan(PlannerInfo *root, Plan *plan, PlannerInfo *subroot,
splan->useHashTable = false;
splan->unknownEqFalse = unknownEqFalse;
splan->parallel_safe = plan->parallel_safe;
+ splan->parallel_safe_ignoring_params = plan->parallel_safe_ignoring_params;
splan->setParam = NIL;
splan->parParam = NIL;
splan->args = NIL;
@@ -1937,6 +1938,7 @@ process_sublinks_mutator(Node *node, process_sublinks_context *context)
{
SubLink *sublink = (SubLink *) node;
Node *testexpr;
+ Node *result;
/*
* First, recursively process the lefthand-side expressions, if any.
@@ -1948,12 +1950,29 @@ process_sublinks_mutator(Node *node, process_sublinks_context *context)
/*
* Now build the SubPlan node and make the expr to return.
*/
- return make_subplan(context->root,
+ result = make_subplan(context->root,
(Query *) sublink->subselect,
sublink->subLinkType,
sublink->subLinkId,
testexpr,
context->isTopQual);
+
+ /*
+ * If planning determined that a subpath was parallel safe as long
+ * as required params are provided by each individual worker then we
+ * can mark the resulting subplan actually parallel safe since we now
+ * know for certain how that path will be used.
+ */
+ if (IsA(result, SubPlan) && !((SubPlan*)result)->parallel_safe
+ && ((SubPlan*)result)->parallel_safe_ignoring_params
+ && enable_parallel_params_recheck)
+ {
+ Plan *subplan = planner_subplan_get_plan(context->root, (SubPlan*)result);
+ ((SubPlan*)result)->parallel_safe = is_parallel_safe(context->root, testexpr, NULL);
+ subplan->parallel_safe = ((SubPlan*)result)->parallel_safe;
+ }
+
+ return result;
}
/*
diff --git a/src/backend/optimizer/prep/prepunion.c b/src/backend/optimizer/prep/prepunion.c
index e9256a2d4d..16cb35c914 100644
--- a/src/backend/optimizer/prep/prepunion.c
+++ b/src/backend/optimizer/prep/prepunion.c
@@ -409,6 +409,7 @@ recurse_set_operations(Node *setOp, PlannerInfo *root,
Assert(subpath->param_info == NULL);
/* avoid apply_projection_to_path, in case of multiple refs */
+ /* TODO: how to we know the target is parallel safe? */
path = (Path *) create_projection_path(root, subpath->parent,
subpath, target);
lfirst(lc) = path;
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 3412d31117..e2cf335f4a 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -55,6 +55,15 @@
#include "utils/syscache.h"
#include "utils/typcache.h"
+bool enable_parallel_params_recheck = true;
+
+typedef struct
+{
+ PlannerInfo *root;
+ AggSplit aggsplit;
+ AggClauseCosts *costs;
+} get_agg_clause_costs_context;
+
typedef struct
{
ParamListInfo boundParams;
@@ -88,6 +97,8 @@ typedef struct
{
char max_hazard; /* worst proparallel hazard found so far */
char max_interesting; /* worst proparallel hazard of interest */
+ char max_hazard_ignoring_params;
+ bool check_params_independently;
List *safe_param_ids; /* PARAM_EXEC Param IDs to treat as safe */
} max_parallel_hazard_context;
@@ -624,19 +635,27 @@ max_parallel_hazard(Query *parse)
context.max_hazard = PROPARALLEL_SAFE;
context.max_interesting = PROPARALLEL_UNSAFE;
context.safe_param_ids = NIL;
+ context.check_params_independently = false;
(void) max_parallel_hazard_walker((Node *) parse, &context);
return context.max_hazard;
}
/*
* is_parallel_safe
- * Detect whether the given expr contains only parallel-safe functions
+ * Detect whether the given expr contains only parallel safe funcions
+ * XXX: It does more than functions?
+ *
+ * If provided, safe_ignoring_params will be set the result of running the same
+ * parallel safety checks with the exception that params will be allowed. This
+ * value is useful since params are not inherently parallel unsafe, but rather
+ * their usage context (whether or not the worker is able to provide the value)
+ * determines parallel safety.
*
* root->glob->maxParallelHazard must previously have been set to the
- * result of max_parallel_hazard() on the whole query.
+ * result of max_parallel_hazard() on the whole query
*/
bool
-is_parallel_safe(PlannerInfo *root, Node *node)
+is_parallel_safe(PlannerInfo *root, Node *node, bool *safe_ignoring_params)
{
max_parallel_hazard_context context;
PlannerInfo *proot;
@@ -653,8 +672,10 @@ is_parallel_safe(PlannerInfo *root, Node *node)
return true;
/* Else use max_parallel_hazard's search logic, but stop on RESTRICTED */
context.max_hazard = PROPARALLEL_SAFE;
+ context.max_hazard_ignoring_params = PROPARALLEL_SAFE;
context.max_interesting = PROPARALLEL_RESTRICTED;
context.safe_param_ids = NIL;
+ context.check_params_independently = safe_ignoring_params != NULL;
/*
* The params that refer to the same or parent query level are considered
@@ -672,12 +693,17 @@ is_parallel_safe(PlannerInfo *root, Node *node)
}
}
- return !max_parallel_hazard_walker(node, &context);
+ (void) max_parallel_hazard_walker(node, &context);
+
+ if (safe_ignoring_params)
+ *safe_ignoring_params = context.max_hazard_ignoring_params == PROPARALLEL_SAFE;
+
+ return context.max_hazard == PROPARALLEL_SAFE;
}
/* core logic for all parallel-hazard checks */
static bool
-max_parallel_hazard_test(char proparallel, max_parallel_hazard_context *context)
+max_parallel_hazard_test(char proparallel, max_parallel_hazard_context *context, bool from_param)
{
switch (proparallel)
{
@@ -688,12 +714,16 @@ max_parallel_hazard_test(char proparallel, max_parallel_hazard_context *context)
/* increase max_hazard to RESTRICTED */
Assert(context->max_hazard != PROPARALLEL_UNSAFE);
context->max_hazard = proparallel;
+ if (!from_param)
+ context->max_hazard_ignoring_params = proparallel;
/* done if we are not expecting any unsafe functions */
if (context->max_interesting == proparallel)
return true;
break;
case PROPARALLEL_UNSAFE:
context->max_hazard = proparallel;
+ if (!from_param)
+ context->max_hazard_ignoring_params = proparallel;
/* we're always done at the first unsafe construct */
return true;
default:
@@ -708,7 +738,41 @@ static bool
max_parallel_hazard_checker(Oid func_id, void *context)
{
return max_parallel_hazard_test(func_parallel(func_id),
- (max_parallel_hazard_context *) context);
+ (max_parallel_hazard_context *) context, false);
+}
+
+static bool
+max_parallel_hazard_walker_can_short_circuit(max_parallel_hazard_context *context)
+{
+ if (!context->check_params_independently)
+ return true;
+
+ switch (context->max_hazard)
+ {
+ case PROPARALLEL_SAFE:
+ /* nothing to see here, move along */
+ break;
+ case PROPARALLEL_RESTRICTED:
+ if (context->max_interesting == PROPARALLEL_RESTRICTED)
+ return context->max_hazard_ignoring_params != PROPARALLEL_SAFE;
+
+ /*
+ * We haven't even met our max interesting yet, so
+ * we certainly can't short-circuit.
+ */
+ break;
+ case PROPARALLEL_UNSAFE:
+ if (context->max_interesting == PROPARALLEL_RESTRICTED)
+ return context->max_hazard_ignoring_params != PROPARALLEL_SAFE;
+ else if (context->max_interesting == PROPARALLEL_UNSAFE)
+ return context->max_hazard_ignoring_params == PROPARALLEL_UNSAFE;
+
+ break;
+ default:
+ elog(ERROR, "unrecognized proparallel value \"%c\"", context->max_hazard);
+ break;
+ }
+ return false;
}
static bool
@@ -720,7 +784,7 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
/* Check for hazardous functions in node itself */
if (check_functions_in_node(node, max_parallel_hazard_checker,
context))
- return true;
+ return max_parallel_hazard_walker_can_short_circuit(context);
/*
* It should be OK to treat MinMaxExpr as parallel-safe, since btree
@@ -735,14 +799,14 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
*/
if (IsA(node, CoerceToDomain))
{
- if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
- return true;
+ if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context, false))
+ return max_parallel_hazard_walker_can_short_circuit(context);
}
else if (IsA(node, NextValueExpr))
{
- if (max_parallel_hazard_test(PROPARALLEL_UNSAFE, context))
- return true;
+ if (max_parallel_hazard_test(PROPARALLEL_UNSAFE, context, false))
+ return max_parallel_hazard_walker_can_short_circuit(context);
}
/*
@@ -755,8 +819,8 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
*/
else if (IsA(node, WindowFunc))
{
- if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
- return true;
+ if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context, false))
+ return max_parallel_hazard_walker_can_short_circuit(context);
}
/*
@@ -775,8 +839,8 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
*/
else if (IsA(node, SubLink))
{
- if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
- return true;
+ if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context, false))
+ return max_parallel_hazard_walker_can_short_circuit(context);
}
/*
@@ -791,18 +855,23 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
List *save_safe_param_ids;
if (!subplan->parallel_safe &&
- max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
+ (!enable_parallel_params_recheck || !subplan->parallel_safe_ignoring_params) &&
+ max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context, false) &&
+ max_parallel_hazard_walker_can_short_circuit(context))
return true;
save_safe_param_ids = context->safe_param_ids;
context->safe_param_ids = list_concat_copy(context->safe_param_ids,
subplan->paramIds);
- if (max_parallel_hazard_walker(subplan->testexpr, context))
- return true; /* no need to restore safe_param_ids */
+ if (max_parallel_hazard_walker(subplan->testexpr, context) &&
+ max_parallel_hazard_walker_can_short_circuit(context))
+ /* no need to restore safe_param_ids */
+ return true;
+
list_free(context->safe_param_ids);
context->safe_param_ids = save_safe_param_ids;
/* we must also check args, but no special Param treatment there */
if (max_parallel_hazard_walker((Node *) subplan->args, context))
- return true;
+ return max_parallel_hazard_walker_can_short_circuit(context);
/* don't want to recurse normally, so we're done */
return false;
}
@@ -824,8 +893,8 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
if (param->paramkind != PARAM_EXEC ||
!list_member_int(context->safe_param_ids, param->paramid))
{
- if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
- return true;
+ if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context, true))
+ return max_parallel_hazard_walker_can_short_circuit(context);
}
return false; /* nothing to recurse to */
}
@@ -843,7 +912,7 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
if (query->rowMarks != NULL)
{
context->max_hazard = PROPARALLEL_UNSAFE;
- return true;
+ return max_parallel_hazard_walker_can_short_circuit(context);
}
/* Recurse into subselects */
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index a53850b370..21be3824c1 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -756,10 +756,10 @@ add_partial_path(RelOptInfo *parent_rel, Path *new_path)
CHECK_FOR_INTERRUPTS();
/* Path to be added must be parallel safe. */
- Assert(new_path->parallel_safe);
+ Assert(new_path->parallel_safe || new_path->parallel_safe_ignoring_params);
/* Relation should be OK for parallelism, too. */
- Assert(parent_rel->consider_parallel);
+ Assert(parent_rel->consider_parallel || parent_rel->consider_parallel_rechecking_params);
/*
* As in add_path, throw out any paths which are dominated by the new
@@ -938,6 +938,7 @@ create_seqscan_path(PlannerInfo *root, RelOptInfo *rel,
required_outer);
pathnode->parallel_aware = parallel_workers > 0 ? true : false;
pathnode->parallel_safe = rel->consider_parallel;
+ pathnode->parallel_safe_ignoring_params = rel->consider_parallel_rechecking_params;
pathnode->parallel_workers = parallel_workers;
pathnode->pathkeys = NIL; /* seqscan has unordered result */
@@ -1016,6 +1017,7 @@ create_index_path(PlannerInfo *root,
required_outer);
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel;
+ pathnode->path.parallel_safe_ignoring_params = rel->consider_parallel_rechecking_params;
pathnode->path.parallel_workers = 0;
pathnode->path.pathkeys = pathkeys;
@@ -1865,7 +1867,7 @@ create_gather_merge_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
Cost input_startup_cost = 0;
Cost input_total_cost = 0;
- Assert(subpath->parallel_safe);
+ Assert(subpath->parallel_safe || subpath->parallel_safe_ignoring_params);
Assert(pathkeys);
pathnode->path.pathtype = T_GatherMerge;
@@ -1953,7 +1955,7 @@ create_gather_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
{
GatherPath *pathnode = makeNode(GatherPath);
- Assert(subpath->parallel_safe);
+ Assert(subpath->parallel_safe || subpath->parallel_safe_ignoring_params);
pathnode->path.pathtype = T_Gather;
pathnode->path.parent = rel;
@@ -2000,6 +2002,8 @@ create_subqueryscan_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe;
+ pathnode->path.parallel_safe_ignoring_params = rel->consider_parallel_rechecking_params &&
+ subpath->parallel_safe_ignoring_params;
pathnode->path.parallel_workers = subpath->parallel_workers;
pathnode->path.pathkeys = pathkeys;
pathnode->subpath = subpath;
@@ -2417,6 +2421,8 @@ create_nestloop_path(PlannerInfo *root,
NestPath *pathnode = makeNode(NestPath);
Relids inner_req_outer = PATH_REQ_OUTER(inner_path);
+ /* TODO: Assert lateral relids subset safety? */
+
/*
* If the inner path is parameterized by the outer, we must drop any
* restrict_clauses that are due to be moved into the inner path. We have
@@ -2457,6 +2463,8 @@ create_nestloop_path(PlannerInfo *root,
pathnode->jpath.path.parallel_aware = false;
pathnode->jpath.path.parallel_safe = joinrel->consider_parallel &&
outer_path->parallel_safe && inner_path->parallel_safe;
+ pathnode->jpath.path.parallel_safe_ignoring_params = joinrel->consider_parallel_rechecking_params &&
+ outer_path->parallel_safe_ignoring_params && inner_path->parallel_safe_ignoring_params;
/* This is a foolish way to estimate parallel_workers, but for now... */
pathnode->jpath.path.parallel_workers = outer_path->parallel_workers;
pathnode->jpath.path.pathkeys = pathkeys;
@@ -2630,6 +2638,8 @@ create_projection_path(PlannerInfo *root,
{
ProjectionPath *pathnode = makeNode(ProjectionPath);
PathTarget *oldtarget;
+ bool target_parallel_safe;
+ bool target_parallel_safe_ignoring_params = false;
/*
* We mustn't put a ProjectionPath directly above another; it's useless
@@ -2653,9 +2663,12 @@ create_projection_path(PlannerInfo *root,
/* For now, assume we are above any joins, so no parameterization */
pathnode->path.param_info = NULL;
pathnode->path.parallel_aware = false;
+ target_parallel_safe = is_parallel_safe(root, (Node *) target->exprs,
+ &target_parallel_safe_ignoring_params);
pathnode->path.parallel_safe = rel->consider_parallel &&
- subpath->parallel_safe &&
- is_parallel_safe(root, (Node *) target->exprs);
+ subpath->parallel_safe && target_parallel_safe;
+ pathnode->path.parallel_safe_ignoring_params = rel->consider_parallel_rechecking_params &&
+ subpath->parallel_safe_ignoring_params && target_parallel_safe_ignoring_params;
pathnode->path.parallel_workers = subpath->parallel_workers;
/* Projection does not change the sort order */
pathnode->path.pathkeys = subpath->pathkeys;
@@ -2763,7 +2776,7 @@ apply_projection_to_path(PlannerInfo *root,
* parallel-safe in the target expressions, then we can't.
*/
if ((IsA(path, GatherPath) || IsA(path, GatherMergePath)) &&
- is_parallel_safe(root, (Node *) target->exprs))
+ is_parallel_safe(root, (Node *) target->exprs, NULL))
{
/*
* We always use create_projection_path here, even if the subpath is
@@ -2797,7 +2810,7 @@ apply_projection_to_path(PlannerInfo *root,
}
}
else if (path->parallel_safe &&
- !is_parallel_safe(root, (Node *) target->exprs))
+ !is_parallel_safe(root, (Node *) target->exprs, NULL))
{
/*
* We're inserting a parallel-restricted target list into a path
@@ -2805,6 +2818,7 @@ apply_projection_to_path(PlannerInfo *root,
* safe.
*/
path->parallel_safe = false;
+ path->parallel_safe_ignoring_params = false; /* TODO */
}
return path;
@@ -2837,7 +2851,7 @@ create_set_projection_path(PlannerInfo *root,
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe &&
- is_parallel_safe(root, (Node *) target->exprs);
+ is_parallel_safe(root, (Node *) target->exprs, NULL);
pathnode->path.parallel_workers = subpath->parallel_workers;
/* Projection does not change the sort order XXX? */
pathnode->path.pathkeys = subpath->pathkeys;
@@ -3114,6 +3128,8 @@ create_agg_path(PlannerInfo *root,
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe;
+ pathnode->path.parallel_safe_ignoring_params = rel->consider_parallel_rechecking_params &&
+ subpath->parallel_safe_ignoring_params;
pathnode->path.parallel_workers = subpath->parallel_workers;
if (aggstrategy == AGG_SORTED)
pathnode->path.pathkeys = subpath->pathkeys; /* preserves order */
@@ -3735,6 +3751,8 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe;
+ pathnode->path.parallel_safe_ignoring_params = rel->consider_parallel_rechecking_params &&
+ subpath->parallel_safe_ignoring_params;
pathnode->path.parallel_workers = subpath->parallel_workers;
pathnode->path.rows = subpath->rows;
pathnode->path.startup_cost = subpath->startup_cost;
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 47769cea45..9d908d0fea 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -213,6 +213,7 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
rel->consider_startup = (root->tuple_fraction > 0);
rel->consider_param_startup = false; /* might get changed later */
rel->consider_parallel = false; /* might get changed later */
+ rel->consider_parallel_rechecking_params = false; /* might get changed later */
rel->reltarget = create_empty_pathtarget();
rel->pathlist = NIL;
rel->ppilist = NIL;
@@ -617,6 +618,7 @@ build_join_rel(PlannerInfo *root,
joinrel->consider_startup = (root->tuple_fraction > 0);
joinrel->consider_param_startup = false;
joinrel->consider_parallel = false;
+ joinrel->consider_parallel_rechecking_params = false;
joinrel->reltarget = create_empty_pathtarget();
joinrel->pathlist = NIL;
joinrel->ppilist = NIL;
@@ -743,10 +745,27 @@ build_join_rel(PlannerInfo *root,
* take; therefore, we should make the same decision here however we get
* here.
*/
- if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
- is_parallel_safe(root, (Node *) restrictlist) &&
- is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
- joinrel->consider_parallel = true;
+ if ((inner_rel->consider_parallel || inner_rel->consider_parallel_rechecking_params)
+ && (outer_rel->consider_parallel || outer_rel->consider_parallel_rechecking_params))
+ {
+ bool restrictlist_parallel_safe;
+ bool restrictlist_parallel_safe_ignoring_params = false;
+ bool target_parallel_safe;
+ bool target_parallel_safe_ignoring_params = false;
+
+ restrictlist_parallel_safe = is_parallel_safe(root, (Node *) restrictlist, &restrictlist_parallel_safe_ignoring_params);
+ target_parallel_safe = is_parallel_safe(root, (Node *) joinrel->reltarget->exprs, &target_parallel_safe_ignoring_params);
+
+ if (inner_rel->consider_parallel && outer_rel->consider_parallel
+ && restrictlist_parallel_safe && target_parallel_safe)
+ joinrel->consider_parallel = true;
+
+ if (inner_rel->consider_parallel_rechecking_params
+ && outer_rel->consider_parallel_rechecking_params
+ && restrictlist_parallel_safe_ignoring_params
+ && target_parallel_safe_ignoring_params)
+ joinrel->consider_parallel_rechecking_params = true;
+ }
/* Add the joinrel to the PlannerInfo. */
add_join_rel(root, joinrel);
@@ -805,6 +824,7 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->consider_startup = (root->tuple_fraction > 0);
joinrel->consider_param_startup = false;
joinrel->consider_parallel = false;
+ joinrel->consider_parallel_rechecking_params = false;
joinrel->reltarget = create_empty_pathtarget();
joinrel->pathlist = NIL;
joinrel->ppilist = NIL;
@@ -892,6 +912,7 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
/* Child joinrel is parallel safe if parent is parallel safe. */
joinrel->consider_parallel = parent_joinrel->consider_parallel;
+ joinrel->consider_parallel_rechecking_params = parent_joinrel->consider_parallel_rechecking_params;
/* Set estimates of the child-joinrel's size. */
set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
@@ -1236,6 +1257,7 @@ fetch_upper_rel(PlannerInfo *root, UpperRelationKind kind, Relids relids)
upperrel->consider_startup = (root->tuple_fraction > 0);
upperrel->consider_param_startup = false;
upperrel->consider_parallel = false; /* might get changed later */
+ upperrel->consider_parallel_rechecking_params = false; /* might get changed later */
upperrel->reltarget = create_empty_pathtarget();
upperrel->pathlist = NIL;
upperrel->cheapest_startup_path = NULL;
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 467b0fd6fe..034510b611 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -57,6 +57,7 @@
#include "libpq/libpq.h"
#include "libpq/pqformat.h"
#include "miscadmin.h"
+#include "optimizer/clauses.h"
#include "optimizer/cost.h"
#include "optimizer/geqo.h"
#include "optimizer/optimizer.h"
@@ -968,6 +969,16 @@ static const unit_conversion time_unit_conversion_table[] =
static struct config_bool ConfigureNamesBool[] =
{
+ {
+ {"enable_parallel_params_recheck", PGC_USERSET, QUERY_TUNING_METHOD,
+ gettext_noop("Enables the planner's rechecking of parallel safety in the presence of PARAM_EXEC params (for correlated subqueries)."),
+ NULL,
+ GUC_EXPLAIN
+ },
+ &enable_parallel_params_recheck,
+ true,
+ NULL, NULL, NULL
+ },
{
{"enable_seqscan", PGC_USERSET, QUERY_TUNING_METHOD,
gettext_noop("Enables the planner's use of sequential-scan plans."),
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 1abe233db2..f2c3d229b0 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -683,6 +683,7 @@ typedef struct RelOptInfo
bool consider_startup; /* keep cheap-startup-cost paths? */
bool consider_param_startup; /* ditto, for parameterized paths? */
bool consider_parallel; /* consider parallel paths? */
+ bool consider_parallel_rechecking_params; /* consider parallel paths? */
/* default result targetlist for Paths scanning this relation */
struct PathTarget *reltarget; /* list of Vars/Exprs, cost, width */
@@ -1182,6 +1183,7 @@ typedef struct Path
bool parallel_aware; /* engage parallel-aware logic? */
bool parallel_safe; /* OK to use as part of parallel plan? */
+ bool parallel_safe_ignoring_params; /* OK to use as part of parallel plan if worker context provides params? */
int parallel_workers; /* desired # of workers; 0 = not parallel */
/* estimated size/costs for path (see costsize.c for more info) */
@@ -2471,7 +2473,7 @@ typedef struct MinMaxAggInfo
* for conflicting purposes.
*
* In addition, PARAM_EXEC slots are assigned for Params representing outputs
- * from subplans (values that are setParam items for those subplans). These
+ * from subplans (values that are setParam items for those subplans). [TODO: is this true, or only for init plans?] These
* IDs need not be tracked via PlannerParamItems, since we do not need any
* duplicate-elimination nor later processing of the represented expressions.
* Instead, we just record the assignment of the slot number by appending to
@@ -2590,6 +2592,7 @@ typedef struct
/* Data which may differ across partitions. */
bool target_parallel_safe;
+ bool target_parallel_safe_ignoring_params;
Node *havingQual;
List *targetList;
PartitionwiseAggregateType patype;
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index ec9a8b0c81..67805f2cfa 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -128,6 +128,7 @@ typedef struct Plan
*/
bool parallel_aware; /* engage parallel-aware logic? */
bool parallel_safe; /* OK to use as part of parallel plan? */
+ bool parallel_safe_ignoring_params; /* OK to use as part of parallel plan if worker context provides params? */
/*
* information needed for asynchronous execution
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
index c04282f91f..df1e2496c7 100644
--- a/src/include/nodes/primnodes.h
+++ b/src/include/nodes/primnodes.h
@@ -764,6 +764,7 @@ typedef struct SubPlan
* spec result is UNKNOWN; this allows much
* simpler handling of null values */
bool parallel_safe; /* is the subplan parallel-safe? */
+ bool parallel_safe_ignoring_params; /* is the subplan parallel-safe when params are provided by the worker context? */
/* Note: parallel_safe does not consider contents of testexpr or args */
/* Information for passing params into and out of the subselect: */
/* setParam and parParam are lists of integers (param IDs) */
diff --git a/src/include/optimizer/clauses.h b/src/include/optimizer/clauses.h
index 0673887a85..df01be2c61 100644
--- a/src/include/optimizer/clauses.h
+++ b/src/include/optimizer/clauses.h
@@ -16,6 +16,8 @@
#include "nodes/pathnodes.h"
+extern PGDLLIMPORT bool enable_parallel_params_recheck;
+
typedef struct
{
int numWindowFuncs; /* total number of WindowFuncs found */
@@ -33,7 +35,7 @@ extern double expression_returns_set_rows(PlannerInfo *root, Node *clause);
extern bool contain_subplans(Node *clause);
extern char max_parallel_hazard(Query *parse);
-extern bool is_parallel_safe(PlannerInfo *root, Node *node);
+extern bool is_parallel_safe(PlannerInfo *root, Node *node, bool *safe_ignoring_params);
extern bool contain_nonstrict_functions(Node *clause);
extern bool contain_exec_param(Node *clause, List *param_ids);
extern bool contain_leaked_vars(Node *clause);
diff --git a/src/test/regress/expected/incremental_sort.out b/src/test/regress/expected/incremental_sort.out
index 545e301e48..8f9ca05e60 100644
--- a/src/test/regress/expected/incremental_sort.out
+++ b/src/test/regress/expected/incremental_sort.out
@@ -1614,16 +1614,16 @@ from tenk1 t, generate_series(1, 1000);
QUERY PLAN
---------------------------------------------------------------------------------
Unique
- -> Sort
- Sort Key: t.unique1, ((SubPlan 1))
- -> Gather
- Workers Planned: 2
+ -> Gather Merge
+ Workers Planned: 2
+ -> Sort
+ Sort Key: t.unique1, ((SubPlan 1))
-> Nested Loop
-> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
-> Function Scan on generate_series
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on tenk1
- Index Cond: (unique1 = t.unique1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on tenk1
+ Index Cond: (unique1 = t.unique1)
(11 rows)
explain (costs off) select
@@ -1633,16 +1633,16 @@ from tenk1 t, generate_series(1, 1000)
order by 1, 2;
QUERY PLAN
---------------------------------------------------------------------------
- Sort
- Sort Key: t.unique1, ((SubPlan 1))
- -> Gather
- Workers Planned: 2
+ Gather Merge
+ Workers Planned: 2
+ -> Sort
+ Sort Key: t.unique1, ((SubPlan 1))
-> Nested Loop
-> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
-> Function Scan on generate_series
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on tenk1
- Index Cond: (unique1 = t.unique1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on tenk1
+ Index Cond: (unique1 = t.unique1)
(10 rows)
-- Parallel sort but with expression not available until the upper rel.
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 2303f70d6e..253f117d7a 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -311,6 +311,131 @@ select count(*) from tenk1 where (two, four) not in
10000
(1 row)
+-- test parallel plans for queries containing correlated subplans
+-- where the subplan only needs params available from the current
+-- worker's scan.
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t, generate_series(1, 10);
+ QUERY PLAN
+----------------------------------------------------------------------------
+ Gather
+ Output: ((SubPlan 1))
+ Workers Planned: 4
+ -> Nested Loop
+ Output: (SubPlan 1)
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ -> Function Scan on pg_catalog.generate_series
+ Output: generate_series.generate_series
+ Function Call: generate_series(1, 10)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(14 rows)
+
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t;
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: ((SubPlan 1))
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(9 rows)
+
+explain (analyze, costs off, summary off, verbose, timing off) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t
+ limit 1;
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------
+ Limit (actual rows=1 loops=1)
+ Output: ((SubPlan 1))
+ -> Gather (actual rows=1 loops=1)
+ Output: ((SubPlan 1))
+ Workers Planned: 4
+ Workers Launched: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t (actual rows=1 loops=5)
+ Output: (SubPlan 1)
+ Heap Fetches: 0
+ Worker 0: actual rows=1 loops=1
+ Worker 1: actual rows=1 loops=1
+ Worker 2: actual rows=1 loops=1
+ Worker 3: actual rows=1 loops=1
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1 (actual rows=1 loops=5)
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+ Heap Fetches: 0
+ Worker 0: actual rows=1 loops=1
+ Worker 1: actual rows=1 loops=1
+ Worker 2: actual rows=1 loops=1
+ Worker 3: actual rows=1 loops=1
+(22 rows)
+
+explain (costs off, verbose) select t.unique1
+ from tenk1 t
+ where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: t.unique1
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ Filter: (t.unique1 = (SubPlan 1))
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(10 rows)
+
+explain (costs off, verbose) select *
+ from tenk1 t
+ order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Gather Merge
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
+ Workers Planned: 4
+ -> Sort
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
+ Sort Key: ((SubPlan 1))
+ -> Parallel Seq Scan on public.tenk1 t
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(12 rows)
+
+-- test subplan in join/lateral join
+explain (costs off, verbose, timing off) select t.unique1, l.*
+ from tenk1 t
+ join lateral (
+ select (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1 offset 0)
+ ) l on true;
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: t.unique1, ((SubPlan 1))
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1, (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(9 rows)
+
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
explain (costs off)
select * from tenk1 where (unique1 + random())::integer not in
@@ -1192,6 +1317,18 @@ EXECUTE pstmt('1', make_some_array(1,2));
DEALLOCATE pstmt;
-- test interaction between subquery and partial_paths
+-- this plan changes to using a non-parallel index only
+-- scan on tenk1_unique1 (the parallel version of the subquery scan
+-- is cheaper, but only by ~30, and cost comparison treats them as equal
+-- since the costs are so large) because set_rel_consider_parallel
+-- called from make_one_rel sees the subplan as parallel safe now
+-- (in context it now knows the params are actually parallel safe).
+-- Because of that the non-parallel index path is now parallel_safe=true,
+-- therefore it wins the COSTS_EQUAL comparison in add_path.
+-- Perhaps any is_parallel_safe calls made for the purpose of determining
+-- consider_parallel should disable that behavior? It's not clear which is
+-- correct.
+set enable_parallel_params_recheck = off;
CREATE VIEW tenk1_vw_sec WITH (security_barrier) AS SELECT * FROM tenk1;
EXPLAIN (COSTS OFF)
SELECT 1 FROM tenk1_vw_sec
diff --git a/src/test/regress/expected/sysviews.out b/src/test/regress/expected/sysviews.out
index 6e54f3e15e..e4b48fc61d 100644
--- a/src/test/regress/expected/sysviews.out
+++ b/src/test/regress/expected/sysviews.out
@@ -109,13 +109,14 @@ select name, setting from pg_settings where name like 'enable%';
enable_nestloop | on
enable_parallel_append | on
enable_parallel_hash | on
+ enable_parallel_params_recheck | on
enable_partition_pruning | on
enable_partitionwise_aggregate | off
enable_partitionwise_join | off
enable_seqscan | on
enable_sort | on
enable_tidscan | on
-(20 rows)
+(21 rows)
-- Test that the pg_timezone_names and pg_timezone_abbrevs views are
-- more-or-less working. We can't test their contents in any great detail
diff --git a/src/test/regress/sql/select_parallel.sql b/src/test/regress/sql/select_parallel.sql
index 019e17e751..f217d0ec9b 100644
--- a/src/test/regress/sql/select_parallel.sql
+++ b/src/test/regress/sql/select_parallel.sql
@@ -111,6 +111,31 @@ explain (costs off)
(select hundred, thousand from tenk2 where thousand > 100);
select count(*) from tenk1 where (two, four) not in
(select hundred, thousand from tenk2 where thousand > 100);
+-- test parallel plans for queries containing correlated subplans
+-- where the subplan only needs params available from the current
+-- worker's scan.
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t, generate_series(1, 10);
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t;
+explain (analyze, costs off, summary off, verbose, timing off) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t
+ limit 1;
+explain (costs off, verbose) select t.unique1
+ from tenk1 t
+ where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+explain (costs off, verbose) select *
+ from tenk1 t
+ order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+-- test subplan in join/lateral join
+explain (costs off, verbose, timing off) select t.unique1, l.*
+ from tenk1 t
+ join lateral (
+ select (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1 offset 0)
+ ) l on true;
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
explain (costs off)
select * from tenk1 where (unique1 + random())::integer not in
@@ -456,6 +481,18 @@ EXECUTE pstmt('1', make_some_array(1,2));
DEALLOCATE pstmt;
-- test interaction between subquery and partial_paths
+-- this plan changes to using a non-parallel index only
+-- scan on tenk1_unique1 (the parallel version of the subquery scan
+-- is cheaper, but only by ~30, and cost comparison treats them as equal
+-- since the costs are so large) because set_rel_consider_parallel
+-- called from make_one_rel sees the subplan as parallel safe now
+-- (in context it now knows the params are actually parallel safe).
+-- Because of that the non-parallel index path is now parallel_safe=true,
+-- therefore it wins the COSTS_EQUAL comparison in add_path.
+-- Perhaps any is_parallel_safe calls made for the purpose of determining
+-- consider_parallel should disable that behavior? It's not clear which is
+-- correct.
+set enable_parallel_params_recheck = off;
CREATE VIEW tenk1_vw_sec WITH (security_barrier) AS SELECT * FROM tenk1;
EXPLAIN (COSTS OFF)
SELECT 1 FROM tenk1_vw_sec
--
2.20.1
v2-0003-Other-places-to-consider-for-completeness.patchapplication/octet-stream; name=v2-0003-Other-places-to-consider-for-completeness.patchDownload
From 4132c2144c71c4dac91a0428e97f6d5d81843f62 Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Fri, 7 May 2021 15:37:22 +0000
Subject: [PATCH v2 3/3] Other places to consider for completeness
---
src/backend/optimizer/path/allpaths.c | 4 ++++
src/backend/optimizer/plan/planmain.c | 2 +-
src/backend/optimizer/plan/planner.c | 3 +++
src/backend/optimizer/plan/subselect.c | 3 +++
src/backend/optimizer/prep/prepunion.c | 2 ++
5 files changed, 13 insertions(+), 1 deletion(-)
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 5ab6d0e23f..17438d2ea1 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -1142,6 +1142,8 @@ set_append_rel_size(PlannerInfo *root, RelOptInfo *rel,
*/
if (!childrel->consider_parallel)
rel->consider_parallel = false;
+ if (!childrel->consider_parallel_rechecking_params)
+ rel->consider_parallel_rechecking_params = false;
/*
* Accumulate size information from each live child.
@@ -1263,6 +1265,8 @@ set_append_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
*/
if (!rel->consider_parallel)
childrel->consider_parallel = false;
+ if (!childrel->consider_parallel_rechecking_params)
+ rel->consider_parallel_rechecking_params = false;
/*
* Compute the child's access paths.
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index bdbce2b87d..5135f9ee97 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -119,7 +119,7 @@ query_planner(PlannerInfo *root,
if (root->glob->parallelModeOK &&
force_parallel_mode != FORCE_PARALLEL_OFF)
final_rel->consider_parallel =
- is_parallel_safe(root, parse->jointree->quals, NULL);
+ is_parallel_safe(root, parse->jointree->quals, &final_rel->consider_parallel_rechecking_params);
/*
* The only path for it is a trivial Result path. We cheat a
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 25ed2b2788..7cde1da3c3 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -4095,6 +4095,7 @@ create_window_paths(PlannerInfo *root,
if (input_rel->consider_parallel && output_target_parallel_safe &&
is_parallel_safe(root, (Node *) activeWindows, NULL))
window_rel->consider_parallel = true;
+ /* consider_parallel_rechecking_params */
/*
* If the input rel belongs to a single FDW, so does the window rel.
@@ -4292,6 +4293,7 @@ create_distinct_paths(PlannerInfo *root, RelOptInfo *input_rel)
* expressions are parallel-safe.
*/
distinct_rel->consider_parallel = input_rel->consider_parallel;
+ distinct_rel->consider_parallel_rechecking_params = input_rel->consider_parallel_rechecking_params;
/*
* If the input rel belongs to a single FDW, so does the distinct_rel.
@@ -4646,6 +4648,7 @@ create_ordered_paths(PlannerInfo *root,
*/
if (input_rel->consider_parallel && target_parallel_safe)
ordered_rel->consider_parallel = true;
+ /* consider_parallel_rechecking_params */
/*
* If the input rel belongs to a single FDW, so does the ordered_rel.
diff --git a/src/backend/optimizer/plan/subselect.c b/src/backend/optimizer/plan/subselect.c
index 393db3b42d..3e1001fa86 100644
--- a/src/backend/optimizer/plan/subselect.c
+++ b/src/backend/optimizer/plan/subselect.c
@@ -1020,6 +1020,7 @@ SS_process_ctes(PlannerInfo *root)
* parallel-safe.
*/
splan->parallel_safe = false;
+ splan->parallel_safe_ignoring_params = false;
splan->setParam = NIL;
splan->parParam = NIL;
splan->args = NIL;
@@ -2176,6 +2177,7 @@ SS_charge_for_initplans(PlannerInfo *root, RelOptInfo *final_rel)
path->startup_cost += initplan_cost;
path->total_cost += initplan_cost;
path->parallel_safe = false;
+ path->parallel_safe_ignoring_params = false;
}
/*
@@ -2184,6 +2186,7 @@ SS_charge_for_initplans(PlannerInfo *root, RelOptInfo *final_rel)
*/
final_rel->partial_pathlist = NIL;
final_rel->consider_parallel = false;
+ final_rel->consider_parallel_rechecking_params = false;
/* We needn't do set_cheapest() here, caller will do it */
}
diff --git a/src/backend/optimizer/prep/prepunion.c b/src/backend/optimizer/prep/prepunion.c
index 16cb35c914..ab92e7d7e4 100644
--- a/src/backend/optimizer/prep/prepunion.c
+++ b/src/backend/optimizer/prep/prepunion.c
@@ -273,6 +273,7 @@ recurse_set_operations(Node *setOp, PlannerInfo *root,
*/
final_rel = fetch_upper_rel(subroot, UPPERREL_FINAL, NULL);
rel->consider_parallel = final_rel->consider_parallel;
+ rel->consider_parallel_rechecking_params = final_rel->consider_parallel_rechecking_params;
/*
* For the moment, we consider only a single Path for the subquery.
@@ -617,6 +618,7 @@ generate_union_paths(SetOperationStmt *op, PlannerInfo *root,
result_rel = fetch_upper_rel(root, UPPERREL_SETOP, relids);
result_rel->reltarget = create_pathtarget(root, tlist);
result_rel->consider_parallel = consider_parallel;
+ /* consider_parallel_rechecking_params */
/*
* Append the child results together.
--
2.20.1
Re: Parallelize correlated subqueries that execute within each worker
On Tue, Sep 7, 2021 at 6:17 AM James Coleman <jtc331@gmail.com> wrote:
On Wed, Sep 1, 2021 at 7:06 AM Daniel Gustafsson <daniel@yesql.se> wrote:
On 7 May 2021, at 18:30, James Coleman <jtc331@gmail.com> wrote:
..here we are now, and I finally have this patch cleaned up
enough to share.This patch no longer applies to HEAD, can you please submit a rebased
version?
See attached.
Thanks,
James
Hi,
For v2-0002-Parallel-query-support-for-basic-correlated-subqu.patch :
+ * is when we're going to execute multiple partial parths in parallel
parths -> paths
if (index->amcanparallel &&
- rel->consider_parallel && outer_relids == NULL &&
- scantype != ST_BITMAPSCAN)
+ rel->consider_parallel && outer_relids == NULL &&
+ scantype != ST_BITMAPSCAN)the change above seems unnecessary since the first line of if condition
doesn't change.
Similar comment for the next hunk.
+ * It's not a partial path; it'a a full path that is executed
as a subquery.
it'a a -> it's a
+ /* rel->consider_parallel_rechecking_params = false; */
+ /* rel->partial_pathlist = NIL; */The commented code can be taken out.
Cheers
Re: Parallelize correlated subqueries that execute within each worker
On Tue, Sep 7, 2021 at 11:06 AM Zhihong Yu <zyu@yugabyte.com> wrote:
On Tue, Sep 7, 2021 at 6:17 AM James Coleman <jtc331@gmail.com> wrote:
On Wed, Sep 1, 2021 at 7:06 AM Daniel Gustafsson <daniel@yesql.se> wrote:
On 7 May 2021, at 18:30, James Coleman <jtc331@gmail.com> wrote:
..here we are now, and I finally have this patch cleaned up
enough to share.This patch no longer applies to HEAD, can you please submit a rebased version?
See attached.
Thanks,
JamesHi,
For v2-0002-Parallel-query-support-for-basic-correlated-subqu.patch :+ * is when we're going to execute multiple partial parths in parallel
parths -> paths
if (index->amcanparallel && - rel->consider_parallel && outer_relids == NULL && - scantype != ST_BITMAPSCAN) + rel->consider_parallel && outer_relids == NULL && + scantype != ST_BITMAPSCAN)the change above seems unnecessary since the first line of if condition doesn't change.
Similar comment for the next hunk.+ * It's not a partial path; it'a a full path that is executed as a subquery.
it'a a -> it's a
+ /* rel->consider_parallel_rechecking_params = false; */ + /* rel->partial_pathlist = NIL; */The commented code can be taken out.
Thanks for taking a look at this.
See updated patch series attached.
James Coleman
Attachments:
v3-0001-Allow-parallel-LATERAL-subqueries-with-LIMIT-OFFS.patchapplication/octet-stream; name=v3-0001-Allow-parallel-LATERAL-subqueries-with-LIMIT-OFFS.patchDownload
From 7ba602847a13fbf7c87aab3651b4304ffba5e9e6 Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Mon, 30 Nov 2020 11:36:35 -0500
Subject: [PATCH v3 1/3] Allow parallel LATERAL subqueries with LIMIT/OFFSET
The code that determined whether or not a rel should be considered for
parallel query excluded subqueries with LIMIT/OFFSET. That's correct in
the general case: as the comment notes that'd mean we have to guarantee
ordering (and claims it's not worth checking that) for results to be
consistent across workers. However there's a simpler case that hasn't
been considered: LATERAL subqueries with LIMIT/OFFSET don't fall under
the same reasoning since they're executed (when not converted to a JOIN)
per tuple anyway, so consistency of results across workers isn't a
factor.
---
src/backend/optimizer/path/allpaths.c | 4 +++-
src/test/regress/expected/select_parallel.out | 15 +++++++++++++++
src/test/regress/sql/select_parallel.sql | 6 ++++++
3 files changed, 24 insertions(+), 1 deletion(-)
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 296dd75c1b..1363f1bc6c 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -682,11 +682,13 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
* inconsistent results at the top-level. (In some cases, where
* the result is ordered, we could relax this restriction. But it
* doesn't currently seem worth expending extra effort to do so.)
+ * LATERAL is an exception: LIMIT/OFFSET is safe to execute within
+ * workers since the sub-select is executed per tuple
*/
{
Query *subquery = castNode(Query, rte->subquery);
- if (limit_needed(subquery))
+ if (!rte->lateral && limit_needed(subquery))
return;
}
break;
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 4ea1aa7dfd..2303f70d6e 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -1040,6 +1040,21 @@ explain (costs off)
Filter: (stringu1 ~~ '%AAAA'::text)
(11 rows)
+-- ...unless it's LATERAL
+savepoint settings;
+set parallel_tuple_cost=0;
+explain (costs off) select t.unique1 from tenk1 t
+join lateral (select t.unique1 from tenk1 offset 0) l on true;
+ QUERY PLAN
+---------------------------------------------------------------------
+ Gather
+ Workers Planned: 4
+ -> Nested Loop
+ -> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
+ -> Index Only Scan using tenk1_hundred on tenk1
+(5 rows)
+
+rollback to savepoint settings;
-- to increase the parallel query test coverage
SAVEPOINT settings;
SET LOCAL force_parallel_mode = 1;
diff --git a/src/test/regress/sql/select_parallel.sql b/src/test/regress/sql/select_parallel.sql
index f924731248..019e17e751 100644
--- a/src/test/regress/sql/select_parallel.sql
+++ b/src/test/regress/sql/select_parallel.sql
@@ -390,6 +390,12 @@ explain (costs off, verbose)
explain (costs off)
select * from tenk1 a where two in
(select two from tenk1 b where stringu1 like '%AAAA' limit 3);
+-- ...unless it's LATERAL
+savepoint settings;
+set parallel_tuple_cost=0;
+explain (costs off) select t.unique1 from tenk1 t
+join lateral (select t.unique1 from tenk1 offset 0) l on true;
+rollback to savepoint settings;
-- to increase the parallel query test coverage
SAVEPOINT settings;
--
2.20.1
v3-0002-Parallel-query-support-for-basic-correlated-subqu.patchapplication/octet-stream; name=v3-0002-Parallel-query-support-for-basic-correlated-subqu.patchDownload
From f89bd7f6af26e16e2f625e6ee05dbc390a300b38 Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Fri, 27 Nov 2020 18:44:30 -0500
Subject: [PATCH v3 2/3] Parallel query support for basic correlated subqueries
Not all Params are inherently parallel-unsafe, but we can't know whether
they're parallel-safe up-front: we need contextual information for a
given path shape. Here we delay the final determination of whether or
not a Param is parallel-safe by initially verifying that it is minimally
parallel-safe for things that are inherent (e.g., no parallel-unsafe
functions or relations involved) and later re-checking that a given
usage is contextually safe (e.g., the Param is for correlation that can
happen entirely within a parallel worker (as opposed to needing to pass
values between workers).
---
doc/src/sgml/parallel.sgml | 3 +-
src/backend/nodes/copyfuncs.c | 2 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/outfuncs.c | 3 +
src/backend/nodes/readfuncs.c | 2 +
src/backend/optimizer/path/allpaths.c | 61 +++++--
src/backend/optimizer/path/equivclass.c | 4 +-
src/backend/optimizer/path/indxpath.c | 12 ++
src/backend/optimizer/path/joinpath.c | 21 ++-
src/backend/optimizer/plan/createplan.c | 2 +
src/backend/optimizer/plan/planmain.c | 2 +-
src/backend/optimizer/plan/planner.c | 158 +++++++++++++++---
src/backend/optimizer/plan/subselect.c | 21 ++-
src/backend/optimizer/prep/prepunion.c | 1 +
src/backend/optimizer/util/clauses.c | 113 ++++++++++---
src/backend/optimizer/util/pathnode.c | 36 +++-
src/backend/optimizer/util/relnode.c | 30 +++-
src/backend/utils/misc/guc.c | 11 ++
src/include/nodes/pathnodes.h | 5 +-
src/include/nodes/plannodes.h | 1 +
src/include/nodes/primnodes.h | 1 +
src/include/optimizer/clauses.h | 4 +-
.../regress/expected/incremental_sort.out | 28 ++--
src/test/regress/expected/select_parallel.out | 137 +++++++++++++++
src/test/regress/expected/sysviews.out | 3 +-
src/test/regress/sql/select_parallel.sql | 37 ++++
26 files changed, 603 insertions(+), 96 deletions(-)
diff --git a/doc/src/sgml/parallel.sgml b/doc/src/sgml/parallel.sgml
index 13479d7e5e..2d924dd2ac 100644
--- a/doc/src/sgml/parallel.sgml
+++ b/doc/src/sgml/parallel.sgml
@@ -517,7 +517,8 @@ EXPLAIN SELECT * FROM pgbench_accounts WHERE filler LIKE '%x%';
<listitem>
<para>
- Plan nodes that reference a correlated <literal>SubPlan</literal>.
+ Plan nodes that reference a correlated <literal>SubPlan</literal> where
+ the result is shared between workers.
</para>
</listitem>
</itemizedlist>
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 38251c2b8e..4042cef2ea 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -124,6 +124,7 @@ CopyPlanFields(const Plan *from, Plan *newnode)
COPY_SCALAR_FIELD(parallel_aware);
COPY_SCALAR_FIELD(parallel_safe);
COPY_SCALAR_FIELD(async_capable);
+ COPY_SCALAR_FIELD(parallel_safe_ignoring_params);
COPY_SCALAR_FIELD(plan_node_id);
COPY_NODE_FIELD(targetlist);
COPY_NODE_FIELD(qual);
@@ -1767,6 +1768,7 @@ _copySubPlan(const SubPlan *from)
COPY_SCALAR_FIELD(useHashTable);
COPY_SCALAR_FIELD(unknownEqFalse);
COPY_SCALAR_FIELD(parallel_safe);
+ COPY_SCALAR_FIELD(parallel_safe_ignoring_params);
COPY_NODE_FIELD(setParam);
COPY_NODE_FIELD(parParam);
COPY_NODE_FIELD(args);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 8a1762000c..482063c066 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -465,6 +465,7 @@ _equalSubPlan(const SubPlan *a, const SubPlan *b)
COMPARE_SCALAR_FIELD(useHashTable);
COMPARE_SCALAR_FIELD(unknownEqFalse);
COMPARE_SCALAR_FIELD(parallel_safe);
+ COMPARE_SCALAR_FIELD(parallel_safe_ignoring_params);
COMPARE_NODE_FIELD(setParam);
COMPARE_NODE_FIELD(parParam);
COMPARE_NODE_FIELD(args);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 87561cbb6f..42dbb05db5 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -334,6 +334,7 @@ _outPlanInfo(StringInfo str, const Plan *node)
WRITE_BOOL_FIELD(parallel_aware);
WRITE_BOOL_FIELD(parallel_safe);
WRITE_BOOL_FIELD(async_capable);
+ WRITE_BOOL_FIELD(parallel_safe_ignoring_params);
WRITE_INT_FIELD(plan_node_id);
WRITE_NODE_FIELD(targetlist);
WRITE_NODE_FIELD(qual);
@@ -1374,6 +1375,7 @@ _outSubPlan(StringInfo str, const SubPlan *node)
WRITE_BOOL_FIELD(useHashTable);
WRITE_BOOL_FIELD(unknownEqFalse);
WRITE_BOOL_FIELD(parallel_safe);
+ WRITE_BOOL_FIELD(parallel_safe_ignoring_params);
WRITE_NODE_FIELD(setParam);
WRITE_NODE_FIELD(parParam);
WRITE_NODE_FIELD(args);
@@ -1772,6 +1774,7 @@ _outPathInfo(StringInfo str, const Path *node)
outBitmapset(str, NULL);
WRITE_BOOL_FIELD(parallel_aware);
WRITE_BOOL_FIELD(parallel_safe);
+ WRITE_BOOL_FIELD(parallel_safe_ignoring_params);
WRITE_INT_FIELD(parallel_workers);
WRITE_FLOAT_FIELD(rows, "%.0f");
WRITE_FLOAT_FIELD(startup_cost, "%.2f");
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 0dd1ad7dfc..b6598726bb 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -1621,6 +1621,7 @@ ReadCommonPlan(Plan *local_node)
READ_BOOL_FIELD(parallel_aware);
READ_BOOL_FIELD(parallel_safe);
READ_BOOL_FIELD(async_capable);
+ READ_BOOL_FIELD(parallel_safe_ignoring_params);
READ_INT_FIELD(plan_node_id);
READ_NODE_FIELD(targetlist);
READ_NODE_FIELD(qual);
@@ -2615,6 +2616,7 @@ _readSubPlan(void)
READ_BOOL_FIELD(useHashTable);
READ_BOOL_FIELD(unknownEqFalse);
READ_BOOL_FIELD(parallel_safe);
+ READ_BOOL_FIELD(parallel_safe_ignoring_params);
READ_NODE_FIELD(setParam);
READ_NODE_FIELD(parParam);
READ_NODE_FIELD(args);
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 1363f1bc6c..6eb39e6a06 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -556,7 +556,8 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
* (see grouping_planner).
*/
if (rel->reloptkind == RELOPT_BASEREL &&
- bms_membership(root->all_baserels) != BMS_SINGLETON)
+ bms_membership(root->all_baserels) != BMS_SINGLETON
+ && (rel->subplan_params == NIL || rte->rtekind != RTE_SUBQUERY))
generate_useful_gather_paths(root, rel, false);
/* Now find the cheapest of the paths for this rel */
@@ -592,6 +593,9 @@ static void
set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte)
{
+ bool parallel_safe;
+ bool parallel_safe_except_in_params;
+
/*
* The flag has previously been initialized to false, so we can just
* return if it becomes clear that we can't safely set it.
@@ -632,7 +636,7 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
if (proparallel != PROPARALLEL_SAFE)
return;
- if (!is_parallel_safe(root, (Node *) rte->tablesample->args))
+ if (!is_parallel_safe(root, (Node *) rte->tablesample->args, NULL))
return;
}
@@ -700,7 +704,7 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
case RTE_FUNCTION:
/* Check for parallel-restricted functions. */
- if (!is_parallel_safe(root, (Node *) rte->functions))
+ if (!is_parallel_safe(root, (Node *) rte->functions, NULL))
return;
break;
@@ -710,7 +714,7 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
case RTE_VALUES:
/* Check for parallel-restricted functions. */
- if (!is_parallel_safe(root, (Node *) rte->values_lists))
+ if (!is_parallel_safe(root, (Node *) rte->values_lists, NULL))
return;
break;
@@ -747,18 +751,28 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
* outer join clauses work correctly. It would likely break equivalence
* classes, too.
*/
- if (!is_parallel_safe(root, (Node *) rel->baserestrictinfo))
- return;
+ parallel_safe = is_parallel_safe(root, (Node *) rel->baserestrictinfo,
+ ¶llel_safe_except_in_params);
/*
* Likewise, if the relation's outputs are not parallel-safe, give up.
* (Usually, they're just Vars, but sometimes they're not.)
*/
- if (!is_parallel_safe(root, (Node *) rel->reltarget->exprs))
- return;
+ if (parallel_safe || parallel_safe_except_in_params)
+ {
+ bool target_parallel_safe;
+ bool target_parallel_safe_ignoring_params = false;
+
+ target_parallel_safe = is_parallel_safe(root,
+ (Node *) rel->reltarget->exprs,
+ &target_parallel_safe_ignoring_params);
+ parallel_safe = parallel_safe && target_parallel_safe;
+ parallel_safe_except_in_params = parallel_safe_except_in_params
+ && target_parallel_safe_ignoring_params;
+ }
- /* We have a winner. */
- rel->consider_parallel = true;
+ rel->consider_parallel = parallel_safe;
+ rel->consider_parallel_rechecking_params = parallel_safe_except_in_params;
}
/*
@@ -2277,9 +2291,21 @@ set_subquery_pathlist(PlannerInfo *root, RelOptInfo *rel,
pathkeys, required_outer));
}
+ /*
+ * XXX: As far as I can tell, the only time partial paths exist here
+ * is when we're going to execute multiple partial paths in parallel
+ * under a gather node (instead of executing paths serially under
+ * an append node). That means that the subquery scan path here
+ * is self-contained at this point -- so by definition it can't be
+ * reliant on lateral relids, which means we'll never have to consider
+ * rechecking params here.
+ */
+ Assert(!(rel->consider_parallel_rechecking_params && rel->partial_pathlist && !rel->consider_parallel));
+
/* If outer rel allows parallelism, do same for partial paths. */
if (rel->consider_parallel && bms_is_empty(required_outer))
{
+
/* If consider_parallel is false, there should be no partial paths. */
Assert(sub_final_rel->consider_parallel ||
sub_final_rel->partial_pathlist == NIL);
@@ -2633,7 +2659,7 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
cheapest_partial_path->rows * cheapest_partial_path->parallel_workers;
simple_gather_path = (Path *)
create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
- NULL, rowsp);
+ rel->lateral_relids, rowsp);
add_path(rel, simple_gather_path);
/*
@@ -2650,7 +2676,7 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
rows = subpath->rows * subpath->parallel_workers;
path = create_gather_merge_path(root, rel, subpath, rel->reltarget,
- subpath->pathkeys, NULL, rowsp);
+ subpath->pathkeys, rel->lateral_relids, rowsp);
add_path(rel, &path->path);
}
}
@@ -2752,11 +2778,15 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
double *rowsp = NULL;
List *useful_pathkeys_list = NIL;
Path *cheapest_partial_path = NULL;
+ Relids required_outer = rel->lateral_relids;
/* If there are no partial paths, there's nothing to do here. */
if (rel->partial_pathlist == NIL)
return;
+ if (!bms_is_subset(required_outer, rel->relids))
+ return;
+
/* Should we override the rel's rowcount estimate? */
if (override_rows)
rowsp = &rows;
@@ -2828,7 +2858,7 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
tmp,
rel->reltarget,
tmp->pathkeys,
- NULL,
+ required_outer,
rowsp);
add_path(rel, &path->path);
@@ -2862,7 +2892,7 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
tmp,
rel->reltarget,
tmp->pathkeys,
- NULL,
+ required_outer,
rowsp);
add_path(rel, &path->path);
@@ -3041,7 +3071,8 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
/*
* Except for the topmost scan/join rel, consider gathering
* partial paths. We'll do the same for the topmost scan/join rel
- * once we know the final targetlist (see grouping_planner).
+ * once we know the final targetlist (see
+ * apply_scanjoin_target_to_paths).
*/
if (lev < levels_needed)
generate_useful_gather_paths(root, rel, false);
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 6f1abbe47d..4c176d2d49 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -897,7 +897,7 @@ find_computable_ec_member(PlannerInfo *root,
* check this last because it's a rather expensive test.
*/
if (require_parallel_safe &&
- !is_parallel_safe(root, (Node *) em->em_expr))
+ !is_parallel_safe(root, (Node *) em->em_expr, NULL))
continue;
return em; /* found usable expression */
@@ -1012,7 +1012,7 @@ relation_can_be_sorted_early(PlannerInfo *root, RelOptInfo *rel,
* check this last because it's a rather expensive test.
*/
if (require_parallel_safe &&
- !is_parallel_safe(root, (Node *) em->em_expr))
+ !is_parallel_safe(root, (Node *) em->em_expr, NULL))
continue;
return true;
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
index 0e4e00eaf0..a094626248 100644
--- a/src/backend/optimizer/path/indxpath.c
+++ b/src/backend/optimizer/path/indxpath.c
@@ -1047,6 +1047,17 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
/*
* If appropriate, consider parallel index scan. We don't allow
* parallel index scan for bitmap index scans.
+ *
+ * XXX: Checking rel->consider_parallel_rechecking_params here resulted
+ * in some odd behavior on:
+ * select (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1) from tenk1 t;
+ * where the total cost on the chosen plan went *up* considering
+ * the extra path.
+ *
+ * Current working theory is that this method is about base relation
+ * scans, and we only want parameterized paths to be parallelized as
+ * companions to existing parallel plans and so don't really care to
+ * consider a separate parallel index scan here.
*/
if (index->amcanparallel &&
rel->consider_parallel && outer_relids == NULL &&
@@ -1100,6 +1111,7 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
result = lappend(result, ipath);
/* If appropriate, consider parallel index scan */
+ /* XXX: As above here for rel->consider_parallel_rechecking_params? */
if (index->amcanparallel &&
rel->consider_parallel && outer_relids == NULL &&
scantype != ST_BITMAPSCAN)
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index 6407ede12a..f8daa7b265 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -722,6 +722,7 @@ try_partial_nestloop_path(PlannerInfo *root,
else
outerrelids = outerrel->relids;
+ /* TODO: recheck parallel safety here? */
if (!bms_is_subset(inner_paramrels, outerrelids))
return;
}
@@ -1746,16 +1747,24 @@ match_unsorted_outer(PlannerInfo *root,
* partial path and the joinrel is parallel-safe. However, we can't
* handle JOIN_UNIQUE_OUTER, because the outer path will be partial, and
* therefore we won't be able to properly guarantee uniqueness. Nor can
- * we handle joins needing lateral rels, since partial paths must not be
- * parameterized. Similarly, we can't handle JOIN_FULL and JOIN_RIGHT,
- * because they can produce false null extended rows.
+ * we handle JOIN_FULL and JOIN_RIGHT, because they can produce false null
+ * extended rows.
+ *
+ * Partial paths may only have parameters in limited cases
+ * where the parameterization is fully satisfied without sharing state
+ * between workers, so we only allow lateral rels on inputs to the join
+ * if the resulting join contains no lateral rels, the inner rel's laterals
+ * are fully satisfied by the outer rel, and the outer rel doesn't depend
+ * on the inner rel to produce any laterals.
*/
if (joinrel->consider_parallel &&
save_jointype != JOIN_UNIQUE_OUTER &&
save_jointype != JOIN_FULL &&
save_jointype != JOIN_RIGHT &&
outerrel->partial_pathlist != NIL &&
- bms_is_empty(joinrel->lateral_relids))
+ bms_is_empty(joinrel->lateral_relids) &&
+ bms_is_subset(innerrel->lateral_relids, outerrel->relids) &&
+ (bms_is_empty(outerrel->lateral_relids) || !bms_is_subset(outerrel->lateral_relids, innerrel->relids)))
{
if (nestjoinOK)
consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
@@ -1870,7 +1879,9 @@ consider_parallel_nestloop(PlannerInfo *root,
Path *mpath;
/* Can't join to an inner path that is not parallel-safe */
- if (!innerpath->parallel_safe)
+ /* TODO: recheck parallel safety of params here? */
+ if (!innerpath->parallel_safe &&
+ !(innerpath->parallel_safe_ignoring_params))
continue;
/*
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index a5f6d678cc..e84834427b 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -5273,6 +5273,7 @@ copy_generic_path_info(Plan *dest, Path *src)
dest->plan_width = src->pathtarget->width;
dest->parallel_aware = src->parallel_aware;
dest->parallel_safe = src->parallel_safe;
+ dest->parallel_safe_ignoring_params = src->parallel_safe_ignoring_params;
}
/*
@@ -5290,6 +5291,7 @@ copy_plan_costsize(Plan *dest, Plan *src)
dest->parallel_aware = false;
/* Assume the inserted node is parallel-safe, if child plan is. */
dest->parallel_safe = src->parallel_safe;
+ dest->parallel_safe_ignoring_params = src->parallel_safe_ignoring_params;
}
/*
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index 273ac0acf7..bdbce2b87d 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -119,7 +119,7 @@ query_planner(PlannerInfo *root,
if (root->glob->parallelModeOK &&
force_parallel_mode != FORCE_PARALLEL_OFF)
final_rel->consider_parallel =
- is_parallel_safe(root, parse->jointree->quals);
+ is_parallel_safe(root, parse->jointree->quals, NULL);
/*
* The only path for it is a trivial Result path. We cheat a
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 1e42d75465..008a096ee7 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -150,6 +150,7 @@ static RelOptInfo *create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
PathTarget *target,
bool target_parallel_safe,
+ bool target_parallel_safe_ignoring_params,
grouping_sets_data *gd);
static bool is_degenerate_grouping(PlannerInfo *root);
static void create_degenerate_grouping_paths(PlannerInfo *root,
@@ -157,6 +158,7 @@ static void create_degenerate_grouping_paths(PlannerInfo *root,
RelOptInfo *grouped_rel);
static RelOptInfo *make_grouping_rel(PlannerInfo *root, RelOptInfo *input_rel,
PathTarget *target, bool target_parallel_safe,
+ bool target_parallel_safe_ignoring_params,
Node *havingQual);
static void create_ordinary_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
@@ -237,6 +239,7 @@ static void apply_scanjoin_target_to_paths(PlannerInfo *root,
List *scanjoin_targets,
List *scanjoin_targets_contain_srfs,
bool scanjoin_target_parallel_safe,
+ bool scanjoin_target_parallel_safe_ignoring_params,
bool tlist_same_exprs);
static void create_partitionwise_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
@@ -1241,6 +1244,7 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
List *final_targets;
List *final_targets_contain_srfs;
bool final_target_parallel_safe;
+ bool final_target_parallel_safe_ignoring_params = false;
RelOptInfo *current_rel;
RelOptInfo *final_rel;
FinalPathExtraData extra;
@@ -1303,7 +1307,7 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
/* And check whether it's parallel safe */
final_target_parallel_safe =
- is_parallel_safe(root, (Node *) final_target->exprs);
+ is_parallel_safe(root, (Node *) final_target->exprs, &final_target_parallel_safe_ignoring_params);
/* The setop result tlist couldn't contain any SRFs */
Assert(!parse->hasTargetSRFs);
@@ -1337,14 +1341,17 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
List *sort_input_targets;
List *sort_input_targets_contain_srfs;
bool sort_input_target_parallel_safe;
+ bool sort_input_target_parallel_safe_ignoring_params = false;
PathTarget *grouping_target;
List *grouping_targets;
List *grouping_targets_contain_srfs;
bool grouping_target_parallel_safe;
+ bool grouping_target_parallel_safe_ignoring_params = false;
PathTarget *scanjoin_target;
List *scanjoin_targets;
List *scanjoin_targets_contain_srfs;
bool scanjoin_target_parallel_safe;
+ bool scanjoin_target_parallel_safe_ignoring_params = false;
bool scanjoin_target_same_exprs;
bool have_grouping;
WindowFuncLists *wflists = NULL;
@@ -1457,7 +1464,7 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
*/
final_target = create_pathtarget(root, root->processed_tlist);
final_target_parallel_safe =
- is_parallel_safe(root, (Node *) final_target->exprs);
+ is_parallel_safe(root, (Node *) final_target->exprs, &final_target_parallel_safe_ignoring_params);
/*
* If ORDER BY was given, consider whether we should use a post-sort
@@ -1470,12 +1477,13 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
final_target,
&have_postponed_srfs);
sort_input_target_parallel_safe =
- is_parallel_safe(root, (Node *) sort_input_target->exprs);
+ is_parallel_safe(root, (Node *) sort_input_target->exprs, &sort_input_target_parallel_safe_ignoring_params);
}
else
{
sort_input_target = final_target;
sort_input_target_parallel_safe = final_target_parallel_safe;
+ sort_input_target_parallel_safe_ignoring_params = final_target_parallel_safe_ignoring_params;
}
/*
@@ -1489,12 +1497,13 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
final_target,
activeWindows);
grouping_target_parallel_safe =
- is_parallel_safe(root, (Node *) grouping_target->exprs);
+ is_parallel_safe(root, (Node *) grouping_target->exprs, &grouping_target_parallel_safe_ignoring_params);
}
else
{
grouping_target = sort_input_target;
grouping_target_parallel_safe = sort_input_target_parallel_safe;
+ grouping_target_parallel_safe_ignoring_params = sort_input_target_parallel_safe_ignoring_params;
}
/*
@@ -1508,12 +1517,13 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
{
scanjoin_target = make_group_input_target(root, final_target);
scanjoin_target_parallel_safe =
- is_parallel_safe(root, (Node *) scanjoin_target->exprs);
+ is_parallel_safe(root, (Node *) scanjoin_target->exprs, &scanjoin_target_parallel_safe_ignoring_params);
}
else
{
scanjoin_target = grouping_target;
scanjoin_target_parallel_safe = grouping_target_parallel_safe;
+ scanjoin_target_parallel_safe_ignoring_params = grouping_target_parallel_safe_ignoring_params;
}
/*
@@ -1565,6 +1575,7 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
apply_scanjoin_target_to_paths(root, current_rel, scanjoin_targets,
scanjoin_targets_contain_srfs,
scanjoin_target_parallel_safe,
+ scanjoin_target_parallel_safe_ignoring_params,
scanjoin_target_same_exprs);
/*
@@ -1592,6 +1603,7 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
current_rel,
grouping_target,
grouping_target_parallel_safe,
+ grouping_target_parallel_safe_ignoring_params,
gset_data);
/* Fix things up if grouping_target contains SRFs */
if (parse->hasTargetSRFs)
@@ -1665,10 +1677,25 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
* not a SELECT, consider_parallel will be false for every relation in the
* query.
*/
- if (current_rel->consider_parallel &&
- is_parallel_safe(root, parse->limitOffset) &&
- is_parallel_safe(root, parse->limitCount))
- final_rel->consider_parallel = true;
+ if (current_rel->consider_parallel || current_rel->consider_parallel_rechecking_params)
+ {
+ bool limit_count_parallel_safe;
+ bool limit_offset_parallel_safe;
+ bool limit_count_parallel_safe_ignoring_params = false;
+ bool limit_offset_parallel_safe_ignoring_params = false;
+
+ limit_count_parallel_safe = is_parallel_safe(root, parse->limitCount, &limit_count_parallel_safe_ignoring_params);
+ limit_offset_parallel_safe = is_parallel_safe(root, parse->limitOffset, &limit_offset_parallel_safe_ignoring_params);
+
+ if (current_rel->consider_parallel &&
+ limit_count_parallel_safe &&
+ limit_offset_parallel_safe)
+ final_rel->consider_parallel = true;
+ if (current_rel->consider_parallel_rechecking_params &&
+ limit_count_parallel_safe_ignoring_params &&
+ limit_offset_parallel_safe_ignoring_params)
+ final_rel->consider_parallel_rechecking_params = true;
+ }
/*
* If the current_rel belongs to a single FDW, so does the final_rel.
@@ -1869,8 +1896,8 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
* Generate partial paths for final_rel, too, if outer query levels might
* be able to make use of them.
*/
- if (final_rel->consider_parallel && root->query_level > 1 &&
- !limit_needed(parse))
+ if ((final_rel->consider_parallel || final_rel->consider_parallel_rechecking_params) &&
+ root->query_level > 1 && !limit_needed(parse))
{
Assert(!parse->rowMarks && parse->commandType == CMD_SELECT);
foreach(lc, current_rel->partial_pathlist)
@@ -3282,6 +3309,7 @@ create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
PathTarget *target,
bool target_parallel_safe,
+ bool target_parallel_safe_ignoring_params,
grouping_sets_data *gd)
{
Query *parse = root->parse;
@@ -3297,7 +3325,9 @@ create_grouping_paths(PlannerInfo *root,
* aggregation paths.
*/
grouped_rel = make_grouping_rel(root, input_rel, target,
- target_parallel_safe, parse->havingQual);
+ target_parallel_safe,
+ target_parallel_safe_ignoring_params,
+ parse->havingQual);
/*
* Create either paths for a degenerate grouping or paths for ordinary
@@ -3358,6 +3388,7 @@ create_grouping_paths(PlannerInfo *root,
extra.flags = flags;
extra.target_parallel_safe = target_parallel_safe;
+ extra.target_parallel_safe_ignoring_params = target_parallel_safe_ignoring_params;
extra.havingQual = parse->havingQual;
extra.targetList = parse->targetList;
extra.partial_costs_set = false;
@@ -3393,7 +3424,7 @@ create_grouping_paths(PlannerInfo *root,
static RelOptInfo *
make_grouping_rel(PlannerInfo *root, RelOptInfo *input_rel,
PathTarget *target, bool target_parallel_safe,
- Node *havingQual)
+ bool target_parallel_safe_ignoring_params, Node *havingQual)
{
RelOptInfo *grouped_rel;
@@ -3421,9 +3452,21 @@ make_grouping_rel(PlannerInfo *root, RelOptInfo *input_rel,
* can't be parallel-safe, either. Otherwise, it's parallel-safe if the
* target list and HAVING quals are parallel-safe.
*/
- if (input_rel->consider_parallel && target_parallel_safe &&
- is_parallel_safe(root, (Node *) havingQual))
- grouped_rel->consider_parallel = true;
+ if ((input_rel->consider_parallel || input_rel->consider_parallel_rechecking_params)
+ && (target_parallel_safe || target_parallel_safe_ignoring_params))
+ {
+ bool having_qual_parallel_safe;
+ bool having_qual_parallel_safe_ignoring_params = false;
+
+ having_qual_parallel_safe = is_parallel_safe(root, (Node *) havingQual,
+ &having_qual_parallel_safe_ignoring_params);
+
+ grouped_rel->consider_parallel = input_rel->consider_parallel &&
+ having_qual_parallel_safe && target_parallel_safe;
+ grouped_rel->consider_parallel_rechecking_params =
+ input_rel->consider_parallel_rechecking_params &&
+ having_qual_parallel_safe_ignoring_params && target_parallel_safe_ignoring_params;
+ }
/*
* If the input rel belongs to a single FDW, so does the grouped rel.
@@ -4050,7 +4093,7 @@ create_window_paths(PlannerInfo *root,
* target list and active windows for non-parallel-safe constructs.
*/
if (input_rel->consider_parallel && output_target_parallel_safe &&
- is_parallel_safe(root, (Node *) activeWindows))
+ is_parallel_safe(root, (Node *) activeWindows, NULL))
window_rel->consider_parallel = true;
/*
@@ -5755,6 +5798,7 @@ adjust_paths_for_srfs(PlannerInfo *root, RelOptInfo *rel,
PathTarget *thistarget = lfirst_node(PathTarget, lc1);
bool contains_srfs = (bool) lfirst_int(lc2);
+ /* TODO: How do we know the new target is parallel safe? */
/* If this level doesn't contain SRFs, do regular projection */
if (contains_srfs)
newpath = (Path *) create_set_projection_path(root,
@@ -6071,8 +6115,8 @@ plan_create_index_workers(Oid tableOid, Oid indexOid)
* safe.
*/
if (heap->rd_rel->relpersistence == RELPERSISTENCE_TEMP ||
- !is_parallel_safe(root, (Node *) RelationGetIndexExpressions(index)) ||
- !is_parallel_safe(root, (Node *) RelationGetIndexPredicate(index)))
+ !is_parallel_safe(root, (Node *) RelationGetIndexExpressions(index), NULL) ||
+ !is_parallel_safe(root, (Node *) RelationGetIndexPredicate(index), NULL))
{
parallel_workers = 0;
goto done;
@@ -6535,6 +6579,8 @@ create_partial_grouping_paths(PlannerInfo *root,
grouped_rel->relids);
partially_grouped_rel->consider_parallel =
grouped_rel->consider_parallel;
+ partially_grouped_rel->consider_parallel_rechecking_params =
+ grouped_rel->consider_parallel_rechecking_params;
partially_grouped_rel->reloptkind = grouped_rel->reloptkind;
partially_grouped_rel->serverid = grouped_rel->serverid;
partially_grouped_rel->userid = grouped_rel->userid;
@@ -6998,6 +7044,7 @@ apply_scanjoin_target_to_paths(PlannerInfo *root,
List *scanjoin_targets,
List *scanjoin_targets_contain_srfs,
bool scanjoin_target_parallel_safe,
+ bool scanjoin_target_parallel_safe_ignoring_params,
bool tlist_same_exprs)
{
bool rel_is_partitioned = IS_PARTITIONED_REL(rel);
@@ -7007,6 +7054,11 @@ apply_scanjoin_target_to_paths(PlannerInfo *root,
/* This recurses, so be paranoid. */
check_stack_depth();
+ /*
+ * TOOD: when/how do we want to generate gather paths if
+ * scanjoin_target_parallel_safe_ignoring_params = true
+ */
+
/*
* If the rel is partitioned, we want to drop its existing paths and
* generate new ones. This function would still be correct if we kept the
@@ -7047,6 +7099,67 @@ apply_scanjoin_target_to_paths(PlannerInfo *root,
generate_useful_gather_paths(root, rel, false);
/* Can't use parallel query above this level. */
+
+ /*
+ * There are cases where:
+ * (rel->consider_parallel &&
+ * !scanjoin_target_parallel_safe &&
+ * scanjoin_target_parallel_safe_ignoring_params)
+ * is true at this point. See longer commment below.
+ */
+ if (!(rel->consider_parallel_rechecking_params && scanjoin_target_parallel_safe_ignoring_params))
+ {
+ /*
+ * TODO: if we limit setting:
+ *
+ * rel->consider_parallel_rechecking_params = false
+ * rel->partial_pathlist = NIL
+ *
+ * to this condition, we're pushing off the checks as to whether or
+ * not a given param usage is safe in the context of a given path
+ * (in the context of a given rel?). That almost certainly means
+ * we'd have to add other checks later (maybe just on
+ * lateral/relids and not parallel safety overall), because at the
+ * end of grouping_planner() we copy partial paths to the
+ * final_rel, and while that path may be acceptable in some
+ * contexts it may not be in all contexts.
+ *
+ * OTOH if we're only dependent on PARAM_EXEC params, and we already
+ * know that subpath->param_info == NULL holds (and that seems like
+ * it must since we were going to replace the path target anyway...
+ * though the one caveat is from the original form of this function
+ * we'd only ever actually assert that for paths not partial paths)
+ * then if a param shows up in the target why would it not be parallel
+ * safe.
+ *
+ * Adding to the mystery even with the original form of this function
+ * we still end up with parallel paths where I'd expect this to
+ * disallow them. For example:
+ *
+ * SELECT '' AS six, f1 AS "Correlated Field", f3 AS "Second Field"
+ * FROM SUBSELECT_TBL upper
+ * WHERE f3 IN (
+ * SELECT upper.f1 + f2
+ * FROM SUBSELECT_TBL
+ * WHERE f2 = CAST(f3 AS integer)
+ * );
+ *
+ * ends up with the correlated query underneath parallel plan despite
+ * its target containing a param, and therefore this function marking
+ * the rel as consider_parallel=false and removing the partial paths.
+ *
+ * But the plan as a whole is parallel safe, and so the subplan is also
+ * parallel safe, which means we can incorporate it into a full parallel
+ * plan. In other words, this is a parallel safe, but not parallel aware
+ * subplan (and regular, not parallel, seq scan inside that subplan).
+ * It's not a partial path; it's a full path that is executed as a subquery.
+ *
+ * Current conclusion: it's fine for subplans, which is the case we're
+ * currently targeting anyway. And it might even be the only case that
+ * matters at all.
+ */
+ }
+ rel->consider_parallel_rechecking_params = false;
rel->partial_pathlist = NIL;
rel->consider_parallel = false;
}
@@ -7182,6 +7295,7 @@ apply_scanjoin_target_to_paths(PlannerInfo *root,
child_scanjoin_targets,
scanjoin_targets_contain_srfs,
scanjoin_target_parallel_safe,
+ scanjoin_target_parallel_safe_ignoring_params,
tlist_same_exprs);
/* Save non-dummy children for Append paths. */
@@ -7199,6 +7313,11 @@ apply_scanjoin_target_to_paths(PlannerInfo *root,
* avoid creating multiple Gather nodes within the same plan. We must do
* this after all paths have been generated and before set_cheapest, since
* one of the generated paths may turn out to be the cheapest one.
+ *
+ * TODO: This is the same problem as earlier in this function: when allowing
+ * "parallel safe ignoring params" paths here we don't actually know we are
+ * safe in any possible context just possibly safe in the context of the
+ * right rel.
*/
if (rel->consider_parallel && !IS_OTHER_REL(rel))
generate_useful_gather_paths(root, rel, false);
@@ -7306,6 +7425,7 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
child_grouped_rel = make_grouping_rel(root, child_input_rel,
child_target,
extra->target_parallel_safe,
+ extra->target_parallel_safe_ignoring_params,
child_extra.havingQual);
/* Create grouping paths for this child relation. */
diff --git a/src/backend/optimizer/plan/subselect.c b/src/backend/optimizer/plan/subselect.c
index c9f7a09d10..393db3b42d 100644
--- a/src/backend/optimizer/plan/subselect.c
+++ b/src/backend/optimizer/plan/subselect.c
@@ -342,6 +342,7 @@ build_subplan(PlannerInfo *root, Plan *plan, PlannerInfo *subroot,
splan->useHashTable = false;
splan->unknownEqFalse = unknownEqFalse;
splan->parallel_safe = plan->parallel_safe;
+ splan->parallel_safe_ignoring_params = plan->parallel_safe_ignoring_params;
splan->setParam = NIL;
splan->parParam = NIL;
splan->args = NIL;
@@ -1937,6 +1938,7 @@ process_sublinks_mutator(Node *node, process_sublinks_context *context)
{
SubLink *sublink = (SubLink *) node;
Node *testexpr;
+ Node *result;
/*
* First, recursively process the lefthand-side expressions, if any.
@@ -1948,12 +1950,29 @@ process_sublinks_mutator(Node *node, process_sublinks_context *context)
/*
* Now build the SubPlan node and make the expr to return.
*/
- return make_subplan(context->root,
+ result = make_subplan(context->root,
(Query *) sublink->subselect,
sublink->subLinkType,
sublink->subLinkId,
testexpr,
context->isTopQual);
+
+ /*
+ * If planning determined that a subpath was parallel safe as long
+ * as required params are provided by each individual worker then we
+ * can mark the resulting subplan actually parallel safe since we now
+ * know for certain how that path will be used.
+ */
+ if (IsA(result, SubPlan) && !((SubPlan*)result)->parallel_safe
+ && ((SubPlan*)result)->parallel_safe_ignoring_params
+ && enable_parallel_params_recheck)
+ {
+ Plan *subplan = planner_subplan_get_plan(context->root, (SubPlan*)result);
+ ((SubPlan*)result)->parallel_safe = is_parallel_safe(context->root, testexpr, NULL);
+ subplan->parallel_safe = ((SubPlan*)result)->parallel_safe;
+ }
+
+ return result;
}
/*
diff --git a/src/backend/optimizer/prep/prepunion.c b/src/backend/optimizer/prep/prepunion.c
index e9256a2d4d..16cb35c914 100644
--- a/src/backend/optimizer/prep/prepunion.c
+++ b/src/backend/optimizer/prep/prepunion.c
@@ -409,6 +409,7 @@ recurse_set_operations(Node *setOp, PlannerInfo *root,
Assert(subpath->param_info == NULL);
/* avoid apply_projection_to_path, in case of multiple refs */
+ /* TODO: how to we know the target is parallel safe? */
path = (Path *) create_projection_path(root, subpath->parent,
subpath, target);
lfirst(lc) = path;
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 3412d31117..e2cf335f4a 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -55,6 +55,15 @@
#include "utils/syscache.h"
#include "utils/typcache.h"
+bool enable_parallel_params_recheck = true;
+
+typedef struct
+{
+ PlannerInfo *root;
+ AggSplit aggsplit;
+ AggClauseCosts *costs;
+} get_agg_clause_costs_context;
+
typedef struct
{
ParamListInfo boundParams;
@@ -88,6 +97,8 @@ typedef struct
{
char max_hazard; /* worst proparallel hazard found so far */
char max_interesting; /* worst proparallel hazard of interest */
+ char max_hazard_ignoring_params;
+ bool check_params_independently;
List *safe_param_ids; /* PARAM_EXEC Param IDs to treat as safe */
} max_parallel_hazard_context;
@@ -624,19 +635,27 @@ max_parallel_hazard(Query *parse)
context.max_hazard = PROPARALLEL_SAFE;
context.max_interesting = PROPARALLEL_UNSAFE;
context.safe_param_ids = NIL;
+ context.check_params_independently = false;
(void) max_parallel_hazard_walker((Node *) parse, &context);
return context.max_hazard;
}
/*
* is_parallel_safe
- * Detect whether the given expr contains only parallel-safe functions
+ * Detect whether the given expr contains only parallel safe funcions
+ * XXX: It does more than functions?
+ *
+ * If provided, safe_ignoring_params will be set the result of running the same
+ * parallel safety checks with the exception that params will be allowed. This
+ * value is useful since params are not inherently parallel unsafe, but rather
+ * their usage context (whether or not the worker is able to provide the value)
+ * determines parallel safety.
*
* root->glob->maxParallelHazard must previously have been set to the
- * result of max_parallel_hazard() on the whole query.
+ * result of max_parallel_hazard() on the whole query
*/
bool
-is_parallel_safe(PlannerInfo *root, Node *node)
+is_parallel_safe(PlannerInfo *root, Node *node, bool *safe_ignoring_params)
{
max_parallel_hazard_context context;
PlannerInfo *proot;
@@ -653,8 +672,10 @@ is_parallel_safe(PlannerInfo *root, Node *node)
return true;
/* Else use max_parallel_hazard's search logic, but stop on RESTRICTED */
context.max_hazard = PROPARALLEL_SAFE;
+ context.max_hazard_ignoring_params = PROPARALLEL_SAFE;
context.max_interesting = PROPARALLEL_RESTRICTED;
context.safe_param_ids = NIL;
+ context.check_params_independently = safe_ignoring_params != NULL;
/*
* The params that refer to the same or parent query level are considered
@@ -672,12 +693,17 @@ is_parallel_safe(PlannerInfo *root, Node *node)
}
}
- return !max_parallel_hazard_walker(node, &context);
+ (void) max_parallel_hazard_walker(node, &context);
+
+ if (safe_ignoring_params)
+ *safe_ignoring_params = context.max_hazard_ignoring_params == PROPARALLEL_SAFE;
+
+ return context.max_hazard == PROPARALLEL_SAFE;
}
/* core logic for all parallel-hazard checks */
static bool
-max_parallel_hazard_test(char proparallel, max_parallel_hazard_context *context)
+max_parallel_hazard_test(char proparallel, max_parallel_hazard_context *context, bool from_param)
{
switch (proparallel)
{
@@ -688,12 +714,16 @@ max_parallel_hazard_test(char proparallel, max_parallel_hazard_context *context)
/* increase max_hazard to RESTRICTED */
Assert(context->max_hazard != PROPARALLEL_UNSAFE);
context->max_hazard = proparallel;
+ if (!from_param)
+ context->max_hazard_ignoring_params = proparallel;
/* done if we are not expecting any unsafe functions */
if (context->max_interesting == proparallel)
return true;
break;
case PROPARALLEL_UNSAFE:
context->max_hazard = proparallel;
+ if (!from_param)
+ context->max_hazard_ignoring_params = proparallel;
/* we're always done at the first unsafe construct */
return true;
default:
@@ -708,7 +738,41 @@ static bool
max_parallel_hazard_checker(Oid func_id, void *context)
{
return max_parallel_hazard_test(func_parallel(func_id),
- (max_parallel_hazard_context *) context);
+ (max_parallel_hazard_context *) context, false);
+}
+
+static bool
+max_parallel_hazard_walker_can_short_circuit(max_parallel_hazard_context *context)
+{
+ if (!context->check_params_independently)
+ return true;
+
+ switch (context->max_hazard)
+ {
+ case PROPARALLEL_SAFE:
+ /* nothing to see here, move along */
+ break;
+ case PROPARALLEL_RESTRICTED:
+ if (context->max_interesting == PROPARALLEL_RESTRICTED)
+ return context->max_hazard_ignoring_params != PROPARALLEL_SAFE;
+
+ /*
+ * We haven't even met our max interesting yet, so
+ * we certainly can't short-circuit.
+ */
+ break;
+ case PROPARALLEL_UNSAFE:
+ if (context->max_interesting == PROPARALLEL_RESTRICTED)
+ return context->max_hazard_ignoring_params != PROPARALLEL_SAFE;
+ else if (context->max_interesting == PROPARALLEL_UNSAFE)
+ return context->max_hazard_ignoring_params == PROPARALLEL_UNSAFE;
+
+ break;
+ default:
+ elog(ERROR, "unrecognized proparallel value \"%c\"", context->max_hazard);
+ break;
+ }
+ return false;
}
static bool
@@ -720,7 +784,7 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
/* Check for hazardous functions in node itself */
if (check_functions_in_node(node, max_parallel_hazard_checker,
context))
- return true;
+ return max_parallel_hazard_walker_can_short_circuit(context);
/*
* It should be OK to treat MinMaxExpr as parallel-safe, since btree
@@ -735,14 +799,14 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
*/
if (IsA(node, CoerceToDomain))
{
- if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
- return true;
+ if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context, false))
+ return max_parallel_hazard_walker_can_short_circuit(context);
}
else if (IsA(node, NextValueExpr))
{
- if (max_parallel_hazard_test(PROPARALLEL_UNSAFE, context))
- return true;
+ if (max_parallel_hazard_test(PROPARALLEL_UNSAFE, context, false))
+ return max_parallel_hazard_walker_can_short_circuit(context);
}
/*
@@ -755,8 +819,8 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
*/
else if (IsA(node, WindowFunc))
{
- if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
- return true;
+ if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context, false))
+ return max_parallel_hazard_walker_can_short_circuit(context);
}
/*
@@ -775,8 +839,8 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
*/
else if (IsA(node, SubLink))
{
- if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
- return true;
+ if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context, false))
+ return max_parallel_hazard_walker_can_short_circuit(context);
}
/*
@@ -791,18 +855,23 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
List *save_safe_param_ids;
if (!subplan->parallel_safe &&
- max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
+ (!enable_parallel_params_recheck || !subplan->parallel_safe_ignoring_params) &&
+ max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context, false) &&
+ max_parallel_hazard_walker_can_short_circuit(context))
return true;
save_safe_param_ids = context->safe_param_ids;
context->safe_param_ids = list_concat_copy(context->safe_param_ids,
subplan->paramIds);
- if (max_parallel_hazard_walker(subplan->testexpr, context))
- return true; /* no need to restore safe_param_ids */
+ if (max_parallel_hazard_walker(subplan->testexpr, context) &&
+ max_parallel_hazard_walker_can_short_circuit(context))
+ /* no need to restore safe_param_ids */
+ return true;
+
list_free(context->safe_param_ids);
context->safe_param_ids = save_safe_param_ids;
/* we must also check args, but no special Param treatment there */
if (max_parallel_hazard_walker((Node *) subplan->args, context))
- return true;
+ return max_parallel_hazard_walker_can_short_circuit(context);
/* don't want to recurse normally, so we're done */
return false;
}
@@ -824,8 +893,8 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
if (param->paramkind != PARAM_EXEC ||
!list_member_int(context->safe_param_ids, param->paramid))
{
- if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
- return true;
+ if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context, true))
+ return max_parallel_hazard_walker_can_short_circuit(context);
}
return false; /* nothing to recurse to */
}
@@ -843,7 +912,7 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
if (query->rowMarks != NULL)
{
context->max_hazard = PROPARALLEL_UNSAFE;
- return true;
+ return max_parallel_hazard_walker_can_short_circuit(context);
}
/* Recurse into subselects */
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index a53850b370..21be3824c1 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -756,10 +756,10 @@ add_partial_path(RelOptInfo *parent_rel, Path *new_path)
CHECK_FOR_INTERRUPTS();
/* Path to be added must be parallel safe. */
- Assert(new_path->parallel_safe);
+ Assert(new_path->parallel_safe || new_path->parallel_safe_ignoring_params);
/* Relation should be OK for parallelism, too. */
- Assert(parent_rel->consider_parallel);
+ Assert(parent_rel->consider_parallel || parent_rel->consider_parallel_rechecking_params);
/*
* As in add_path, throw out any paths which are dominated by the new
@@ -938,6 +938,7 @@ create_seqscan_path(PlannerInfo *root, RelOptInfo *rel,
required_outer);
pathnode->parallel_aware = parallel_workers > 0 ? true : false;
pathnode->parallel_safe = rel->consider_parallel;
+ pathnode->parallel_safe_ignoring_params = rel->consider_parallel_rechecking_params;
pathnode->parallel_workers = parallel_workers;
pathnode->pathkeys = NIL; /* seqscan has unordered result */
@@ -1016,6 +1017,7 @@ create_index_path(PlannerInfo *root,
required_outer);
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel;
+ pathnode->path.parallel_safe_ignoring_params = rel->consider_parallel_rechecking_params;
pathnode->path.parallel_workers = 0;
pathnode->path.pathkeys = pathkeys;
@@ -1865,7 +1867,7 @@ create_gather_merge_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
Cost input_startup_cost = 0;
Cost input_total_cost = 0;
- Assert(subpath->parallel_safe);
+ Assert(subpath->parallel_safe || subpath->parallel_safe_ignoring_params);
Assert(pathkeys);
pathnode->path.pathtype = T_GatherMerge;
@@ -1953,7 +1955,7 @@ create_gather_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
{
GatherPath *pathnode = makeNode(GatherPath);
- Assert(subpath->parallel_safe);
+ Assert(subpath->parallel_safe || subpath->parallel_safe_ignoring_params);
pathnode->path.pathtype = T_Gather;
pathnode->path.parent = rel;
@@ -2000,6 +2002,8 @@ create_subqueryscan_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe;
+ pathnode->path.parallel_safe_ignoring_params = rel->consider_parallel_rechecking_params &&
+ subpath->parallel_safe_ignoring_params;
pathnode->path.parallel_workers = subpath->parallel_workers;
pathnode->path.pathkeys = pathkeys;
pathnode->subpath = subpath;
@@ -2417,6 +2421,8 @@ create_nestloop_path(PlannerInfo *root,
NestPath *pathnode = makeNode(NestPath);
Relids inner_req_outer = PATH_REQ_OUTER(inner_path);
+ /* TODO: Assert lateral relids subset safety? */
+
/*
* If the inner path is parameterized by the outer, we must drop any
* restrict_clauses that are due to be moved into the inner path. We have
@@ -2457,6 +2463,8 @@ create_nestloop_path(PlannerInfo *root,
pathnode->jpath.path.parallel_aware = false;
pathnode->jpath.path.parallel_safe = joinrel->consider_parallel &&
outer_path->parallel_safe && inner_path->parallel_safe;
+ pathnode->jpath.path.parallel_safe_ignoring_params = joinrel->consider_parallel_rechecking_params &&
+ outer_path->parallel_safe_ignoring_params && inner_path->parallel_safe_ignoring_params;
/* This is a foolish way to estimate parallel_workers, but for now... */
pathnode->jpath.path.parallel_workers = outer_path->parallel_workers;
pathnode->jpath.path.pathkeys = pathkeys;
@@ -2630,6 +2638,8 @@ create_projection_path(PlannerInfo *root,
{
ProjectionPath *pathnode = makeNode(ProjectionPath);
PathTarget *oldtarget;
+ bool target_parallel_safe;
+ bool target_parallel_safe_ignoring_params = false;
/*
* We mustn't put a ProjectionPath directly above another; it's useless
@@ -2653,9 +2663,12 @@ create_projection_path(PlannerInfo *root,
/* For now, assume we are above any joins, so no parameterization */
pathnode->path.param_info = NULL;
pathnode->path.parallel_aware = false;
+ target_parallel_safe = is_parallel_safe(root, (Node *) target->exprs,
+ &target_parallel_safe_ignoring_params);
pathnode->path.parallel_safe = rel->consider_parallel &&
- subpath->parallel_safe &&
- is_parallel_safe(root, (Node *) target->exprs);
+ subpath->parallel_safe && target_parallel_safe;
+ pathnode->path.parallel_safe_ignoring_params = rel->consider_parallel_rechecking_params &&
+ subpath->parallel_safe_ignoring_params && target_parallel_safe_ignoring_params;
pathnode->path.parallel_workers = subpath->parallel_workers;
/* Projection does not change the sort order */
pathnode->path.pathkeys = subpath->pathkeys;
@@ -2763,7 +2776,7 @@ apply_projection_to_path(PlannerInfo *root,
* parallel-safe in the target expressions, then we can't.
*/
if ((IsA(path, GatherPath) || IsA(path, GatherMergePath)) &&
- is_parallel_safe(root, (Node *) target->exprs))
+ is_parallel_safe(root, (Node *) target->exprs, NULL))
{
/*
* We always use create_projection_path here, even if the subpath is
@@ -2797,7 +2810,7 @@ apply_projection_to_path(PlannerInfo *root,
}
}
else if (path->parallel_safe &&
- !is_parallel_safe(root, (Node *) target->exprs))
+ !is_parallel_safe(root, (Node *) target->exprs, NULL))
{
/*
* We're inserting a parallel-restricted target list into a path
@@ -2805,6 +2818,7 @@ apply_projection_to_path(PlannerInfo *root,
* safe.
*/
path->parallel_safe = false;
+ path->parallel_safe_ignoring_params = false; /* TODO */
}
return path;
@@ -2837,7 +2851,7 @@ create_set_projection_path(PlannerInfo *root,
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe &&
- is_parallel_safe(root, (Node *) target->exprs);
+ is_parallel_safe(root, (Node *) target->exprs, NULL);
pathnode->path.parallel_workers = subpath->parallel_workers;
/* Projection does not change the sort order XXX? */
pathnode->path.pathkeys = subpath->pathkeys;
@@ -3114,6 +3128,8 @@ create_agg_path(PlannerInfo *root,
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe;
+ pathnode->path.parallel_safe_ignoring_params = rel->consider_parallel_rechecking_params &&
+ subpath->parallel_safe_ignoring_params;
pathnode->path.parallel_workers = subpath->parallel_workers;
if (aggstrategy == AGG_SORTED)
pathnode->path.pathkeys = subpath->pathkeys; /* preserves order */
@@ -3735,6 +3751,8 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe;
+ pathnode->path.parallel_safe_ignoring_params = rel->consider_parallel_rechecking_params &&
+ subpath->parallel_safe_ignoring_params;
pathnode->path.parallel_workers = subpath->parallel_workers;
pathnode->path.rows = subpath->rows;
pathnode->path.startup_cost = subpath->startup_cost;
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 47769cea45..9d908d0fea 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -213,6 +213,7 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
rel->consider_startup = (root->tuple_fraction > 0);
rel->consider_param_startup = false; /* might get changed later */
rel->consider_parallel = false; /* might get changed later */
+ rel->consider_parallel_rechecking_params = false; /* might get changed later */
rel->reltarget = create_empty_pathtarget();
rel->pathlist = NIL;
rel->ppilist = NIL;
@@ -617,6 +618,7 @@ build_join_rel(PlannerInfo *root,
joinrel->consider_startup = (root->tuple_fraction > 0);
joinrel->consider_param_startup = false;
joinrel->consider_parallel = false;
+ joinrel->consider_parallel_rechecking_params = false;
joinrel->reltarget = create_empty_pathtarget();
joinrel->pathlist = NIL;
joinrel->ppilist = NIL;
@@ -743,10 +745,27 @@ build_join_rel(PlannerInfo *root,
* take; therefore, we should make the same decision here however we get
* here.
*/
- if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
- is_parallel_safe(root, (Node *) restrictlist) &&
- is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
- joinrel->consider_parallel = true;
+ if ((inner_rel->consider_parallel || inner_rel->consider_parallel_rechecking_params)
+ && (outer_rel->consider_parallel || outer_rel->consider_parallel_rechecking_params))
+ {
+ bool restrictlist_parallel_safe;
+ bool restrictlist_parallel_safe_ignoring_params = false;
+ bool target_parallel_safe;
+ bool target_parallel_safe_ignoring_params = false;
+
+ restrictlist_parallel_safe = is_parallel_safe(root, (Node *) restrictlist, &restrictlist_parallel_safe_ignoring_params);
+ target_parallel_safe = is_parallel_safe(root, (Node *) joinrel->reltarget->exprs, &target_parallel_safe_ignoring_params);
+
+ if (inner_rel->consider_parallel && outer_rel->consider_parallel
+ && restrictlist_parallel_safe && target_parallel_safe)
+ joinrel->consider_parallel = true;
+
+ if (inner_rel->consider_parallel_rechecking_params
+ && outer_rel->consider_parallel_rechecking_params
+ && restrictlist_parallel_safe_ignoring_params
+ && target_parallel_safe_ignoring_params)
+ joinrel->consider_parallel_rechecking_params = true;
+ }
/* Add the joinrel to the PlannerInfo. */
add_join_rel(root, joinrel);
@@ -805,6 +824,7 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->consider_startup = (root->tuple_fraction > 0);
joinrel->consider_param_startup = false;
joinrel->consider_parallel = false;
+ joinrel->consider_parallel_rechecking_params = false;
joinrel->reltarget = create_empty_pathtarget();
joinrel->pathlist = NIL;
joinrel->ppilist = NIL;
@@ -892,6 +912,7 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
/* Child joinrel is parallel safe if parent is parallel safe. */
joinrel->consider_parallel = parent_joinrel->consider_parallel;
+ joinrel->consider_parallel_rechecking_params = parent_joinrel->consider_parallel_rechecking_params;
/* Set estimates of the child-joinrel's size. */
set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
@@ -1236,6 +1257,7 @@ fetch_upper_rel(PlannerInfo *root, UpperRelationKind kind, Relids relids)
upperrel->consider_startup = (root->tuple_fraction > 0);
upperrel->consider_param_startup = false;
upperrel->consider_parallel = false; /* might get changed later */
+ upperrel->consider_parallel_rechecking_params = false; /* might get changed later */
upperrel->reltarget = create_empty_pathtarget();
upperrel->pathlist = NIL;
upperrel->cheapest_startup_path = NULL;
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 467b0fd6fe..034510b611 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -57,6 +57,7 @@
#include "libpq/libpq.h"
#include "libpq/pqformat.h"
#include "miscadmin.h"
+#include "optimizer/clauses.h"
#include "optimizer/cost.h"
#include "optimizer/geqo.h"
#include "optimizer/optimizer.h"
@@ -968,6 +969,16 @@ static const unit_conversion time_unit_conversion_table[] =
static struct config_bool ConfigureNamesBool[] =
{
+ {
+ {"enable_parallel_params_recheck", PGC_USERSET, QUERY_TUNING_METHOD,
+ gettext_noop("Enables the planner's rechecking of parallel safety in the presence of PARAM_EXEC params (for correlated subqueries)."),
+ NULL,
+ GUC_EXPLAIN
+ },
+ &enable_parallel_params_recheck,
+ true,
+ NULL, NULL, NULL
+ },
{
{"enable_seqscan", PGC_USERSET, QUERY_TUNING_METHOD,
gettext_noop("Enables the planner's use of sequential-scan plans."),
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 1abe233db2..f2c3d229b0 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -683,6 +683,7 @@ typedef struct RelOptInfo
bool consider_startup; /* keep cheap-startup-cost paths? */
bool consider_param_startup; /* ditto, for parameterized paths? */
bool consider_parallel; /* consider parallel paths? */
+ bool consider_parallel_rechecking_params; /* consider parallel paths? */
/* default result targetlist for Paths scanning this relation */
struct PathTarget *reltarget; /* list of Vars/Exprs, cost, width */
@@ -1182,6 +1183,7 @@ typedef struct Path
bool parallel_aware; /* engage parallel-aware logic? */
bool parallel_safe; /* OK to use as part of parallel plan? */
+ bool parallel_safe_ignoring_params; /* OK to use as part of parallel plan if worker context provides params? */
int parallel_workers; /* desired # of workers; 0 = not parallel */
/* estimated size/costs for path (see costsize.c for more info) */
@@ -2471,7 +2473,7 @@ typedef struct MinMaxAggInfo
* for conflicting purposes.
*
* In addition, PARAM_EXEC slots are assigned for Params representing outputs
- * from subplans (values that are setParam items for those subplans). These
+ * from subplans (values that are setParam items for those subplans). [TODO: is this true, or only for init plans?] These
* IDs need not be tracked via PlannerParamItems, since we do not need any
* duplicate-elimination nor later processing of the represented expressions.
* Instead, we just record the assignment of the slot number by appending to
@@ -2590,6 +2592,7 @@ typedef struct
/* Data which may differ across partitions. */
bool target_parallel_safe;
+ bool target_parallel_safe_ignoring_params;
Node *havingQual;
List *targetList;
PartitionwiseAggregateType patype;
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index ec9a8b0c81..67805f2cfa 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -128,6 +128,7 @@ typedef struct Plan
*/
bool parallel_aware; /* engage parallel-aware logic? */
bool parallel_safe; /* OK to use as part of parallel plan? */
+ bool parallel_safe_ignoring_params; /* OK to use as part of parallel plan if worker context provides params? */
/*
* information needed for asynchronous execution
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
index c04282f91f..df1e2496c7 100644
--- a/src/include/nodes/primnodes.h
+++ b/src/include/nodes/primnodes.h
@@ -764,6 +764,7 @@ typedef struct SubPlan
* spec result is UNKNOWN; this allows much
* simpler handling of null values */
bool parallel_safe; /* is the subplan parallel-safe? */
+ bool parallel_safe_ignoring_params; /* is the subplan parallel-safe when params are provided by the worker context? */
/* Note: parallel_safe does not consider contents of testexpr or args */
/* Information for passing params into and out of the subselect: */
/* setParam and parParam are lists of integers (param IDs) */
diff --git a/src/include/optimizer/clauses.h b/src/include/optimizer/clauses.h
index 0673887a85..df01be2c61 100644
--- a/src/include/optimizer/clauses.h
+++ b/src/include/optimizer/clauses.h
@@ -16,6 +16,8 @@
#include "nodes/pathnodes.h"
+extern PGDLLIMPORT bool enable_parallel_params_recheck;
+
typedef struct
{
int numWindowFuncs; /* total number of WindowFuncs found */
@@ -33,7 +35,7 @@ extern double expression_returns_set_rows(PlannerInfo *root, Node *clause);
extern bool contain_subplans(Node *clause);
extern char max_parallel_hazard(Query *parse);
-extern bool is_parallel_safe(PlannerInfo *root, Node *node);
+extern bool is_parallel_safe(PlannerInfo *root, Node *node, bool *safe_ignoring_params);
extern bool contain_nonstrict_functions(Node *clause);
extern bool contain_exec_param(Node *clause, List *param_ids);
extern bool contain_leaked_vars(Node *clause);
diff --git a/src/test/regress/expected/incremental_sort.out b/src/test/regress/expected/incremental_sort.out
index 545e301e48..8f9ca05e60 100644
--- a/src/test/regress/expected/incremental_sort.out
+++ b/src/test/regress/expected/incremental_sort.out
@@ -1614,16 +1614,16 @@ from tenk1 t, generate_series(1, 1000);
QUERY PLAN
---------------------------------------------------------------------------------
Unique
- -> Sort
- Sort Key: t.unique1, ((SubPlan 1))
- -> Gather
- Workers Planned: 2
+ -> Gather Merge
+ Workers Planned: 2
+ -> Sort
+ Sort Key: t.unique1, ((SubPlan 1))
-> Nested Loop
-> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
-> Function Scan on generate_series
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on tenk1
- Index Cond: (unique1 = t.unique1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on tenk1
+ Index Cond: (unique1 = t.unique1)
(11 rows)
explain (costs off) select
@@ -1633,16 +1633,16 @@ from tenk1 t, generate_series(1, 1000)
order by 1, 2;
QUERY PLAN
---------------------------------------------------------------------------
- Sort
- Sort Key: t.unique1, ((SubPlan 1))
- -> Gather
- Workers Planned: 2
+ Gather Merge
+ Workers Planned: 2
+ -> Sort
+ Sort Key: t.unique1, ((SubPlan 1))
-> Nested Loop
-> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
-> Function Scan on generate_series
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on tenk1
- Index Cond: (unique1 = t.unique1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on tenk1
+ Index Cond: (unique1 = t.unique1)
(10 rows)
-- Parallel sort but with expression not available until the upper rel.
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 2303f70d6e..253f117d7a 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -311,6 +311,131 @@ select count(*) from tenk1 where (two, four) not in
10000
(1 row)
+-- test parallel plans for queries containing correlated subplans
+-- where the subplan only needs params available from the current
+-- worker's scan.
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t, generate_series(1, 10);
+ QUERY PLAN
+----------------------------------------------------------------------------
+ Gather
+ Output: ((SubPlan 1))
+ Workers Planned: 4
+ -> Nested Loop
+ Output: (SubPlan 1)
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ -> Function Scan on pg_catalog.generate_series
+ Output: generate_series.generate_series
+ Function Call: generate_series(1, 10)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(14 rows)
+
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t;
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: ((SubPlan 1))
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(9 rows)
+
+explain (analyze, costs off, summary off, verbose, timing off) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t
+ limit 1;
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------
+ Limit (actual rows=1 loops=1)
+ Output: ((SubPlan 1))
+ -> Gather (actual rows=1 loops=1)
+ Output: ((SubPlan 1))
+ Workers Planned: 4
+ Workers Launched: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t (actual rows=1 loops=5)
+ Output: (SubPlan 1)
+ Heap Fetches: 0
+ Worker 0: actual rows=1 loops=1
+ Worker 1: actual rows=1 loops=1
+ Worker 2: actual rows=1 loops=1
+ Worker 3: actual rows=1 loops=1
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1 (actual rows=1 loops=5)
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+ Heap Fetches: 0
+ Worker 0: actual rows=1 loops=1
+ Worker 1: actual rows=1 loops=1
+ Worker 2: actual rows=1 loops=1
+ Worker 3: actual rows=1 loops=1
+(22 rows)
+
+explain (costs off, verbose) select t.unique1
+ from tenk1 t
+ where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: t.unique1
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ Filter: (t.unique1 = (SubPlan 1))
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(10 rows)
+
+explain (costs off, verbose) select *
+ from tenk1 t
+ order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Gather Merge
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
+ Workers Planned: 4
+ -> Sort
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
+ Sort Key: ((SubPlan 1))
+ -> Parallel Seq Scan on public.tenk1 t
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(12 rows)
+
+-- test subplan in join/lateral join
+explain (costs off, verbose, timing off) select t.unique1, l.*
+ from tenk1 t
+ join lateral (
+ select (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1 offset 0)
+ ) l on true;
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: t.unique1, ((SubPlan 1))
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1, (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(9 rows)
+
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
explain (costs off)
select * from tenk1 where (unique1 + random())::integer not in
@@ -1192,6 +1317,18 @@ EXECUTE pstmt('1', make_some_array(1,2));
DEALLOCATE pstmt;
-- test interaction between subquery and partial_paths
+-- this plan changes to using a non-parallel index only
+-- scan on tenk1_unique1 (the parallel version of the subquery scan
+-- is cheaper, but only by ~30, and cost comparison treats them as equal
+-- since the costs are so large) because set_rel_consider_parallel
+-- called from make_one_rel sees the subplan as parallel safe now
+-- (in context it now knows the params are actually parallel safe).
+-- Because of that the non-parallel index path is now parallel_safe=true,
+-- therefore it wins the COSTS_EQUAL comparison in add_path.
+-- Perhaps any is_parallel_safe calls made for the purpose of determining
+-- consider_parallel should disable that behavior? It's not clear which is
+-- correct.
+set enable_parallel_params_recheck = off;
CREATE VIEW tenk1_vw_sec WITH (security_barrier) AS SELECT * FROM tenk1;
EXPLAIN (COSTS OFF)
SELECT 1 FROM tenk1_vw_sec
diff --git a/src/test/regress/expected/sysviews.out b/src/test/regress/expected/sysviews.out
index 6e54f3e15e..e4b48fc61d 100644
--- a/src/test/regress/expected/sysviews.out
+++ b/src/test/regress/expected/sysviews.out
@@ -109,13 +109,14 @@ select name, setting from pg_settings where name like 'enable%';
enable_nestloop | on
enable_parallel_append | on
enable_parallel_hash | on
+ enable_parallel_params_recheck | on
enable_partition_pruning | on
enable_partitionwise_aggregate | off
enable_partitionwise_join | off
enable_seqscan | on
enable_sort | on
enable_tidscan | on
-(20 rows)
+(21 rows)
-- Test that the pg_timezone_names and pg_timezone_abbrevs views are
-- more-or-less working. We can't test their contents in any great detail
diff --git a/src/test/regress/sql/select_parallel.sql b/src/test/regress/sql/select_parallel.sql
index 019e17e751..f217d0ec9b 100644
--- a/src/test/regress/sql/select_parallel.sql
+++ b/src/test/regress/sql/select_parallel.sql
@@ -111,6 +111,31 @@ explain (costs off)
(select hundred, thousand from tenk2 where thousand > 100);
select count(*) from tenk1 where (two, four) not in
(select hundred, thousand from tenk2 where thousand > 100);
+-- test parallel plans for queries containing correlated subplans
+-- where the subplan only needs params available from the current
+-- worker's scan.
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t, generate_series(1, 10);
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t;
+explain (analyze, costs off, summary off, verbose, timing off) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t
+ limit 1;
+explain (costs off, verbose) select t.unique1
+ from tenk1 t
+ where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+explain (costs off, verbose) select *
+ from tenk1 t
+ order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+-- test subplan in join/lateral join
+explain (costs off, verbose, timing off) select t.unique1, l.*
+ from tenk1 t
+ join lateral (
+ select (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1 offset 0)
+ ) l on true;
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
explain (costs off)
select * from tenk1 where (unique1 + random())::integer not in
@@ -456,6 +481,18 @@ EXECUTE pstmt('1', make_some_array(1,2));
DEALLOCATE pstmt;
-- test interaction between subquery and partial_paths
+-- this plan changes to using a non-parallel index only
+-- scan on tenk1_unique1 (the parallel version of the subquery scan
+-- is cheaper, but only by ~30, and cost comparison treats them as equal
+-- since the costs are so large) because set_rel_consider_parallel
+-- called from make_one_rel sees the subplan as parallel safe now
+-- (in context it now knows the params are actually parallel safe).
+-- Because of that the non-parallel index path is now parallel_safe=true,
+-- therefore it wins the COSTS_EQUAL comparison in add_path.
+-- Perhaps any is_parallel_safe calls made for the purpose of determining
+-- consider_parallel should disable that behavior? It's not clear which is
+-- correct.
+set enable_parallel_params_recheck = off;
CREATE VIEW tenk1_vw_sec WITH (security_barrier) AS SELECT * FROM tenk1;
EXPLAIN (COSTS OFF)
SELECT 1 FROM tenk1_vw_sec
--
2.20.1
v3-0003-Other-places-to-consider-for-completeness.patchapplication/octet-stream; name=v3-0003-Other-places-to-consider-for-completeness.patchDownload
From c69f27574a664546d7a7e3fb3aaa98f74d8e4eba Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Fri, 7 May 2021 15:37:22 +0000
Subject: [PATCH v3 3/3] Other places to consider for completeness
---
src/backend/optimizer/path/allpaths.c | 4 ++++
src/backend/optimizer/plan/planmain.c | 2 +-
src/backend/optimizer/plan/planner.c | 3 +++
src/backend/optimizer/plan/subselect.c | 3 +++
src/backend/optimizer/prep/prepunion.c | 2 ++
5 files changed, 13 insertions(+), 1 deletion(-)
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 6eb39e6a06..9849d3712a 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -1142,6 +1142,8 @@ set_append_rel_size(PlannerInfo *root, RelOptInfo *rel,
*/
if (!childrel->consider_parallel)
rel->consider_parallel = false;
+ if (!childrel->consider_parallel_rechecking_params)
+ rel->consider_parallel_rechecking_params = false;
/*
* Accumulate size information from each live child.
@@ -1263,6 +1265,8 @@ set_append_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
*/
if (!rel->consider_parallel)
childrel->consider_parallel = false;
+ if (!childrel->consider_parallel_rechecking_params)
+ rel->consider_parallel_rechecking_params = false;
/*
* Compute the child's access paths.
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index bdbce2b87d..5135f9ee97 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -119,7 +119,7 @@ query_planner(PlannerInfo *root,
if (root->glob->parallelModeOK &&
force_parallel_mode != FORCE_PARALLEL_OFF)
final_rel->consider_parallel =
- is_parallel_safe(root, parse->jointree->quals, NULL);
+ is_parallel_safe(root, parse->jointree->quals, &final_rel->consider_parallel_rechecking_params);
/*
* The only path for it is a trivial Result path. We cheat a
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 008a096ee7..3d8102e9f7 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -4095,6 +4095,7 @@ create_window_paths(PlannerInfo *root,
if (input_rel->consider_parallel && output_target_parallel_safe &&
is_parallel_safe(root, (Node *) activeWindows, NULL))
window_rel->consider_parallel = true;
+ /* consider_parallel_rechecking_params */
/*
* If the input rel belongs to a single FDW, so does the window rel.
@@ -4292,6 +4293,7 @@ create_distinct_paths(PlannerInfo *root, RelOptInfo *input_rel)
* expressions are parallel-safe.
*/
distinct_rel->consider_parallel = input_rel->consider_parallel;
+ distinct_rel->consider_parallel_rechecking_params = input_rel->consider_parallel_rechecking_params;
/*
* If the input rel belongs to a single FDW, so does the distinct_rel.
@@ -4646,6 +4648,7 @@ create_ordered_paths(PlannerInfo *root,
*/
if (input_rel->consider_parallel && target_parallel_safe)
ordered_rel->consider_parallel = true;
+ /* consider_parallel_rechecking_params */
/*
* If the input rel belongs to a single FDW, so does the ordered_rel.
diff --git a/src/backend/optimizer/plan/subselect.c b/src/backend/optimizer/plan/subselect.c
index 393db3b42d..3e1001fa86 100644
--- a/src/backend/optimizer/plan/subselect.c
+++ b/src/backend/optimizer/plan/subselect.c
@@ -1020,6 +1020,7 @@ SS_process_ctes(PlannerInfo *root)
* parallel-safe.
*/
splan->parallel_safe = false;
+ splan->parallel_safe_ignoring_params = false;
splan->setParam = NIL;
splan->parParam = NIL;
splan->args = NIL;
@@ -2176,6 +2177,7 @@ SS_charge_for_initplans(PlannerInfo *root, RelOptInfo *final_rel)
path->startup_cost += initplan_cost;
path->total_cost += initplan_cost;
path->parallel_safe = false;
+ path->parallel_safe_ignoring_params = false;
}
/*
@@ -2184,6 +2186,7 @@ SS_charge_for_initplans(PlannerInfo *root, RelOptInfo *final_rel)
*/
final_rel->partial_pathlist = NIL;
final_rel->consider_parallel = false;
+ final_rel->consider_parallel_rechecking_params = false;
/* We needn't do set_cheapest() here, caller will do it */
}
diff --git a/src/backend/optimizer/prep/prepunion.c b/src/backend/optimizer/prep/prepunion.c
index 16cb35c914..ab92e7d7e4 100644
--- a/src/backend/optimizer/prep/prepunion.c
+++ b/src/backend/optimizer/prep/prepunion.c
@@ -273,6 +273,7 @@ recurse_set_operations(Node *setOp, PlannerInfo *root,
*/
final_rel = fetch_upper_rel(subroot, UPPERREL_FINAL, NULL);
rel->consider_parallel = final_rel->consider_parallel;
+ rel->consider_parallel_rechecking_params = final_rel->consider_parallel_rechecking_params;
/*
* For the moment, we consider only a single Path for the subquery.
@@ -617,6 +618,7 @@ generate_union_paths(SetOperationStmt *op, PlannerInfo *root,
result_rel = fetch_upper_rel(root, UPPERREL_SETOP, relids);
result_rel->reltarget = create_pathtarget(root, tlist);
result_rel->consider_parallel = consider_parallel;
+ /* consider_parallel_rechecking_params */
/*
* Append the child results together.
--
2.20.1
Re: Parallelize correlated subqueries that execute within each worker
On Wed, Sep 8, 2021 at 8:47 AM James Coleman <jtc331@gmail.com> wrote:
See updated patch series attached.
Jaime,
I noticed on 3-October you moved this into "waiting on author"; I
don't see anything waiting in this thread, however. Am I missing
something?
I'm planning to change it back to "needs review".
Thanks,
James
Re: Parallelize correlated subqueries that execute within each worker
As a preliminary comment, it would be quite useful to get Tom Lane's
opinion on this, since it's not an area I understand especially well,
and I think he understands it better than anyone.
On Fri, May 7, 2021 at 12:30 PM James Coleman <jtc331@gmail.com> wrote:
The basic idea is that we need to track (both on nodes and relations)
not only whether that node or rel is parallel safe but also whether
it's parallel safe assuming params are rechecked in the using context.
That allows us to delay making a final decision until we have
sufficient context to conclude that a given usage of a Param is
actually parallel safe or unsafe.
I don't really understand what you mean by "assuming params are
rechecked in the using context." However, I think that a possibly
better approach to this whole area would be to try to solve the
problem by putting limits on where you can insert a Gather node.
Consider:
Nested Loop
-> Seq Scan on x
-> Index Scan on y
Index Cond: y.q = x.q
If you insert a Gather node atop the Index Scan, necessarily changing
it to a Parallel Index Scan, then you need to pass values around. For
every value we get for x.q, we would need to start workers, sending
them the value of x.q, and they do a parallel index scan working
together to find all rows where y.q = x.q, and then exit. We repeat
this for every tuple from x.q. In the absence of infrastructure to
pass those parameters, we can't put the Gather there. We also don't
want to, because it would be really slow.
If you insert the Gather node atop the Seq Scan or the Nested Loop, in
either case necessarily changing the Seq Scan to a Parallel Seq Scan,
you have no problem. If you put it on top of the Nested Loop, the
parameter will be set in the workers and used in the workers and
everything is fine. If you put it on top of the Seq Scan, the
parameter will be set in the leader -- by the Nested Loop -- and used
in the leader, and again you have no problem.
So in my view of the world, the parameter just acts as an additional
constraint on where Gather nodes can be placed. I don't see that there
are any parameters that are unsafe categorically -- they're just
unsafe if the place where they are set is on a different side of the
Gather from the place where they are used. So I don't understand --
possibly just because I'm dumb -- the idea behind
consider_parallel_rechecking_params, because that seems to be making a
sort of overall judgement about the safety or unsafety of the
parameter on its own merits, rather than thinking about the Gather
placement.
When I last worked on this, I had hoped that extParam or allParam
would be the thing that would answer the question: are there any
parameters used under this node that are not also set under this node?
But I seem to recall that neither seemed to be answering precisely
that question, and the lousy naming of those fields and limited
documentation of their intended purpose did not help.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: Parallelize correlated subqueries that execute within each worker
Robert Haas <robertmhaas@gmail.com> writes:
When I last worked on this, I had hoped that extParam or allParam
would be the thing that would answer the question: are there any
parameters used under this node that are not also set under this node?
But I seem to recall that neither seemed to be answering precisely
that question, and the lousy naming of those fields and limited
documentation of their intended purpose did not help.
FWIW, I've never been very happy with those fields either. IIRC the
design in that area was all Vadim's, but to the extent that there's
any usable documentation of extParam/allParam, it was filled in by me
while trying to understand what Vadim did. If somebody wants to step
up and do a rewrite to make the planner's Param management more useful
or at least easier to understand, I think that'd be great.
But anyway: yeah, those fields as currently constituted don't help
much. They tell you which Params are consumed by this node or its
subnodes, but not where those Params came from. The planner's
plan_params and outer_params fields might be more nearly the right
thing, but I'm not sure they're spot-on either, nor that they're
up-to-date at the point where you'd want to make decisions about
Gather safety.
regards, tom lane
Re: Parallelize correlated subqueries that execute within each worker
On Wed, Nov 3, 2021 at 11:14 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
FWIW, I've never been very happy with those fields either. IIRC the
design in that area was all Vadim's, but to the extent that there's
any usable documentation of extParam/allParam, it was filled in by me
while trying to understand what Vadim did. If somebody wants to step
up and do a rewrite to make the planner's Param management more useful
or at least easier to understand, I think that'd be great.
Good to know, thanks.
But anyway: yeah, those fields as currently constituted don't help
much. They tell you which Params are consumed by this node or its
subnodes, but not where those Params came from. The planner's
plan_params and outer_params fields might be more nearly the right
thing, but I'm not sure they're spot-on either, nor that they're
up-to-date at the point where you'd want to make decisions about
Gather safety.
One thing I discovered when I was looking at this a few years ago is
that there was only one query in the regression tests where extParam
and allParam were not the same. The offending query was select 1 =
all(select (select 1)), and the resulting plan has a Materialize node
with an attached InitPlan. For that Materialize node, extParam = {}
and allParam = {$0}, with $0 also being the output parameter of the
InitPlan attached that that Materialize node. In every other node in
that plan and in every node of every other plan generated by the
regression tests, the values were identical. So it's extremely niche
that these fields are even different from each other, and it's unclear
to me that we really need both of them.
What's also interesting is that extParam is computed (by
finalize_plan) as plan->extParam = bms_del_members(plan->extParam,
initSetParam). So I think it mostly ends up that extParam for a node
is not exactly all the parameters that anything under that node cares
about, but rather - approximately - all the things that anything under
that node cares about that aren't also set someplace under that node.
If it were exactly that, I think it would be perfect for our needs
here: if the set of things used but not set below the current level is
empty, it's OK to insert a Gather node; otherwise, it's not, at least,
not unless we find a way to pipe parameters from the leader into the
workers. But I think there's some reason that I no longer remember why
it's not exactly that, and therefore the idea doesn't work.
One problem I do remember is that attaching initplans at the top of
each subquery level as we presently do is really not good for this
kind of thing. Suppose you have several levels of Nested Loop and
someplace down in the plan you reference an InitPlan. The planner sees
no harm in attaching the InitPlan at the top level, which makes it
unsafe to put the Gather any place but at the top level. If you
attached the InitPlan to the lowest node in the plan tree that is high
enough to be above all the places that use the value from that
parameter, you could potentially shift the Gather down the plan tree,
which would be great if, for example, there's exactly one
parallel-restricted join and the rest are parallel-safe. The best plan
might be to do all the other joins under a Gather and then perform the
parallel-restricted join above it.
But I found it very hard to figure out how to rejigger the logic that
places InitPlans to be more intelligent, and eventually gave up.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: Parallelize correlated subqueries that execute within each worker
Robert Haas <robertmhaas@gmail.com> writes:
One thing I discovered when I was looking at this a few years ago is
that there was only one query in the regression tests where extParam
and allParam were not the same. The offending query was select 1 =
all(select (select 1)), and the resulting plan has a Materialize node
with an attached InitPlan. For that Materialize node, extParam = {}
and allParam = {$0}, with $0 also being the output parameter of the
InitPlan attached that that Materialize node. In every other node in
that plan and in every node of every other plan generated by the
regression tests, the values were identical. So it's extremely niche
that these fields are even different from each other, and it's unclear
to me that we really need both of them.
Yeah, I've had that nagging feeling about them too. But ISTR trying to
reduce them to one value years ago, and finding that it didn't quite work,
or at least would result in more subquery-re-evaluation than we do today.
You have to dig into what the executor uses these values for to really
grok them. I'm afraid that that detail is all swapped out right now, so
I can't say much more.
regards, tom lane
Re: Parallelize correlated subqueries that execute within each worker
Hi Robert, thanks for the detailed reply.
On Wed, Nov 3, 2021 at 10:48 AM Robert Haas <robertmhaas@gmail.com> wrote:
As a preliminary comment, it would be quite useful to get Tom Lane's
opinion on this, since it's not an area I understand especially well,
and I think he understands it better than anyone.On Fri, May 7, 2021 at 12:30 PM James Coleman <jtc331@gmail.com> wrote:
The basic idea is that we need to track (both on nodes and relations)
not only whether that node or rel is parallel safe but also whether
it's parallel safe assuming params are rechecked in the using context.
That allows us to delay making a final decision until we have
sufficient context to conclude that a given usage of a Param is
actually parallel safe or unsafe.I don't really understand what you mean by "assuming params are
rechecked in the using context." However, I think that a possibly
better approach to this whole area would be to try to solve the
problem by putting limits on where you can insert a Gather node.
Consider:Nested Loop
-> Seq Scan on x
-> Index Scan on y
Index Cond: y.q = x.qIf you insert a Gather node atop the Index Scan, necessarily changing
it to a Parallel Index Scan, then you need to pass values around. For
every value we get for x.q, we would need to start workers, sending
them the value of x.q, and they do a parallel index scan working
together to find all rows where y.q = x.q, and then exit. We repeat
this for every tuple from x.q. In the absence of infrastructure to
pass those parameters, we can't put the Gather there. We also don't
want to, because it would be really slow.If you insert the Gather node atop the Seq Scan or the Nested Loop, in
either case necessarily changing the Seq Scan to a Parallel Seq Scan,
you have no problem. If you put it on top of the Nested Loop, the
parameter will be set in the workers and used in the workers and
everything is fine. If you put it on top of the Seq Scan, the
parameter will be set in the leader -- by the Nested Loop -- and used
in the leader, and again you have no problem.So in my view of the world, the parameter just acts as an additional
constraint on where Gather nodes can be placed. I don't see that there
are any parameters that are unsafe categorically -- they're just
unsafe if the place where they are set is on a different side of the
Gather from the place where they are used. So I don't understand --
possibly just because I'm dumb -- the idea behind
consider_parallel_rechecking_params, because that seems to be making a
sort of overall judgement about the safety or unsafety of the
parameter on its own merits, rather than thinking about the Gather
placement.
I had to read through this several times before I understood the point
(not your fault, this is, as you note, a complicated area). I *think*
if I grok it properly you're effectively describing what this patch
results in conceptually (but possibly solving it from a different
direction).
As I understand the current code, parallel plans are largely chosen
based not on where it's safe to insert a Gather node but rather by
determining if a given path is parallel safe. Through that lens params
are a bit of an odd man out -- they aren't inherently unsafe in the
way a parallel-unsafe function is, but they can only be used in
parallel plans under certain conditions (whether because of project
policy, performance, or missing infrastructure).
Under that paradigm the existing consider_parallel and parallel_safe
boolean values imply everything is about whether a plan is inherently
parallel safe. Thus the current doesn't have the context to handle the
nuance of params (as they are not inherently parallel-safe or unsafe).
Introducing consider_parallel_rechecking_params and
parallel_safe_ignoring_params allows us to keep more context on params
and make a more nuanced decision at the proper level of the plan. This
is what I mean by "rechecked in the using context", though I realize
now that both "recheck" and "context" are overloaded terms in the
project, so don't describe the concept particularly clearly. When a
path relies on params we can only make a final determination about its
parallel safety if we know whether or not the current parallel node
can provide the param's value. We don't necessarily know that
information until we attempt to generate a full parallel node in the
plan (I think what you're describing as "inserting a Gather node")
since the param may come from another node in the plan. These new
values allow us to do that by tracking tentatively parallel-safe
subplans (given proper Gather node placement) and delaying the
parallel-safety determination until the point at which a param is
available (or not).
Is that a more helpful framing of what my goal is here?
When I last worked on this, I had hoped that extParam or allParam
would be the thing that would answer the question: are there any
parameters used under this node that are not also set under this node?
But I seem to recall that neither seemed to be answering precisely
that question, and the lousy naming of those fields and limited
documentation of their intended purpose did not help.
I don't really know anything about extParam or allParam, so I can't
offer any insight here.
Thanks,
James Coleman
Re: Parallelize correlated subqueries that execute within each worker
On Wed, Nov 3, 2021 at 1:34 PM James Coleman <jtc331@gmail.com> wrote:
As I understand the current code, parallel plans are largely chosen
based not on where it's safe to insert a Gather node but rather by
determining if a given path is parallel safe. Through that lens params
are a bit of an odd man out -- they aren't inherently unsafe in the
way a parallel-unsafe function is, but they can only be used in
parallel plans under certain conditions (whether because of project
policy, performance, or missing infrastructure).
Right.
Introducing consider_parallel_rechecking_params and
parallel_safe_ignoring_params allows us to keep more context on params
and make a more nuanced decision at the proper level of the plan. This
is what I mean by "rechecked in the using context", though I realize
now that both "recheck" and "context" are overloaded terms in the
project, so don't describe the concept particularly clearly. When a
path relies on params we can only make a final determination about its
parallel safety if we know whether or not the current parallel node
can provide the param's value. We don't necessarily know that
information until we attempt to generate a full parallel node in the
plan (I think what you're describing as "inserting a Gather node")
since the param may come from another node in the plan. These new
values allow us to do that by tracking tentatively parallel-safe
subplans (given proper Gather node placement) and delaying the
parallel-safety determination until the point at which a param is
available (or not).
So I think I agree with you here. But I don't like all of this
"ignoring_params" stuff and I don't see why it's necessary. Say we
don't have both parallel_safe and parallel_safe_ignoring_params. Say
we just have parallel_safe. If the plan will be parallel safe if the
params are available, we label it parallel safe. If the plan will not
be parallel safe even if the params are available, we say it's not
parallel safe. Then, when we get to generate_gather_paths(), we don't
generate any paths if there are required parameters that are not
available. What's wrong with that approach?
Maybe it's clearer to say this: I feel like one extra Boolean is
either too much or too little. I think maybe it's not even needed. But
if it is needed, then why just a bool instead of, say, a Bitmapset of
params that are needed, or something?
I'm sort of speaking from intuition here rather than sure knowledge. I
might be totally wrong.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: Parallelize correlated subqueries that execute within each worker
On Mon, Nov 15, 2021 at 10:01:37AM -0500, Robert Haas wrote:
On Wed, Nov 3, 2021 at 1:34 PM James Coleman <jtc331@gmail.com> wrote:
As I understand the current code, parallel plans are largely chosen
based not on where it's safe to insert a Gather node but rather by
determining if a given path is parallel safe. Through that lens params
are a bit of an odd man out -- they aren't inherently unsafe in the
way a parallel-unsafe function is, but they can only be used in
parallel plans under certain conditions (whether because of project
policy, performance, or missing infrastructure).Right.
Please note that the CF bot is complaining here, so I have moved this
patch to the next CF, but changed the status as waiting on author.
--
Michael
Re: Parallelize correlated subqueries that execute within each worker
On Fri, Dec 3, 2021 at 2:35 AM Michael Paquier <michael@paquier.xyz> wrote:
On Mon, Nov 15, 2021 at 10:01:37AM -0500, Robert Haas wrote:
On Wed, Nov 3, 2021 at 1:34 PM James Coleman <jtc331@gmail.com> wrote:
As I understand the current code, parallel plans are largely chosen
based not on where it's safe to insert a Gather node but rather by
determining if a given path is parallel safe. Through that lens params
are a bit of an odd man out -- they aren't inherently unsafe in the
way a parallel-unsafe function is, but they can only be used in
parallel plans under certain conditions (whether because of project
policy, performance, or missing infrastructure).Right.
Please note that the CF bot is complaining here, so I have moved this
patch to the next CF, but changed the status as waiting on author.
I rebased this back in December, but somehow forgot to reply with the
updated patch, so, here it is finally.
Thanks,
James Coleman
Attachments:
v4-0001-Allow-parallel-LATERAL-subqueries-with-LIMIT-OFFS.patchapplication/octet-stream; name=v4-0001-Allow-parallel-LATERAL-subqueries-with-LIMIT-OFFS.patchDownload
From 6596bb6909b9dfaadd6e6e834be4032e7903c4e9 Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Mon, 30 Nov 2020 11:36:35 -0500
Subject: [PATCH v4 1/3] Allow parallel LATERAL subqueries with LIMIT/OFFSET
The code that determined whether or not a rel should be considered for
parallel query excluded subqueries with LIMIT/OFFSET. That's correct in
the general case: as the comment notes that'd mean we have to guarantee
ordering (and claims it's not worth checking that) for results to be
consistent across workers. However there's a simpler case that hasn't
been considered: LATERAL subqueries with LIMIT/OFFSET don't fall under
the same reasoning since they're executed (when not converted to a JOIN)
per tuple anyway, so consistency of results across workers isn't a
factor.
---
src/backend/optimizer/path/allpaths.c | 4 +++-
src/test/regress/expected/select_parallel.out | 15 +++++++++++++++
src/test/regress/sql/select_parallel.sql | 6 ++++++
3 files changed, 24 insertions(+), 1 deletion(-)
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 169b1d53fc..6a581e20fa 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -682,11 +682,13 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
* inconsistent results at the top-level. (In some cases, where
* the result is ordered, we could relax this restriction. But it
* doesn't currently seem worth expending extra effort to do so.)
+ * LATERAL is an exception: LIMIT/OFFSET is safe to execute within
+ * workers since the sub-select is executed per tuple
*/
{
Query *subquery = castNode(Query, rte->subquery);
- if (limit_needed(subquery))
+ if (!rte->lateral && limit_needed(subquery))
return;
}
break;
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 4ea1aa7dfd..2303f70d6e 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -1040,6 +1040,21 @@ explain (costs off)
Filter: (stringu1 ~~ '%AAAA'::text)
(11 rows)
+-- ...unless it's LATERAL
+savepoint settings;
+set parallel_tuple_cost=0;
+explain (costs off) select t.unique1 from tenk1 t
+join lateral (select t.unique1 from tenk1 offset 0) l on true;
+ QUERY PLAN
+---------------------------------------------------------------------
+ Gather
+ Workers Planned: 4
+ -> Nested Loop
+ -> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
+ -> Index Only Scan using tenk1_hundred on tenk1
+(5 rows)
+
+rollback to savepoint settings;
-- to increase the parallel query test coverage
SAVEPOINT settings;
SET LOCAL force_parallel_mode = 1;
diff --git a/src/test/regress/sql/select_parallel.sql b/src/test/regress/sql/select_parallel.sql
index f924731248..019e17e751 100644
--- a/src/test/regress/sql/select_parallel.sql
+++ b/src/test/regress/sql/select_parallel.sql
@@ -390,6 +390,12 @@ explain (costs off, verbose)
explain (costs off)
select * from tenk1 a where two in
(select two from tenk1 b where stringu1 like '%AAAA' limit 3);
+-- ...unless it's LATERAL
+savepoint settings;
+set parallel_tuple_cost=0;
+explain (costs off) select t.unique1 from tenk1 t
+join lateral (select t.unique1 from tenk1 offset 0) l on true;
+rollback to savepoint settings;
-- to increase the parallel query test coverage
SAVEPOINT settings;
--
2.20.1
v4-0002-Parallel-query-support-for-basic-correlated-subqu.patchapplication/octet-stream; name=v4-0002-Parallel-query-support-for-basic-correlated-subqu.patchDownload
From 694bd60ed35d7e6ada19280db2abd24ff4272cad Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Fri, 27 Nov 2020 18:44:30 -0500
Subject: [PATCH v4 2/3] Parallel query support for basic correlated subqueries
Not all Params are inherently parallel-unsafe, but we can't know whether
they're parallel-safe up-front: we need contextual information for a
given path shape. Here we delay the final determination of whether or
not a Param is parallel-safe by initially verifying that it is minimally
parallel-safe for things that are inherent (e.g., no parallel-unsafe
functions or relations involved) and later re-checking that a given
usage is contextually safe (e.g., the Param is for correlation that can
happen entirely within a parallel worker (as opposed to needing to pass
values between workers).
---
doc/src/sgml/parallel.sgml | 3 +-
src/backend/nodes/copyfuncs.c | 2 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/outfuncs.c | 3 +
src/backend/nodes/readfuncs.c | 2 +
src/backend/optimizer/path/allpaths.c | 61 +++++--
src/backend/optimizer/path/equivclass.c | 4 +-
src/backend/optimizer/path/indxpath.c | 12 ++
src/backend/optimizer/path/joinpath.c | 21 ++-
src/backend/optimizer/plan/createplan.c | 2 +
src/backend/optimizer/plan/planmain.c | 2 +-
src/backend/optimizer/plan/planner.c | 158 +++++++++++++++---
src/backend/optimizer/plan/subselect.c | 21 ++-
src/backend/optimizer/prep/prepunion.c | 1 +
src/backend/optimizer/util/clauses.c | 113 ++++++++++---
src/backend/optimizer/util/pathnode.c | 36 +++-
src/backend/optimizer/util/relnode.c | 30 +++-
src/backend/utils/misc/guc.c | 11 ++
src/include/nodes/pathnodes.h | 5 +-
src/include/nodes/plannodes.h | 1 +
src/include/nodes/primnodes.h | 1 +
src/include/optimizer/clauses.h | 4 +-
.../regress/expected/incremental_sort.out | 28 ++--
src/test/regress/expected/select_parallel.out | 137 +++++++++++++++
src/test/regress/expected/sysviews.out | 3 +-
src/test/regress/sql/select_parallel.sql | 37 ++++
26 files changed, 603 insertions(+), 96 deletions(-)
diff --git a/doc/src/sgml/parallel.sgml b/doc/src/sgml/parallel.sgml
index 13479d7e5e..2d924dd2ac 100644
--- a/doc/src/sgml/parallel.sgml
+++ b/doc/src/sgml/parallel.sgml
@@ -517,7 +517,8 @@ EXPLAIN SELECT * FROM pgbench_accounts WHERE filler LIKE '%x%';
<listitem>
<para>
- Plan nodes that reference a correlated <literal>SubPlan</literal>.
+ Plan nodes that reference a correlated <literal>SubPlan</literal> where
+ the result is shared between workers.
</para>
</listitem>
</itemizedlist>
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index b105c26381..91f229c0a0 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -128,6 +128,7 @@ CopyPlanFields(const Plan *from, Plan *newnode)
COPY_SCALAR_FIELD(parallel_aware);
COPY_SCALAR_FIELD(parallel_safe);
COPY_SCALAR_FIELD(async_capable);
+ COPY_SCALAR_FIELD(parallel_safe_ignoring_params);
COPY_SCALAR_FIELD(plan_node_id);
COPY_NODE_FIELD(targetlist);
COPY_NODE_FIELD(qual);
@@ -1774,6 +1775,7 @@ _copySubPlan(const SubPlan *from)
COPY_SCALAR_FIELD(useHashTable);
COPY_SCALAR_FIELD(unknownEqFalse);
COPY_SCALAR_FIELD(parallel_safe);
+ COPY_SCALAR_FIELD(parallel_safe_ignoring_params);
COPY_NODE_FIELD(setParam);
COPY_NODE_FIELD(parParam);
COPY_NODE_FIELD(args);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index ae37ea9464..59658029a8 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -472,6 +472,7 @@ _equalSubPlan(const SubPlan *a, const SubPlan *b)
COMPARE_SCALAR_FIELD(useHashTable);
COMPARE_SCALAR_FIELD(unknownEqFalse);
COMPARE_SCALAR_FIELD(parallel_safe);
+ COMPARE_SCALAR_FIELD(parallel_safe_ignoring_params);
COMPARE_NODE_FIELD(setParam);
COMPARE_NODE_FIELD(parParam);
COMPARE_NODE_FIELD(args);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index d28cea1567..860dfa4645 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -341,6 +341,7 @@ _outPlanInfo(StringInfo str, const Plan *node)
WRITE_BOOL_FIELD(parallel_aware);
WRITE_BOOL_FIELD(parallel_safe);
WRITE_BOOL_FIELD(async_capable);
+ WRITE_BOOL_FIELD(parallel_safe_ignoring_params);
WRITE_INT_FIELD(plan_node_id);
WRITE_NODE_FIELD(targetlist);
WRITE_NODE_FIELD(qual);
@@ -1384,6 +1385,7 @@ _outSubPlan(StringInfo str, const SubPlan *node)
WRITE_BOOL_FIELD(useHashTable);
WRITE_BOOL_FIELD(unknownEqFalse);
WRITE_BOOL_FIELD(parallel_safe);
+ WRITE_BOOL_FIELD(parallel_safe_ignoring_params);
WRITE_NODE_FIELD(setParam);
WRITE_NODE_FIELD(parParam);
WRITE_NODE_FIELD(args);
@@ -1782,6 +1784,7 @@ _outPathInfo(StringInfo str, const Path *node)
outBitmapset(str, NULL);
WRITE_BOOL_FIELD(parallel_aware);
WRITE_BOOL_FIELD(parallel_safe);
+ WRITE_BOOL_FIELD(parallel_safe_ignoring_params);
WRITE_INT_FIELD(parallel_workers);
WRITE_FLOAT_FIELD(rows, "%.0f");
WRITE_FLOAT_FIELD(startup_cost, "%.2f");
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 3f68f7c18d..0d8cc96fbc 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -1619,6 +1619,7 @@ ReadCommonPlan(Plan *local_node)
READ_BOOL_FIELD(parallel_aware);
READ_BOOL_FIELD(parallel_safe);
READ_BOOL_FIELD(async_capable);
+ READ_BOOL_FIELD(parallel_safe_ignoring_params);
READ_INT_FIELD(plan_node_id);
READ_NODE_FIELD(targetlist);
READ_NODE_FIELD(qual);
@@ -2616,6 +2617,7 @@ _readSubPlan(void)
READ_BOOL_FIELD(useHashTable);
READ_BOOL_FIELD(unknownEqFalse);
READ_BOOL_FIELD(parallel_safe);
+ READ_BOOL_FIELD(parallel_safe_ignoring_params);
READ_NODE_FIELD(setParam);
READ_NODE_FIELD(parParam);
READ_NODE_FIELD(args);
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 6a581e20fa..e419773dcf 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -556,7 +556,8 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
* (see grouping_planner).
*/
if (rel->reloptkind == RELOPT_BASEREL &&
- bms_membership(root->all_baserels) != BMS_SINGLETON)
+ bms_membership(root->all_baserels) != BMS_SINGLETON
+ && (rel->subplan_params == NIL || rte->rtekind != RTE_SUBQUERY))
generate_useful_gather_paths(root, rel, false);
/* Now find the cheapest of the paths for this rel */
@@ -592,6 +593,9 @@ static void
set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte)
{
+ bool parallel_safe;
+ bool parallel_safe_except_in_params;
+
/*
* The flag has previously been initialized to false, so we can just
* return if it becomes clear that we can't safely set it.
@@ -632,7 +636,7 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
if (proparallel != PROPARALLEL_SAFE)
return;
- if (!is_parallel_safe(root, (Node *) rte->tablesample->args))
+ if (!is_parallel_safe(root, (Node *) rte->tablesample->args, NULL))
return;
}
@@ -700,7 +704,7 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
case RTE_FUNCTION:
/* Check for parallel-restricted functions. */
- if (!is_parallel_safe(root, (Node *) rte->functions))
+ if (!is_parallel_safe(root, (Node *) rte->functions, NULL))
return;
break;
@@ -710,7 +714,7 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
case RTE_VALUES:
/* Check for parallel-restricted functions. */
- if (!is_parallel_safe(root, (Node *) rte->values_lists))
+ if (!is_parallel_safe(root, (Node *) rte->values_lists, NULL))
return;
break;
@@ -747,18 +751,28 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
* outer join clauses work correctly. It would likely break equivalence
* classes, too.
*/
- if (!is_parallel_safe(root, (Node *) rel->baserestrictinfo))
- return;
+ parallel_safe = is_parallel_safe(root, (Node *) rel->baserestrictinfo,
+ ¶llel_safe_except_in_params);
/*
* Likewise, if the relation's outputs are not parallel-safe, give up.
* (Usually, they're just Vars, but sometimes they're not.)
*/
- if (!is_parallel_safe(root, (Node *) rel->reltarget->exprs))
- return;
+ if (parallel_safe || parallel_safe_except_in_params)
+ {
+ bool target_parallel_safe;
+ bool target_parallel_safe_ignoring_params = false;
+
+ target_parallel_safe = is_parallel_safe(root,
+ (Node *) rel->reltarget->exprs,
+ &target_parallel_safe_ignoring_params);
+ parallel_safe = parallel_safe && target_parallel_safe;
+ parallel_safe_except_in_params = parallel_safe_except_in_params
+ && target_parallel_safe_ignoring_params;
+ }
- /* We have a winner. */
- rel->consider_parallel = true;
+ rel->consider_parallel = parallel_safe;
+ rel->consider_parallel_rechecking_params = parallel_safe_except_in_params;
}
/*
@@ -2344,9 +2358,21 @@ set_subquery_pathlist(PlannerInfo *root, RelOptInfo *rel,
pathkeys, required_outer));
}
+ /*
+ * XXX: As far as I can tell, the only time partial paths exist here
+ * is when we're going to execute multiple partial paths in parallel
+ * under a gather node (instead of executing paths serially under
+ * an append node). That means that the subquery scan path here
+ * is self-contained at this point -- so by definition it can't be
+ * reliant on lateral relids, which means we'll never have to consider
+ * rechecking params here.
+ */
+ Assert(!(rel->consider_parallel_rechecking_params && rel->partial_pathlist && !rel->consider_parallel));
+
/* If outer rel allows parallelism, do same for partial paths. */
if (rel->consider_parallel && bms_is_empty(required_outer))
{
+
/* If consider_parallel is false, there should be no partial paths. */
Assert(sub_final_rel->consider_parallel ||
sub_final_rel->partial_pathlist == NIL);
@@ -2700,7 +2726,7 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
cheapest_partial_path->rows * cheapest_partial_path->parallel_workers;
simple_gather_path = (Path *)
create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
- NULL, rowsp);
+ rel->lateral_relids, rowsp);
add_path(rel, simple_gather_path);
/*
@@ -2717,7 +2743,7 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
rows = subpath->rows * subpath->parallel_workers;
path = create_gather_merge_path(root, rel, subpath, rel->reltarget,
- subpath->pathkeys, NULL, rowsp);
+ subpath->pathkeys, rel->lateral_relids, rowsp);
add_path(rel, &path->path);
}
}
@@ -2819,11 +2845,15 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
double *rowsp = NULL;
List *useful_pathkeys_list = NIL;
Path *cheapest_partial_path = NULL;
+ Relids required_outer = rel->lateral_relids;
/* If there are no partial paths, there's nothing to do here. */
if (rel->partial_pathlist == NIL)
return;
+ if (!bms_is_subset(required_outer, rel->relids))
+ return;
+
/* Should we override the rel's rowcount estimate? */
if (override_rows)
rowsp = &rows;
@@ -2895,7 +2925,7 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
tmp,
rel->reltarget,
tmp->pathkeys,
- NULL,
+ required_outer,
rowsp);
add_path(rel, &path->path);
@@ -2929,7 +2959,7 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
tmp,
rel->reltarget,
tmp->pathkeys,
- NULL,
+ required_outer,
rowsp);
add_path(rel, &path->path);
@@ -3108,7 +3138,8 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
/*
* Except for the topmost scan/join rel, consider gathering
* partial paths. We'll do the same for the topmost scan/join rel
- * once we know the final targetlist (see grouping_planner).
+ * once we know the final targetlist (see
+ * apply_scanjoin_target_to_paths).
*/
if (lev < levels_needed)
generate_useful_gather_paths(root, rel, false);
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 8c6770de97..51f58adc16 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -897,7 +897,7 @@ find_computable_ec_member(PlannerInfo *root,
* check this last because it's a rather expensive test.
*/
if (require_parallel_safe &&
- !is_parallel_safe(root, (Node *) em->em_expr))
+ !is_parallel_safe(root, (Node *) em->em_expr, NULL))
continue;
return em; /* found usable expression */
@@ -1012,7 +1012,7 @@ relation_can_be_sorted_early(PlannerInfo *root, RelOptInfo *rel,
* check this last because it's a rather expensive test.
*/
if (require_parallel_safe &&
- !is_parallel_safe(root, (Node *) em->em_expr))
+ !is_parallel_safe(root, (Node *) em->em_expr, NULL))
continue;
return true;
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
index 0ef70ad7f1..4327d3f18e 100644
--- a/src/backend/optimizer/path/indxpath.c
+++ b/src/backend/optimizer/path/indxpath.c
@@ -1047,6 +1047,17 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
/*
* If appropriate, consider parallel index scan. We don't allow
* parallel index scan for bitmap index scans.
+ *
+ * XXX: Checking rel->consider_parallel_rechecking_params here resulted
+ * in some odd behavior on:
+ * select (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1) from tenk1 t;
+ * where the total cost on the chosen plan went *up* considering
+ * the extra path.
+ *
+ * Current working theory is that this method is about base relation
+ * scans, and we only want parameterized paths to be parallelized as
+ * companions to existing parallel plans and so don't really care to
+ * consider a separate parallel index scan here.
*/
if (index->amcanparallel &&
rel->consider_parallel && outer_relids == NULL &&
@@ -1100,6 +1111,7 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
result = lappend(result, ipath);
/* If appropriate, consider parallel index scan */
+ /* XXX: As above here for rel->consider_parallel_rechecking_params? */
if (index->amcanparallel &&
rel->consider_parallel && outer_relids == NULL &&
scantype != ST_BITMAPSCAN)
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index f96fc9fd28..7a16880bfb 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -767,6 +767,7 @@ try_partial_nestloop_path(PlannerInfo *root,
else
outerrelids = outerrel->relids;
+ /* TODO: recheck parallel safety here? */
if (!bms_is_subset(inner_paramrels, outerrelids))
return;
}
@@ -1791,16 +1792,24 @@ match_unsorted_outer(PlannerInfo *root,
* partial path and the joinrel is parallel-safe. However, we can't
* handle JOIN_UNIQUE_OUTER, because the outer path will be partial, and
* therefore we won't be able to properly guarantee uniqueness. Nor can
- * we handle joins needing lateral rels, since partial paths must not be
- * parameterized. Similarly, we can't handle JOIN_FULL and JOIN_RIGHT,
- * because they can produce false null extended rows.
+ * we handle JOIN_FULL and JOIN_RIGHT, because they can produce false null
+ * extended rows.
+ *
+ * Partial paths may only have parameters in limited cases
+ * where the parameterization is fully satisfied without sharing state
+ * between workers, so we only allow lateral rels on inputs to the join
+ * if the resulting join contains no lateral rels, the inner rel's laterals
+ * are fully satisfied by the outer rel, and the outer rel doesn't depend
+ * on the inner rel to produce any laterals.
*/
if (joinrel->consider_parallel &&
save_jointype != JOIN_UNIQUE_OUTER &&
save_jointype != JOIN_FULL &&
save_jointype != JOIN_RIGHT &&
outerrel->partial_pathlist != NIL &&
- bms_is_empty(joinrel->lateral_relids))
+ bms_is_empty(joinrel->lateral_relids) &&
+ bms_is_subset(innerrel->lateral_relids, outerrel->relids) &&
+ (bms_is_empty(outerrel->lateral_relids) || !bms_is_subset(outerrel->lateral_relids, innerrel->relids)))
{
if (nestjoinOK)
consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
@@ -1915,7 +1924,9 @@ consider_parallel_nestloop(PlannerInfo *root,
Path *mpath;
/* Can't join to an inner path that is not parallel-safe */
- if (!innerpath->parallel_safe)
+ /* TODO: recheck parallel safety of params here? */
+ if (!innerpath->parallel_safe &&
+ !(innerpath->parallel_safe_ignoring_params))
continue;
/*
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index cd6d72c763..4fb3dcc8a0 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -5300,6 +5300,7 @@ copy_generic_path_info(Plan *dest, Path *src)
dest->plan_width = src->pathtarget->width;
dest->parallel_aware = src->parallel_aware;
dest->parallel_safe = src->parallel_safe;
+ dest->parallel_safe_ignoring_params = src->parallel_safe_ignoring_params;
}
/*
@@ -5317,6 +5318,7 @@ copy_plan_costsize(Plan *dest, Plan *src)
dest->parallel_aware = false;
/* Assume the inserted node is parallel-safe, if child plan is. */
dest->parallel_safe = src->parallel_safe;
+ dest->parallel_safe_ignoring_params = src->parallel_safe_ignoring_params;
}
/*
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index c92ddd27ed..e3f4214d68 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -119,7 +119,7 @@ query_planner(PlannerInfo *root,
if (root->glob->parallelModeOK &&
force_parallel_mode != FORCE_PARALLEL_OFF)
final_rel->consider_parallel =
- is_parallel_safe(root, parse->jointree->quals);
+ is_parallel_safe(root, parse->jointree->quals, NULL);
/*
* The only path for it is a trivial Result path. We cheat a
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index bd09f85aea..a4057ab48c 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -150,6 +150,7 @@ static RelOptInfo *create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
PathTarget *target,
bool target_parallel_safe,
+ bool target_parallel_safe_ignoring_params,
grouping_sets_data *gd);
static bool is_degenerate_grouping(PlannerInfo *root);
static void create_degenerate_grouping_paths(PlannerInfo *root,
@@ -157,6 +158,7 @@ static void create_degenerate_grouping_paths(PlannerInfo *root,
RelOptInfo *grouped_rel);
static RelOptInfo *make_grouping_rel(PlannerInfo *root, RelOptInfo *input_rel,
PathTarget *target, bool target_parallel_safe,
+ bool target_parallel_safe_ignoring_params,
Node *havingQual);
static void create_ordinary_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
@@ -237,6 +239,7 @@ static void apply_scanjoin_target_to_paths(PlannerInfo *root,
List *scanjoin_targets,
List *scanjoin_targets_contain_srfs,
bool scanjoin_target_parallel_safe,
+ bool scanjoin_target_parallel_safe_ignoring_params,
bool tlist_same_exprs);
static void create_partitionwise_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
@@ -1242,6 +1245,7 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
List *final_targets;
List *final_targets_contain_srfs;
bool final_target_parallel_safe;
+ bool final_target_parallel_safe_ignoring_params = false;
RelOptInfo *current_rel;
RelOptInfo *final_rel;
FinalPathExtraData extra;
@@ -1304,7 +1308,7 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
/* And check whether it's parallel safe */
final_target_parallel_safe =
- is_parallel_safe(root, (Node *) final_target->exprs);
+ is_parallel_safe(root, (Node *) final_target->exprs, &final_target_parallel_safe_ignoring_params);
/* The setop result tlist couldn't contain any SRFs */
Assert(!parse->hasTargetSRFs);
@@ -1338,14 +1342,17 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
List *sort_input_targets;
List *sort_input_targets_contain_srfs;
bool sort_input_target_parallel_safe;
+ bool sort_input_target_parallel_safe_ignoring_params = false;
PathTarget *grouping_target;
List *grouping_targets;
List *grouping_targets_contain_srfs;
bool grouping_target_parallel_safe;
+ bool grouping_target_parallel_safe_ignoring_params = false;
PathTarget *scanjoin_target;
List *scanjoin_targets;
List *scanjoin_targets_contain_srfs;
bool scanjoin_target_parallel_safe;
+ bool scanjoin_target_parallel_safe_ignoring_params = false;
bool scanjoin_target_same_exprs;
bool have_grouping;
WindowFuncLists *wflists = NULL;
@@ -1458,7 +1465,7 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
*/
final_target = create_pathtarget(root, root->processed_tlist);
final_target_parallel_safe =
- is_parallel_safe(root, (Node *) final_target->exprs);
+ is_parallel_safe(root, (Node *) final_target->exprs, &final_target_parallel_safe_ignoring_params);
/*
* If ORDER BY was given, consider whether we should use a post-sort
@@ -1471,12 +1478,13 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
final_target,
&have_postponed_srfs);
sort_input_target_parallel_safe =
- is_parallel_safe(root, (Node *) sort_input_target->exprs);
+ is_parallel_safe(root, (Node *) sort_input_target->exprs, &sort_input_target_parallel_safe_ignoring_params);
}
else
{
sort_input_target = final_target;
sort_input_target_parallel_safe = final_target_parallel_safe;
+ sort_input_target_parallel_safe_ignoring_params = final_target_parallel_safe_ignoring_params;
}
/*
@@ -1490,12 +1498,13 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
final_target,
activeWindows);
grouping_target_parallel_safe =
- is_parallel_safe(root, (Node *) grouping_target->exprs);
+ is_parallel_safe(root, (Node *) grouping_target->exprs, &grouping_target_parallel_safe_ignoring_params);
}
else
{
grouping_target = sort_input_target;
grouping_target_parallel_safe = sort_input_target_parallel_safe;
+ grouping_target_parallel_safe_ignoring_params = sort_input_target_parallel_safe_ignoring_params;
}
/*
@@ -1509,12 +1518,13 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
{
scanjoin_target = make_group_input_target(root, final_target);
scanjoin_target_parallel_safe =
- is_parallel_safe(root, (Node *) scanjoin_target->exprs);
+ is_parallel_safe(root, (Node *) scanjoin_target->exprs, &scanjoin_target_parallel_safe_ignoring_params);
}
else
{
scanjoin_target = grouping_target;
scanjoin_target_parallel_safe = grouping_target_parallel_safe;
+ scanjoin_target_parallel_safe_ignoring_params = grouping_target_parallel_safe_ignoring_params;
}
/*
@@ -1566,6 +1576,7 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
apply_scanjoin_target_to_paths(root, current_rel, scanjoin_targets,
scanjoin_targets_contain_srfs,
scanjoin_target_parallel_safe,
+ scanjoin_target_parallel_safe_ignoring_params,
scanjoin_target_same_exprs);
/*
@@ -1593,6 +1604,7 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
current_rel,
grouping_target,
grouping_target_parallel_safe,
+ grouping_target_parallel_safe_ignoring_params,
gset_data);
/* Fix things up if grouping_target contains SRFs */
if (parse->hasTargetSRFs)
@@ -1666,10 +1678,25 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
* not a SELECT, consider_parallel will be false for every relation in the
* query.
*/
- if (current_rel->consider_parallel &&
- is_parallel_safe(root, parse->limitOffset) &&
- is_parallel_safe(root, parse->limitCount))
- final_rel->consider_parallel = true;
+ if (current_rel->consider_parallel || current_rel->consider_parallel_rechecking_params)
+ {
+ bool limit_count_parallel_safe;
+ bool limit_offset_parallel_safe;
+ bool limit_count_parallel_safe_ignoring_params = false;
+ bool limit_offset_parallel_safe_ignoring_params = false;
+
+ limit_count_parallel_safe = is_parallel_safe(root, parse->limitCount, &limit_count_parallel_safe_ignoring_params);
+ limit_offset_parallel_safe = is_parallel_safe(root, parse->limitOffset, &limit_offset_parallel_safe_ignoring_params);
+
+ if (current_rel->consider_parallel &&
+ limit_count_parallel_safe &&
+ limit_offset_parallel_safe)
+ final_rel->consider_parallel = true;
+ if (current_rel->consider_parallel_rechecking_params &&
+ limit_count_parallel_safe_ignoring_params &&
+ limit_offset_parallel_safe_ignoring_params)
+ final_rel->consider_parallel_rechecking_params = true;
+ }
/*
* If the current_rel belongs to a single FDW, so does the final_rel.
@@ -1870,8 +1897,8 @@ grouping_planner(PlannerInfo *root, double tuple_fraction)
* Generate partial paths for final_rel, too, if outer query levels might
* be able to make use of them.
*/
- if (final_rel->consider_parallel && root->query_level > 1 &&
- !limit_needed(parse))
+ if ((final_rel->consider_parallel || final_rel->consider_parallel_rechecking_params) &&
+ root->query_level > 1 && !limit_needed(parse))
{
Assert(!parse->rowMarks && parse->commandType == CMD_SELECT);
foreach(lc, current_rel->partial_pathlist)
@@ -3283,6 +3310,7 @@ create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
PathTarget *target,
bool target_parallel_safe,
+ bool target_parallel_safe_ignoring_params,
grouping_sets_data *gd)
{
Query *parse = root->parse;
@@ -3298,7 +3326,9 @@ create_grouping_paths(PlannerInfo *root,
* aggregation paths.
*/
grouped_rel = make_grouping_rel(root, input_rel, target,
- target_parallel_safe, parse->havingQual);
+ target_parallel_safe,
+ target_parallel_safe_ignoring_params,
+ parse->havingQual);
/*
* Create either paths for a degenerate grouping or paths for ordinary
@@ -3359,6 +3389,7 @@ create_grouping_paths(PlannerInfo *root,
extra.flags = flags;
extra.target_parallel_safe = target_parallel_safe;
+ extra.target_parallel_safe_ignoring_params = target_parallel_safe_ignoring_params;
extra.havingQual = parse->havingQual;
extra.targetList = parse->targetList;
extra.partial_costs_set = false;
@@ -3394,7 +3425,7 @@ create_grouping_paths(PlannerInfo *root,
static RelOptInfo *
make_grouping_rel(PlannerInfo *root, RelOptInfo *input_rel,
PathTarget *target, bool target_parallel_safe,
- Node *havingQual)
+ bool target_parallel_safe_ignoring_params, Node *havingQual)
{
RelOptInfo *grouped_rel;
@@ -3422,9 +3453,21 @@ make_grouping_rel(PlannerInfo *root, RelOptInfo *input_rel,
* can't be parallel-safe, either. Otherwise, it's parallel-safe if the
* target list and HAVING quals are parallel-safe.
*/
- if (input_rel->consider_parallel && target_parallel_safe &&
- is_parallel_safe(root, (Node *) havingQual))
- grouped_rel->consider_parallel = true;
+ if ((input_rel->consider_parallel || input_rel->consider_parallel_rechecking_params)
+ && (target_parallel_safe || target_parallel_safe_ignoring_params))
+ {
+ bool having_qual_parallel_safe;
+ bool having_qual_parallel_safe_ignoring_params = false;
+
+ having_qual_parallel_safe = is_parallel_safe(root, (Node *) havingQual,
+ &having_qual_parallel_safe_ignoring_params);
+
+ grouped_rel->consider_parallel = input_rel->consider_parallel &&
+ having_qual_parallel_safe && target_parallel_safe;
+ grouped_rel->consider_parallel_rechecking_params =
+ input_rel->consider_parallel_rechecking_params &&
+ having_qual_parallel_safe_ignoring_params && target_parallel_safe_ignoring_params;
+ }
/*
* If the input rel belongs to a single FDW, so does the grouped rel.
@@ -4051,7 +4094,7 @@ create_window_paths(PlannerInfo *root,
* target list and active windows for non-parallel-safe constructs.
*/
if (input_rel->consider_parallel && output_target_parallel_safe &&
- is_parallel_safe(root, (Node *) activeWindows))
+ is_parallel_safe(root, (Node *) activeWindows, NULL))
window_rel->consider_parallel = true;
/*
@@ -5756,6 +5799,7 @@ adjust_paths_for_srfs(PlannerInfo *root, RelOptInfo *rel,
PathTarget *thistarget = lfirst_node(PathTarget, lc1);
bool contains_srfs = (bool) lfirst_int(lc2);
+ /* TODO: How do we know the new target is parallel safe? */
/* If this level doesn't contain SRFs, do regular projection */
if (contains_srfs)
newpath = (Path *) create_set_projection_path(root,
@@ -6072,8 +6116,8 @@ plan_create_index_workers(Oid tableOid, Oid indexOid)
* safe.
*/
if (heap->rd_rel->relpersistence == RELPERSISTENCE_TEMP ||
- !is_parallel_safe(root, (Node *) RelationGetIndexExpressions(index)) ||
- !is_parallel_safe(root, (Node *) RelationGetIndexPredicate(index)))
+ !is_parallel_safe(root, (Node *) RelationGetIndexExpressions(index), NULL) ||
+ !is_parallel_safe(root, (Node *) RelationGetIndexPredicate(index), NULL))
{
parallel_workers = 0;
goto done;
@@ -6536,6 +6580,8 @@ create_partial_grouping_paths(PlannerInfo *root,
grouped_rel->relids);
partially_grouped_rel->consider_parallel =
grouped_rel->consider_parallel;
+ partially_grouped_rel->consider_parallel_rechecking_params =
+ grouped_rel->consider_parallel_rechecking_params;
partially_grouped_rel->reloptkind = grouped_rel->reloptkind;
partially_grouped_rel->serverid = grouped_rel->serverid;
partially_grouped_rel->userid = grouped_rel->userid;
@@ -6999,6 +7045,7 @@ apply_scanjoin_target_to_paths(PlannerInfo *root,
List *scanjoin_targets,
List *scanjoin_targets_contain_srfs,
bool scanjoin_target_parallel_safe,
+ bool scanjoin_target_parallel_safe_ignoring_params,
bool tlist_same_exprs)
{
bool rel_is_partitioned = IS_PARTITIONED_REL(rel);
@@ -7008,6 +7055,11 @@ apply_scanjoin_target_to_paths(PlannerInfo *root,
/* This recurses, so be paranoid. */
check_stack_depth();
+ /*
+ * TOOD: when/how do we want to generate gather paths if
+ * scanjoin_target_parallel_safe_ignoring_params = true
+ */
+
/*
* If the rel is partitioned, we want to drop its existing paths and
* generate new ones. This function would still be correct if we kept the
@@ -7048,6 +7100,67 @@ apply_scanjoin_target_to_paths(PlannerInfo *root,
generate_useful_gather_paths(root, rel, false);
/* Can't use parallel query above this level. */
+
+ /*
+ * There are cases where:
+ * (rel->consider_parallel &&
+ * !scanjoin_target_parallel_safe &&
+ * scanjoin_target_parallel_safe_ignoring_params)
+ * is true at this point. See longer commment below.
+ */
+ if (!(rel->consider_parallel_rechecking_params && scanjoin_target_parallel_safe_ignoring_params))
+ {
+ /*
+ * TODO: if we limit setting:
+ *
+ * rel->consider_parallel_rechecking_params = false
+ * rel->partial_pathlist = NIL
+ *
+ * to this condition, we're pushing off the checks as to whether or
+ * not a given param usage is safe in the context of a given path
+ * (in the context of a given rel?). That almost certainly means
+ * we'd have to add other checks later (maybe just on
+ * lateral/relids and not parallel safety overall), because at the
+ * end of grouping_planner() we copy partial paths to the
+ * final_rel, and while that path may be acceptable in some
+ * contexts it may not be in all contexts.
+ *
+ * OTOH if we're only dependent on PARAM_EXEC params, and we already
+ * know that subpath->param_info == NULL holds (and that seems like
+ * it must since we were going to replace the path target anyway...
+ * though the one caveat is from the original form of this function
+ * we'd only ever actually assert that for paths not partial paths)
+ * then if a param shows up in the target why would it not be parallel
+ * safe.
+ *
+ * Adding to the mystery even with the original form of this function
+ * we still end up with parallel paths where I'd expect this to
+ * disallow them. For example:
+ *
+ * SELECT '' AS six, f1 AS "Correlated Field", f3 AS "Second Field"
+ * FROM SUBSELECT_TBL upper
+ * WHERE f3 IN (
+ * SELECT upper.f1 + f2
+ * FROM SUBSELECT_TBL
+ * WHERE f2 = CAST(f3 AS integer)
+ * );
+ *
+ * ends up with the correlated query underneath parallel plan despite
+ * its target containing a param, and therefore this function marking
+ * the rel as consider_parallel=false and removing the partial paths.
+ *
+ * But the plan as a whole is parallel safe, and so the subplan is also
+ * parallel safe, which means we can incorporate it into a full parallel
+ * plan. In other words, this is a parallel safe, but not parallel aware
+ * subplan (and regular, not parallel, seq scan inside that subplan).
+ * It's not a partial path; it's a full path that is executed as a subquery.
+ *
+ * Current conclusion: it's fine for subplans, which is the case we're
+ * currently targeting anyway. And it might even be the only case that
+ * matters at all.
+ */
+ }
+ rel->consider_parallel_rechecking_params = false;
rel->partial_pathlist = NIL;
rel->consider_parallel = false;
}
@@ -7183,6 +7296,7 @@ apply_scanjoin_target_to_paths(PlannerInfo *root,
child_scanjoin_targets,
scanjoin_targets_contain_srfs,
scanjoin_target_parallel_safe,
+ scanjoin_target_parallel_safe_ignoring_params,
tlist_same_exprs);
/* Save non-dummy children for Append paths. */
@@ -7200,6 +7314,11 @@ apply_scanjoin_target_to_paths(PlannerInfo *root,
* avoid creating multiple Gather nodes within the same plan. We must do
* this after all paths have been generated and before set_cheapest, since
* one of the generated paths may turn out to be the cheapest one.
+ *
+ * TODO: This is the same problem as earlier in this function: when allowing
+ * "parallel safe ignoring params" paths here we don't actually know we are
+ * safe in any possible context just possibly safe in the context of the
+ * right rel.
*/
if (rel->consider_parallel && !IS_OTHER_REL(rel))
generate_useful_gather_paths(root, rel, false);
@@ -7307,6 +7426,7 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
child_grouped_rel = make_grouping_rel(root, child_input_rel,
child_target,
extra->target_parallel_safe,
+ extra->target_parallel_safe_ignoring_params,
child_extra.havingQual);
/* Create grouping paths for this child relation. */
diff --git a/src/backend/optimizer/plan/subselect.c b/src/backend/optimizer/plan/subselect.c
index 8c9408d372..6e5a8b09ca 100644
--- a/src/backend/optimizer/plan/subselect.c
+++ b/src/backend/optimizer/plan/subselect.c
@@ -342,6 +342,7 @@ build_subplan(PlannerInfo *root, Plan *plan, PlannerInfo *subroot,
splan->useHashTable = false;
splan->unknownEqFalse = unknownEqFalse;
splan->parallel_safe = plan->parallel_safe;
+ splan->parallel_safe_ignoring_params = plan->parallel_safe_ignoring_params;
splan->setParam = NIL;
splan->parParam = NIL;
splan->args = NIL;
@@ -1937,6 +1938,7 @@ process_sublinks_mutator(Node *node, process_sublinks_context *context)
{
SubLink *sublink = (SubLink *) node;
Node *testexpr;
+ Node *result;
/*
* First, recursively process the lefthand-side expressions, if any.
@@ -1948,12 +1950,29 @@ process_sublinks_mutator(Node *node, process_sublinks_context *context)
/*
* Now build the SubPlan node and make the expr to return.
*/
- return make_subplan(context->root,
+ result = make_subplan(context->root,
(Query *) sublink->subselect,
sublink->subLinkType,
sublink->subLinkId,
testexpr,
context->isTopQual);
+
+ /*
+ * If planning determined that a subpath was parallel safe as long
+ * as required params are provided by each individual worker then we
+ * can mark the resulting subplan actually parallel safe since we now
+ * know for certain how that path will be used.
+ */
+ if (IsA(result, SubPlan) && !((SubPlan*)result)->parallel_safe
+ && ((SubPlan*)result)->parallel_safe_ignoring_params
+ && enable_parallel_params_recheck)
+ {
+ Plan *subplan = planner_subplan_get_plan(context->root, (SubPlan*)result);
+ ((SubPlan*)result)->parallel_safe = is_parallel_safe(context->root, testexpr, NULL);
+ subplan->parallel_safe = ((SubPlan*)result)->parallel_safe;
+ }
+
+ return result;
}
/*
diff --git a/src/backend/optimizer/prep/prepunion.c b/src/backend/optimizer/prep/prepunion.c
index f004fad1d9..20074660e4 100644
--- a/src/backend/optimizer/prep/prepunion.c
+++ b/src/backend/optimizer/prep/prepunion.c
@@ -409,6 +409,7 @@ recurse_set_operations(Node *setOp, PlannerInfo *root,
Assert(subpath->param_info == NULL);
/* avoid apply_projection_to_path, in case of multiple refs */
+ /* TODO: how to we know the target is parallel safe? */
path = (Path *) create_projection_path(root, subpath->parent,
subpath, target);
lfirst(lc) = path;
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index a707dc9f26..611bd3f871 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -55,6 +55,15 @@
#include "utils/syscache.h"
#include "utils/typcache.h"
+bool enable_parallel_params_recheck = true;
+
+typedef struct
+{
+ PlannerInfo *root;
+ AggSplit aggsplit;
+ AggClauseCosts *costs;
+} get_agg_clause_costs_context;
+
typedef struct
{
ParamListInfo boundParams;
@@ -88,6 +97,8 @@ typedef struct
{
char max_hazard; /* worst proparallel hazard found so far */
char max_interesting; /* worst proparallel hazard of interest */
+ char max_hazard_ignoring_params;
+ bool check_params_independently;
List *safe_param_ids; /* PARAM_EXEC Param IDs to treat as safe */
} max_parallel_hazard_context;
@@ -625,19 +636,27 @@ max_parallel_hazard(Query *parse)
context.max_hazard = PROPARALLEL_SAFE;
context.max_interesting = PROPARALLEL_UNSAFE;
context.safe_param_ids = NIL;
+ context.check_params_independently = false;
(void) max_parallel_hazard_walker((Node *) parse, &context);
return context.max_hazard;
}
/*
* is_parallel_safe
- * Detect whether the given expr contains only parallel-safe functions
+ * Detect whether the given expr contains only parallel safe funcions
+ * XXX: It does more than functions?
+ *
+ * If provided, safe_ignoring_params will be set the result of running the same
+ * parallel safety checks with the exception that params will be allowed. This
+ * value is useful since params are not inherently parallel unsafe, but rather
+ * their usage context (whether or not the worker is able to provide the value)
+ * determines parallel safety.
*
* root->glob->maxParallelHazard must previously have been set to the
- * result of max_parallel_hazard() on the whole query.
+ * result of max_parallel_hazard() on the whole query
*/
bool
-is_parallel_safe(PlannerInfo *root, Node *node)
+is_parallel_safe(PlannerInfo *root, Node *node, bool *safe_ignoring_params)
{
max_parallel_hazard_context context;
PlannerInfo *proot;
@@ -654,8 +673,10 @@ is_parallel_safe(PlannerInfo *root, Node *node)
return true;
/* Else use max_parallel_hazard's search logic, but stop on RESTRICTED */
context.max_hazard = PROPARALLEL_SAFE;
+ context.max_hazard_ignoring_params = PROPARALLEL_SAFE;
context.max_interesting = PROPARALLEL_RESTRICTED;
context.safe_param_ids = NIL;
+ context.check_params_independently = safe_ignoring_params != NULL;
/*
* The params that refer to the same or parent query level are considered
@@ -673,12 +694,17 @@ is_parallel_safe(PlannerInfo *root, Node *node)
}
}
- return !max_parallel_hazard_walker(node, &context);
+ (void) max_parallel_hazard_walker(node, &context);
+
+ if (safe_ignoring_params)
+ *safe_ignoring_params = context.max_hazard_ignoring_params == PROPARALLEL_SAFE;
+
+ return context.max_hazard == PROPARALLEL_SAFE;
}
/* core logic for all parallel-hazard checks */
static bool
-max_parallel_hazard_test(char proparallel, max_parallel_hazard_context *context)
+max_parallel_hazard_test(char proparallel, max_parallel_hazard_context *context, bool from_param)
{
switch (proparallel)
{
@@ -689,12 +715,16 @@ max_parallel_hazard_test(char proparallel, max_parallel_hazard_context *context)
/* increase max_hazard to RESTRICTED */
Assert(context->max_hazard != PROPARALLEL_UNSAFE);
context->max_hazard = proparallel;
+ if (!from_param)
+ context->max_hazard_ignoring_params = proparallel;
/* done if we are not expecting any unsafe functions */
if (context->max_interesting == proparallel)
return true;
break;
case PROPARALLEL_UNSAFE:
context->max_hazard = proparallel;
+ if (!from_param)
+ context->max_hazard_ignoring_params = proparallel;
/* we're always done at the first unsafe construct */
return true;
default:
@@ -709,7 +739,41 @@ static bool
max_parallel_hazard_checker(Oid func_id, void *context)
{
return max_parallel_hazard_test(func_parallel(func_id),
- (max_parallel_hazard_context *) context);
+ (max_parallel_hazard_context *) context, false);
+}
+
+static bool
+max_parallel_hazard_walker_can_short_circuit(max_parallel_hazard_context *context)
+{
+ if (!context->check_params_independently)
+ return true;
+
+ switch (context->max_hazard)
+ {
+ case PROPARALLEL_SAFE:
+ /* nothing to see here, move along */
+ break;
+ case PROPARALLEL_RESTRICTED:
+ if (context->max_interesting == PROPARALLEL_RESTRICTED)
+ return context->max_hazard_ignoring_params != PROPARALLEL_SAFE;
+
+ /*
+ * We haven't even met our max interesting yet, so
+ * we certainly can't short-circuit.
+ */
+ break;
+ case PROPARALLEL_UNSAFE:
+ if (context->max_interesting == PROPARALLEL_RESTRICTED)
+ return context->max_hazard_ignoring_params != PROPARALLEL_SAFE;
+ else if (context->max_interesting == PROPARALLEL_UNSAFE)
+ return context->max_hazard_ignoring_params == PROPARALLEL_UNSAFE;
+
+ break;
+ default:
+ elog(ERROR, "unrecognized proparallel value \"%c\"", context->max_hazard);
+ break;
+ }
+ return false;
}
static bool
@@ -721,7 +785,7 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
/* Check for hazardous functions in node itself */
if (check_functions_in_node(node, max_parallel_hazard_checker,
context))
- return true;
+ return max_parallel_hazard_walker_can_short_circuit(context);
/*
* It should be OK to treat MinMaxExpr as parallel-safe, since btree
@@ -736,14 +800,14 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
*/
if (IsA(node, CoerceToDomain))
{
- if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
- return true;
+ if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context, false))
+ return max_parallel_hazard_walker_can_short_circuit(context);
}
else if (IsA(node, NextValueExpr))
{
- if (max_parallel_hazard_test(PROPARALLEL_UNSAFE, context))
- return true;
+ if (max_parallel_hazard_test(PROPARALLEL_UNSAFE, context, false))
+ return max_parallel_hazard_walker_can_short_circuit(context);
}
/*
@@ -756,8 +820,8 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
*/
else if (IsA(node, WindowFunc))
{
- if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
- return true;
+ if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context, false))
+ return max_parallel_hazard_walker_can_short_circuit(context);
}
/*
@@ -776,8 +840,8 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
*/
else if (IsA(node, SubLink))
{
- if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
- return true;
+ if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context, false))
+ return max_parallel_hazard_walker_can_short_circuit(context);
}
/*
@@ -792,18 +856,23 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
List *save_safe_param_ids;
if (!subplan->parallel_safe &&
- max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
+ (!enable_parallel_params_recheck || !subplan->parallel_safe_ignoring_params) &&
+ max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context, false) &&
+ max_parallel_hazard_walker_can_short_circuit(context))
return true;
save_safe_param_ids = context->safe_param_ids;
context->safe_param_ids = list_concat_copy(context->safe_param_ids,
subplan->paramIds);
- if (max_parallel_hazard_walker(subplan->testexpr, context))
- return true; /* no need to restore safe_param_ids */
+ if (max_parallel_hazard_walker(subplan->testexpr, context) &&
+ max_parallel_hazard_walker_can_short_circuit(context))
+ /* no need to restore safe_param_ids */
+ return true;
+
list_free(context->safe_param_ids);
context->safe_param_ids = save_safe_param_ids;
/* we must also check args, but no special Param treatment there */
if (max_parallel_hazard_walker((Node *) subplan->args, context))
- return true;
+ return max_parallel_hazard_walker_can_short_circuit(context);
/* don't want to recurse normally, so we're done */
return false;
}
@@ -825,8 +894,8 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
if (param->paramkind != PARAM_EXEC ||
!list_member_int(context->safe_param_ids, param->paramid))
{
- if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
- return true;
+ if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context, true))
+ return max_parallel_hazard_walker_can_short_circuit(context);
}
return false; /* nothing to recurse to */
}
@@ -844,7 +913,7 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
if (query->rowMarks != NULL)
{
context->max_hazard = PROPARALLEL_UNSAFE;
- return true;
+ return max_parallel_hazard_walker_can_short_circuit(context);
}
/* Recurse into subselects */
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 5c32c96b71..2c9764aa2b 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -756,10 +756,10 @@ add_partial_path(RelOptInfo *parent_rel, Path *new_path)
CHECK_FOR_INTERRUPTS();
/* Path to be added must be parallel safe. */
- Assert(new_path->parallel_safe);
+ Assert(new_path->parallel_safe || new_path->parallel_safe_ignoring_params);
/* Relation should be OK for parallelism, too. */
- Assert(parent_rel->consider_parallel);
+ Assert(parent_rel->consider_parallel || parent_rel->consider_parallel_rechecking_params);
/*
* As in add_path, throw out any paths which are dominated by the new
@@ -938,6 +938,7 @@ create_seqscan_path(PlannerInfo *root, RelOptInfo *rel,
required_outer);
pathnode->parallel_aware = (parallel_workers > 0);
pathnode->parallel_safe = rel->consider_parallel;
+ pathnode->parallel_safe_ignoring_params = rel->consider_parallel_rechecking_params;
pathnode->parallel_workers = parallel_workers;
pathnode->pathkeys = NIL; /* seqscan has unordered result */
@@ -1016,6 +1017,7 @@ create_index_path(PlannerInfo *root,
required_outer);
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel;
+ pathnode->path.parallel_safe_ignoring_params = rel->consider_parallel_rechecking_params;
pathnode->path.parallel_workers = 0;
pathnode->path.pathkeys = pathkeys;
@@ -1866,7 +1868,7 @@ create_gather_merge_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
Cost input_startup_cost = 0;
Cost input_total_cost = 0;
- Assert(subpath->parallel_safe);
+ Assert(subpath->parallel_safe || subpath->parallel_safe_ignoring_params);
Assert(pathkeys);
pathnode->path.pathtype = T_GatherMerge;
@@ -1954,7 +1956,7 @@ create_gather_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
{
GatherPath *pathnode = makeNode(GatherPath);
- Assert(subpath->parallel_safe);
+ Assert(subpath->parallel_safe || subpath->parallel_safe_ignoring_params);
pathnode->path.pathtype = T_Gather;
pathnode->path.parent = rel;
@@ -2001,6 +2003,8 @@ create_subqueryscan_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe;
+ pathnode->path.parallel_safe_ignoring_params = rel->consider_parallel_rechecking_params &&
+ subpath->parallel_safe_ignoring_params;
pathnode->path.parallel_workers = subpath->parallel_workers;
pathnode->path.pathkeys = pathkeys;
pathnode->subpath = subpath;
@@ -2418,6 +2422,8 @@ create_nestloop_path(PlannerInfo *root,
NestPath *pathnode = makeNode(NestPath);
Relids inner_req_outer = PATH_REQ_OUTER(inner_path);
+ /* TODO: Assert lateral relids subset safety? */
+
/*
* If the inner path is parameterized by the outer, we must drop any
* restrict_clauses that are due to be moved into the inner path. We have
@@ -2458,6 +2464,8 @@ create_nestloop_path(PlannerInfo *root,
pathnode->jpath.path.parallel_aware = false;
pathnode->jpath.path.parallel_safe = joinrel->consider_parallel &&
outer_path->parallel_safe && inner_path->parallel_safe;
+ pathnode->jpath.path.parallel_safe_ignoring_params = joinrel->consider_parallel_rechecking_params &&
+ outer_path->parallel_safe_ignoring_params && inner_path->parallel_safe_ignoring_params;
/* This is a foolish way to estimate parallel_workers, but for now... */
pathnode->jpath.path.parallel_workers = outer_path->parallel_workers;
pathnode->jpath.path.pathkeys = pathkeys;
@@ -2631,6 +2639,8 @@ create_projection_path(PlannerInfo *root,
{
ProjectionPath *pathnode = makeNode(ProjectionPath);
PathTarget *oldtarget;
+ bool target_parallel_safe;
+ bool target_parallel_safe_ignoring_params = false;
/*
* We mustn't put a ProjectionPath directly above another; it's useless
@@ -2654,9 +2664,12 @@ create_projection_path(PlannerInfo *root,
/* For now, assume we are above any joins, so no parameterization */
pathnode->path.param_info = NULL;
pathnode->path.parallel_aware = false;
+ target_parallel_safe = is_parallel_safe(root, (Node *) target->exprs,
+ &target_parallel_safe_ignoring_params);
pathnode->path.parallel_safe = rel->consider_parallel &&
- subpath->parallel_safe &&
- is_parallel_safe(root, (Node *) target->exprs);
+ subpath->parallel_safe && target_parallel_safe;
+ pathnode->path.parallel_safe_ignoring_params = rel->consider_parallel_rechecking_params &&
+ subpath->parallel_safe_ignoring_params && target_parallel_safe_ignoring_params;
pathnode->path.parallel_workers = subpath->parallel_workers;
/* Projection does not change the sort order */
pathnode->path.pathkeys = subpath->pathkeys;
@@ -2764,7 +2777,7 @@ apply_projection_to_path(PlannerInfo *root,
* parallel-safe in the target expressions, then we can't.
*/
if ((IsA(path, GatherPath) || IsA(path, GatherMergePath)) &&
- is_parallel_safe(root, (Node *) target->exprs))
+ is_parallel_safe(root, (Node *) target->exprs, NULL))
{
/*
* We always use create_projection_path here, even if the subpath is
@@ -2798,7 +2811,7 @@ apply_projection_to_path(PlannerInfo *root,
}
}
else if (path->parallel_safe &&
- !is_parallel_safe(root, (Node *) target->exprs))
+ !is_parallel_safe(root, (Node *) target->exprs, NULL))
{
/*
* We're inserting a parallel-restricted target list into a path
@@ -2806,6 +2819,7 @@ apply_projection_to_path(PlannerInfo *root,
* safe.
*/
path->parallel_safe = false;
+ path->parallel_safe_ignoring_params = false; /* TODO */
}
return path;
@@ -2838,7 +2852,7 @@ create_set_projection_path(PlannerInfo *root,
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe &&
- is_parallel_safe(root, (Node *) target->exprs);
+ is_parallel_safe(root, (Node *) target->exprs, NULL);
pathnode->path.parallel_workers = subpath->parallel_workers;
/* Projection does not change the sort order XXX? */
pathnode->path.pathkeys = subpath->pathkeys;
@@ -3115,6 +3129,8 @@ create_agg_path(PlannerInfo *root,
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe;
+ pathnode->path.parallel_safe_ignoring_params = rel->consider_parallel_rechecking_params &&
+ subpath->parallel_safe_ignoring_params;
pathnode->path.parallel_workers = subpath->parallel_workers;
if (aggstrategy == AGG_SORTED)
pathnode->path.pathkeys = subpath->pathkeys; /* preserves order */
@@ -3736,6 +3752,8 @@ create_limit_path(PlannerInfo *root, RelOptInfo *rel,
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe;
+ pathnode->path.parallel_safe_ignoring_params = rel->consider_parallel_rechecking_params &&
+ subpath->parallel_safe_ignoring_params;
pathnode->path.parallel_workers = subpath->parallel_workers;
pathnode->path.rows = subpath->rows;
pathnode->path.startup_cost = subpath->startup_cost;
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 520409f4ba..b1503226f9 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -213,6 +213,7 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
rel->consider_startup = (root->tuple_fraction > 0);
rel->consider_param_startup = false; /* might get changed later */
rel->consider_parallel = false; /* might get changed later */
+ rel->consider_parallel_rechecking_params = false; /* might get changed later */
rel->reltarget = create_empty_pathtarget();
rel->pathlist = NIL;
rel->ppilist = NIL;
@@ -617,6 +618,7 @@ build_join_rel(PlannerInfo *root,
joinrel->consider_startup = (root->tuple_fraction > 0);
joinrel->consider_param_startup = false;
joinrel->consider_parallel = false;
+ joinrel->consider_parallel_rechecking_params = false;
joinrel->reltarget = create_empty_pathtarget();
joinrel->pathlist = NIL;
joinrel->ppilist = NIL;
@@ -743,10 +745,27 @@ build_join_rel(PlannerInfo *root,
* take; therefore, we should make the same decision here however we get
* here.
*/
- if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
- is_parallel_safe(root, (Node *) restrictlist) &&
- is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
- joinrel->consider_parallel = true;
+ if ((inner_rel->consider_parallel || inner_rel->consider_parallel_rechecking_params)
+ && (outer_rel->consider_parallel || outer_rel->consider_parallel_rechecking_params))
+ {
+ bool restrictlist_parallel_safe;
+ bool restrictlist_parallel_safe_ignoring_params = false;
+ bool target_parallel_safe;
+ bool target_parallel_safe_ignoring_params = false;
+
+ restrictlist_parallel_safe = is_parallel_safe(root, (Node *) restrictlist, &restrictlist_parallel_safe_ignoring_params);
+ target_parallel_safe = is_parallel_safe(root, (Node *) joinrel->reltarget->exprs, &target_parallel_safe_ignoring_params);
+
+ if (inner_rel->consider_parallel && outer_rel->consider_parallel
+ && restrictlist_parallel_safe && target_parallel_safe)
+ joinrel->consider_parallel = true;
+
+ if (inner_rel->consider_parallel_rechecking_params
+ && outer_rel->consider_parallel_rechecking_params
+ && restrictlist_parallel_safe_ignoring_params
+ && target_parallel_safe_ignoring_params)
+ joinrel->consider_parallel_rechecking_params = true;
+ }
/* Add the joinrel to the PlannerInfo. */
add_join_rel(root, joinrel);
@@ -805,6 +824,7 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->consider_startup = (root->tuple_fraction > 0);
joinrel->consider_param_startup = false;
joinrel->consider_parallel = false;
+ joinrel->consider_parallel_rechecking_params = false;
joinrel->reltarget = create_empty_pathtarget();
joinrel->pathlist = NIL;
joinrel->ppilist = NIL;
@@ -892,6 +912,7 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
/* Child joinrel is parallel safe if parent is parallel safe. */
joinrel->consider_parallel = parent_joinrel->consider_parallel;
+ joinrel->consider_parallel_rechecking_params = parent_joinrel->consider_parallel_rechecking_params;
/* Set estimates of the child-joinrel's size. */
set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
@@ -1236,6 +1257,7 @@ fetch_upper_rel(PlannerInfo *root, UpperRelationKind kind, Relids relids)
upperrel->consider_startup = (root->tuple_fraction > 0);
upperrel->consider_param_startup = false;
upperrel->consider_parallel = false; /* might get changed later */
+ upperrel->consider_parallel_rechecking_params = false; /* might get changed later */
upperrel->reltarget = create_empty_pathtarget();
upperrel->pathlist = NIL;
upperrel->cheapest_startup_path = NULL;
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index effb9d03a0..ddac1aac62 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -58,6 +58,7 @@
#include "libpq/libpq.h"
#include "libpq/pqformat.h"
#include "miscadmin.h"
+#include "optimizer/clauses.h"
#include "optimizer/cost.h"
#include "optimizer/geqo.h"
#include "optimizer/optimizer.h"
@@ -972,6 +973,16 @@ static const unit_conversion time_unit_conversion_table[] =
static struct config_bool ConfigureNamesBool[] =
{
+ {
+ {"enable_parallel_params_recheck", PGC_USERSET, QUERY_TUNING_METHOD,
+ gettext_noop("Enables the planner's rechecking of parallel safety in the presence of PARAM_EXEC params (for correlated subqueries)."),
+ NULL,
+ GUC_EXPLAIN
+ },
+ &enable_parallel_params_recheck,
+ true,
+ NULL, NULL, NULL
+ },
{
{"enable_seqscan", PGC_USERSET, QUERY_TUNING_METHOD,
gettext_noop("Enables the planner's use of sequential-scan plans."),
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 1f33fe13c1..dc036863c9 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -687,6 +687,7 @@ typedef struct RelOptInfo
bool consider_startup; /* keep cheap-startup-cost paths? */
bool consider_param_startup; /* ditto, for parameterized paths? */
bool consider_parallel; /* consider parallel paths? */
+ bool consider_parallel_rechecking_params; /* consider parallel paths? */
/* default result targetlist for Paths scanning this relation */
struct PathTarget *reltarget; /* list of Vars/Exprs, cost, width */
@@ -1186,6 +1187,7 @@ typedef struct Path
bool parallel_aware; /* engage parallel-aware logic? */
bool parallel_safe; /* OK to use as part of parallel plan? */
+ bool parallel_safe_ignoring_params; /* OK to use as part of parallel plan if worker context provides params? */
int parallel_workers; /* desired # of workers; 0 = not parallel */
/* estimated size/costs for path (see costsize.c for more info) */
@@ -2478,7 +2480,7 @@ typedef struct MinMaxAggInfo
* for conflicting purposes.
*
* In addition, PARAM_EXEC slots are assigned for Params representing outputs
- * from subplans (values that are setParam items for those subplans). These
+ * from subplans (values that are setParam items for those subplans). [TODO: is this true, or only for init plans?] These
* IDs need not be tracked via PlannerParamItems, since we do not need any
* duplicate-elimination nor later processing of the represented expressions.
* Instead, we just record the assignment of the slot number by appending to
@@ -2597,6 +2599,7 @@ typedef struct
/* Data which may differ across partitions. */
bool target_parallel_safe;
+ bool target_parallel_safe_ignoring_params;
Node *havingQual;
List *targetList;
PartitionwiseAggregateType patype;
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 0b518ce6b2..48a0738e44 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -128,6 +128,7 @@ typedef struct Plan
*/
bool parallel_aware; /* engage parallel-aware logic? */
bool parallel_safe; /* OK to use as part of parallel plan? */
+ bool parallel_safe_ignoring_params; /* OK to use as part of parallel plan if worker context provides params? */
/*
* information needed for asynchronous execution
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
index dab5c4ff5d..a5884164bf 100644
--- a/src/include/nodes/primnodes.h
+++ b/src/include/nodes/primnodes.h
@@ -764,6 +764,7 @@ typedef struct SubPlan
* spec result is UNKNOWN; this allows much
* simpler handling of null values */
bool parallel_safe; /* is the subplan parallel-safe? */
+ bool parallel_safe_ignoring_params; /* is the subplan parallel-safe when params are provided by the worker context? */
/* Note: parallel_safe does not consider contents of testexpr or args */
/* Information for passing params into and out of the subselect: */
/* setParam and parParam are lists of integers (param IDs) */
diff --git a/src/include/optimizer/clauses.h b/src/include/optimizer/clauses.h
index 6c5203dc44..5005d4927b 100644
--- a/src/include/optimizer/clauses.h
+++ b/src/include/optimizer/clauses.h
@@ -16,6 +16,8 @@
#include "nodes/pathnodes.h"
+extern PGDLLIMPORT bool enable_parallel_params_recheck;
+
typedef struct
{
int numWindowFuncs; /* total number of WindowFuncs found */
@@ -33,7 +35,7 @@ extern double expression_returns_set_rows(PlannerInfo *root, Node *clause);
extern bool contain_subplans(Node *clause);
extern char max_parallel_hazard(Query *parse);
-extern bool is_parallel_safe(PlannerInfo *root, Node *node);
+extern bool is_parallel_safe(PlannerInfo *root, Node *node, bool *safe_ignoring_params);
extern bool contain_nonstrict_functions(Node *clause);
extern bool contain_exec_param(Node *clause, List *param_ids);
extern bool contain_leaked_vars(Node *clause);
diff --git a/src/test/regress/expected/incremental_sort.out b/src/test/regress/expected/incremental_sort.out
index 545e301e48..8f9ca05e60 100644
--- a/src/test/regress/expected/incremental_sort.out
+++ b/src/test/regress/expected/incremental_sort.out
@@ -1614,16 +1614,16 @@ from tenk1 t, generate_series(1, 1000);
QUERY PLAN
---------------------------------------------------------------------------------
Unique
- -> Sort
- Sort Key: t.unique1, ((SubPlan 1))
- -> Gather
- Workers Planned: 2
+ -> Gather Merge
+ Workers Planned: 2
+ -> Sort
+ Sort Key: t.unique1, ((SubPlan 1))
-> Nested Loop
-> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
-> Function Scan on generate_series
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on tenk1
- Index Cond: (unique1 = t.unique1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on tenk1
+ Index Cond: (unique1 = t.unique1)
(11 rows)
explain (costs off) select
@@ -1633,16 +1633,16 @@ from tenk1 t, generate_series(1, 1000)
order by 1, 2;
QUERY PLAN
---------------------------------------------------------------------------
- Sort
- Sort Key: t.unique1, ((SubPlan 1))
- -> Gather
- Workers Planned: 2
+ Gather Merge
+ Workers Planned: 2
+ -> Sort
+ Sort Key: t.unique1, ((SubPlan 1))
-> Nested Loop
-> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
-> Function Scan on generate_series
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on tenk1
- Index Cond: (unique1 = t.unique1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on tenk1
+ Index Cond: (unique1 = t.unique1)
(10 rows)
-- Parallel sort but with expression not available until the upper rel.
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 2303f70d6e..253f117d7a 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -311,6 +311,131 @@ select count(*) from tenk1 where (two, four) not in
10000
(1 row)
+-- test parallel plans for queries containing correlated subplans
+-- where the subplan only needs params available from the current
+-- worker's scan.
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t, generate_series(1, 10);
+ QUERY PLAN
+----------------------------------------------------------------------------
+ Gather
+ Output: ((SubPlan 1))
+ Workers Planned: 4
+ -> Nested Loop
+ Output: (SubPlan 1)
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ -> Function Scan on pg_catalog.generate_series
+ Output: generate_series.generate_series
+ Function Call: generate_series(1, 10)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(14 rows)
+
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t;
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: ((SubPlan 1))
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(9 rows)
+
+explain (analyze, costs off, summary off, verbose, timing off) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t
+ limit 1;
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------
+ Limit (actual rows=1 loops=1)
+ Output: ((SubPlan 1))
+ -> Gather (actual rows=1 loops=1)
+ Output: ((SubPlan 1))
+ Workers Planned: 4
+ Workers Launched: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t (actual rows=1 loops=5)
+ Output: (SubPlan 1)
+ Heap Fetches: 0
+ Worker 0: actual rows=1 loops=1
+ Worker 1: actual rows=1 loops=1
+ Worker 2: actual rows=1 loops=1
+ Worker 3: actual rows=1 loops=1
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1 (actual rows=1 loops=5)
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+ Heap Fetches: 0
+ Worker 0: actual rows=1 loops=1
+ Worker 1: actual rows=1 loops=1
+ Worker 2: actual rows=1 loops=1
+ Worker 3: actual rows=1 loops=1
+(22 rows)
+
+explain (costs off, verbose) select t.unique1
+ from tenk1 t
+ where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: t.unique1
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ Filter: (t.unique1 = (SubPlan 1))
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(10 rows)
+
+explain (costs off, verbose) select *
+ from tenk1 t
+ order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Gather Merge
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
+ Workers Planned: 4
+ -> Sort
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
+ Sort Key: ((SubPlan 1))
+ -> Parallel Seq Scan on public.tenk1 t
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(12 rows)
+
+-- test subplan in join/lateral join
+explain (costs off, verbose, timing off) select t.unique1, l.*
+ from tenk1 t
+ join lateral (
+ select (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1 offset 0)
+ ) l on true;
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: t.unique1, ((SubPlan 1))
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1, (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(9 rows)
+
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
explain (costs off)
select * from tenk1 where (unique1 + random())::integer not in
@@ -1192,6 +1317,18 @@ EXECUTE pstmt('1', make_some_array(1,2));
DEALLOCATE pstmt;
-- test interaction between subquery and partial_paths
+-- this plan changes to using a non-parallel index only
+-- scan on tenk1_unique1 (the parallel version of the subquery scan
+-- is cheaper, but only by ~30, and cost comparison treats them as equal
+-- since the costs are so large) because set_rel_consider_parallel
+-- called from make_one_rel sees the subplan as parallel safe now
+-- (in context it now knows the params are actually parallel safe).
+-- Because of that the non-parallel index path is now parallel_safe=true,
+-- therefore it wins the COSTS_EQUAL comparison in add_path.
+-- Perhaps any is_parallel_safe calls made for the purpose of determining
+-- consider_parallel should disable that behavior? It's not clear which is
+-- correct.
+set enable_parallel_params_recheck = off;
CREATE VIEW tenk1_vw_sec WITH (security_barrier) AS SELECT * FROM tenk1;
EXPLAIN (COSTS OFF)
SELECT 1 FROM tenk1_vw_sec
diff --git a/src/test/regress/expected/sysviews.out b/src/test/regress/expected/sysviews.out
index 2088857615..297d2bbdf4 100644
--- a/src/test/regress/expected/sysviews.out
+++ b/src/test/regress/expected/sysviews.out
@@ -116,13 +116,14 @@ select name, setting from pg_settings where name like 'enable%';
enable_nestloop | on
enable_parallel_append | on
enable_parallel_hash | on
+ enable_parallel_params_recheck | on
enable_partition_pruning | on
enable_partitionwise_aggregate | off
enable_partitionwise_join | off
enable_seqscan | on
enable_sort | on
enable_tidscan | on
-(20 rows)
+(21 rows)
-- Test that the pg_timezone_names and pg_timezone_abbrevs views are
-- more-or-less working. We can't test their contents in any great detail
diff --git a/src/test/regress/sql/select_parallel.sql b/src/test/regress/sql/select_parallel.sql
index 019e17e751..f217d0ec9b 100644
--- a/src/test/regress/sql/select_parallel.sql
+++ b/src/test/regress/sql/select_parallel.sql
@@ -111,6 +111,31 @@ explain (costs off)
(select hundred, thousand from tenk2 where thousand > 100);
select count(*) from tenk1 where (two, four) not in
(select hundred, thousand from tenk2 where thousand > 100);
+-- test parallel plans for queries containing correlated subplans
+-- where the subplan only needs params available from the current
+-- worker's scan.
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t, generate_series(1, 10);
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t;
+explain (analyze, costs off, summary off, verbose, timing off) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t
+ limit 1;
+explain (costs off, verbose) select t.unique1
+ from tenk1 t
+ where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+explain (costs off, verbose) select *
+ from tenk1 t
+ order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+-- test subplan in join/lateral join
+explain (costs off, verbose, timing off) select t.unique1, l.*
+ from tenk1 t
+ join lateral (
+ select (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1 offset 0)
+ ) l on true;
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
explain (costs off)
select * from tenk1 where (unique1 + random())::integer not in
@@ -456,6 +481,18 @@ EXECUTE pstmt('1', make_some_array(1,2));
DEALLOCATE pstmt;
-- test interaction between subquery and partial_paths
+-- this plan changes to using a non-parallel index only
+-- scan on tenk1_unique1 (the parallel version of the subquery scan
+-- is cheaper, but only by ~30, and cost comparison treats them as equal
+-- since the costs are so large) because set_rel_consider_parallel
+-- called from make_one_rel sees the subplan as parallel safe now
+-- (in context it now knows the params are actually parallel safe).
+-- Because of that the non-parallel index path is now parallel_safe=true,
+-- therefore it wins the COSTS_EQUAL comparison in add_path.
+-- Perhaps any is_parallel_safe calls made for the purpose of determining
+-- consider_parallel should disable that behavior? It's not clear which is
+-- correct.
+set enable_parallel_params_recheck = off;
CREATE VIEW tenk1_vw_sec WITH (security_barrier) AS SELECT * FROM tenk1;
EXPLAIN (COSTS OFF)
SELECT 1 FROM tenk1_vw_sec
--
2.20.1
v4-0003-Other-places-to-consider-for-completeness.patchapplication/octet-stream; name=v4-0003-Other-places-to-consider-for-completeness.patchDownload
From 8c9e1f4d75be114fb4e0878c0624bcb2be37f214 Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Fri, 7 May 2021 15:37:22 +0000
Subject: [PATCH v4 3/3] Other places to consider for completeness
---
src/backend/optimizer/path/allpaths.c | 4 ++++
src/backend/optimizer/plan/planmain.c | 2 +-
src/backend/optimizer/plan/planner.c | 3 +++
src/backend/optimizer/plan/subselect.c | 3 +++
src/backend/optimizer/prep/prepunion.c | 2 ++
5 files changed, 13 insertions(+), 1 deletion(-)
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index e419773dcf..93d522b141 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -1142,6 +1142,8 @@ set_append_rel_size(PlannerInfo *root, RelOptInfo *rel,
*/
if (!childrel->consider_parallel)
rel->consider_parallel = false;
+ if (!childrel->consider_parallel_rechecking_params)
+ rel->consider_parallel_rechecking_params = false;
/*
* Accumulate size information from each live child.
@@ -1263,6 +1265,8 @@ set_append_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
*/
if (!rel->consider_parallel)
childrel->consider_parallel = false;
+ if (!childrel->consider_parallel_rechecking_params)
+ rel->consider_parallel_rechecking_params = false;
/*
* Compute the child's access paths.
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index e3f4214d68..c2276fdfcf 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -119,7 +119,7 @@ query_planner(PlannerInfo *root,
if (root->glob->parallelModeOK &&
force_parallel_mode != FORCE_PARALLEL_OFF)
final_rel->consider_parallel =
- is_parallel_safe(root, parse->jointree->quals, NULL);
+ is_parallel_safe(root, parse->jointree->quals, &final_rel->consider_parallel_rechecking_params);
/*
* The only path for it is a trivial Result path. We cheat a
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index a4057ab48c..32ba4a2cb9 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -4096,6 +4096,7 @@ create_window_paths(PlannerInfo *root,
if (input_rel->consider_parallel && output_target_parallel_safe &&
is_parallel_safe(root, (Node *) activeWindows, NULL))
window_rel->consider_parallel = true;
+ /* consider_parallel_rechecking_params */
/*
* If the input rel belongs to a single FDW, so does the window rel.
@@ -4293,6 +4294,7 @@ create_distinct_paths(PlannerInfo *root, RelOptInfo *input_rel)
* expressions are parallel-safe.
*/
distinct_rel->consider_parallel = input_rel->consider_parallel;
+ distinct_rel->consider_parallel_rechecking_params = input_rel->consider_parallel_rechecking_params;
/*
* If the input rel belongs to a single FDW, so does the distinct_rel.
@@ -4647,6 +4649,7 @@ create_ordered_paths(PlannerInfo *root,
*/
if (input_rel->consider_parallel && target_parallel_safe)
ordered_rel->consider_parallel = true;
+ /* consider_parallel_rechecking_params */
/*
* If the input rel belongs to a single FDW, so does the ordered_rel.
diff --git a/src/backend/optimizer/plan/subselect.c b/src/backend/optimizer/plan/subselect.c
index 6e5a8b09ca..bcd0bfa580 100644
--- a/src/backend/optimizer/plan/subselect.c
+++ b/src/backend/optimizer/plan/subselect.c
@@ -1020,6 +1020,7 @@ SS_process_ctes(PlannerInfo *root)
* parallel-safe.
*/
splan->parallel_safe = false;
+ splan->parallel_safe_ignoring_params = false;
splan->setParam = NIL;
splan->parParam = NIL;
splan->args = NIL;
@@ -2176,6 +2177,7 @@ SS_charge_for_initplans(PlannerInfo *root, RelOptInfo *final_rel)
path->startup_cost += initplan_cost;
path->total_cost += initplan_cost;
path->parallel_safe = false;
+ path->parallel_safe_ignoring_params = false;
}
/*
@@ -2184,6 +2186,7 @@ SS_charge_for_initplans(PlannerInfo *root, RelOptInfo *final_rel)
*/
final_rel->partial_pathlist = NIL;
final_rel->consider_parallel = false;
+ final_rel->consider_parallel_rechecking_params = false;
/* We needn't do set_cheapest() here, caller will do it */
}
diff --git a/src/backend/optimizer/prep/prepunion.c b/src/backend/optimizer/prep/prepunion.c
index 20074660e4..486520fe22 100644
--- a/src/backend/optimizer/prep/prepunion.c
+++ b/src/backend/optimizer/prep/prepunion.c
@@ -273,6 +273,7 @@ recurse_set_operations(Node *setOp, PlannerInfo *root,
*/
final_rel = fetch_upper_rel(subroot, UPPERREL_FINAL, NULL);
rel->consider_parallel = final_rel->consider_parallel;
+ rel->consider_parallel_rechecking_params = final_rel->consider_parallel_rechecking_params;
/*
* For the moment, we consider only a single Path for the subquery.
@@ -617,6 +618,7 @@ generate_union_paths(SetOperationStmt *op, PlannerInfo *root,
result_rel = fetch_upper_rel(root, UPPERREL_SETOP, relids);
result_rel->reltarget = create_pathtarget(root, tlist);
result_rel->consider_parallel = consider_parallel;
+ /* consider_parallel_rechecking_params */
/*
* Append the child results together.
--
2.20.1
Re: Parallelize correlated subqueries that execute within each worker
On Mon, Nov 15, 2021 at 10:01 AM Robert Haas <robertmhaas@gmail.com> wrote:
On Wed, Nov 3, 2021 at 1:34 PM James Coleman <jtc331@gmail.com> wrote:
As I understand the current code, parallel plans are largely chosen
based not on where it's safe to insert a Gather node but rather by
determining if a given path is parallel safe. Through that lens params
are a bit of an odd man out -- they aren't inherently unsafe in the
way a parallel-unsafe function is, but they can only be used in
parallel plans under certain conditions (whether because of project
policy, performance, or missing infrastructure).Right.
Introducing consider_parallel_rechecking_params and
parallel_safe_ignoring_params allows us to keep more context on params
and make a more nuanced decision at the proper level of the plan. This
is what I mean by "rechecked in the using context", though I realize
now that both "recheck" and "context" are overloaded terms in the
project, so don't describe the concept particularly clearly. When a
path relies on params we can only make a final determination about its
parallel safety if we know whether or not the current parallel node
can provide the param's value. We don't necessarily know that
information until we attempt to generate a full parallel node in the
plan (I think what you're describing as "inserting a Gather node")
since the param may come from another node in the plan. These new
values allow us to do that by tracking tentatively parallel-safe
subplans (given proper Gather node placement) and delaying the
parallel-safety determination until the point at which a param is
available (or not).So I think I agree with you here. But I don't like all of this
"ignoring_params" stuff and I don't see why it's necessary. Say we
don't have both parallel_safe and parallel_safe_ignoring_params. Say
we just have parallel_safe. If the plan will be parallel safe if the
params are available, we label it parallel safe. If the plan will not
be parallel safe even if the params are available, we say it's not
parallel safe. Then, when we get to generate_gather_paths(), we don't
generate any paths if there are required parameters that are not
available. What's wrong with that approach?Maybe it's clearer to say this: I feel like one extra Boolean is
either too much or too little. I think maybe it's not even needed. But
if it is needed, then why just a bool instead of, say, a Bitmapset of
params that are needed, or something?I'm sort of speaking from intuition here rather than sure knowledge. I
might be totally wrong.
Apologies for quite the delay responding to this.
I've been chewing on this a bit, and I was about to go re-read the
code and see how easy it'd be to do exactly what you're suggesting in
generate_gather_paths() (and verifying it doesn't need to happen in
other places). However there's one (I think large) gotcha with that
approach (assuming it otherwise makes sense): it means we do
unnecessary work. In the current patch series we only need to recheck
parallel safety if we're in a situation where we might actually
benefit from doing that work (namely when we have a correlated
subquery we might otherwise be able to execute in a parallel plan). If
we don't track that status we'd have to recheck the full parallel
safety of the path for all paths -- even without correlated
subqueries.
Alternatively we could merge these fields into a single enum field
that tracked these states. Even better, we could use a bitmap to
signify what items are/aren't parallel safe. I'm not sure if that'd
create even larger churn in the patch, but maybe it's worth it either
way. In theory it'd open up further expansions to this concept later
(though I'm not aware of any such ideas).
If you think such an approach would be an improvement I'd be happy to
take a pass at a revised patch.
Thoughts?
Thanks,
James Coleman
Re: Parallelize correlated subqueries that execute within each worker
On Fri, Jan 14, 2022 at 2:25 PM James Coleman <jtc331@gmail.com> wrote:
I've been chewing on this a bit, and I was about to go re-read the
code and see how easy it'd be to do exactly what you're suggesting in
generate_gather_paths() (and verifying it doesn't need to happen in
other places). However there's one (I think large) gotcha with that
approach (assuming it otherwise makes sense): it means we do
unnecessary work. In the current patch series we only need to recheck
parallel safety if we're in a situation where we might actually
benefit from doing that work (namely when we have a correlated
subquery we might otherwise be able to execute in a parallel plan). If
we don't track that status we'd have to recheck the full parallel
safety of the path for all paths -- even without correlated
subqueries.
I don't think there's an intrinsic problem with the idea of making a
tentative determination about parallel safety and then refining it
later, but I'm not sure why you think it would be a lot of work to
figure this out at the point where we generate gather paths. I think
it's just a matter of testing whether the set of parameters that the
path needs as input is the empty set. It may be that neither extParam
nor allParam are precisely that thing, but I think both are very
close, and it seems to me that there's no theoretical reason why we
can't know for every path the set of inputs that it requires "from the
outside."
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: Parallelize correlated subqueries that execute within each worker
Robert Haas <robertmhaas@gmail.com> writes:
I don't think there's an intrinsic problem with the idea of making a
tentative determination about parallel safety and then refining it
later, but I'm not sure why you think it would be a lot of work to
figure this out at the point where we generate gather paths. I think
it's just a matter of testing whether the set of parameters that the
path needs as input is the empty set. It may be that neither extParam
nor allParam are precisely that thing, but I think both are very
close, and it seems to me that there's no theoretical reason why we
can't know for every path the set of inputs that it requires "from the
outside."
I'd be very happy if someone redesigned the extParam/allParam mechanism,
or at least documented it better. It's confusing and I've never been
able to escape the feeling that it's somewhat redundant.
The real problem with it though is that we don't compute those values
until much too late to be useful in path construction; see comments
for SS_identify_outer_params. To be helpful to the planner, we'd have
to rejigger things at least enough to calculate them earlier -- or
maybe better, calculate what the planner wants earlier, and then transform
to what the executor wants later.
regards, tom lane
Re: Parallelize correlated subqueries that execute within each worker
On Fri, Jan 21, 2022 at 3:20 PM Robert Haas <robertmhaas@gmail.com> wrote:
On Fri, Jan 14, 2022 at 2:25 PM James Coleman <jtc331@gmail.com> wrote:
I've been chewing on this a bit, and I was about to go re-read the
code and see how easy it'd be to do exactly what you're suggesting in
generate_gather_paths() (and verifying it doesn't need to happen in
other places). However there's one (I think large) gotcha with that
approach (assuming it otherwise makes sense): it means we do
unnecessary work. In the current patch series we only need to recheck
parallel safety if we're in a situation where we might actually
benefit from doing that work (namely when we have a correlated
subquery we might otherwise be able to execute in a parallel plan). If
we don't track that status we'd have to recheck the full parallel
safety of the path for all paths -- even without correlated
subqueries.I don't think there's an intrinsic problem with the idea of making a
tentative determination about parallel safety and then refining it
later, but I'm not sure why you think it would be a lot of work to
figure this out at the point where we generate gather paths. I think
it's just a matter of testing whether the set of parameters that the
path needs as input is the empty set. It may be that neither extParam
nor allParam are precisely that thing, but I think both are very
close, and it seems to me that there's no theoretical reason why we
can't know for every path the set of inputs that it requires "from the
outside."
As I understand it now (not sure I realized this before) you're
suggesting that *even when there are required params* marking it as
parallel safe, and then checking the params for parallel safety later.
From a purely theoretical perspective that seemed reasonable, so I
took a pass at that approach.
The first, and likely most interesting, thing I discovered was that
the vast majority of what the patch accomplishes it does so not via
the delayed params safety checking but rather via the required outer
relids checks I'd added to generate_useful_gather_paths.
For that to happen I did have to mark PARAM_EXEC params as presumed
parallel safe. That means that parallel_safe now doesn't strictly mean
"parallel safe in the current context" but "parallel safe as long as
any params are provided". That's a real change, but probably
acceptable as long as a project policy decision is made in that
direction.
There are a few concerns I have (and I'm not sure what level they rise to):
1. From what I can tell we don't have access on a path to the set of
params required by that path (I believe this is what Tom was
referencing in his sister reply at this point in the thread). That
means we have to rely on checking that the required outer relids are
provided by the current query level. I'm not quite sure yet whether or
not that guarantees (or if the rest of the path construction logic
guarantees for us) that the params provided by the outer rel are used
in a correlated way that isn't shared across workers. And because we
don't have the param information available we can't add additional
checks (that I can tell) to verify that.
2. Are we excluding any paths (by having one that will always be
invalid win the cost comparisons in add_partial_path)? I suppose this
danger actually exists in the previous version of the patch as well,
and I don't actually have any examples of this being a problem. Also
maybe this can only be a problem if (1) reveals a bug.
3. The new patch series actually ends up allowing parallelization of
correlated params in a few more places than the original patch series.
From what I can tell all of the cases are in fact safe to execute in
parallel, which, if true, means this is a feature not a concern. The
changed query plans fall into two categories: a.) putting a gather
inside a subplan and b.) correlated param usages in a subquery scan
path on the inner side of a join. I've separated out those specific
changes in a separate patch to make it easier to tell which ones I'm
referencing.
On the other hand this is a dramatically simpler patch series.
Assuming the approach is sound, it should much easier to maintain than
the previous version.
The final patch in the series is a set of additional checks I could
imagine to try to be more explicit, but at least in the current test
suite there isn't anything at all they affect.
Does this look at least somewhat more like what you'd envisionsed
(granting the need to squint hard given the relids checks instead of
directly checking params)?
Thanks,
James Coleman
Attachments:
v4-0001-Allow-parallel-LATERAL-subqueries-with-LIMIT-OFFS.patchtext/x-patch; charset=US-ASCII; name=v4-0001-Allow-parallel-LATERAL-subqueries-with-LIMIT-OFFS.patchDownload
From 6596bb6909b9dfaadd6e6e834be4032e7903c4e9 Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Mon, 30 Nov 2020 11:36:35 -0500
Subject: [PATCH v4 1/4] Allow parallel LATERAL subqueries with LIMIT/OFFSET
The code that determined whether or not a rel should be considered for
parallel query excluded subqueries with LIMIT/OFFSET. That's correct in
the general case: as the comment notes that'd mean we have to guarantee
ordering (and claims it's not worth checking that) for results to be
consistent across workers. However there's a simpler case that hasn't
been considered: LATERAL subqueries with LIMIT/OFFSET don't fall under
the same reasoning since they're executed (when not converted to a JOIN)
per tuple anyway, so consistency of results across workers isn't a
factor.
---
src/backend/optimizer/path/allpaths.c | 4 +++-
src/test/regress/expected/select_parallel.out | 15 +++++++++++++++
src/test/regress/sql/select_parallel.sql | 6 ++++++
3 files changed, 24 insertions(+), 1 deletion(-)
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 169b1d53fc..6a581e20fa 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -682,11 +682,13 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
* inconsistent results at the top-level. (In some cases, where
* the result is ordered, we could relax this restriction. But it
* doesn't currently seem worth expending extra effort to do so.)
+ * LATERAL is an exception: LIMIT/OFFSET is safe to execute within
+ * workers since the sub-select is executed per tuple
*/
{
Query *subquery = castNode(Query, rte->subquery);
- if (limit_needed(subquery))
+ if (!rte->lateral && limit_needed(subquery))
return;
}
break;
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 4ea1aa7dfd..2303f70d6e 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -1040,6 +1040,21 @@ explain (costs off)
Filter: (stringu1 ~~ '%AAAA'::text)
(11 rows)
+-- ...unless it's LATERAL
+savepoint settings;
+set parallel_tuple_cost=0;
+explain (costs off) select t.unique1 from tenk1 t
+join lateral (select t.unique1 from tenk1 offset 0) l on true;
+ QUERY PLAN
+---------------------------------------------------------------------
+ Gather
+ Workers Planned: 4
+ -> Nested Loop
+ -> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
+ -> Index Only Scan using tenk1_hundred on tenk1
+(5 rows)
+
+rollback to savepoint settings;
-- to increase the parallel query test coverage
SAVEPOINT settings;
SET LOCAL force_parallel_mode = 1;
diff --git a/src/test/regress/sql/select_parallel.sql b/src/test/regress/sql/select_parallel.sql
index f924731248..019e17e751 100644
--- a/src/test/regress/sql/select_parallel.sql
+++ b/src/test/regress/sql/select_parallel.sql
@@ -390,6 +390,12 @@ explain (costs off, verbose)
explain (costs off)
select * from tenk1 a where two in
(select two from tenk1 b where stringu1 like '%AAAA' limit 3);
+-- ...unless it's LATERAL
+savepoint settings;
+set parallel_tuple_cost=0;
+explain (costs off) select t.unique1 from tenk1 t
+join lateral (select t.unique1 from tenk1 offset 0) l on true;
+rollback to savepoint settings;
-- to increase the parallel query test coverage
SAVEPOINT settings;
--
2.17.1
v4-0003-Changed-queries.patchtext/x-patch; charset=US-ASCII; name=v4-0003-Changed-queries.patchDownload
From 200c4ed4118f92014253b49faa41a76ecec84c96 Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Sat, 22 Jan 2022 18:34:54 -0500
Subject: [PATCH v4 3/4] Changed queries
---
src/test/regress/expected/partition_prune.out | 104 +++++++++---------
src/test/regress/expected/select_parallel.out | 26 +++--
2 files changed, 69 insertions(+), 61 deletions(-)
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index 7555764c77..5c45f9c0a5 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -1284,60 +1284,64 @@ EXPLAIN (VERBOSE, COSTS OFF) SELECT * FROM part p(x) ORDER BY x;
--
-- pruning won't work for mc3p, because some keys are Params
explain (costs off) select * from mc2p t1, lateral (select count(*) from mc3p t2 where t2.a = t1.b and abs(t2.b) = 1 and t2.c = 1) s where t1.a = 1;
- QUERY PLAN
------------------------------------------------------------------------
- Nested Loop
- -> Append
- -> Seq Scan on mc2p1 t1_1
- Filter: (a = 1)
- -> Seq Scan on mc2p2 t1_2
- Filter: (a = 1)
- -> Seq Scan on mc2p_default t1_3
- Filter: (a = 1)
- -> Aggregate
- -> Append
- -> Seq Scan on mc3p0 t2_1
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p1 t2_2
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p2 t2_3
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p3 t2_4
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p4 t2_5
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p5 t2_6
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p6 t2_7
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p7 t2_8
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p_default t2_9
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
-(28 rows)
+ QUERY PLAN
+-----------------------------------------------------------------------------
+ Gather
+ Workers Planned: 2
+ -> Nested Loop
+ -> Parallel Append
+ -> Parallel Seq Scan on mc2p1 t1_1
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p2 t1_2
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p_default t1_3
+ Filter: (a = 1)
+ -> Aggregate
+ -> Append
+ -> Seq Scan on mc3p0 t2_1
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p1 t2_2
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p2 t2_3
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p3 t2_4
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p4 t2_5
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p5 t2_6
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p6 t2_7
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p7 t2_8
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p_default t2_9
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+(30 rows)
-- pruning should work fine, because values for a prefix of keys (a, b) are
-- available
explain (costs off) select * from mc2p t1, lateral (select count(*) from mc3p t2 where t2.c = t1.b and abs(t2.b) = 1 and t2.a = 1) s where t1.a = 1;
- QUERY PLAN
------------------------------------------------------------------------
- Nested Loop
- -> Append
- -> Seq Scan on mc2p1 t1_1
- Filter: (a = 1)
- -> Seq Scan on mc2p2 t1_2
- Filter: (a = 1)
- -> Seq Scan on mc2p_default t1_3
- Filter: (a = 1)
- -> Aggregate
- -> Append
- -> Seq Scan on mc3p0 t2_1
- Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p1 t2_2
- Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p_default t2_3
- Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
-(16 rows)
+ QUERY PLAN
+-----------------------------------------------------------------------------
+ Gather
+ Workers Planned: 2
+ -> Nested Loop
+ -> Parallel Append
+ -> Parallel Seq Scan on mc2p1 t1_1
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p2 t1_2
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p_default t1_3
+ Filter: (a = 1)
+ -> Aggregate
+ -> Append
+ -> Seq Scan on mc3p0 t2_1
+ Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p1 t2_2
+ Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p_default t2_3
+ Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
+(18 rows)
-- also here, because values for all keys are provided
explain (costs off) select * from mc2p t1, lateral (select count(*) from mc3p t2 where t2.a = 1 and abs(t2.b) = 1 and t2.c = 1) s where t1.a = 1;
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 124fe9fec5..9bb60c2c1e 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -137,8 +137,8 @@ create table part_pa_test_p2 partition of part_pa_test for values from (0) to (m
explain (costs off)
select (select max((select pa1.b from part_pa_test pa1 where pa1.a = pa2.a)))
from part_pa_test pa2;
- QUERY PLAN
---------------------------------------------------------------
+ QUERY PLAN
+----------------------------------------------------------------
Aggregate
-> Gather
Workers Planned: 3
@@ -148,12 +148,14 @@ explain (costs off)
SubPlan 2
-> Result
SubPlan 1
- -> Append
- -> Seq Scan on part_pa_test_p1 pa1_1
- Filter: (a = pa2.a)
- -> Seq Scan on part_pa_test_p2 pa1_2
- Filter: (a = pa2.a)
-(14 rows)
+ -> Gather
+ Workers Planned: 3
+ -> Parallel Append
+ -> Parallel Seq Scan on part_pa_test_p1 pa1_1
+ Filter: (a = pa2.a)
+ -> Parallel Seq Scan on part_pa_test_p2 pa1_2
+ Filter: (a = pa2.a)
+(16 rows)
drop table part_pa_test;
-- test with leader participation disabled
@@ -1330,8 +1332,10 @@ SELECT 1 FROM tenk1_vw_sec
-> Parallel Index Only Scan using tenk1_unique1 on tenk1
SubPlan 1
-> Aggregate
- -> Seq Scan on int4_tbl
- Filter: (f1 < tenk1_vw_sec.unique1)
-(9 rows)
+ -> Gather
+ Workers Planned: 1
+ -> Parallel Seq Scan on int4_tbl
+ Filter: (f1 < tenk1_vw_sec.unique1)
+(11 rows)
rollback;
--
2.17.1
v4-0004-Possible-additional-checks.patchtext/x-patch; charset=US-ASCII; name=v4-0004-Possible-additional-checks.patchDownload
From c11d38b8a88e8f7a46528712fc9341de34171f8d Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Sat, 22 Jan 2022 17:33:23 -0500
Subject: [PATCH v4 4/4] Possible additional checks
---
src/backend/optimizer/path/allpaths.c | 33 ++++++++++++++++++++++-----
1 file changed, 27 insertions(+), 6 deletions(-)
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 776f002054..7e30824320 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -2682,11 +2682,16 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
ListCell *lc;
double rows;
double *rowsp = NULL;
+ Relids required_outer = rel->lateral_relids;
/* If there are no partial paths, there's nothing to do here. */
if (rel->partial_pathlist == NIL)
return;
+ if (!bms_is_subset(required_outer, rel->relids))
+ return;
+
+
/* Should we override the rel's rowcount estimate? */
if (override_rows)
rowsp = &rows;
@@ -2697,12 +2702,16 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
* of partial_pathlist because of the way add_partial_path works.
*/
cheapest_partial_path = linitial(rel->partial_pathlist);
- rows =
- cheapest_partial_path->rows * cheapest_partial_path->parallel_workers;
- simple_gather_path = (Path *)
- create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
- rel->lateral_relids, rowsp);
- add_path(rel, simple_gather_path);
+ if (cheapest_partial_path->param_info == NULL ||
+ bms_is_subset(cheapest_partial_path->param_info->ppi_req_outer, rel->relids))
+ {
+ rows =
+ cheapest_partial_path->rows * cheapest_partial_path->parallel_workers;
+ simple_gather_path = (Path *)
+ create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
+ rel->lateral_relids, rowsp);
+ add_path(rel, simple_gather_path);
+ }
/*
* For each useful ordering, we can consider an order-preserving Gather
@@ -2716,6 +2725,10 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
if (subpath->pathkeys == NIL)
continue;
+ if (subpath->param_info != NULL &&
+ !bms_is_subset(subpath->param_info->ppi_req_outer, rel->relids))
+ break;
+
rows = subpath->rows * subpath->parallel_workers;
path = create_gather_merge_path(root, rel, subpath, rel->reltarget,
subpath->pathkeys, rel->lateral_relids, rowsp);
@@ -2888,6 +2901,10 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
{
Path *tmp;
+ if (subpath->param_info != NULL &&
+ !bms_is_subset(subpath->param_info->ppi_req_outer, rel->relids))
+ break;
+
tmp = (Path *) create_sort_path(root,
rel,
subpath,
@@ -2916,6 +2933,10 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
{
Path *tmp;
+ if (subpath->param_info != NULL &&
+ !bms_is_subset(subpath->param_info->ppi_req_outer, rel->relids))
+ break;
+
/*
* We should have already excluded pathkeys of length 1
* because then presorted_keys > 0 would imply is_sorted was
--
2.17.1
v4-0002-Parallelize-correlated-subqueries.patchtext/x-patch; charset=US-ASCII; name=v4-0002-Parallelize-correlated-subqueries.patchDownload
From ba785f2abca7c9f5199f0fc27c3b71f6d1d8010b Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Fri, 21 Jan 2022 22:38:46 -0500
Subject: [PATCH v4 2/4] Parallelize correlated subqueries
When params are provided at the current query level (i.e., are generated
within a single worker and not shared across workers) we can safely
execute these in parallel.
Alternative approach using just relids subset check
---
doc/src/sgml/parallel.sgml | 3 +-
src/backend/optimizer/path/allpaths.c | 18 ++-
src/backend/optimizer/path/joinpath.c | 16 ++-
src/backend/optimizer/util/clauses.c | 3 +
src/backend/optimizer/util/pathnode.c | 2 +
src/backend/utils/misc/guc.c | 1 +
src/include/nodes/pathnodes.h | 2 +-
.../regress/expected/incremental_sort.out | 28 ++--
src/test/regress/expected/select_parallel.out | 125 ++++++++++++++++++
src/test/regress/sql/select_parallel.sql | 25 ++++
10 files changed, 197 insertions(+), 26 deletions(-)
diff --git a/doc/src/sgml/parallel.sgml b/doc/src/sgml/parallel.sgml
index 13479d7e5e..2d924dd2ac 100644
--- a/doc/src/sgml/parallel.sgml
+++ b/doc/src/sgml/parallel.sgml
@@ -517,7 +517,8 @@ EXPLAIN SELECT * FROM pgbench_accounts WHERE filler LIKE '%x%';
<listitem>
<para>
- Plan nodes that reference a correlated <literal>SubPlan</literal>.
+ Plan nodes that reference a correlated <literal>SubPlan</literal> where
+ the result is shared between workers.
</para>
</listitem>
</itemizedlist>
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 6a581e20fa..776f002054 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -556,7 +556,8 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
* (see grouping_planner).
*/
if (rel->reloptkind == RELOPT_BASEREL &&
- bms_membership(root->all_baserels) != BMS_SINGLETON)
+ bms_membership(root->all_baserels) != BMS_SINGLETON
+ && (rel->subplan_params == NIL || rte->rtekind != RTE_SUBQUERY))
generate_useful_gather_paths(root, rel, false);
/* Now find the cheapest of the paths for this rel */
@@ -2700,7 +2701,7 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
cheapest_partial_path->rows * cheapest_partial_path->parallel_workers;
simple_gather_path = (Path *)
create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
- NULL, rowsp);
+ rel->lateral_relids, rowsp);
add_path(rel, simple_gather_path);
/*
@@ -2717,7 +2718,7 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
rows = subpath->rows * subpath->parallel_workers;
path = create_gather_merge_path(root, rel, subpath, rel->reltarget,
- subpath->pathkeys, NULL, rowsp);
+ subpath->pathkeys, rel->lateral_relids, rowsp);
add_path(rel, &path->path);
}
}
@@ -2819,11 +2820,15 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
double *rowsp = NULL;
List *useful_pathkeys_list = NIL;
Path *cheapest_partial_path = NULL;
+ Relids required_outer = rel->lateral_relids;
/* If there are no partial paths, there's nothing to do here. */
if (rel->partial_pathlist == NIL)
return;
+ if (!bms_is_subset(required_outer, rel->relids))
+ return;
+
/* Should we override the rel's rowcount estimate? */
if (override_rows)
rowsp = &rows;
@@ -2895,7 +2900,7 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
tmp,
rel->reltarget,
tmp->pathkeys,
- NULL,
+ required_outer,
rowsp);
add_path(rel, &path->path);
@@ -2929,7 +2934,7 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
tmp,
rel->reltarget,
tmp->pathkeys,
- NULL,
+ required_outer,
rowsp);
add_path(rel, &path->path);
@@ -3108,7 +3113,8 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
/*
* Except for the topmost scan/join rel, consider gathering
* partial paths. We'll do the same for the topmost scan/join rel
- * once we know the final targetlist (see grouping_planner).
+ * once we know the final targetlist (see
+ * apply_scanjoin_target_to_paths).
*/
if (lev < levels_needed)
generate_useful_gather_paths(root, rel, false);
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index f96fc9fd28..e85b5449ea 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -1791,16 +1791,24 @@ match_unsorted_outer(PlannerInfo *root,
* partial path and the joinrel is parallel-safe. However, we can't
* handle JOIN_UNIQUE_OUTER, because the outer path will be partial, and
* therefore we won't be able to properly guarantee uniqueness. Nor can
- * we handle joins needing lateral rels, since partial paths must not be
- * parameterized. Similarly, we can't handle JOIN_FULL and JOIN_RIGHT,
- * because they can produce false null extended rows.
+ * we handle JOIN_FULL and JOIN_RIGHT, because they can produce false null
+ * extended rows.
+ *
+ * Partial paths may only have parameters in limited cases
+ * where the parameterization is fully satisfied without sharing state
+ * between workers, so we only allow lateral rels on inputs to the join
+ * if the resulting join contains no lateral rels, the inner rel's laterals
+ * are fully satisfied by the outer rel, and the outer rel doesn't depend
+ * on the inner rel to produce any laterals.
*/
if (joinrel->consider_parallel &&
save_jointype != JOIN_UNIQUE_OUTER &&
save_jointype != JOIN_FULL &&
save_jointype != JOIN_RIGHT &&
outerrel->partial_pathlist != NIL &&
- bms_is_empty(joinrel->lateral_relids))
+ bms_is_empty(joinrel->lateral_relids) &&
+ bms_is_subset(innerrel->lateral_relids, outerrel->relids) &&
+ (bms_is_empty(outerrel->lateral_relids) || !bms_is_subset(outerrel->lateral_relids, innerrel->relids)))
{
if (nestjoinOK)
consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index a707dc9f26..f0002f6887 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -822,6 +822,9 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
if (param->paramkind == PARAM_EXTERN)
return false;
+ if (param->paramkind == PARAM_EXEC)
+ return false;
+
if (param->paramkind != PARAM_EXEC ||
!list_member_int(context->safe_param_ids, param->paramid))
{
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 5c32c96b71..144b2c485d 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -2418,6 +2418,8 @@ create_nestloop_path(PlannerInfo *root,
NestPath *pathnode = makeNode(NestPath);
Relids inner_req_outer = PATH_REQ_OUTER(inner_path);
+ /* TODO: Assert lateral relids subset safety? */
+
/*
* If the inner path is parameterized by the outer, we must drop any
* restrict_clauses that are due to be moved into the inner path. We have
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index effb9d03a0..52cd3512b3 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -58,6 +58,7 @@
#include "libpq/libpq.h"
#include "libpq/pqformat.h"
#include "miscadmin.h"
+#include "optimizer/clauses.h"
#include "optimizer/cost.h"
#include "optimizer/geqo.h"
#include "optimizer/optimizer.h"
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 1f33fe13c1..75681d6fb9 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -2478,7 +2478,7 @@ typedef struct MinMaxAggInfo
* for conflicting purposes.
*
* In addition, PARAM_EXEC slots are assigned for Params representing outputs
- * from subplans (values that are setParam items for those subplans). These
+ * from subplans (values that are setParam items for those subplans). [TODO: is this true, or only for init plans?] These
* IDs need not be tracked via PlannerParamItems, since we do not need any
* duplicate-elimination nor later processing of the represented expressions.
* Instead, we just record the assignment of the slot number by appending to
diff --git a/src/test/regress/expected/incremental_sort.out b/src/test/regress/expected/incremental_sort.out
index 545e301e48..8f9ca05e60 100644
--- a/src/test/regress/expected/incremental_sort.out
+++ b/src/test/regress/expected/incremental_sort.out
@@ -1614,16 +1614,16 @@ from tenk1 t, generate_series(1, 1000);
QUERY PLAN
---------------------------------------------------------------------------------
Unique
- -> Sort
- Sort Key: t.unique1, ((SubPlan 1))
- -> Gather
- Workers Planned: 2
+ -> Gather Merge
+ Workers Planned: 2
+ -> Sort
+ Sort Key: t.unique1, ((SubPlan 1))
-> Nested Loop
-> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
-> Function Scan on generate_series
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on tenk1
- Index Cond: (unique1 = t.unique1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on tenk1
+ Index Cond: (unique1 = t.unique1)
(11 rows)
explain (costs off) select
@@ -1633,16 +1633,16 @@ from tenk1 t, generate_series(1, 1000)
order by 1, 2;
QUERY PLAN
---------------------------------------------------------------------------
- Sort
- Sort Key: t.unique1, ((SubPlan 1))
- -> Gather
- Workers Planned: 2
+ Gather Merge
+ Workers Planned: 2
+ -> Sort
+ Sort Key: t.unique1, ((SubPlan 1))
-> Nested Loop
-> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
-> Function Scan on generate_series
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on tenk1
- Index Cond: (unique1 = t.unique1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on tenk1
+ Index Cond: (unique1 = t.unique1)
(10 rows)
-- Parallel sort but with expression not available until the upper rel.
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 2303f70d6e..124fe9fec5 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -311,6 +311,131 @@ select count(*) from tenk1 where (two, four) not in
10000
(1 row)
+-- test parallel plans for queries containing correlated subplans
+-- where the subplan only needs params available from the current
+-- worker's scan.
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t, generate_series(1, 10);
+ QUERY PLAN
+----------------------------------------------------------------------------
+ Gather
+ Output: ((SubPlan 1))
+ Workers Planned: 4
+ -> Nested Loop
+ Output: (SubPlan 1)
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ -> Function Scan on pg_catalog.generate_series
+ Output: generate_series.generate_series
+ Function Call: generate_series(1, 10)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(14 rows)
+
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t;
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: ((SubPlan 1))
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(9 rows)
+
+explain (analyze, costs off, summary off, verbose, timing off) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t
+ limit 1;
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------
+ Limit (actual rows=1 loops=1)
+ Output: ((SubPlan 1))
+ -> Gather (actual rows=1 loops=1)
+ Output: ((SubPlan 1))
+ Workers Planned: 4
+ Workers Launched: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t (actual rows=1 loops=5)
+ Output: (SubPlan 1)
+ Heap Fetches: 0
+ Worker 0: actual rows=1 loops=1
+ Worker 1: actual rows=1 loops=1
+ Worker 2: actual rows=1 loops=1
+ Worker 3: actual rows=1 loops=1
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1 (actual rows=1 loops=5)
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+ Heap Fetches: 0
+ Worker 0: actual rows=1 loops=1
+ Worker 1: actual rows=1 loops=1
+ Worker 2: actual rows=1 loops=1
+ Worker 3: actual rows=1 loops=1
+(22 rows)
+
+explain (costs off, verbose) select t.unique1
+ from tenk1 t
+ where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: t.unique1
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ Filter: (t.unique1 = (SubPlan 1))
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(10 rows)
+
+explain (costs off, verbose) select *
+ from tenk1 t
+ order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Gather Merge
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
+ Workers Planned: 4
+ -> Sort
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
+ Sort Key: ((SubPlan 1))
+ -> Parallel Seq Scan on public.tenk1 t
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(12 rows)
+
+-- test subplan in join/lateral join
+explain (costs off, verbose, timing off) select t.unique1, l.*
+ from tenk1 t
+ join lateral (
+ select (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1 offset 0)
+ ) l on true;
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: t.unique1, ((SubPlan 1))
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1, (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(9 rows)
+
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
explain (costs off)
select * from tenk1 where (unique1 + random())::integer not in
diff --git a/src/test/regress/sql/select_parallel.sql b/src/test/regress/sql/select_parallel.sql
index 019e17e751..c49799b6d4 100644
--- a/src/test/regress/sql/select_parallel.sql
+++ b/src/test/regress/sql/select_parallel.sql
@@ -111,6 +111,31 @@ explain (costs off)
(select hundred, thousand from tenk2 where thousand > 100);
select count(*) from tenk1 where (two, four) not in
(select hundred, thousand from tenk2 where thousand > 100);
+-- test parallel plans for queries containing correlated subplans
+-- where the subplan only needs params available from the current
+-- worker's scan.
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t, generate_series(1, 10);
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t;
+explain (analyze, costs off, summary off, verbose, timing off) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t
+ limit 1;
+explain (costs off, verbose) select t.unique1
+ from tenk1 t
+ where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+explain (costs off, verbose) select *
+ from tenk1 t
+ order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+-- test subplan in join/lateral join
+explain (costs off, verbose, timing off) select t.unique1, l.*
+ from tenk1 t
+ join lateral (
+ select (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1 offset 0)
+ ) l on true;
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
explain (costs off)
select * from tenk1 where (unique1 + random())::integer not in
--
2.17.1
Re: Parallelize correlated subqueries that execute within each worker
Hi,
On 2022-01-22 20:25:19 -0500, James Coleman wrote:
On the other hand this is a dramatically simpler patch series.
Assuming the approach is sound, it should much easier to maintain than
the previous version.The final patch in the series is a set of additional checks I could
imagine to try to be more explicit, but at least in the current test
suite there isn't anything at all they affect.Does this look at least somewhat more like what you'd envisionsed
(granting the need to squint hard given the relids checks instead of
directly checking params)?
This fails on freebsd (so likely a timing issue): https://cirrus-ci.com/task/4758411492458496?logs=test_world#L2225
Marked as waiting on author.
Greetings,
Andres Freund
Re: Parallelize correlated subqueries that execute within each worker
This entry has been waiting on author input for a while (our current
threshold is roughly two weeks), so I've marked it Returned with
Feedback.
Once you think the patchset is ready for review again, you (or any
interested party) can resurrect the patch entry by visiting
https://commitfest.postgresql.org/38/3246/
and changing the status to "Needs Review", and then changing the
status again to "Move to next CF". (Don't forget the second step;
hopefully we will have streamlined this in the near future!)
Thanks,
--Jacob
Re: Parallelize correlated subqueries that execute within each worker
On Mon, Mar 21, 2022 at 8:48 PM Andres Freund <andres@anarazel.de> wrote:
Hi,
On 2022-01-22 20:25:19 -0500, James Coleman wrote:
On the other hand this is a dramatically simpler patch series.
Assuming the approach is sound, it should much easier to maintain than
the previous version.The final patch in the series is a set of additional checks I could
imagine to try to be more explicit, but at least in the current test
suite there isn't anything at all they affect.Does this look at least somewhat more like what you'd envisionsed
(granting the need to squint hard given the relids checks instead of
directly checking params)?This fails on freebsd (so likely a timing issue): https://cirrus-ci.com/task/4758411492458496?logs=test_world#L2225
Marked as waiting on author.
I've finally gotten around to checking this out, and the issue was an
"explain analyze" test that had actual loops different on FreeBSD.
There doesn't seem to be a way to disable loop output, but instead of
processing the explain output with e.g. a function (as we do some
other places) to remove the offending and unnecessary output I've just
removed the "analyze" (as I don't believe it was actually necessary).
Attached is an updated patch series. In this version I've removed the
"parallelize some subqueries with limit" patch since discussion is
proceeding in the spun off thread. The first patch adds additional
tests so that you can see how those new tests change with the code
changes in the 2nd patch in the series. As before the final patch in
the series includes changes where we may also want to verify
correctness but don't have a test demonstrating the need.
Thanks,
James Coleman
Attachments:
v5-0002-Parallelize-correlated-subqueries.patchapplication/octet-stream; name=v5-0002-Parallelize-correlated-subqueries.patchDownload
From 07ccd5b8d1776b5109c54d20ed3dcaef22d752f9 Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Fri, 21 Jan 2022 22:38:46 -0500
Subject: [PATCH v5 2/3] Parallelize correlated subqueries
When params are provided at the current query level (i.e., are generated
within a single worker and not shared across workers) we can safely
execute these in parallel.
Alternative approach using just relids subset check
---
doc/src/sgml/parallel.sgml | 3 +-
src/backend/optimizer/path/allpaths.c | 18 ++-
src/backend/optimizer/path/joinpath.c | 16 ++-
src/backend/optimizer/util/clauses.c | 3 +
src/backend/optimizer/util/pathnode.c | 2 +
src/include/nodes/pathnodes.h | 2 +-
.../regress/expected/incremental_sort.out | 28 ++--
src/test/regress/expected/partition_prune.out | 104 +++++++-------
src/test/regress/expected/select_parallel.out | 128 ++++++++++--------
9 files changed, 169 insertions(+), 135 deletions(-)
diff --git a/doc/src/sgml/parallel.sgml b/doc/src/sgml/parallel.sgml
index c37fb67065..d44325dd89 100644
--- a/doc/src/sgml/parallel.sgml
+++ b/doc/src/sgml/parallel.sgml
@@ -517,7 +517,8 @@ EXPLAIN SELECT * FROM pgbench_accounts WHERE filler LIKE '%x%';
<listitem>
<para>
- Plan nodes that reference a correlated <literal>SubPlan</literal>.
+ Plan nodes that reference a correlated <literal>SubPlan</literal> where
+ the result is shared between workers.
</para>
</listitem>
</itemizedlist>
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 8fc28007f5..e1ad9ae372 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -558,7 +558,8 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
* the final scan/join targetlist is available (see grouping_planner).
*/
if (rel->reloptkind == RELOPT_BASEREL &&
- !bms_equal(rel->relids, root->all_baserels))
+ !bms_equal(rel->relids, root->all_baserels)
+ && (rel->subplan_params == NIL || rte->rtekind != RTE_SUBQUERY))
generate_useful_gather_paths(root, rel, false);
/* Now find the cheapest of the paths for this rel */
@@ -3025,7 +3026,7 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
cheapest_partial_path->rows * cheapest_partial_path->parallel_workers;
simple_gather_path = (Path *)
create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
- NULL, rowsp);
+ rel->lateral_relids, rowsp);
add_path(rel, simple_gather_path);
/*
@@ -3042,7 +3043,7 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
rows = subpath->rows * subpath->parallel_workers;
path = create_gather_merge_path(root, rel, subpath, rel->reltarget,
- subpath->pathkeys, NULL, rowsp);
+ subpath->pathkeys, rel->lateral_relids, rowsp);
add_path(rel, &path->path);
}
}
@@ -3144,11 +3145,15 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
double *rowsp = NULL;
List *useful_pathkeys_list = NIL;
Path *cheapest_partial_path = NULL;
+ Relids required_outer = rel->lateral_relids;
/* If there are no partial paths, there's nothing to do here. */
if (rel->partial_pathlist == NIL)
return;
+ if (!bms_is_subset(required_outer, rel->relids))
+ return;
+
/* Should we override the rel's rowcount estimate? */
if (override_rows)
rowsp = &rows;
@@ -3220,7 +3225,7 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
tmp,
rel->reltarget,
tmp->pathkeys,
- NULL,
+ required_outer,
rowsp);
add_path(rel, &path->path);
@@ -3254,7 +3259,7 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
tmp,
rel->reltarget,
tmp->pathkeys,
- NULL,
+ required_outer,
rowsp);
add_path(rel, &path->path);
@@ -3433,7 +3438,8 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
/*
* Except for the topmost scan/join rel, consider gathering
* partial paths. We'll do the same for the topmost scan/join rel
- * once we know the final targetlist (see grouping_planner).
+ * once we know the final targetlist (see
+ * apply_scanjoin_target_to_paths).
*/
if (!bms_equal(rel->relids, root->all_baserels))
generate_useful_gather_paths(root, rel, false);
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index 2a3f0ab7bf..21606f6e59 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -1791,16 +1791,24 @@ match_unsorted_outer(PlannerInfo *root,
* partial path and the joinrel is parallel-safe. However, we can't
* handle JOIN_UNIQUE_OUTER, because the outer path will be partial, and
* therefore we won't be able to properly guarantee uniqueness. Nor can
- * we handle joins needing lateral rels, since partial paths must not be
- * parameterized. Similarly, we can't handle JOIN_FULL and JOIN_RIGHT,
- * because they can produce false null extended rows.
+ * we handle JOIN_FULL and JOIN_RIGHT, because they can produce false null
+ * extended rows.
+ *
+ * Partial paths may only have parameters in limited cases
+ * where the parameterization is fully satisfied without sharing state
+ * between workers, so we only allow lateral rels on inputs to the join
+ * if the resulting join contains no lateral rels, the inner rel's laterals
+ * are fully satisfied by the outer rel, and the outer rel doesn't depend
+ * on the inner rel to produce any laterals.
*/
if (joinrel->consider_parallel &&
save_jointype != JOIN_UNIQUE_OUTER &&
save_jointype != JOIN_FULL &&
save_jointype != JOIN_RIGHT &&
outerrel->partial_pathlist != NIL &&
- bms_is_empty(joinrel->lateral_relids))
+ bms_is_empty(joinrel->lateral_relids) &&
+ bms_is_subset(innerrel->lateral_relids, outerrel->relids) &&
+ (bms_is_empty(outerrel->lateral_relids) || !bms_is_subset(outerrel->lateral_relids, innerrel->relids)))
{
if (nestjoinOK)
consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index bf3a7cae60..11bab7fa7e 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -822,6 +822,9 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
if (param->paramkind == PARAM_EXTERN)
return false;
+ if (param->paramkind == PARAM_EXEC)
+ return false;
+
if (param->paramkind != PARAM_EXEC ||
!list_member_int(context->safe_param_ids, param->paramid))
{
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index e10561d843..e8309342f0 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -2437,6 +2437,8 @@ create_nestloop_path(PlannerInfo *root,
NestPath *pathnode = makeNode(NestPath);
Relids inner_req_outer = PATH_REQ_OUTER(inner_path);
+ /* TODO: Assert lateral relids subset safety? */
+
/*
* If the inner path is parameterized by the outer, we must drop any
* restrict_clauses that are due to be moved into the inner path. We have
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 294cfe9c47..b69d4e2692 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -2964,7 +2964,7 @@ typedef struct MinMaxAggInfo
* for conflicting purposes.
*
* In addition, PARAM_EXEC slots are assigned for Params representing outputs
- * from subplans (values that are setParam items for those subplans). These
+ * from subplans (values that are setParam items for those subplans). [TODO: is this true, or only for init plans?] These
* IDs need not be tracked via PlannerParamItems, since we do not need any
* duplicate-elimination nor later processing of the represented expressions.
* Instead, we just record the assignment of the slot number by appending to
diff --git a/src/test/regress/expected/incremental_sort.out b/src/test/regress/expected/incremental_sort.out
index 49953eaade..66c09381f1 100644
--- a/src/test/regress/expected/incremental_sort.out
+++ b/src/test/regress/expected/incremental_sort.out
@@ -1612,16 +1612,16 @@ from tenk1 t, generate_series(1, 1000);
QUERY PLAN
---------------------------------------------------------------------------------
Unique
- -> Sort
- Sort Key: t.unique1, ((SubPlan 1))
- -> Gather
- Workers Planned: 2
+ -> Gather Merge
+ Workers Planned: 2
+ -> Sort
+ Sort Key: t.unique1, ((SubPlan 1))
-> Nested Loop
-> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
-> Function Scan on generate_series
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on tenk1
- Index Cond: (unique1 = t.unique1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on tenk1
+ Index Cond: (unique1 = t.unique1)
(11 rows)
explain (costs off) select
@@ -1631,16 +1631,16 @@ from tenk1 t, generate_series(1, 1000)
order by 1, 2;
QUERY PLAN
---------------------------------------------------------------------------
- Sort
- Sort Key: t.unique1, ((SubPlan 1))
- -> Gather
- Workers Planned: 2
+ Gather Merge
+ Workers Planned: 2
+ -> Sort
+ Sort Key: t.unique1, ((SubPlan 1))
-> Nested Loop
-> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
-> Function Scan on generate_series
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on tenk1
- Index Cond: (unique1 = t.unique1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on tenk1
+ Index Cond: (unique1 = t.unique1)
(10 rows)
-- Parallel sort but with expression not available until the upper rel.
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index 7555764c77..5c45f9c0a5 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -1284,60 +1284,64 @@ EXPLAIN (VERBOSE, COSTS OFF) SELECT * FROM part p(x) ORDER BY x;
--
-- pruning won't work for mc3p, because some keys are Params
explain (costs off) select * from mc2p t1, lateral (select count(*) from mc3p t2 where t2.a = t1.b and abs(t2.b) = 1 and t2.c = 1) s where t1.a = 1;
- QUERY PLAN
------------------------------------------------------------------------
- Nested Loop
- -> Append
- -> Seq Scan on mc2p1 t1_1
- Filter: (a = 1)
- -> Seq Scan on mc2p2 t1_2
- Filter: (a = 1)
- -> Seq Scan on mc2p_default t1_3
- Filter: (a = 1)
- -> Aggregate
- -> Append
- -> Seq Scan on mc3p0 t2_1
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p1 t2_2
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p2 t2_3
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p3 t2_4
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p4 t2_5
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p5 t2_6
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p6 t2_7
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p7 t2_8
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p_default t2_9
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
-(28 rows)
+ QUERY PLAN
+-----------------------------------------------------------------------------
+ Gather
+ Workers Planned: 2
+ -> Nested Loop
+ -> Parallel Append
+ -> Parallel Seq Scan on mc2p1 t1_1
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p2 t1_2
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p_default t1_3
+ Filter: (a = 1)
+ -> Aggregate
+ -> Append
+ -> Seq Scan on mc3p0 t2_1
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p1 t2_2
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p2 t2_3
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p3 t2_4
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p4 t2_5
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p5 t2_6
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p6 t2_7
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p7 t2_8
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p_default t2_9
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+(30 rows)
-- pruning should work fine, because values for a prefix of keys (a, b) are
-- available
explain (costs off) select * from mc2p t1, lateral (select count(*) from mc3p t2 where t2.c = t1.b and abs(t2.b) = 1 and t2.a = 1) s where t1.a = 1;
- QUERY PLAN
------------------------------------------------------------------------
- Nested Loop
- -> Append
- -> Seq Scan on mc2p1 t1_1
- Filter: (a = 1)
- -> Seq Scan on mc2p2 t1_2
- Filter: (a = 1)
- -> Seq Scan on mc2p_default t1_3
- Filter: (a = 1)
- -> Aggregate
- -> Append
- -> Seq Scan on mc3p0 t2_1
- Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p1 t2_2
- Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p_default t2_3
- Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
-(16 rows)
+ QUERY PLAN
+-----------------------------------------------------------------------------
+ Gather
+ Workers Planned: 2
+ -> Nested Loop
+ -> Parallel Append
+ -> Parallel Seq Scan on mc2p1 t1_1
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p2 t1_2
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p_default t1_3
+ Filter: (a = 1)
+ -> Aggregate
+ -> Append
+ -> Seq Scan on mc3p0 t2_1
+ Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p1 t2_2
+ Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p_default t2_3
+ Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
+(18 rows)
-- also here, because values for all keys are provided
explain (costs off) select * from mc2p t1, lateral (select count(*) from mc3p t2 where t2.a = 1 and abs(t2.b) = 1 and t2.c = 1) s where t1.a = 1;
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 9b4d7dd44a..01443e2ffb 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -137,8 +137,8 @@ create table part_pa_test_p2 partition of part_pa_test for values from (0) to (m
explain (costs off)
select (select max((select pa1.b from part_pa_test pa1 where pa1.a = pa2.a)))
from part_pa_test pa2;
- QUERY PLAN
---------------------------------------------------------------
+ QUERY PLAN
+----------------------------------------------------------------
Aggregate
-> Gather
Workers Planned: 3
@@ -148,12 +148,14 @@ explain (costs off)
SubPlan 2
-> Result
SubPlan 1
- -> Append
- -> Seq Scan on part_pa_test_p1 pa1_1
- Filter: (a = pa2.a)
- -> Seq Scan on part_pa_test_p2 pa1_2
- Filter: (a = pa2.a)
-(14 rows)
+ -> Gather
+ Workers Planned: 3
+ -> Parallel Append
+ -> Parallel Seq Scan on part_pa_test_p1 pa1_1
+ Filter: (a = pa2.a)
+ -> Parallel Seq Scan on part_pa_test_p2 pa1_2
+ Filter: (a = pa2.a)
+(16 rows)
drop table part_pa_test;
-- test with leader participation disabled
@@ -320,19 +322,19 @@ explain (costs off, verbose) select
QUERY PLAN
----------------------------------------------------------------------------
Gather
- Output: (SubPlan 1)
+ Output: ((SubPlan 1))
Workers Planned: 4
-> Nested Loop
- Output: t.unique1
+ Output: (SubPlan 1)
-> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
Output: t.unique1
-> Function Scan on pg_catalog.generate_series
Output: generate_series.generate_series
Function Call: generate_series(1, 10)
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(14 rows)
explain (costs off, verbose) select
@@ -341,63 +343,69 @@ explain (costs off, verbose) select
QUERY PLAN
----------------------------------------------------------------------
Gather
- Output: (SubPlan 1)
+ Output: ((SubPlan 1))
Workers Planned: 4
-> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
- Output: t.unique1
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ Output: (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(9 rows)
explain (costs off, verbose) select
(select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
from tenk1 t
limit 1;
- QUERY PLAN
--------------------------------------------------------------------
+ QUERY PLAN
+----------------------------------------------------------------------------
Limit
Output: ((SubPlan 1))
- -> Seq Scan on public.tenk1 t
- Output: (SubPlan 1)
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
-(8 rows)
+ -> Gather
+ Output: ((SubPlan 1))
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(11 rows)
explain (costs off, verbose) select t.unique1
from tenk1 t
where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
- QUERY PLAN
--------------------------------------------------------------
- Seq Scan on public.tenk1 t
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
Output: t.unique1
- Filter: (t.unique1 = (SubPlan 1))
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
-(7 rows)
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ Filter: (t.unique1 = (SubPlan 1))
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(10 rows)
explain (costs off, verbose) select *
from tenk1 t
order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
- QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
- Sort
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Gather Merge
Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
- Sort Key: ((SubPlan 1))
- -> Gather
- Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, (SubPlan 1)
- Workers Planned: 4
+ Workers Planned: 4
+ -> Sort
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
+ Sort Key: ((SubPlan 1))
-> Parallel Seq Scan on public.tenk1 t
- Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(12 rows)
-- test subplan in join/lateral join
@@ -409,14 +417,14 @@ explain (costs off, verbose, timing off) select t.unique1, l.*
QUERY PLAN
----------------------------------------------------------------------
Gather
- Output: t.unique1, (SubPlan 1)
+ Output: t.unique1, ((SubPlan 1))
Workers Planned: 4
-> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
- Output: t.unique1
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ Output: t.unique1, (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(9 rows)
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
@@ -1322,8 +1330,10 @@ SELECT 1 FROM tenk1_vw_sec
-> Parallel Index Only Scan using tenk1_unique1 on tenk1
SubPlan 1
-> Aggregate
- -> Seq Scan on int4_tbl
- Filter: (f1 < tenk1_vw_sec.unique1)
-(9 rows)
+ -> Gather
+ Workers Planned: 1
+ -> Parallel Seq Scan on int4_tbl
+ Filter: (f1 < tenk1_vw_sec.unique1)
+(11 rows)
rollback;
--
2.32.1 (Apple Git-133)
v5-0003-Possible-additional-checks.patchapplication/octet-stream; name=v5-0003-Possible-additional-checks.patchDownload
From bb794cf9b4fd6cd33e01ea8aa89e863fffade13f Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Sat, 22 Jan 2022 17:33:23 -0500
Subject: [PATCH v5 3/3] Possible additional checks
---
src/backend/optimizer/path/allpaths.c | 33 ++++++++++++++++++++++-----
1 file changed, 27 insertions(+), 6 deletions(-)
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index e1ad9ae372..e19cbbe90f 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -3007,11 +3007,16 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
ListCell *lc;
double rows;
double *rowsp = NULL;
+ Relids required_outer = rel->lateral_relids;
/* If there are no partial paths, there's nothing to do here. */
if (rel->partial_pathlist == NIL)
return;
+ if (!bms_is_subset(required_outer, rel->relids))
+ return;
+
+
/* Should we override the rel's rowcount estimate? */
if (override_rows)
rowsp = &rows;
@@ -3022,12 +3027,16 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
* of partial_pathlist because of the way add_partial_path works.
*/
cheapest_partial_path = linitial(rel->partial_pathlist);
- rows =
- cheapest_partial_path->rows * cheapest_partial_path->parallel_workers;
- simple_gather_path = (Path *)
- create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
- rel->lateral_relids, rowsp);
- add_path(rel, simple_gather_path);
+ if (cheapest_partial_path->param_info == NULL ||
+ bms_is_subset(cheapest_partial_path->param_info->ppi_req_outer, rel->relids))
+ {
+ rows =
+ cheapest_partial_path->rows * cheapest_partial_path->parallel_workers;
+ simple_gather_path = (Path *)
+ create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
+ rel->lateral_relids, rowsp);
+ add_path(rel, simple_gather_path);
+ }
/*
* For each useful ordering, we can consider an order-preserving Gather
@@ -3041,6 +3050,10 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
if (subpath->pathkeys == NIL)
continue;
+ if (subpath->param_info != NULL &&
+ !bms_is_subset(subpath->param_info->ppi_req_outer, rel->relids))
+ break;
+
rows = subpath->rows * subpath->parallel_workers;
path = create_gather_merge_path(root, rel, subpath, rel->reltarget,
subpath->pathkeys, rel->lateral_relids, rowsp);
@@ -3213,6 +3226,10 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
{
Path *tmp;
+ if (subpath->param_info != NULL &&
+ !bms_is_subset(subpath->param_info->ppi_req_outer, rel->relids))
+ break;
+
tmp = (Path *) create_sort_path(root,
rel,
subpath,
@@ -3241,6 +3258,10 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
{
Path *tmp;
+ if (subpath->param_info != NULL &&
+ !bms_is_subset(subpath->param_info->ppi_req_outer, rel->relids))
+ break;
+
/*
* We should have already excluded pathkeys of length 1
* because then presorted_keys > 0 would imply is_sorted was
--
2.32.1 (Apple Git-133)
v5-0001-Add-tests-before-change.patchapplication/octet-stream; name=v5-0001-Add-tests-before-change.patchDownload
From edc5bb3c0ba9d1aaa5d85e9716ce84856ad40514 Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Mon, 26 Sep 2022 20:30:23 -0400
Subject: [PATCH v5 1/3] Add tests before change
---
src/test/regress/expected/select_parallel.out | 108 ++++++++++++++++++
src/test/regress/sql/select_parallel.sql | 25 ++++
2 files changed, 133 insertions(+)
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 91f74fe47a..9b4d7dd44a 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -311,6 +311,114 @@ select count(*) from tenk1 where (two, four) not in
10000
(1 row)
+-- test parallel plans for queries containing correlated subplans
+-- where the subplan only needs params available from the current
+-- worker's scan.
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t, generate_series(1, 10);
+ QUERY PLAN
+----------------------------------------------------------------------------
+ Gather
+ Output: (SubPlan 1)
+ Workers Planned: 4
+ -> Nested Loop
+ Output: t.unique1
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ -> Function Scan on pg_catalog.generate_series
+ Output: generate_series.generate_series
+ Function Call: generate_series(1, 10)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(14 rows)
+
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t;
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: (SubPlan 1)
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(9 rows)
+
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t
+ limit 1;
+ QUERY PLAN
+-------------------------------------------------------------------
+ Limit
+ Output: ((SubPlan 1))
+ -> Seq Scan on public.tenk1 t
+ Output: (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(8 rows)
+
+explain (costs off, verbose) select t.unique1
+ from tenk1 t
+ where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+ QUERY PLAN
+-------------------------------------------------------------
+ Seq Scan on public.tenk1 t
+ Output: t.unique1
+ Filter: (t.unique1 = (SubPlan 1))
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(7 rows)
+
+explain (costs off, verbose) select *
+ from tenk1 t
+ order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Sort
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
+ Sort Key: ((SubPlan 1))
+ -> Gather
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, (SubPlan 1)
+ Workers Planned: 4
+ -> Parallel Seq Scan on public.tenk1 t
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(12 rows)
+
+-- test subplan in join/lateral join
+explain (costs off, verbose, timing off) select t.unique1, l.*
+ from tenk1 t
+ join lateral (
+ select (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1 offset 0)
+ ) l on true;
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: t.unique1, (SubPlan 1)
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(9 rows)
+
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
explain (costs off)
select * from tenk1 where (unique1 + random())::integer not in
diff --git a/src/test/regress/sql/select_parallel.sql b/src/test/regress/sql/select_parallel.sql
index 62fb68c7a0..21c2f1c742 100644
--- a/src/test/regress/sql/select_parallel.sql
+++ b/src/test/regress/sql/select_parallel.sql
@@ -111,6 +111,31 @@ explain (costs off)
(select hundred, thousand from tenk2 where thousand > 100);
select count(*) from tenk1 where (two, four) not in
(select hundred, thousand from tenk2 where thousand > 100);
+-- test parallel plans for queries containing correlated subplans
+-- where the subplan only needs params available from the current
+-- worker's scan.
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t, generate_series(1, 10);
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t;
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t
+ limit 1;
+explain (costs off, verbose) select t.unique1
+ from tenk1 t
+ where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+explain (costs off, verbose) select *
+ from tenk1 t
+ order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+-- test subplan in join/lateral join
+explain (costs off, verbose, timing off) select t.unique1, l.*
+ from tenk1 t
+ join lateral (
+ select (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1 offset 0)
+ ) l on true;
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
explain (costs off)
select * from tenk1 where (unique1 + random())::integer not in
--
2.32.1 (Apple Git-133)
Re: Parallelize correlated subqueries that execute within each worker
On Tue, 27 Sept 2022 at 08:26, James Coleman <jtc331@gmail.com> wrote:
On Mon, Mar 21, 2022 at 8:48 PM Andres Freund <andres@anarazel.de> wrote:
Hi,
On 2022-01-22 20:25:19 -0500, James Coleman wrote:
On the other hand this is a dramatically simpler patch series.
Assuming the approach is sound, it should much easier to maintain than
the previous version.The final patch in the series is a set of additional checks I could
imagine to try to be more explicit, but at least in the current test
suite there isn't anything at all they affect.Does this look at least somewhat more like what you'd envisionsed
(granting the need to squint hard given the relids checks instead of
directly checking params)?This fails on freebsd (so likely a timing issue): https://cirrus-ci.com/task/4758411492458496?logs=test_world#L2225
Marked as waiting on author.
I've finally gotten around to checking this out, and the issue was an
"explain analyze" test that had actual loops different on FreeBSD.
There doesn't seem to be a way to disable loop output, but instead of
processing the explain output with e.g. a function (as we do some
other places) to remove the offending and unnecessary output I've just
removed the "analyze" (as I don't believe it was actually necessary).Attached is an updated patch series. In this version I've removed the
"parallelize some subqueries with limit" patch since discussion is
proceeding in the spun off thread. The first patch adds additional
tests so that you can see how those new tests change with the code
changes in the 2nd patch in the series. As before the final patch in
the series includes changes where we may also want to verify
correctness but don't have a test demonstrating the need.
The patch does not apply on top of HEAD as in [1]http://cfbot.cputube.org/patch_41_3246.log, please post a rebased patch:
=== Applying patches on top of PostgreSQL commit ID
bf03cfd162176d543da79f9398131abc251ddbb9 ===
=== applying patch ./v5-0002-Parallelize-correlated-subqueries.patch
patching file src/backend/optimizer/path/allpaths.c
...
Hunk #5 FAILED at 3225.
Hunk #6 FAILED at 3259.
Hunk #7 succeeded at 3432 (offset -6 lines).
2 out of 7 hunks FAILED -- saving rejects to file
src/backend/optimizer/path/allpaths.c.rej
[1]: http://cfbot.cputube.org/patch_41_3246.log
Regards,
Vignesh
Re: Parallelize correlated subqueries that execute within each worker
Hi,
This patch hasn't been updated since September, and it got broken by
4a29eabd1d91c5484426bc5836e0a7143b064f5a which the incremental sort
stuff a little bit. But the breakage was rather limited, so I took a
stab at fixing it - attached is the result, hopefully correct.
I also added a couple minor comments about stuff I noticed while
rebasing and skimming the patch, I kept those in separate commits.
There's also a couple pre-existing TODOs.
James, what's your plan with this patch. Do you intend to work on it for
PG16, or are there some issues I missed in the thread?
One of the queries in in incremental_sort changed plans a little bit:
explain (costs off) select distinct
unique1,
(select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
from tenk1 t, generate_series(1, 1000);
switched from
Unique (cost=18582710.41..18747375.21 rows=10000 width=8)
-> Gather Merge (cost=18582710.41..18697375.21 rows=10000000 ...)
Workers Planned: 2
-> Sort (cost=18582710.39..18593127.06 rows=4166667 ...)
Sort Key: t.unique1, ((SubPlan 1))
...
to
Unique (cost=18582710.41..18614268.91 rows=10000 ...)
-> Gather Merge (cost=18582710.41..18614168.91 rows=20000 ...)
Workers Planned: 2
-> Unique (cost=18582710.39..18613960.39 rows=10000 ...)
-> Sort (cost=18582710.39..18593127.06 ...)
Sort Key: t.unique1, ((SubPlan 1))
...
which probably makes sense, as the cost estimate decreases a bit.
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
0001-Add-tests-before-change-v6.patchtext/x-patch; charset=UTF-8; name=0001-Add-tests-before-change-v6.patchDownload
From 95db15fe16303ed3f4fdea52af3b8d6d05a8d7a6 Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Mon, 26 Sep 2022 20:30:23 -0400
Subject: [PATCH 1/5] Add tests before change
---
src/test/regress/expected/select_parallel.out | 108 ++++++++++++++++++
src/test/regress/sql/select_parallel.sql | 25 ++++
2 files changed, 133 insertions(+)
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 91f74fe47a3..9b4d7dd44a4 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -311,6 +311,114 @@ select count(*) from tenk1 where (two, four) not in
10000
(1 row)
+-- test parallel plans for queries containing correlated subplans
+-- where the subplan only needs params available from the current
+-- worker's scan.
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t, generate_series(1, 10);
+ QUERY PLAN
+----------------------------------------------------------------------------
+ Gather
+ Output: (SubPlan 1)
+ Workers Planned: 4
+ -> Nested Loop
+ Output: t.unique1
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ -> Function Scan on pg_catalog.generate_series
+ Output: generate_series.generate_series
+ Function Call: generate_series(1, 10)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(14 rows)
+
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t;
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: (SubPlan 1)
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(9 rows)
+
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t
+ limit 1;
+ QUERY PLAN
+-------------------------------------------------------------------
+ Limit
+ Output: ((SubPlan 1))
+ -> Seq Scan on public.tenk1 t
+ Output: (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(8 rows)
+
+explain (costs off, verbose) select t.unique1
+ from tenk1 t
+ where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+ QUERY PLAN
+-------------------------------------------------------------
+ Seq Scan on public.tenk1 t
+ Output: t.unique1
+ Filter: (t.unique1 = (SubPlan 1))
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(7 rows)
+
+explain (costs off, verbose) select *
+ from tenk1 t
+ order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Sort
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
+ Sort Key: ((SubPlan 1))
+ -> Gather
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, (SubPlan 1)
+ Workers Planned: 4
+ -> Parallel Seq Scan on public.tenk1 t
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(12 rows)
+
+-- test subplan in join/lateral join
+explain (costs off, verbose, timing off) select t.unique1, l.*
+ from tenk1 t
+ join lateral (
+ select (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1 offset 0)
+ ) l on true;
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: t.unique1, (SubPlan 1)
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(9 rows)
+
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
explain (costs off)
select * from tenk1 where (unique1 + random())::integer not in
diff --git a/src/test/regress/sql/select_parallel.sql b/src/test/regress/sql/select_parallel.sql
index 62fb68c7a04..21c2f1c7424 100644
--- a/src/test/regress/sql/select_parallel.sql
+++ b/src/test/regress/sql/select_parallel.sql
@@ -111,6 +111,31 @@ explain (costs off)
(select hundred, thousand from tenk2 where thousand > 100);
select count(*) from tenk1 where (two, four) not in
(select hundred, thousand from tenk2 where thousand > 100);
+-- test parallel plans for queries containing correlated subplans
+-- where the subplan only needs params available from the current
+-- worker's scan.
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t, generate_series(1, 10);
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t;
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t
+ limit 1;
+explain (costs off, verbose) select t.unique1
+ from tenk1 t
+ where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+explain (costs off, verbose) select *
+ from tenk1 t
+ order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+-- test subplan in join/lateral join
+explain (costs off, verbose, timing off) select t.unique1, l.*
+ from tenk1 t
+ join lateral (
+ select (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1 offset 0)
+ ) l on true;
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
explain (costs off)
select * from tenk1 where (unique1 + random())::integer not in
--
2.39.0
0002-Parallelize-correlated-subqueries-v6.patchtext/x-patch; charset=UTF-8; name=0002-Parallelize-correlated-subqueries-v6.patchDownload
From cd379afbe22a9b41ff2b5dd5d86719a19f15301f Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas.vondra@postgresql.org>
Date: Wed, 18 Jan 2023 19:15:06 +0100
Subject: [PATCH 2/5] Parallelize correlated subqueries
When params are provided at the current query level (i.e., are generated
within a single worker and not shared across workers) we can safely
execute these in parallel.
Alternative approach using just relids subset check
---
doc/src/sgml/parallel.sgml | 3 +-
src/backend/optimizer/path/allpaths.c | 16 ++-
src/backend/optimizer/path/joinpath.c | 16 ++-
src/backend/optimizer/util/clauses.c | 3 +
src/backend/optimizer/util/pathnode.c | 2 +
src/include/nodes/pathnodes.h | 2 +-
.../regress/expected/incremental_sort.out | 28 ++--
src/test/regress/expected/partition_prune.out | 104 +++++++-------
src/test/regress/expected/select_parallel.out | 128 ++++++++++--------
9 files changed, 168 insertions(+), 134 deletions(-)
diff --git a/doc/src/sgml/parallel.sgml b/doc/src/sgml/parallel.sgml
index 5acc9537d6f..fd32572ec8b 100644
--- a/doc/src/sgml/parallel.sgml
+++ b/doc/src/sgml/parallel.sgml
@@ -518,7 +518,8 @@ EXPLAIN SELECT * FROM pgbench_accounts WHERE filler LIKE '%x%';
<listitem>
<para>
- Plan nodes that reference a correlated <literal>SubPlan</literal>.
+ Plan nodes that reference a correlated <literal>SubPlan</literal> where
+ the result is shared between workers.
</para>
</listitem>
</itemizedlist>
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index c2fc568dc8a..7108501b9a7 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -558,7 +558,8 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
* the final scan/join targetlist is available (see grouping_planner).
*/
if (rel->reloptkind == RELOPT_BASEREL &&
- !bms_equal(rel->relids, root->all_baserels))
+ !bms_equal(rel->relids, root->all_baserels)
+ && (rel->subplan_params == NIL || rte->rtekind != RTE_SUBQUERY))
generate_useful_gather_paths(root, rel, false);
/* Now find the cheapest of the paths for this rel */
@@ -3037,7 +3038,7 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
cheapest_partial_path->rows * cheapest_partial_path->parallel_workers;
simple_gather_path = (Path *)
create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
- NULL, rowsp);
+ rel->lateral_relids, rowsp);
add_path(rel, simple_gather_path);
/*
@@ -3054,7 +3055,7 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
rows = subpath->rows * subpath->parallel_workers;
path = create_gather_merge_path(root, rel, subpath, rel->reltarget,
- subpath->pathkeys, NULL, rowsp);
+ subpath->pathkeys, rel->lateral_relids, rowsp);
add_path(rel, &path->path);
}
}
@@ -3156,11 +3157,15 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
double *rowsp = NULL;
List *useful_pathkeys_list = NIL;
Path *cheapest_partial_path = NULL;
+ Relids required_outer = rel->lateral_relids;
/* If there are no partial paths, there's nothing to do here. */
if (rel->partial_pathlist == NIL)
return;
+ if (!bms_is_subset(required_outer, rel->relids))
+ return;
+
/* Should we override the rel's rowcount estimate? */
if (override_rows)
rowsp = &rows;
@@ -3249,7 +3254,7 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
subpath,
rel->reltarget,
subpath->pathkeys,
- NULL,
+ required_outer,
rowsp);
add_path(rel, &path->path);
@@ -3427,7 +3432,8 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
/*
* Except for the topmost scan/join rel, consider gathering
* partial paths. We'll do the same for the topmost scan/join rel
- * once we know the final targetlist (see grouping_planner).
+ * once we know the final targetlist (see
+ * apply_scanjoin_target_to_paths).
*/
if (!bms_equal(rel->relids, root->all_baserels))
generate_useful_gather_paths(root, rel, false);
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index d345c0437a4..000e3ca9a25 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -1792,16 +1792,24 @@ match_unsorted_outer(PlannerInfo *root,
* partial path and the joinrel is parallel-safe. However, we can't
* handle JOIN_UNIQUE_OUTER, because the outer path will be partial, and
* therefore we won't be able to properly guarantee uniqueness. Nor can
- * we handle joins needing lateral rels, since partial paths must not be
- * parameterized. Similarly, we can't handle JOIN_FULL and JOIN_RIGHT,
- * because they can produce false null extended rows.
+ * we handle JOIN_FULL and JOIN_RIGHT, because they can produce false null
+ * extended rows.
+ *
+ * Partial paths may only have parameters in limited cases
+ * where the parameterization is fully satisfied without sharing state
+ * between workers, so we only allow lateral rels on inputs to the join
+ * if the resulting join contains no lateral rels, the inner rel's laterals
+ * are fully satisfied by the outer rel, and the outer rel doesn't depend
+ * on the inner rel to produce any laterals.
*/
if (joinrel->consider_parallel &&
save_jointype != JOIN_UNIQUE_OUTER &&
save_jointype != JOIN_FULL &&
save_jointype != JOIN_RIGHT &&
outerrel->partial_pathlist != NIL &&
- bms_is_empty(joinrel->lateral_relids))
+ bms_is_empty(joinrel->lateral_relids) &&
+ bms_is_subset(innerrel->lateral_relids, outerrel->relids) &&
+ (bms_is_empty(outerrel->lateral_relids) || !bms_is_subset(outerrel->lateral_relids, innerrel->relids)))
{
if (nestjoinOK)
consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index aa584848cf9..035471d05d0 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -816,6 +816,9 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
if (param->paramkind == PARAM_EXTERN)
return false;
+ if (param->paramkind == PARAM_EXEC)
+ return false;
+
if (param->paramkind != PARAM_EXEC ||
!list_member_int(context->safe_param_ids, param->paramid))
{
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 4478036bb6a..5667222e925 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -2437,6 +2437,8 @@ create_nestloop_path(PlannerInfo *root,
NestPath *pathnode = makeNode(NestPath);
Relids inner_req_outer = PATH_REQ_OUTER(inner_path);
+ /* TODO: Assert lateral relids subset safety? */
+
/*
* If the inner path is parameterized by the outer, we must drop any
* restrict_clauses that are due to be moved into the inner path. We have
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index c20b7298a3d..a5ba65a5616 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -2962,7 +2962,7 @@ typedef struct MinMaxAggInfo
* for conflicting purposes.
*
* In addition, PARAM_EXEC slots are assigned for Params representing outputs
- * from subplans (values that are setParam items for those subplans). These
+ * from subplans (values that are setParam items for those subplans). [TODO: is this true, or only for init plans?] These
* IDs need not be tracked via PlannerParamItems, since we do not need any
* duplicate-elimination nor later processing of the represented expressions.
* Instead, we just record the assignment of the slot number by appending to
diff --git a/src/test/regress/expected/incremental_sort.out b/src/test/regress/expected/incremental_sort.out
index 0c3433f8e58..0cb7c1a49c0 100644
--- a/src/test/regress/expected/incremental_sort.out
+++ b/src/test/regress/expected/incremental_sort.out
@@ -1600,16 +1600,16 @@ from tenk1 t, generate_series(1, 1000);
QUERY PLAN
---------------------------------------------------------------------------------
Unique
- -> Sort
- Sort Key: t.unique1, ((SubPlan 1))
- -> Gather
- Workers Planned: 2
+ -> Gather Merge
+ Workers Planned: 2
+ -> Sort
+ Sort Key: t.unique1, ((SubPlan 1))
-> Nested Loop
-> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
-> Function Scan on generate_series
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on tenk1
- Index Cond: (unique1 = t.unique1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on tenk1
+ Index Cond: (unique1 = t.unique1)
(11 rows)
explain (costs off) select
@@ -1619,16 +1619,16 @@ from tenk1 t, generate_series(1, 1000)
order by 1, 2;
QUERY PLAN
---------------------------------------------------------------------------
- Sort
- Sort Key: t.unique1, ((SubPlan 1))
- -> Gather
- Workers Planned: 2
+ Gather Merge
+ Workers Planned: 2
+ -> Sort
+ Sort Key: t.unique1, ((SubPlan 1))
-> Nested Loop
-> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
-> Function Scan on generate_series
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on tenk1
- Index Cond: (unique1 = t.unique1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on tenk1
+ Index Cond: (unique1 = t.unique1)
(10 rows)
-- Parallel sort but with expression not available until the upper rel.
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index 7555764c779..5c45f9c0a50 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -1284,60 +1284,64 @@ EXPLAIN (VERBOSE, COSTS OFF) SELECT * FROM part p(x) ORDER BY x;
--
-- pruning won't work for mc3p, because some keys are Params
explain (costs off) select * from mc2p t1, lateral (select count(*) from mc3p t2 where t2.a = t1.b and abs(t2.b) = 1 and t2.c = 1) s where t1.a = 1;
- QUERY PLAN
------------------------------------------------------------------------
- Nested Loop
- -> Append
- -> Seq Scan on mc2p1 t1_1
- Filter: (a = 1)
- -> Seq Scan on mc2p2 t1_2
- Filter: (a = 1)
- -> Seq Scan on mc2p_default t1_3
- Filter: (a = 1)
- -> Aggregate
- -> Append
- -> Seq Scan on mc3p0 t2_1
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p1 t2_2
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p2 t2_3
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p3 t2_4
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p4 t2_5
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p5 t2_6
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p6 t2_7
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p7 t2_8
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p_default t2_9
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
-(28 rows)
+ QUERY PLAN
+-----------------------------------------------------------------------------
+ Gather
+ Workers Planned: 2
+ -> Nested Loop
+ -> Parallel Append
+ -> Parallel Seq Scan on mc2p1 t1_1
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p2 t1_2
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p_default t1_3
+ Filter: (a = 1)
+ -> Aggregate
+ -> Append
+ -> Seq Scan on mc3p0 t2_1
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p1 t2_2
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p2 t2_3
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p3 t2_4
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p4 t2_5
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p5 t2_6
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p6 t2_7
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p7 t2_8
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p_default t2_9
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+(30 rows)
-- pruning should work fine, because values for a prefix of keys (a, b) are
-- available
explain (costs off) select * from mc2p t1, lateral (select count(*) from mc3p t2 where t2.c = t1.b and abs(t2.b) = 1 and t2.a = 1) s where t1.a = 1;
- QUERY PLAN
------------------------------------------------------------------------
- Nested Loop
- -> Append
- -> Seq Scan on mc2p1 t1_1
- Filter: (a = 1)
- -> Seq Scan on mc2p2 t1_2
- Filter: (a = 1)
- -> Seq Scan on mc2p_default t1_3
- Filter: (a = 1)
- -> Aggregate
- -> Append
- -> Seq Scan on mc3p0 t2_1
- Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p1 t2_2
- Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p_default t2_3
- Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
-(16 rows)
+ QUERY PLAN
+-----------------------------------------------------------------------------
+ Gather
+ Workers Planned: 2
+ -> Nested Loop
+ -> Parallel Append
+ -> Parallel Seq Scan on mc2p1 t1_1
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p2 t1_2
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p_default t1_3
+ Filter: (a = 1)
+ -> Aggregate
+ -> Append
+ -> Seq Scan on mc3p0 t2_1
+ Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p1 t2_2
+ Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p_default t2_3
+ Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
+(18 rows)
-- also here, because values for all keys are provided
explain (costs off) select * from mc2p t1, lateral (select count(*) from mc3p t2 where t2.a = 1 and abs(t2.b) = 1 and t2.c = 1) s where t1.a = 1;
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 9b4d7dd44a4..01443e2ffbe 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -137,8 +137,8 @@ create table part_pa_test_p2 partition of part_pa_test for values from (0) to (m
explain (costs off)
select (select max((select pa1.b from part_pa_test pa1 where pa1.a = pa2.a)))
from part_pa_test pa2;
- QUERY PLAN
---------------------------------------------------------------
+ QUERY PLAN
+----------------------------------------------------------------
Aggregate
-> Gather
Workers Planned: 3
@@ -148,12 +148,14 @@ explain (costs off)
SubPlan 2
-> Result
SubPlan 1
- -> Append
- -> Seq Scan on part_pa_test_p1 pa1_1
- Filter: (a = pa2.a)
- -> Seq Scan on part_pa_test_p2 pa1_2
- Filter: (a = pa2.a)
-(14 rows)
+ -> Gather
+ Workers Planned: 3
+ -> Parallel Append
+ -> Parallel Seq Scan on part_pa_test_p1 pa1_1
+ Filter: (a = pa2.a)
+ -> Parallel Seq Scan on part_pa_test_p2 pa1_2
+ Filter: (a = pa2.a)
+(16 rows)
drop table part_pa_test;
-- test with leader participation disabled
@@ -320,19 +322,19 @@ explain (costs off, verbose) select
QUERY PLAN
----------------------------------------------------------------------------
Gather
- Output: (SubPlan 1)
+ Output: ((SubPlan 1))
Workers Planned: 4
-> Nested Loop
- Output: t.unique1
+ Output: (SubPlan 1)
-> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
Output: t.unique1
-> Function Scan on pg_catalog.generate_series
Output: generate_series.generate_series
Function Call: generate_series(1, 10)
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(14 rows)
explain (costs off, verbose) select
@@ -341,63 +343,69 @@ explain (costs off, verbose) select
QUERY PLAN
----------------------------------------------------------------------
Gather
- Output: (SubPlan 1)
+ Output: ((SubPlan 1))
Workers Planned: 4
-> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
- Output: t.unique1
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ Output: (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(9 rows)
explain (costs off, verbose) select
(select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
from tenk1 t
limit 1;
- QUERY PLAN
--------------------------------------------------------------------
+ QUERY PLAN
+----------------------------------------------------------------------------
Limit
Output: ((SubPlan 1))
- -> Seq Scan on public.tenk1 t
- Output: (SubPlan 1)
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
-(8 rows)
+ -> Gather
+ Output: ((SubPlan 1))
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(11 rows)
explain (costs off, verbose) select t.unique1
from tenk1 t
where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
- QUERY PLAN
--------------------------------------------------------------
- Seq Scan on public.tenk1 t
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
Output: t.unique1
- Filter: (t.unique1 = (SubPlan 1))
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
-(7 rows)
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ Filter: (t.unique1 = (SubPlan 1))
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(10 rows)
explain (costs off, verbose) select *
from tenk1 t
order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
- QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
- Sort
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Gather Merge
Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
- Sort Key: ((SubPlan 1))
- -> Gather
- Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, (SubPlan 1)
- Workers Planned: 4
+ Workers Planned: 4
+ -> Sort
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
+ Sort Key: ((SubPlan 1))
-> Parallel Seq Scan on public.tenk1 t
- Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(12 rows)
-- test subplan in join/lateral join
@@ -409,14 +417,14 @@ explain (costs off, verbose, timing off) select t.unique1, l.*
QUERY PLAN
----------------------------------------------------------------------
Gather
- Output: t.unique1, (SubPlan 1)
+ Output: t.unique1, ((SubPlan 1))
Workers Planned: 4
-> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
- Output: t.unique1
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ Output: t.unique1, (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(9 rows)
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
@@ -1322,8 +1330,10 @@ SELECT 1 FROM tenk1_vw_sec
-> Parallel Index Only Scan using tenk1_unique1 on tenk1
SubPlan 1
-> Aggregate
- -> Seq Scan on int4_tbl
- Filter: (f1 < tenk1_vw_sec.unique1)
-(9 rows)
+ -> Gather
+ Workers Planned: 1
+ -> Parallel Seq Scan on int4_tbl
+ Filter: (f1 < tenk1_vw_sec.unique1)
+(11 rows)
rollback;
--
2.39.0
0003-review-comments-v6.patchtext/x-patch; charset=UTF-8; name=0003-review-comments-v6.patchDownload
From 1aa279e8e82a4e9771fb1a5068e9a5792a824a54 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas.vondra@postgresql.org>
Date: Wed, 18 Jan 2023 19:51:38 +0100
Subject: [PATCH 3/5] review comments
---
src/backend/optimizer/path/allpaths.c | 5 +++--
src/backend/optimizer/util/clauses.c | 5 +++++
2 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 7108501b9a7..357dfaab3e8 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -558,8 +558,8 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
* the final scan/join targetlist is available (see grouping_planner).
*/
if (rel->reloptkind == RELOPT_BASEREL &&
- !bms_equal(rel->relids, root->all_baserels)
- && (rel->subplan_params == NIL || rte->rtekind != RTE_SUBQUERY))
+ !bms_equal(rel->relids, root->all_baserels) &&
+ (rel->subplan_params == NIL || rte->rtekind != RTE_SUBQUERY))
generate_useful_gather_paths(root, rel, false);
/* Now find the cheapest of the paths for this rel */
@@ -3163,6 +3163,7 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
if (rel->partial_pathlist == NIL)
return;
+ /* FIXME ??? */
if (!bms_is_subset(required_outer, rel->relids))
return;
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 035471d05d0..9585acf8e69 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -819,6 +819,11 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
if (param->paramkind == PARAM_EXEC)
return false;
+ /*
+ * XXX The first condition is certainly true, thanks to the preceding
+ * check. The comment above should be updated to reflect this change,
+ * probably.
+ */
if (param->paramkind != PARAM_EXEC ||
!list_member_int(context->safe_param_ids, param->paramid))
{
--
2.39.0
0004-Possible-additional-checks-v6.patchtext/x-patch; charset=UTF-8; name=0004-Possible-additional-checks-v6.patchDownload
From 45da97c1141ead1fe757fcff549e68fbd4c1ded9 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas.vondra@postgresql.org>
Date: Wed, 18 Jan 2023 19:21:24 +0100
Subject: [PATCH 4/5] Possible additional checks
---
src/backend/optimizer/path/allpaths.c | 29 +++++++++++++++++++++------
1 file changed, 23 insertions(+), 6 deletions(-)
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 357dfaab3e8..eaa972ec64b 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -3019,11 +3019,16 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
ListCell *lc;
double rows;
double *rowsp = NULL;
+ Relids required_outer = rel->lateral_relids;
/* If there are no partial paths, there's nothing to do here. */
if (rel->partial_pathlist == NIL)
return;
+ if (!bms_is_subset(required_outer, rel->relids))
+ return;
+
+
/* Should we override the rel's rowcount estimate? */
if (override_rows)
rowsp = &rows;
@@ -3034,12 +3039,16 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
* of partial_pathlist because of the way add_partial_path works.
*/
cheapest_partial_path = linitial(rel->partial_pathlist);
- rows =
- cheapest_partial_path->rows * cheapest_partial_path->parallel_workers;
- simple_gather_path = (Path *)
- create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
- rel->lateral_relids, rowsp);
- add_path(rel, simple_gather_path);
+ if (cheapest_partial_path->param_info == NULL ||
+ bms_is_subset(cheapest_partial_path->param_info->ppi_req_outer, rel->relids))
+ {
+ rows =
+ cheapest_partial_path->rows * cheapest_partial_path->parallel_workers;
+ simple_gather_path = (Path *)
+ create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
+ rel->lateral_relids, rowsp);
+ add_path(rel, simple_gather_path);
+ }
/*
* For each useful ordering, we can consider an order-preserving Gather
@@ -3053,6 +3062,10 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
if (subpath->pathkeys == NIL)
continue;
+ if (subpath->param_info != NULL &&
+ !bms_is_subset(subpath->param_info->ppi_req_outer, rel->relids))
+ break;
+
rows = subpath->rows * subpath->parallel_workers;
path = create_gather_merge_path(root, rel, subpath, rel->reltarget,
subpath->pathkeys, rel->lateral_relids, rowsp);
@@ -3223,6 +3236,10 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
(presorted_keys == 0 || !enable_incremental_sort))
continue;
+ if (subpath->param_info != NULL &&
+ !bms_is_subset(subpath->param_info->ppi_req_outer, rel->relids))
+ break;
+
/*
* Consider regular sort for any path that's not presorted or if
* incremental sort is disabled. We've no need to consider both
--
2.39.0
0005-review-comments-v6.patchtext/x-patch; charset=UTF-8; name=0005-review-comments-v6.patchDownload
From 2ca25bd5bbe1aa6849c55cab58dafe202adacce9 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas.vondra@postgresql.org>
Date: Wed, 18 Jan 2023 19:52:46 +0100
Subject: [PATCH 5/5] review comments
---
src/backend/optimizer/path/allpaths.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index eaa972ec64b..9dd10a1274c 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -3025,10 +3025,10 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
if (rel->partial_pathlist == NIL)
return;
+ /* FIXME ??? */
if (!bms_is_subset(required_outer, rel->relids))
return;
-
/* Should we override the rel's rowcount estimate? */
if (override_rows)
rowsp = &rows;
@@ -3062,6 +3062,7 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
if (subpath->pathkeys == NIL)
continue;
+ /* FIXME ??? */
if (subpath->param_info != NULL &&
!bms_is_subset(subpath->param_info->ppi_req_outer, rel->relids))
break;
--
2.39.0
Re: Parallelize correlated subqueries that execute within each worker
On Wed, Jan 18, 2023 at 2:09 PM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:
Hi,
This patch hasn't been updated since September, and it got broken by
4a29eabd1d91c5484426bc5836e0a7143b064f5a which the incremental sort
stuff a little bit. But the breakage was rather limited, so I took a
stab at fixing it - attached is the result, hopefully correct.
Thanks for fixing this up; the changes look correct to me.
I also added a couple minor comments about stuff I noticed while
rebasing and skimming the patch, I kept those in separate commits.
There's also a couple pre-existing TODOs.
I started work on some of these, but wasn't able to finish this
evening, so I don't have an updated series yet.
James, what's your plan with this patch. Do you intend to work on it for
PG16, or are there some issues I missed in the thread?
I'd love to see it get into PG16. I don't have any known issues, but
reviewing activity has been light. Originally Robert had had some
concerns about my original approach; I think my updated approach
resolves those issues, but it'd be good to have that sign-off.
Beyond that I'm mostly looking for review and evaluation of the
approach I've taken; of note is my description of that in [1].
One of the queries in in incremental_sort changed plans a little bit:
explain (costs off) select distinct
unique1,
(select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
from tenk1 t, generate_series(1, 1000);switched from
Unique (cost=18582710.41..18747375.21 rows=10000 width=8)
-> Gather Merge (cost=18582710.41..18697375.21 rows=10000000 ...)
Workers Planned: 2
-> Sort (cost=18582710.39..18593127.06 rows=4166667 ...)
Sort Key: t.unique1, ((SubPlan 1))
...to
Unique (cost=18582710.41..18614268.91 rows=10000 ...)
-> Gather Merge (cost=18582710.41..18614168.91 rows=20000 ...)
Workers Planned: 2
-> Unique (cost=18582710.39..18613960.39 rows=10000 ...)
-> Sort (cost=18582710.39..18593127.06 ...)
Sort Key: t.unique1, ((SubPlan 1))
...which probably makes sense, as the cost estimate decreases a bit.
Off the cuff that seems fine. I'll read it over again when I send the
updated series.
James Coleman
1: /messages/by-id/CAAaqYe8m0DHUWk7gLKb_C4abTD4nMkU26ErE=ahow4zNMZbzPQ@mail.gmail.com
Re: Parallelize correlated subqueries that execute within each worker
On Wed, Jan 18, 2023 at 9:34 PM James Coleman <jtc331@gmail.com> wrote:
On Wed, Jan 18, 2023 at 2:09 PM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:Hi,
This patch hasn't been updated since September, and it got broken by
4a29eabd1d91c5484426bc5836e0a7143b064f5a which the incremental sort
stuff a little bit. But the breakage was rather limited, so I took a
stab at fixing it - attached is the result, hopefully correct.Thanks for fixing this up; the changes look correct to me.
I also added a couple minor comments about stuff I noticed while
rebasing and skimming the patch, I kept those in separate commits.
There's also a couple pre-existing TODOs.I started work on some of these, but wasn't able to finish this
evening, so I don't have an updated series yet.James, what's your plan with this patch. Do you intend to work on it for
PG16, or are there some issues I missed in the thread?I'd love to see it get into PG16. I don't have any known issues, but
reviewing activity has been light. Originally Robert had had some
concerns about my original approach; I think my updated approach
resolves those issues, but it'd be good to have that sign-off.Beyond that I'm mostly looking for review and evaluation of the
approach I've taken; of note is my description of that in [1].
Here's an updated patch version incorporating feedback from Tomas as
well as some additional comments and tweaks.
While working through Tomas's comment about a conditional in the
max_parallel_hazard_waker being guaranteed true I realized that in the
current version of the patch the safe_param_ids tracking in
is_parallel_safe isn't actually needed any longer. That seemed
suspicious, and so I started digging, and I found out that in the
current approach all of the tests pass with only the changes in
clauses.c. I don't believe that the other changes aren't needed;
rather I believe there isn't yet a test case exercising them, but I
realize that means I can't prove they're needed. I spent some time
poking at this, but at least with my current level of imagination I
haven't stumbled across a query that would exercise these checks. One
of the reasons I'm fairly confident that this is true is that the
original approach (which was significantly more invasive) definitely
required rechecking parallel safety at each level until we reached the
point where the subquery was known to be generated within the current
worker through the safe_param_ids tracking mechanism. Of course it is
possible that that complexity is actually required and this simplified
approach isn't feasible (but I don't have a good reason to suspect
that currently). It's also possible that the restrictions on
subqueries just aren't necessary...but that isn't compelling because
it would require proving that you can never have a query level with
as-yet unsatisfied lateral rels.
Note: All of the existing tests for "you can't parallelize a
correlated subquery" are all simple versions which are not actually
parallel unsafe in theory. I assume they were added to show that the
code excluded that broad case, and there wasn't any finer grain of
detail required since the code simply didn't support making the
decision with that granularity anyway. But that means there weren't
any existing test cases to exercise the granularity I'm now trying to
achieve.
If someone is willing to help out what I'd like help with currently is
finding such a test case (where a gather or gather merge path would
otherwise be created but at the current plan level not all of the
required lateral rels are yet available -- meaning that we can't
perform all of the subqueries within the current worker). In support
of that patch 0004 converts several of the new parallel safety checks
into WARNING messages instead to make it easy to see if a query
happens to encounter any of those checks.
James Coleman
Attachments:
v7-0004-Warning-messages-for-finding-test-cases.patchapplication/octet-stream; name=v7-0004-Warning-messages-for-finding-test-cases.patchDownload
From a72ab86abc9f0f081b70d68f8505cf9eda0c3e19 Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Sat, 21 Jan 2023 19:41:28 -0500
Subject: [PATCH v7 4/4] Warning messages for finding test cases
---
src/backend/optimizer/path/allpaths.c | 12 ++++++++----
src/backend/optimizer/path/joinpath.c | 13 +++++++++----
2 files changed, 17 insertions(+), 8 deletions(-)
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 58433b762e..01ebbb943b 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -3030,7 +3030,8 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
* params used in a worker are generated within that worker.
*/
if (!bms_is_subset(required_outer, rel->relids))
- return;
+ elog(WARNING, "generate_gather_paths bms_is_subset");
+ /* return; */
/* Should we override the rel's rowcount estimate? */
if (override_rows)
@@ -3067,7 +3068,8 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
if (subpath->param_info != NULL &&
!bms_is_subset(subpath->param_info->ppi_req_outer, rel->relids))
- continue;
+ elog(WARNING, "generate_gather_paths bms_is_subset subpath");
+ /* continue; */
rows = subpath->rows * subpath->parallel_workers;
path = create_gather_merge_path(root, rel, subpath, rel->reltarget,
@@ -3184,7 +3186,8 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
* params used in a worker are generated within that worker.
*/
if (!bms_is_subset(required_outer, rel->relids))
- return;
+ elog(WARNING, "generate_useful_gather_paths bms_is_subset");
+ /* return; */
/* Should we override the rel's rowcount estimate? */
if (override_rows)
@@ -3244,7 +3247,8 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
if (subpath->param_info != NULL &&
!bms_is_subset(subpath->param_info->ppi_req_outer, rel->relids))
- continue;
+ elog(WARNING, "generate_useful_gather_paths bms_is_subset subpath");
+ /* continue; */
/*
* Consider regular sort for any path that's not presorted or if
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index 000e3ca9a2..be17b6e492 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -1806,11 +1806,16 @@ match_unsorted_outer(PlannerInfo *root,
save_jointype != JOIN_UNIQUE_OUTER &&
save_jointype != JOIN_FULL &&
save_jointype != JOIN_RIGHT &&
- outerrel->partial_pathlist != NIL &&
- bms_is_empty(joinrel->lateral_relids) &&
- bms_is_subset(innerrel->lateral_relids, outerrel->relids) &&
- (bms_is_empty(outerrel->lateral_relids) || !bms_is_subset(outerrel->lateral_relids, innerrel->relids)))
+ outerrel->partial_pathlist != NIL)
{
+ if (!(bms_is_empty(joinrel->lateral_relids) &&
+ bms_is_subset(innerrel->lateral_relids, outerrel->relids) &&
+ (bms_is_empty(outerrel->lateral_relids) || !bms_is_subset(outerrel->lateral_relids, innerrel->relids))))
+ {
+ elog(WARNING, "lateral relids violation on joinrel");
+ /* return; */
+ }
+
if (nestjoinOK)
consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
save_jointype, extra);
--
2.32.1 (Apple Git-133)
v7-0001-Add-tests-before-change.patchapplication/octet-stream; name=v7-0001-Add-tests-before-change.patchDownload
From d7a74b2f26edaf36d5098b4b45e19c437a4ebbdf Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Mon, 26 Sep 2022 20:30:23 -0400
Subject: [PATCH v7 1/4] Add tests before change
---
src/test/regress/expected/select_parallel.out | 108 ++++++++++++++++++
src/test/regress/sql/select_parallel.sql | 25 ++++
2 files changed, 133 insertions(+)
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 91f74fe47a..9b4d7dd44a 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -311,6 +311,114 @@ select count(*) from tenk1 where (two, four) not in
10000
(1 row)
+-- test parallel plans for queries containing correlated subplans
+-- where the subplan only needs params available from the current
+-- worker's scan.
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t, generate_series(1, 10);
+ QUERY PLAN
+----------------------------------------------------------------------------
+ Gather
+ Output: (SubPlan 1)
+ Workers Planned: 4
+ -> Nested Loop
+ Output: t.unique1
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ -> Function Scan on pg_catalog.generate_series
+ Output: generate_series.generate_series
+ Function Call: generate_series(1, 10)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(14 rows)
+
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t;
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: (SubPlan 1)
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(9 rows)
+
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t
+ limit 1;
+ QUERY PLAN
+-------------------------------------------------------------------
+ Limit
+ Output: ((SubPlan 1))
+ -> Seq Scan on public.tenk1 t
+ Output: (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(8 rows)
+
+explain (costs off, verbose) select t.unique1
+ from tenk1 t
+ where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+ QUERY PLAN
+-------------------------------------------------------------
+ Seq Scan on public.tenk1 t
+ Output: t.unique1
+ Filter: (t.unique1 = (SubPlan 1))
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(7 rows)
+
+explain (costs off, verbose) select *
+ from tenk1 t
+ order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Sort
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
+ Sort Key: ((SubPlan 1))
+ -> Gather
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, (SubPlan 1)
+ Workers Planned: 4
+ -> Parallel Seq Scan on public.tenk1 t
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(12 rows)
+
+-- test subplan in join/lateral join
+explain (costs off, verbose, timing off) select t.unique1, l.*
+ from tenk1 t
+ join lateral (
+ select (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1 offset 0)
+ ) l on true;
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: t.unique1, (SubPlan 1)
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(9 rows)
+
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
explain (costs off)
select * from tenk1 where (unique1 + random())::integer not in
diff --git a/src/test/regress/sql/select_parallel.sql b/src/test/regress/sql/select_parallel.sql
index 62fb68c7a0..21c2f1c742 100644
--- a/src/test/regress/sql/select_parallel.sql
+++ b/src/test/regress/sql/select_parallel.sql
@@ -111,6 +111,31 @@ explain (costs off)
(select hundred, thousand from tenk2 where thousand > 100);
select count(*) from tenk1 where (two, four) not in
(select hundred, thousand from tenk2 where thousand > 100);
+-- test parallel plans for queries containing correlated subplans
+-- where the subplan only needs params available from the current
+-- worker's scan.
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t, generate_series(1, 10);
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t;
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t
+ limit 1;
+explain (costs off, verbose) select t.unique1
+ from tenk1 t
+ where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+explain (costs off, verbose) select *
+ from tenk1 t
+ order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+-- test subplan in join/lateral join
+explain (costs off, verbose, timing off) select t.unique1, l.*
+ from tenk1 t
+ join lateral (
+ select (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1 offset 0)
+ ) l on true;
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
explain (costs off)
select * from tenk1 where (unique1 + random())::integer not in
--
2.32.1 (Apple Git-133)
v7-0003-Possible-additional-checks.patchapplication/octet-stream; name=v7-0003-Possible-additional-checks.patchDownload
From f2c2e5d64130a876dde539d09d049508820c5063 Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Wed, 18 Jan 2023 20:49:42 -0500
Subject: [PATCH v7 3/4] Possible additional checks
---
src/backend/optimizer/path/allpaths.c | 32 ++++++++++++++++++++++-----
1 file changed, 26 insertions(+), 6 deletions(-)
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index f93c6927b7..58433b762e 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -3019,11 +3019,19 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
ListCell *lc;
double rows;
double *rowsp = NULL;
+ Relids required_outer = rel->lateral_relids;
/* If there are no partial paths, there's nothing to do here. */
if (rel->partial_pathlist == NIL)
return;
+ /*
+ * Delay gather path creation until the level in the join tree where all
+ * params used in a worker are generated within that worker.
+ */
+ if (!bms_is_subset(required_outer, rel->relids))
+ return;
+
/* Should we override the rel's rowcount estimate? */
if (override_rows)
rowsp = &rows;
@@ -3034,12 +3042,16 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
* of partial_pathlist because of the way add_partial_path works.
*/
cheapest_partial_path = linitial(rel->partial_pathlist);
- rows =
- cheapest_partial_path->rows * cheapest_partial_path->parallel_workers;
- simple_gather_path = (Path *)
- create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
- rel->lateral_relids, rowsp);
- add_path(rel, simple_gather_path);
+ if (cheapest_partial_path->param_info == NULL ||
+ bms_is_subset(cheapest_partial_path->param_info->ppi_req_outer, rel->relids))
+ {
+ rows =
+ cheapest_partial_path->rows * cheapest_partial_path->parallel_workers;
+ simple_gather_path = (Path *)
+ create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
+ rel->lateral_relids, rowsp);
+ add_path(rel, simple_gather_path);
+ }
/*
* For each useful ordering, we can consider an order-preserving Gather
@@ -3053,6 +3065,10 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
if (subpath->pathkeys == NIL)
continue;
+ if (subpath->param_info != NULL &&
+ !bms_is_subset(subpath->param_info->ppi_req_outer, rel->relids))
+ continue;
+
rows = subpath->rows * subpath->parallel_workers;
path = create_gather_merge_path(root, rel, subpath, rel->reltarget,
subpath->pathkeys, rel->lateral_relids, rowsp);
@@ -3226,6 +3242,10 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
(presorted_keys == 0 || !enable_incremental_sort))
continue;
+ if (subpath->param_info != NULL &&
+ !bms_is_subset(subpath->param_info->ppi_req_outer, rel->relids))
+ continue;
+
/*
* Consider regular sort for any path that's not presorted or if
* incremental sort is disabled. We've no need to consider both
--
2.32.1 (Apple Git-133)
v7-0002-Parallelize-correlated-subqueries.patchapplication/octet-stream; name=v7-0002-Parallelize-correlated-subqueries.patchDownload
From 30a04905599bf9b65a657b8e272dd73dfc89c723 Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Wed, 18 Jan 2023 20:43:26 -0500
Subject: [PATCH v7 2/4] Parallelize correlated subqueries
When params are provided at the current query level (i.e., are generated
within a single worker and not shared across workers) we can safely
execute these in parallel.
Alternative approach using just relids subset check
---
doc/src/sgml/parallel.sgml | 3 +-
src/backend/optimizer/path/allpaths.c | 20 ++-
src/backend/optimizer/path/joinpath.c | 16 ++-
src/backend/optimizer/util/clauses.c | 74 ++++------
src/backend/optimizer/util/pathnode.c | 2 +
src/include/nodes/pathnodes.h | 2 +-
.../regress/expected/incremental_sort.out | 41 +++---
src/test/regress/expected/partition_prune.out | 104 +++++++-------
src/test/regress/expected/select_parallel.out | 128 ++++++++++--------
9 files changed, 200 insertions(+), 190 deletions(-)
diff --git a/doc/src/sgml/parallel.sgml b/doc/src/sgml/parallel.sgml
index 5acc9537d6..fd32572ec8 100644
--- a/doc/src/sgml/parallel.sgml
+++ b/doc/src/sgml/parallel.sgml
@@ -518,7 +518,8 @@ EXPLAIN SELECT * FROM pgbench_accounts WHERE filler LIKE '%x%';
<listitem>
<para>
- Plan nodes that reference a correlated <literal>SubPlan</literal>.
+ Plan nodes that reference a correlated <literal>SubPlan</literal> where
+ the result is shared between workers.
</para>
</listitem>
</itemizedlist>
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index c2fc568dc8..f93c6927b7 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -558,7 +558,8 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
* the final scan/join targetlist is available (see grouping_planner).
*/
if (rel->reloptkind == RELOPT_BASEREL &&
- !bms_equal(rel->relids, root->all_baserels))
+ !bms_equal(rel->relids, root->all_baserels) &&
+ (rel->subplan_params == NIL || rte->rtekind != RTE_SUBQUERY))
generate_useful_gather_paths(root, rel, false);
/* Now find the cheapest of the paths for this rel */
@@ -3037,7 +3038,7 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
cheapest_partial_path->rows * cheapest_partial_path->parallel_workers;
simple_gather_path = (Path *)
create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
- NULL, rowsp);
+ rel->lateral_relids, rowsp);
add_path(rel, simple_gather_path);
/*
@@ -3054,7 +3055,7 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
rows = subpath->rows * subpath->parallel_workers;
path = create_gather_merge_path(root, rel, subpath, rel->reltarget,
- subpath->pathkeys, NULL, rowsp);
+ subpath->pathkeys, rel->lateral_relids, rowsp);
add_path(rel, &path->path);
}
}
@@ -3156,11 +3157,19 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
double *rowsp = NULL;
List *useful_pathkeys_list = NIL;
Path *cheapest_partial_path = NULL;
+ Relids required_outer = rel->lateral_relids;
/* If there are no partial paths, there's nothing to do here. */
if (rel->partial_pathlist == NIL)
return;
+ /*
+ * Delay gather path creation until the level in the join tree where all
+ * params used in a worker are generated within that worker.
+ */
+ if (!bms_is_subset(required_outer, rel->relids))
+ return;
+
/* Should we override the rel's rowcount estimate? */
if (override_rows)
rowsp = &rows;
@@ -3249,7 +3258,7 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
subpath,
rel->reltarget,
subpath->pathkeys,
- NULL,
+ required_outer,
rowsp);
add_path(rel, &path->path);
@@ -3427,7 +3436,8 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
/*
* Except for the topmost scan/join rel, consider gathering
* partial paths. We'll do the same for the topmost scan/join rel
- * once we know the final targetlist (see grouping_planner).
+ * once we know the final targetlist (see
+ * apply_scanjoin_target_to_paths).
*/
if (!bms_equal(rel->relids, root->all_baserels))
generate_useful_gather_paths(root, rel, false);
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index d345c0437a..000e3ca9a2 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -1792,16 +1792,24 @@ match_unsorted_outer(PlannerInfo *root,
* partial path and the joinrel is parallel-safe. However, we can't
* handle JOIN_UNIQUE_OUTER, because the outer path will be partial, and
* therefore we won't be able to properly guarantee uniqueness. Nor can
- * we handle joins needing lateral rels, since partial paths must not be
- * parameterized. Similarly, we can't handle JOIN_FULL and JOIN_RIGHT,
- * because they can produce false null extended rows.
+ * we handle JOIN_FULL and JOIN_RIGHT, because they can produce false null
+ * extended rows.
+ *
+ * Partial paths may only have parameters in limited cases
+ * where the parameterization is fully satisfied without sharing state
+ * between workers, so we only allow lateral rels on inputs to the join
+ * if the resulting join contains no lateral rels, the inner rel's laterals
+ * are fully satisfied by the outer rel, and the outer rel doesn't depend
+ * on the inner rel to produce any laterals.
*/
if (joinrel->consider_parallel &&
save_jointype != JOIN_UNIQUE_OUTER &&
save_jointype != JOIN_FULL &&
save_jointype != JOIN_RIGHT &&
outerrel->partial_pathlist != NIL &&
- bms_is_empty(joinrel->lateral_relids))
+ bms_is_empty(joinrel->lateral_relids) &&
+ bms_is_subset(innerrel->lateral_relids, outerrel->relids) &&
+ (bms_is_empty(outerrel->lateral_relids) || !bms_is_subset(outerrel->lateral_relids, innerrel->relids)))
{
if (nestjoinOK)
consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index aa584848cf..4a3a0489a8 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -89,7 +89,6 @@ typedef struct
{
char max_hazard; /* worst proparallel hazard found so far */
char max_interesting; /* worst proparallel hazard of interest */
- List *safe_param_ids; /* PARAM_EXEC Param IDs to treat as safe */
} max_parallel_hazard_context;
static bool contain_agg_clause_walker(Node *node, void *context);
@@ -618,7 +617,6 @@ max_parallel_hazard(Query *parse)
context.max_hazard = PROPARALLEL_SAFE;
context.max_interesting = PROPARALLEL_UNSAFE;
- context.safe_param_ids = NIL;
(void) max_parallel_hazard_walker((Node *) parse, &context);
return context.max_hazard;
}
@@ -629,43 +627,24 @@ max_parallel_hazard(Query *parse)
*
* root->glob->maxParallelHazard must previously have been set to the
* result of max_parallel_hazard() on the whole query.
+ *
+ * The caller is responsible for verifying that PARAM_EXEC Params are generated
+ * at the current plan level.
*/
bool
is_parallel_safe(PlannerInfo *root, Node *node)
{
max_parallel_hazard_context context;
- PlannerInfo *proot;
- ListCell *l;
/*
- * Even if the original querytree contained nothing unsafe, we need to
- * search the expression if we have generated any PARAM_EXEC Params while
- * planning, because those are parallel-restricted and there might be one
- * in this expression. But otherwise we don't need to look.
+ * If we've already checked the querytree don't burn cycles doing it again.
*/
- if (root->glob->maxParallelHazard == PROPARALLEL_SAFE &&
- root->glob->paramExecTypes == NIL)
+ if (root->glob->maxParallelHazard == PROPARALLEL_SAFE)
return true;
+
/* Else use max_parallel_hazard's search logic, but stop on RESTRICTED */
context.max_hazard = PROPARALLEL_SAFE;
context.max_interesting = PROPARALLEL_RESTRICTED;
- context.safe_param_ids = NIL;
-
- /*
- * The params that refer to the same or parent query level are considered
- * parallel-safe. The idea is that we compute such params at Gather or
- * Gather Merge node and pass their value to workers.
- */
- for (proot = root; proot != NULL; proot = proot->parent_root)
- {
- foreach(l, proot->init_plans)
- {
- SubPlan *initsubplan = (SubPlan *) lfirst(l);
-
- context.safe_param_ids = list_concat(context.safe_param_ids,
- initsubplan->setParam);
- }
- }
return !max_parallel_hazard_walker(node, &context);
}
@@ -775,39 +754,34 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
}
/*
- * Only parallel-safe SubPlans can be sent to workers. Within the
- * testexpr of the SubPlan, Params representing the output columns of the
- * subplan can be treated as parallel-safe, so temporarily add their IDs
- * to the safe_param_ids list while examining the testexpr.
+ * Only parallel-safe SubPlans can be sent to workers.
*/
else if (IsA(node, SubPlan))
{
SubPlan *subplan = (SubPlan *) node;
- List *save_safe_param_ids;
if (!subplan->parallel_safe &&
max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
return true;
- save_safe_param_ids = context->safe_param_ids;
- context->safe_param_ids = list_concat_copy(context->safe_param_ids,
- subplan->paramIds);
+
if (max_parallel_hazard_walker(subplan->testexpr, context))
- return true; /* no need to restore safe_param_ids */
- list_free(context->safe_param_ids);
- context->safe_param_ids = save_safe_param_ids;
- /* we must also check args, but no special Param treatment there */
+ return true;
+
if (max_parallel_hazard_walker((Node *) subplan->args, context))
return true;
+
/* don't want to recurse normally, so we're done */
return false;
}
/*
- * We can't pass Params to workers at the moment either, so they are also
- * parallel-restricted, unless they are PARAM_EXTERN Params or are
- * PARAM_EXEC Params listed in safe_param_ids, meaning they could be
- * either generated within workers or can be computed by the leader and
- * then their value can be passed to workers.
+ * We can't pass all types of Params to workers at the moment either.
+ * PARAM_EXTERN Params are always allowed. PARAM_EXEC Params are parallel-
+ * safe when they can be computed by the leader and their value passed to
+ * workers or are generated within a worker. However we don't always know
+ * whether a param will be generated within a worker when we are parsing a
+ * querytree. In that case we leave it to the consumer to verify that the
+ * current plan level provides these params.
*/
else if (IsA(node, Param))
{
@@ -816,12 +790,12 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
if (param->paramkind == PARAM_EXTERN)
return false;
- if (param->paramkind != PARAM_EXEC ||
- !list_member_int(context->safe_param_ids, param->paramid))
- {
- if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
- return true;
- }
+ if (param->paramkind == PARAM_EXEC)
+ return false;
+
+ if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
+ return true;
+
return false; /* nothing to recurse to */
}
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 4478036bb6..5667222e92 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -2437,6 +2437,8 @@ create_nestloop_path(PlannerInfo *root,
NestPath *pathnode = makeNode(NestPath);
Relids inner_req_outer = PATH_REQ_OUTER(inner_path);
+ /* TODO: Assert lateral relids subset safety? */
+
/*
* If the inner path is parameterized by the outer, we must drop any
* restrict_clauses that are due to be moved into the inner path. We have
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 2d1d8f4bcd..2b2a72f802 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -2990,7 +2990,7 @@ typedef struct MinMaxAggInfo
* for conflicting purposes.
*
* In addition, PARAM_EXEC slots are assigned for Params representing outputs
- * from subplans (values that are setParam items for those subplans). These
+ * from subplans (values that are setParam items for those subplans). [TODO: is this true, or only for init plans?] These
* IDs need not be tracked via PlannerParamItems, since we do not need any
* duplicate-elimination nor later processing of the represented expressions.
* Instead, we just record the assignment of the slot number by appending to
diff --git a/src/test/regress/expected/incremental_sort.out b/src/test/regress/expected/incremental_sort.out
index 0c3433f8e5..2b52b0ca3c 100644
--- a/src/test/regress/expected/incremental_sort.out
+++ b/src/test/regress/expected/incremental_sort.out
@@ -1597,20 +1597,21 @@ explain (costs off) select distinct
unique1,
(select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
from tenk1 t, generate_series(1, 1000);
- QUERY PLAN
----------------------------------------------------------------------------------
+ QUERY PLAN
+---------------------------------------------------------------------------------------
Unique
- -> Sort
- Sort Key: t.unique1, ((SubPlan 1))
- -> Gather
- Workers Planned: 2
- -> Nested Loop
- -> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
- -> Function Scan on generate_series
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on tenk1
- Index Cond: (unique1 = t.unique1)
-(11 rows)
+ -> Gather Merge
+ Workers Planned: 2
+ -> Unique
+ -> Sort
+ Sort Key: t.unique1, ((SubPlan 1))
+ -> Nested Loop
+ -> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
+ -> Function Scan on generate_series
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on tenk1
+ Index Cond: (unique1 = t.unique1)
+(12 rows)
explain (costs off) select
unique1,
@@ -1619,16 +1620,16 @@ from tenk1 t, generate_series(1, 1000)
order by 1, 2;
QUERY PLAN
---------------------------------------------------------------------------
- Sort
- Sort Key: t.unique1, ((SubPlan 1))
- -> Gather
- Workers Planned: 2
+ Gather Merge
+ Workers Planned: 2
+ -> Sort
+ Sort Key: t.unique1, ((SubPlan 1))
-> Nested Loop
-> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
-> Function Scan on generate_series
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on tenk1
- Index Cond: (unique1 = t.unique1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on tenk1
+ Index Cond: (unique1 = t.unique1)
(10 rows)
-- Parallel sort but with expression not available until the upper rel.
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index 7555764c77..5c45f9c0a5 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -1284,60 +1284,64 @@ EXPLAIN (VERBOSE, COSTS OFF) SELECT * FROM part p(x) ORDER BY x;
--
-- pruning won't work for mc3p, because some keys are Params
explain (costs off) select * from mc2p t1, lateral (select count(*) from mc3p t2 where t2.a = t1.b and abs(t2.b) = 1 and t2.c = 1) s where t1.a = 1;
- QUERY PLAN
------------------------------------------------------------------------
- Nested Loop
- -> Append
- -> Seq Scan on mc2p1 t1_1
- Filter: (a = 1)
- -> Seq Scan on mc2p2 t1_2
- Filter: (a = 1)
- -> Seq Scan on mc2p_default t1_3
- Filter: (a = 1)
- -> Aggregate
- -> Append
- -> Seq Scan on mc3p0 t2_1
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p1 t2_2
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p2 t2_3
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p3 t2_4
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p4 t2_5
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p5 t2_6
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p6 t2_7
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p7 t2_8
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p_default t2_9
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
-(28 rows)
+ QUERY PLAN
+-----------------------------------------------------------------------------
+ Gather
+ Workers Planned: 2
+ -> Nested Loop
+ -> Parallel Append
+ -> Parallel Seq Scan on mc2p1 t1_1
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p2 t1_2
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p_default t1_3
+ Filter: (a = 1)
+ -> Aggregate
+ -> Append
+ -> Seq Scan on mc3p0 t2_1
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p1 t2_2
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p2 t2_3
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p3 t2_4
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p4 t2_5
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p5 t2_6
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p6 t2_7
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p7 t2_8
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p_default t2_9
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+(30 rows)
-- pruning should work fine, because values for a prefix of keys (a, b) are
-- available
explain (costs off) select * from mc2p t1, lateral (select count(*) from mc3p t2 where t2.c = t1.b and abs(t2.b) = 1 and t2.a = 1) s where t1.a = 1;
- QUERY PLAN
------------------------------------------------------------------------
- Nested Loop
- -> Append
- -> Seq Scan on mc2p1 t1_1
- Filter: (a = 1)
- -> Seq Scan on mc2p2 t1_2
- Filter: (a = 1)
- -> Seq Scan on mc2p_default t1_3
- Filter: (a = 1)
- -> Aggregate
- -> Append
- -> Seq Scan on mc3p0 t2_1
- Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p1 t2_2
- Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p_default t2_3
- Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
-(16 rows)
+ QUERY PLAN
+-----------------------------------------------------------------------------
+ Gather
+ Workers Planned: 2
+ -> Nested Loop
+ -> Parallel Append
+ -> Parallel Seq Scan on mc2p1 t1_1
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p2 t1_2
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p_default t1_3
+ Filter: (a = 1)
+ -> Aggregate
+ -> Append
+ -> Seq Scan on mc3p0 t2_1
+ Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p1 t2_2
+ Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p_default t2_3
+ Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
+(18 rows)
-- also here, because values for all keys are provided
explain (costs off) select * from mc2p t1, lateral (select count(*) from mc3p t2 where t2.a = 1 and abs(t2.b) = 1 and t2.c = 1) s where t1.a = 1;
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 9b4d7dd44a..01443e2ffb 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -137,8 +137,8 @@ create table part_pa_test_p2 partition of part_pa_test for values from (0) to (m
explain (costs off)
select (select max((select pa1.b from part_pa_test pa1 where pa1.a = pa2.a)))
from part_pa_test pa2;
- QUERY PLAN
---------------------------------------------------------------
+ QUERY PLAN
+----------------------------------------------------------------
Aggregate
-> Gather
Workers Planned: 3
@@ -148,12 +148,14 @@ explain (costs off)
SubPlan 2
-> Result
SubPlan 1
- -> Append
- -> Seq Scan on part_pa_test_p1 pa1_1
- Filter: (a = pa2.a)
- -> Seq Scan on part_pa_test_p2 pa1_2
- Filter: (a = pa2.a)
-(14 rows)
+ -> Gather
+ Workers Planned: 3
+ -> Parallel Append
+ -> Parallel Seq Scan on part_pa_test_p1 pa1_1
+ Filter: (a = pa2.a)
+ -> Parallel Seq Scan on part_pa_test_p2 pa1_2
+ Filter: (a = pa2.a)
+(16 rows)
drop table part_pa_test;
-- test with leader participation disabled
@@ -320,19 +322,19 @@ explain (costs off, verbose) select
QUERY PLAN
----------------------------------------------------------------------------
Gather
- Output: (SubPlan 1)
+ Output: ((SubPlan 1))
Workers Planned: 4
-> Nested Loop
- Output: t.unique1
+ Output: (SubPlan 1)
-> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
Output: t.unique1
-> Function Scan on pg_catalog.generate_series
Output: generate_series.generate_series
Function Call: generate_series(1, 10)
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(14 rows)
explain (costs off, verbose) select
@@ -341,63 +343,69 @@ explain (costs off, verbose) select
QUERY PLAN
----------------------------------------------------------------------
Gather
- Output: (SubPlan 1)
+ Output: ((SubPlan 1))
Workers Planned: 4
-> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
- Output: t.unique1
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ Output: (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(9 rows)
explain (costs off, verbose) select
(select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
from tenk1 t
limit 1;
- QUERY PLAN
--------------------------------------------------------------------
+ QUERY PLAN
+----------------------------------------------------------------------------
Limit
Output: ((SubPlan 1))
- -> Seq Scan on public.tenk1 t
- Output: (SubPlan 1)
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
-(8 rows)
+ -> Gather
+ Output: ((SubPlan 1))
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(11 rows)
explain (costs off, verbose) select t.unique1
from tenk1 t
where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
- QUERY PLAN
--------------------------------------------------------------
- Seq Scan on public.tenk1 t
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
Output: t.unique1
- Filter: (t.unique1 = (SubPlan 1))
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
-(7 rows)
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ Filter: (t.unique1 = (SubPlan 1))
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(10 rows)
explain (costs off, verbose) select *
from tenk1 t
order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
- QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
- Sort
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Gather Merge
Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
- Sort Key: ((SubPlan 1))
- -> Gather
- Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, (SubPlan 1)
- Workers Planned: 4
+ Workers Planned: 4
+ -> Sort
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
+ Sort Key: ((SubPlan 1))
-> Parallel Seq Scan on public.tenk1 t
- Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(12 rows)
-- test subplan in join/lateral join
@@ -409,14 +417,14 @@ explain (costs off, verbose, timing off) select t.unique1, l.*
QUERY PLAN
----------------------------------------------------------------------
Gather
- Output: t.unique1, (SubPlan 1)
+ Output: t.unique1, ((SubPlan 1))
Workers Planned: 4
-> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
- Output: t.unique1
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ Output: t.unique1, (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(9 rows)
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
@@ -1322,8 +1330,10 @@ SELECT 1 FROM tenk1_vw_sec
-> Parallel Index Only Scan using tenk1_unique1 on tenk1
SubPlan 1
-> Aggregate
- -> Seq Scan on int4_tbl
- Filter: (f1 < tenk1_vw_sec.unique1)
-(9 rows)
+ -> Gather
+ Workers Planned: 1
+ -> Parallel Seq Scan on int4_tbl
+ Filter: (f1 < tenk1_vw_sec.unique1)
+(11 rows)
rollback;
--
2.32.1 (Apple Git-133)
Re: Parallelize correlated subqueries that execute within each worker
On Sat, Jan 21, 2023 at 10:07 PM James Coleman <jtc331@gmail.com> wrote:
...
While working through Tomas's comment about a conditional in the
max_parallel_hazard_waker being guaranteed true I realized that in the
current version of the patch the safe_param_ids tracking in
is_parallel_safe isn't actually needed any longer. That seemed
suspicious, and so I started digging, and I found out that in the
current approach all of the tests pass with only the changes in
clauses.c. I don't believe that the other changes aren't needed;
rather I believe there isn't yet a test case exercising them, but I
realize that means I can't prove they're needed. I spent some time
poking at this, but at least with my current level of imagination I
haven't stumbled across a query that would exercise these checks.
I played with this a good bit more yesterday, I'm now a good bit more
confident this is correct. I've cleaned up the patch; see attached for
v7.
Here's some of my thought process:
The comments in src/include/nodes/pathnodes.h:2953 tell us that
PARAM_EXEC params are used to pass values around from one plan node to
another in the following ways:
1. Values down into subqueries (for outer references in subqueries)
2. Up out of subqueries (for the results of a subplan)
3. From a NestLoop plan node into its inner relation (when the inner
scan is parameterized with values from the outer relation)
Case (2) is already known to be safe (we currently add these params to
safe_param_ids in max_parallel_hazard_walker when we encounter a
SubPlan node).
I also believe case (3) is already handled. We don't build partial
paths for joins when joinrel->lateral_relids is non-empty, and join
order calculations already require that parameterization here go the
correct way (i.e., inner depends on outer rather than the other way
around).
That leaves us with only case (1) to consider in this patch. Another
way of saying this is that this is really the only thing the
safe_param_ids tracking is guarding against. For params passed down
into subqueries we can further distinguish between init plans and
"regular" subplans. We already know that params from init plans are
safe (at the right level). So we're concerned here with a way to know
if the params passed to subplans are safe. We already track required
rels in ParamPathInfo, so it's fairly simple to do this test.
Which this patch we do in fact now see (as expected) rels with
non-empty lateral_relids showing up in generate_[useful_]gather_paths.
And the partial paths can now have non-empty required outer rels. I'm
not able to come up with a plan that would actually be caught by those
checks; I theorize that because of the few places we actually call
generate_[useful_]gather_paths we are in practice already excluding
those, but for now I've left these as a conditional rather than an
assertion because it seems like the kind of guard we'd want to ensure
those methods are safe.
The other other place that we actually create gather[_merge] paths is
gather_grouping_paths(), and there I've chosen to use assertions,
because the point at which grouping happens in planning suggests to me
that we shouldn't have lateral dependencies at that point. If someone
is concerned about that, I'd be happy to change those to conditionals
also.
James Coleman
Attachments:
v8-0001-Add-tests-before-change.patchapplication/octet-stream; name=v8-0001-Add-tests-before-change.patchDownload
From d7a74b2f26edaf36d5098b4b45e19c437a4ebbdf Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Mon, 26 Sep 2022 20:30:23 -0400
Subject: [PATCH v8 1/2] Add tests before change
---
src/test/regress/expected/select_parallel.out | 108 ++++++++++++++++++
src/test/regress/sql/select_parallel.sql | 25 ++++
2 files changed, 133 insertions(+)
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 91f74fe47a..9b4d7dd44a 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -311,6 +311,114 @@ select count(*) from tenk1 where (two, four) not in
10000
(1 row)
+-- test parallel plans for queries containing correlated subplans
+-- where the subplan only needs params available from the current
+-- worker's scan.
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t, generate_series(1, 10);
+ QUERY PLAN
+----------------------------------------------------------------------------
+ Gather
+ Output: (SubPlan 1)
+ Workers Planned: 4
+ -> Nested Loop
+ Output: t.unique1
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ -> Function Scan on pg_catalog.generate_series
+ Output: generate_series.generate_series
+ Function Call: generate_series(1, 10)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(14 rows)
+
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t;
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: (SubPlan 1)
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(9 rows)
+
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t
+ limit 1;
+ QUERY PLAN
+-------------------------------------------------------------------
+ Limit
+ Output: ((SubPlan 1))
+ -> Seq Scan on public.tenk1 t
+ Output: (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(8 rows)
+
+explain (costs off, verbose) select t.unique1
+ from tenk1 t
+ where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+ QUERY PLAN
+-------------------------------------------------------------
+ Seq Scan on public.tenk1 t
+ Output: t.unique1
+ Filter: (t.unique1 = (SubPlan 1))
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(7 rows)
+
+explain (costs off, verbose) select *
+ from tenk1 t
+ order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Sort
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
+ Sort Key: ((SubPlan 1))
+ -> Gather
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, (SubPlan 1)
+ Workers Planned: 4
+ -> Parallel Seq Scan on public.tenk1 t
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(12 rows)
+
+-- test subplan in join/lateral join
+explain (costs off, verbose, timing off) select t.unique1, l.*
+ from tenk1 t
+ join lateral (
+ select (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1 offset 0)
+ ) l on true;
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: t.unique1, (SubPlan 1)
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(9 rows)
+
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
explain (costs off)
select * from tenk1 where (unique1 + random())::integer not in
diff --git a/src/test/regress/sql/select_parallel.sql b/src/test/regress/sql/select_parallel.sql
index 62fb68c7a0..21c2f1c742 100644
--- a/src/test/regress/sql/select_parallel.sql
+++ b/src/test/regress/sql/select_parallel.sql
@@ -111,6 +111,31 @@ explain (costs off)
(select hundred, thousand from tenk2 where thousand > 100);
select count(*) from tenk1 where (two, four) not in
(select hundred, thousand from tenk2 where thousand > 100);
+-- test parallel plans for queries containing correlated subplans
+-- where the subplan only needs params available from the current
+-- worker's scan.
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t, generate_series(1, 10);
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t;
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t
+ limit 1;
+explain (costs off, verbose) select t.unique1
+ from tenk1 t
+ where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+explain (costs off, verbose) select *
+ from tenk1 t
+ order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+-- test subplan in join/lateral join
+explain (costs off, verbose, timing off) select t.unique1, l.*
+ from tenk1 t
+ join lateral (
+ select (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1 offset 0)
+ ) l on true;
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
explain (costs off)
select * from tenk1 where (unique1 + random())::integer not in
--
2.32.1 (Apple Git-133)
v8-0002-Parallelize-correlated-subqueries.patchapplication/octet-stream; name=v8-0002-Parallelize-correlated-subqueries.patchDownload
From 3ddb8ea8ba949965b01cf8fd741f84d60f891945 Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Wed, 18 Jan 2023 20:43:26 -0500
Subject: [PATCH v8 2/2] Parallelize correlated subqueries
When params are provided at the current query level (i.e., are generated
within a single worker and not shared across workers) we can safely
execute these in parallel.
Alternative approach using just relids subset check
---
doc/src/sgml/parallel.sgml | 3 +-
src/backend/optimizer/path/allpaths.c | 45 ++++--
src/backend/optimizer/path/joinpath.c | 10 +-
src/backend/optimizer/plan/planner.c | 8 ++
src/backend/optimizer/util/clauses.c | 74 ++++------
.../regress/expected/incremental_sort.out | 41 +++---
src/test/regress/expected/partition_prune.out | 104 +++++++-------
src/test/regress/expected/select_parallel.out | 128 ++++++++++--------
8 files changed, 221 insertions(+), 192 deletions(-)
diff --git a/doc/src/sgml/parallel.sgml b/doc/src/sgml/parallel.sgml
index 5acc9537d6..fd32572ec8 100644
--- a/doc/src/sgml/parallel.sgml
+++ b/doc/src/sgml/parallel.sgml
@@ -518,7 +518,8 @@ EXPLAIN SELECT * FROM pgbench_accounts WHERE filler LIKE '%x%';
<listitem>
<para>
- Plan nodes that reference a correlated <literal>SubPlan</literal>.
+ Plan nodes that reference a correlated <literal>SubPlan</literal> where
+ the result is shared between workers.
</para>
</listitem>
</itemizedlist>
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index c2fc568dc8..0ff94690c3 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -3018,11 +3018,19 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
ListCell *lc;
double rows;
double *rowsp = NULL;
+ Relids required_outer = rel->lateral_relids;
/* If there are no partial paths, there's nothing to do here. */
if (rel->partial_pathlist == NIL)
return;
+ /*
+ * Delay gather path creation until the level in the join tree where all
+ * params used in a worker are generated within that worker.
+ */
+ if (!bms_is_subset(required_outer, rel->relids))
+ return;
+
/* Should we override the rel's rowcount estimate? */
if (override_rows)
rowsp = &rows;
@@ -3033,12 +3041,17 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
* of partial_pathlist because of the way add_partial_path works.
*/
cheapest_partial_path = linitial(rel->partial_pathlist);
- rows =
- cheapest_partial_path->rows * cheapest_partial_path->parallel_workers;
- simple_gather_path = (Path *)
- create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
- NULL, rowsp);
- add_path(rel, simple_gather_path);
+
+ /* We can't pass params to workers. */
+ if (bms_is_subset(PATH_REQ_OUTER(cheapest_partial_path), rel->relids))
+ {
+ rows =
+ cheapest_partial_path->rows * cheapest_partial_path->parallel_workers;
+ simple_gather_path = (Path *)
+ create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
+ required_outer, rowsp);
+ add_path(rel, simple_gather_path);
+ }
/*
* For each useful ordering, we can consider an order-preserving Gather
@@ -3052,9 +3065,13 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
if (subpath->pathkeys == NIL)
continue;
+ /* We can't pass params to workers. */
+ if (!bms_is_subset(PATH_REQ_OUTER(subpath), rel->relids))
+ continue;
+
rows = subpath->rows * subpath->parallel_workers;
path = create_gather_merge_path(root, rel, subpath, rel->reltarget,
- subpath->pathkeys, NULL, rowsp);
+ subpath->pathkeys, required_outer, rowsp);
add_path(rel, &path->path);
}
}
@@ -3156,11 +3173,19 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
double *rowsp = NULL;
List *useful_pathkeys_list = NIL;
Path *cheapest_partial_path = NULL;
+ Relids required_outer = rel->lateral_relids;
/* If there are no partial paths, there's nothing to do here. */
if (rel->partial_pathlist == NIL)
return;
+ /*
+ * Delay gather path creation until the level in the join tree where all
+ * params used in a worker are generated within that worker.
+ */
+ if (!bms_is_subset(required_outer, rel->relids))
+ return;
+
/* Should we override the rel's rowcount estimate? */
if (override_rows)
rowsp = &rows;
@@ -3217,6 +3242,10 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
(presorted_keys == 0 || !enable_incremental_sort))
continue;
+ /* We can't pass params to workers. */
+ if (!bms_is_subset(PATH_REQ_OUTER(subpath), rel->relids))
+ continue;
+
/*
* Consider regular sort for any path that's not presorted or if
* incremental sort is disabled. We've no need to consider both
@@ -3249,7 +3278,7 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
subpath,
rel->reltarget,
subpath->pathkeys,
- NULL,
+ required_outer,
rowsp);
add_path(rel, &path->path);
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index d345c0437a..644b90ad1e 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -1791,10 +1791,12 @@ match_unsorted_outer(PlannerInfo *root,
* Consider partial nestloop and mergejoin plan if outerrel has any
* partial path and the joinrel is parallel-safe. However, we can't
* handle JOIN_UNIQUE_OUTER, because the outer path will be partial, and
- * therefore we won't be able to properly guarantee uniqueness. Nor can
- * we handle joins needing lateral rels, since partial paths must not be
- * parameterized. Similarly, we can't handle JOIN_FULL and JOIN_RIGHT,
- * because they can produce false null extended rows.
+ * therefore we won't be able to properly guarantee uniqueness. Similarly,
+ * we can't handle JOIN_FULL and JOIN_RIGHT, because they can produce false
+ * null extended rows.
+ *
+ * While partial paths may now be parameterized so long as all of the params
+ * can be generated wholly within a worker we punt on supporting that here.
*/
if (joinrel->consider_parallel &&
save_jointype != JOIN_UNIQUE_OUTER &&
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 05f44faf6e..fcde41d9f3 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -7344,11 +7344,16 @@ gather_grouping_paths(PlannerInfo *root, RelOptInfo *rel)
ListCell *lc;
Path *cheapest_partial_path;
+ /* By grouping time we shouldn't have any lateral dependencies. */
+ Assert(rel->lateral_relids == NULL);
+
/* Try Gather for unordered paths and Gather Merge for ordered ones. */
generate_useful_gather_paths(root, rel, true);
/* Try cheapest partial path + explicit Sort + Gather Merge. */
cheapest_partial_path = linitial(rel->partial_pathlist);
+ /* By grouping time we shouldn't have any lateral dependencies. */
+ Assert(PATH_REQ_OUTER(cheapest_partial_path) == NULL);
if (!pathkeys_contained_in(root->group_pathkeys,
cheapest_partial_path->pathkeys))
{
@@ -7400,6 +7405,9 @@ gather_grouping_paths(PlannerInfo *root, RelOptInfo *rel)
if (presorted_keys == 0)
continue;
+ /* By grouping time we shouldn't have any lateral dependencies. */
+ Assert(PATH_REQ_OUTER(path) == NULL);
+
path = (Path *) create_incremental_sort_path(root,
rel,
path,
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index aa584848cf..4a3a0489a8 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -89,7 +89,6 @@ typedef struct
{
char max_hazard; /* worst proparallel hazard found so far */
char max_interesting; /* worst proparallel hazard of interest */
- List *safe_param_ids; /* PARAM_EXEC Param IDs to treat as safe */
} max_parallel_hazard_context;
static bool contain_agg_clause_walker(Node *node, void *context);
@@ -618,7 +617,6 @@ max_parallel_hazard(Query *parse)
context.max_hazard = PROPARALLEL_SAFE;
context.max_interesting = PROPARALLEL_UNSAFE;
- context.safe_param_ids = NIL;
(void) max_parallel_hazard_walker((Node *) parse, &context);
return context.max_hazard;
}
@@ -629,43 +627,24 @@ max_parallel_hazard(Query *parse)
*
* root->glob->maxParallelHazard must previously have been set to the
* result of max_parallel_hazard() on the whole query.
+ *
+ * The caller is responsible for verifying that PARAM_EXEC Params are generated
+ * at the current plan level.
*/
bool
is_parallel_safe(PlannerInfo *root, Node *node)
{
max_parallel_hazard_context context;
- PlannerInfo *proot;
- ListCell *l;
/*
- * Even if the original querytree contained nothing unsafe, we need to
- * search the expression if we have generated any PARAM_EXEC Params while
- * planning, because those are parallel-restricted and there might be one
- * in this expression. But otherwise we don't need to look.
+ * If we've already checked the querytree don't burn cycles doing it again.
*/
- if (root->glob->maxParallelHazard == PROPARALLEL_SAFE &&
- root->glob->paramExecTypes == NIL)
+ if (root->glob->maxParallelHazard == PROPARALLEL_SAFE)
return true;
+
/* Else use max_parallel_hazard's search logic, but stop on RESTRICTED */
context.max_hazard = PROPARALLEL_SAFE;
context.max_interesting = PROPARALLEL_RESTRICTED;
- context.safe_param_ids = NIL;
-
- /*
- * The params that refer to the same or parent query level are considered
- * parallel-safe. The idea is that we compute such params at Gather or
- * Gather Merge node and pass their value to workers.
- */
- for (proot = root; proot != NULL; proot = proot->parent_root)
- {
- foreach(l, proot->init_plans)
- {
- SubPlan *initsubplan = (SubPlan *) lfirst(l);
-
- context.safe_param_ids = list_concat(context.safe_param_ids,
- initsubplan->setParam);
- }
- }
return !max_parallel_hazard_walker(node, &context);
}
@@ -775,39 +754,34 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
}
/*
- * Only parallel-safe SubPlans can be sent to workers. Within the
- * testexpr of the SubPlan, Params representing the output columns of the
- * subplan can be treated as parallel-safe, so temporarily add their IDs
- * to the safe_param_ids list while examining the testexpr.
+ * Only parallel-safe SubPlans can be sent to workers.
*/
else if (IsA(node, SubPlan))
{
SubPlan *subplan = (SubPlan *) node;
- List *save_safe_param_ids;
if (!subplan->parallel_safe &&
max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
return true;
- save_safe_param_ids = context->safe_param_ids;
- context->safe_param_ids = list_concat_copy(context->safe_param_ids,
- subplan->paramIds);
+
if (max_parallel_hazard_walker(subplan->testexpr, context))
- return true; /* no need to restore safe_param_ids */
- list_free(context->safe_param_ids);
- context->safe_param_ids = save_safe_param_ids;
- /* we must also check args, but no special Param treatment there */
+ return true;
+
if (max_parallel_hazard_walker((Node *) subplan->args, context))
return true;
+
/* don't want to recurse normally, so we're done */
return false;
}
/*
- * We can't pass Params to workers at the moment either, so they are also
- * parallel-restricted, unless they are PARAM_EXTERN Params or are
- * PARAM_EXEC Params listed in safe_param_ids, meaning they could be
- * either generated within workers or can be computed by the leader and
- * then their value can be passed to workers.
+ * We can't pass all types of Params to workers at the moment either.
+ * PARAM_EXTERN Params are always allowed. PARAM_EXEC Params are parallel-
+ * safe when they can be computed by the leader and their value passed to
+ * workers or are generated within a worker. However we don't always know
+ * whether a param will be generated within a worker when we are parsing a
+ * querytree. In that case we leave it to the consumer to verify that the
+ * current plan level provides these params.
*/
else if (IsA(node, Param))
{
@@ -816,12 +790,12 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
if (param->paramkind == PARAM_EXTERN)
return false;
- if (param->paramkind != PARAM_EXEC ||
- !list_member_int(context->safe_param_ids, param->paramid))
- {
- if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
- return true;
- }
+ if (param->paramkind == PARAM_EXEC)
+ return false;
+
+ if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
+ return true;
+
return false; /* nothing to recurse to */
}
diff --git a/src/test/regress/expected/incremental_sort.out b/src/test/regress/expected/incremental_sort.out
index 0c3433f8e5..2b52b0ca3c 100644
--- a/src/test/regress/expected/incremental_sort.out
+++ b/src/test/regress/expected/incremental_sort.out
@@ -1597,20 +1597,21 @@ explain (costs off) select distinct
unique1,
(select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
from tenk1 t, generate_series(1, 1000);
- QUERY PLAN
----------------------------------------------------------------------------------
+ QUERY PLAN
+---------------------------------------------------------------------------------------
Unique
- -> Sort
- Sort Key: t.unique1, ((SubPlan 1))
- -> Gather
- Workers Planned: 2
- -> Nested Loop
- -> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
- -> Function Scan on generate_series
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on tenk1
- Index Cond: (unique1 = t.unique1)
-(11 rows)
+ -> Gather Merge
+ Workers Planned: 2
+ -> Unique
+ -> Sort
+ Sort Key: t.unique1, ((SubPlan 1))
+ -> Nested Loop
+ -> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
+ -> Function Scan on generate_series
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on tenk1
+ Index Cond: (unique1 = t.unique1)
+(12 rows)
explain (costs off) select
unique1,
@@ -1619,16 +1620,16 @@ from tenk1 t, generate_series(1, 1000)
order by 1, 2;
QUERY PLAN
---------------------------------------------------------------------------
- Sort
- Sort Key: t.unique1, ((SubPlan 1))
- -> Gather
- Workers Planned: 2
+ Gather Merge
+ Workers Planned: 2
+ -> Sort
+ Sort Key: t.unique1, ((SubPlan 1))
-> Nested Loop
-> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
-> Function Scan on generate_series
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on tenk1
- Index Cond: (unique1 = t.unique1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on tenk1
+ Index Cond: (unique1 = t.unique1)
(10 rows)
-- Parallel sort but with expression not available until the upper rel.
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index 7555764c77..5c45f9c0a5 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -1284,60 +1284,64 @@ EXPLAIN (VERBOSE, COSTS OFF) SELECT * FROM part p(x) ORDER BY x;
--
-- pruning won't work for mc3p, because some keys are Params
explain (costs off) select * from mc2p t1, lateral (select count(*) from mc3p t2 where t2.a = t1.b and abs(t2.b) = 1 and t2.c = 1) s where t1.a = 1;
- QUERY PLAN
------------------------------------------------------------------------
- Nested Loop
- -> Append
- -> Seq Scan on mc2p1 t1_1
- Filter: (a = 1)
- -> Seq Scan on mc2p2 t1_2
- Filter: (a = 1)
- -> Seq Scan on mc2p_default t1_3
- Filter: (a = 1)
- -> Aggregate
- -> Append
- -> Seq Scan on mc3p0 t2_1
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p1 t2_2
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p2 t2_3
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p3 t2_4
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p4 t2_5
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p5 t2_6
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p6 t2_7
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p7 t2_8
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p_default t2_9
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
-(28 rows)
+ QUERY PLAN
+-----------------------------------------------------------------------------
+ Gather
+ Workers Planned: 2
+ -> Nested Loop
+ -> Parallel Append
+ -> Parallel Seq Scan on mc2p1 t1_1
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p2 t1_2
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p_default t1_3
+ Filter: (a = 1)
+ -> Aggregate
+ -> Append
+ -> Seq Scan on mc3p0 t2_1
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p1 t2_2
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p2 t2_3
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p3 t2_4
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p4 t2_5
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p5 t2_6
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p6 t2_7
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p7 t2_8
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p_default t2_9
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+(30 rows)
-- pruning should work fine, because values for a prefix of keys (a, b) are
-- available
explain (costs off) select * from mc2p t1, lateral (select count(*) from mc3p t2 where t2.c = t1.b and abs(t2.b) = 1 and t2.a = 1) s where t1.a = 1;
- QUERY PLAN
------------------------------------------------------------------------
- Nested Loop
- -> Append
- -> Seq Scan on mc2p1 t1_1
- Filter: (a = 1)
- -> Seq Scan on mc2p2 t1_2
- Filter: (a = 1)
- -> Seq Scan on mc2p_default t1_3
- Filter: (a = 1)
- -> Aggregate
- -> Append
- -> Seq Scan on mc3p0 t2_1
- Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p1 t2_2
- Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p_default t2_3
- Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
-(16 rows)
+ QUERY PLAN
+-----------------------------------------------------------------------------
+ Gather
+ Workers Planned: 2
+ -> Nested Loop
+ -> Parallel Append
+ -> Parallel Seq Scan on mc2p1 t1_1
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p2 t1_2
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p_default t1_3
+ Filter: (a = 1)
+ -> Aggregate
+ -> Append
+ -> Seq Scan on mc3p0 t2_1
+ Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p1 t2_2
+ Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p_default t2_3
+ Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
+(18 rows)
-- also here, because values for all keys are provided
explain (costs off) select * from mc2p t1, lateral (select count(*) from mc3p t2 where t2.a = 1 and abs(t2.b) = 1 and t2.c = 1) s where t1.a = 1;
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 9b4d7dd44a..01443e2ffb 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -137,8 +137,8 @@ create table part_pa_test_p2 partition of part_pa_test for values from (0) to (m
explain (costs off)
select (select max((select pa1.b from part_pa_test pa1 where pa1.a = pa2.a)))
from part_pa_test pa2;
- QUERY PLAN
---------------------------------------------------------------
+ QUERY PLAN
+----------------------------------------------------------------
Aggregate
-> Gather
Workers Planned: 3
@@ -148,12 +148,14 @@ explain (costs off)
SubPlan 2
-> Result
SubPlan 1
- -> Append
- -> Seq Scan on part_pa_test_p1 pa1_1
- Filter: (a = pa2.a)
- -> Seq Scan on part_pa_test_p2 pa1_2
- Filter: (a = pa2.a)
-(14 rows)
+ -> Gather
+ Workers Planned: 3
+ -> Parallel Append
+ -> Parallel Seq Scan on part_pa_test_p1 pa1_1
+ Filter: (a = pa2.a)
+ -> Parallel Seq Scan on part_pa_test_p2 pa1_2
+ Filter: (a = pa2.a)
+(16 rows)
drop table part_pa_test;
-- test with leader participation disabled
@@ -320,19 +322,19 @@ explain (costs off, verbose) select
QUERY PLAN
----------------------------------------------------------------------------
Gather
- Output: (SubPlan 1)
+ Output: ((SubPlan 1))
Workers Planned: 4
-> Nested Loop
- Output: t.unique1
+ Output: (SubPlan 1)
-> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
Output: t.unique1
-> Function Scan on pg_catalog.generate_series
Output: generate_series.generate_series
Function Call: generate_series(1, 10)
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(14 rows)
explain (costs off, verbose) select
@@ -341,63 +343,69 @@ explain (costs off, verbose) select
QUERY PLAN
----------------------------------------------------------------------
Gather
- Output: (SubPlan 1)
+ Output: ((SubPlan 1))
Workers Planned: 4
-> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
- Output: t.unique1
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ Output: (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(9 rows)
explain (costs off, verbose) select
(select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
from tenk1 t
limit 1;
- QUERY PLAN
--------------------------------------------------------------------
+ QUERY PLAN
+----------------------------------------------------------------------------
Limit
Output: ((SubPlan 1))
- -> Seq Scan on public.tenk1 t
- Output: (SubPlan 1)
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
-(8 rows)
+ -> Gather
+ Output: ((SubPlan 1))
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(11 rows)
explain (costs off, verbose) select t.unique1
from tenk1 t
where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
- QUERY PLAN
--------------------------------------------------------------
- Seq Scan on public.tenk1 t
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
Output: t.unique1
- Filter: (t.unique1 = (SubPlan 1))
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
-(7 rows)
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ Filter: (t.unique1 = (SubPlan 1))
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(10 rows)
explain (costs off, verbose) select *
from tenk1 t
order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
- QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
- Sort
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Gather Merge
Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
- Sort Key: ((SubPlan 1))
- -> Gather
- Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, (SubPlan 1)
- Workers Planned: 4
+ Workers Planned: 4
+ -> Sort
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
+ Sort Key: ((SubPlan 1))
-> Parallel Seq Scan on public.tenk1 t
- Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(12 rows)
-- test subplan in join/lateral join
@@ -409,14 +417,14 @@ explain (costs off, verbose, timing off) select t.unique1, l.*
QUERY PLAN
----------------------------------------------------------------------
Gather
- Output: t.unique1, (SubPlan 1)
+ Output: t.unique1, ((SubPlan 1))
Workers Planned: 4
-> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
- Output: t.unique1
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ Output: t.unique1, (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(9 rows)
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
@@ -1322,8 +1330,10 @@ SELECT 1 FROM tenk1_vw_sec
-> Parallel Index Only Scan using tenk1_unique1 on tenk1
SubPlan 1
-> Aggregate
- -> Seq Scan on int4_tbl
- Filter: (f1 < tenk1_vw_sec.unique1)
-(9 rows)
+ -> Gather
+ Workers Planned: 1
+ -> Parallel Seq Scan on int4_tbl
+ Filter: (f1 < tenk1_vw_sec.unique1)
+(11 rows)
rollback;
--
2.32.1 (Apple Git-133)
Re: Parallelize correlated subqueries that execute within each worker
James Coleman <jtc331@gmail.com> wrote:
On Sat, Jan 21, 2023 at 10:07 PM James Coleman <jtc331@gmail.com> wrote:
...
While working through Tomas's comment about a conditional in the
max_parallel_hazard_waker being guaranteed true I realized that in the
current version of the patch the safe_param_ids tracking in
is_parallel_safe isn't actually needed any longer. That seemed
suspicious, and so I started digging, and I found out that in the
current approach all of the tests pass with only the changes in
clauses.c. I don't believe that the other changes aren't needed;
rather I believe there isn't yet a test case exercising them, but I
realize that means I can't prove they're needed. I spent some time
poking at this, but at least with my current level of imagination I
haven't stumbled across a query that would exercise these checks.I played with this a good bit more yesterday, I'm now a good bit more
confident this is correct. I've cleaned up the patch; see attached for
v7.Here's some of my thought process:
The comments in src/include/nodes/pathnodes.h:2953 tell us that
PARAM_EXEC params are used to pass values around from one plan node to
another in the following ways:
1. Values down into subqueries (for outer references in subqueries)
2. Up out of subqueries (for the results of a subplan)
3. From a NestLoop plan node into its inner relation (when the inner
scan is parameterized with values from the outer relation)Case (2) is already known to be safe (we currently add these params to
safe_param_ids in max_parallel_hazard_walker when we encounter a
SubPlan node).I also believe case (3) is already handled. We don't build partial
paths for joins when joinrel->lateral_relids is non-empty, and join
order calculations already require that parameterization here go the
correct way (i.e., inner depends on outer rather than the other way
around).That leaves us with only case (1) to consider in this patch. Another
way of saying this is that this is really the only thing the
safe_param_ids tracking is guarding against. For params passed down
into subqueries we can further distinguish between init plans and
"regular" subplans. We already know that params from init plans are
safe (at the right level). So we're concerned here with a way to know
if the params passed to subplans are safe. We already track required
rels in ParamPathInfo, so it's fairly simple to do this test.Which this patch we do in fact now see (as expected) rels with
non-empty lateral_relids showing up in generate_[useful_]gather_paths.
And the partial paths can now have non-empty required outer rels. I'm
not able to come up with a plan that would actually be caught by those
checks; I theorize that because of the few places we actually call
generate_[useful_]gather_paths we are in practice already excluding
those, but for now I've left these as a conditional rather than an
assertion because it seems like the kind of guard we'd want to ensure
those methods are safe.
Maybe we can later (in separate patches) relax the restrictions imposed on
partial path creation a little bit, so that more parameterized partial paths
are created.
One particular case that should be rejected by your checks is a partial index
path, which can be parameterized, but I couldn't construct a query that makes
your checks fire. Maybe the reason is that a parameterized index path is
mostly used on the inner side of a NL join, however no partial path can be
used there. (The problem is that each worker evaluating the NL join would only
see a subset of the inner relation, which whould lead to incorrect results.)
So I'd also choose conditions rather than assert statements.
Following are my (minor) findings:
In generate_gather_paths() you added this test
/*
* Delay gather path creation until the level in the join tree where all
* params used in a worker are generated within that worker.
*/
if (!bms_is_subset(required_outer, rel->relids))
return;
but I'm not sure if required_outer can contain anything of rel->relids. How
about using bms_is_empty(required) outer, or even this?
if (required_outer)
return;
Similarly,
/* We can't pass params to workers. */
if (!bms_is_subset(PATH_REQ_OUTER(cheapest_partial_path), rel->relids))
might look like
if (!bms_is_empty(PATH_REQ_OUTER(cheapest_partial_path)))
or
if (PATH_REQ_OUTER(cheapest_partial_path))
In particular, build_index_paths() does the following when setting
outer_relids (which eventually becomes (path->param_info->ppi_req_outer):
/* Enforce convention that outer_relids is exactly NULL if empty */
if (bms_is_empty(outer_relids))
outer_relids = NULL;
Another question is whether in this call
simple_gather_path = (Path *)
create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
required_outer, rowsp);
required_outer should be passed to create_gather_path(). Shouldn't it rather
be PATH_REQ_OUTER(cheapest_partial_path) that you test just above? Again,
build_index_paths() initializes outer_relids this way
outer_relids = bms_copy(rel->lateral_relids);
but then it may add some more relations:
/* OK to include this clause */
index_clauses = lappend(index_clauses, iclause);
outer_relids = bms_add_members(outer_relids,
rinfo->clause_relids);
So I think that PATH_REQ_OUTER(cheapest_partial_path) in
generate_gather_paths() can eventually contain more relations than
required_outer, and therefore it's safer to check the first.
Similar comments might apply to generate_useful_gather_paths(). Here I also
suggest to move this test
/* We can't pass params to workers. */
if (!bms_is_subset(PATH_REQ_OUTER(subpath), rel->relids))
continue;
to the top of the loop because it's relatively cheap.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
Re: Parallelize correlated subqueries that execute within each worker
On Mon, Feb 6, 2023 at 11:39 AM Antonin Houska <ah@cybertec.at> wrote:
James Coleman <jtc331@gmail.com> wrote:
Which this patch we do in fact now see (as expected) rels with
non-empty lateral_relids showing up in generate_[useful_]gather_paths.
And the partial paths can now have non-empty required outer rels. I'm
not able to come up with a plan that would actually be caught by those
checks; I theorize that because of the few places we actually call
generate_[useful_]gather_paths we are in practice already excluding
those, but for now I've left these as a conditional rather than an
assertion because it seems like the kind of guard we'd want to ensure
those methods are safe.Maybe we can later (in separate patches) relax the restrictions imposed on
partial path creation a little bit, so that more parameterized partial paths
are created.One particular case that should be rejected by your checks is a partial index
path, which can be parameterized, but I couldn't construct a query that makes
your checks fire. Maybe the reason is that a parameterized index path is
mostly used on the inner side of a NL join, however no partial path can be
used there. (The problem is that each worker evaluating the NL join would only
see a subset of the inner relation, which whould lead to incorrect results.)So I'd also choose conditions rather than assert statements.
Thanks for confirming.
Following are my (minor) findings:
In generate_gather_paths() you added this test
/*
* Delay gather path creation until the level in the join tree where all
* params used in a worker are generated within that worker.
*/
if (!bms_is_subset(required_outer, rel->relids))
return;but I'm not sure if required_outer can contain anything of rel->relids. How
about using bms_is_empty(required) outer, or even this?if (required_outer)
return;Similarly,
/* We can't pass params to workers. */
if (!bms_is_subset(PATH_REQ_OUTER(cheapest_partial_path), rel->relids))might look like
if (!bms_is_empty(PATH_REQ_OUTER(cheapest_partial_path)))
or
if (PATH_REQ_OUTER(cheapest_partial_path))
I'm not sure about this change. Deciding is difficult given the fact
that we don't seem to currently generate these paths, but I don't see
a reason why lateral_relids can't be present on the rel, and if so,
then we need to check to see if they're a subset of relids we can
satisfy rather than checking that they don't exist.
In particular, build_index_paths() does the following when setting
outer_relids (which eventually becomes (path->param_info->ppi_req_outer):/* Enforce convention that outer_relids is exactly NULL if empty */
if (bms_is_empty(outer_relids))
outer_relids = NULL;Another question is whether in this call
simple_gather_path = (Path *)
create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
required_outer, rowsp);required_outer should be passed to create_gather_path(). Shouldn't it rather
be PATH_REQ_OUTER(cheapest_partial_path) that you test just above? Again,
build_index_paths() initializes outer_relids this wayouter_relids = bms_copy(rel->lateral_relids);
but then it may add some more relations:
/* OK to include this clause */
index_clauses = lappend(index_clauses, iclause);
outer_relids = bms_add_members(outer_relids,
rinfo->clause_relids);So I think that PATH_REQ_OUTER(cheapest_partial_path) in
generate_gather_paths() can eventually contain more relations than
required_outer, and therefore it's safer to check the first.
Yes, this is a good catch. Originally I didn't know about
PATH_REQ_OUTER, and I'd missed using it in these places.
Similar comments might apply to generate_useful_gather_paths(). Here I also
suggest to move this test/* We can't pass params to workers. */
if (!bms_is_subset(PATH_REQ_OUTER(subpath), rel->relids))
continue;to the top of the loop because it's relatively cheap.
Moved.
Attached is v9.
James Coleman
Attachments:
v9-0001-Add-tests-before-change.patchapplication/octet-stream; name=v9-0001-Add-tests-before-change.patchDownload
From d7a74b2f26edaf36d5098b4b45e19c437a4ebbdf Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Mon, 26 Sep 2022 20:30:23 -0400
Subject: [PATCH v9 1/2] Add tests before change
---
src/test/regress/expected/select_parallel.out | 108 ++++++++++++++++++
src/test/regress/sql/select_parallel.sql | 25 ++++
2 files changed, 133 insertions(+)
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 91f74fe47a..9b4d7dd44a 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -311,6 +311,114 @@ select count(*) from tenk1 where (two, four) not in
10000
(1 row)
+-- test parallel plans for queries containing correlated subplans
+-- where the subplan only needs params available from the current
+-- worker's scan.
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t, generate_series(1, 10);
+ QUERY PLAN
+----------------------------------------------------------------------------
+ Gather
+ Output: (SubPlan 1)
+ Workers Planned: 4
+ -> Nested Loop
+ Output: t.unique1
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ -> Function Scan on pg_catalog.generate_series
+ Output: generate_series.generate_series
+ Function Call: generate_series(1, 10)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(14 rows)
+
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t;
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: (SubPlan 1)
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(9 rows)
+
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t
+ limit 1;
+ QUERY PLAN
+-------------------------------------------------------------------
+ Limit
+ Output: ((SubPlan 1))
+ -> Seq Scan on public.tenk1 t
+ Output: (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(8 rows)
+
+explain (costs off, verbose) select t.unique1
+ from tenk1 t
+ where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+ QUERY PLAN
+-------------------------------------------------------------
+ Seq Scan on public.tenk1 t
+ Output: t.unique1
+ Filter: (t.unique1 = (SubPlan 1))
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(7 rows)
+
+explain (costs off, verbose) select *
+ from tenk1 t
+ order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Sort
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
+ Sort Key: ((SubPlan 1))
+ -> Gather
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, (SubPlan 1)
+ Workers Planned: 4
+ -> Parallel Seq Scan on public.tenk1 t
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(12 rows)
+
+-- test subplan in join/lateral join
+explain (costs off, verbose, timing off) select t.unique1, l.*
+ from tenk1 t
+ join lateral (
+ select (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1 offset 0)
+ ) l on true;
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: t.unique1, (SubPlan 1)
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(9 rows)
+
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
explain (costs off)
select * from tenk1 where (unique1 + random())::integer not in
diff --git a/src/test/regress/sql/select_parallel.sql b/src/test/regress/sql/select_parallel.sql
index 62fb68c7a0..21c2f1c742 100644
--- a/src/test/regress/sql/select_parallel.sql
+++ b/src/test/regress/sql/select_parallel.sql
@@ -111,6 +111,31 @@ explain (costs off)
(select hundred, thousand from tenk2 where thousand > 100);
select count(*) from tenk1 where (two, four) not in
(select hundred, thousand from tenk2 where thousand > 100);
+-- test parallel plans for queries containing correlated subplans
+-- where the subplan only needs params available from the current
+-- worker's scan.
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t, generate_series(1, 10);
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t;
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t
+ limit 1;
+explain (costs off, verbose) select t.unique1
+ from tenk1 t
+ where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+explain (costs off, verbose) select *
+ from tenk1 t
+ order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+-- test subplan in join/lateral join
+explain (costs off, verbose, timing off) select t.unique1, l.*
+ from tenk1 t
+ join lateral (
+ select (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1 offset 0)
+ ) l on true;
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
explain (costs off)
select * from tenk1 where (unique1 + random())::integer not in
--
2.32.1 (Apple Git-133)
v9-0002-Parallelize-correlated-subqueries.patchapplication/octet-stream; name=v9-0002-Parallelize-correlated-subqueries.patchDownload
From 8b36aa8fa43bfe6a32a1dbce0623ad9243f0b303 Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Wed, 18 Jan 2023 20:43:26 -0500
Subject: [PATCH v9 2/2] Parallelize correlated subqueries
When params are provided at the current query level (i.e., are generated
within a single worker and not shared across workers) we can safely
execute these in parallel.
Alternative approach using just relids subset check
---
doc/src/sgml/parallel.sgml | 3 +-
src/backend/optimizer/path/allpaths.c | 46 +++++--
src/backend/optimizer/path/joinpath.c | 10 +-
src/backend/optimizer/plan/planner.c | 8 ++
src/backend/optimizer/util/clauses.c | 74 ++++------
.../regress/expected/incremental_sort.out | 41 +++---
src/test/regress/expected/partition_prune.out | 104 +++++++-------
src/test/regress/expected/select_parallel.out | 128 ++++++++++--------
8 files changed, 222 insertions(+), 192 deletions(-)
diff --git a/doc/src/sgml/parallel.sgml b/doc/src/sgml/parallel.sgml
index 5acc9537d6..fd32572ec8 100644
--- a/doc/src/sgml/parallel.sgml
+++ b/doc/src/sgml/parallel.sgml
@@ -518,7 +518,8 @@ EXPLAIN SELECT * FROM pgbench_accounts WHERE filler LIKE '%x%';
<listitem>
<para>
- Plan nodes that reference a correlated <literal>SubPlan</literal>.
+ Plan nodes that reference a correlated <literal>SubPlan</literal> where
+ the result is shared between workers.
</para>
</listitem>
</itemizedlist>
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index c2fc568dc8..a579d4c092 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -3018,11 +3018,19 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
ListCell *lc;
double rows;
double *rowsp = NULL;
+ Relids required_outer = rel->lateral_relids;
/* If there are no partial paths, there's nothing to do here. */
if (rel->partial_pathlist == NIL)
return;
+ /*
+ * Delay gather path creation until the level in the join tree where all
+ * params used in a worker are generated within that worker.
+ */
+ if (!bms_is_subset(required_outer, rel->relids))
+ return;
+
/* Should we override the rel's rowcount estimate? */
if (override_rows)
rowsp = &rows;
@@ -3033,12 +3041,17 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
* of partial_pathlist because of the way add_partial_path works.
*/
cheapest_partial_path = linitial(rel->partial_pathlist);
- rows =
- cheapest_partial_path->rows * cheapest_partial_path->parallel_workers;
- simple_gather_path = (Path *)
- create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
- NULL, rowsp);
- add_path(rel, simple_gather_path);
+
+ /* We can't pass params to workers. */
+ if (bms_is_subset(PATH_REQ_OUTER(cheapest_partial_path), rel->relids))
+ {
+ rows =
+ cheapest_partial_path->rows * cheapest_partial_path->parallel_workers;
+ simple_gather_path = (Path *)
+ create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
+ PATH_REQ_OUTER(cheapest_partial_path), rowsp);
+ add_path(rel, simple_gather_path);
+ }
/*
* For each useful ordering, we can consider an order-preserving Gather
@@ -3052,9 +3065,14 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
if (subpath->pathkeys == NIL)
continue;
+ /* We can't pass params to workers. */
+ if (!bms_is_subset(PATH_REQ_OUTER(subpath), rel->relids))
+ continue;
+
rows = subpath->rows * subpath->parallel_workers;
path = create_gather_merge_path(root, rel, subpath, rel->reltarget,
- subpath->pathkeys, NULL, rowsp);
+ subpath->pathkeys,
+ PATH_REQ_OUTER(subpath), rowsp);
add_path(rel, &path->path);
}
}
@@ -3156,11 +3174,19 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
double *rowsp = NULL;
List *useful_pathkeys_list = NIL;
Path *cheapest_partial_path = NULL;
+ Relids required_outer = rel->lateral_relids;
/* If there are no partial paths, there's nothing to do here. */
if (rel->partial_pathlist == NIL)
return;
+ /*
+ * Delay gather path creation until the level in the join tree where all
+ * params used in a worker are generated within that worker.
+ */
+ if (!bms_is_subset(required_outer, rel->relids))
+ return;
+
/* Should we override the rel's rowcount estimate? */
if (override_rows)
rowsp = &rows;
@@ -3190,6 +3216,10 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
Path *subpath = (Path *) lfirst(lc2);
GatherMergePath *path;
+ /* We can't pass params to workers. */
+ if (!bms_is_subset(PATH_REQ_OUTER(subpath), rel->relids))
+ continue;
+
is_sorted = pathkeys_count_contained_in(useful_pathkeys,
subpath->pathkeys,
&presorted_keys);
@@ -3249,7 +3279,7 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
subpath,
rel->reltarget,
subpath->pathkeys,
- NULL,
+ PATH_REQ_OUTER(subpath),
rowsp);
add_path(rel, &path->path);
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index d345c0437a..644b90ad1e 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -1791,10 +1791,12 @@ match_unsorted_outer(PlannerInfo *root,
* Consider partial nestloop and mergejoin plan if outerrel has any
* partial path and the joinrel is parallel-safe. However, we can't
* handle JOIN_UNIQUE_OUTER, because the outer path will be partial, and
- * therefore we won't be able to properly guarantee uniqueness. Nor can
- * we handle joins needing lateral rels, since partial paths must not be
- * parameterized. Similarly, we can't handle JOIN_FULL and JOIN_RIGHT,
- * because they can produce false null extended rows.
+ * therefore we won't be able to properly guarantee uniqueness. Similarly,
+ * we can't handle JOIN_FULL and JOIN_RIGHT, because they can produce false
+ * null extended rows.
+ *
+ * While partial paths may now be parameterized so long as all of the params
+ * can be generated wholly within a worker we punt on supporting that here.
*/
if (joinrel->consider_parallel &&
save_jointype != JOIN_UNIQUE_OUTER &&
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 05f44faf6e..fcde41d9f3 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -7344,11 +7344,16 @@ gather_grouping_paths(PlannerInfo *root, RelOptInfo *rel)
ListCell *lc;
Path *cheapest_partial_path;
+ /* By grouping time we shouldn't have any lateral dependencies. */
+ Assert(rel->lateral_relids == NULL);
+
/* Try Gather for unordered paths and Gather Merge for ordered ones. */
generate_useful_gather_paths(root, rel, true);
/* Try cheapest partial path + explicit Sort + Gather Merge. */
cheapest_partial_path = linitial(rel->partial_pathlist);
+ /* By grouping time we shouldn't have any lateral dependencies. */
+ Assert(PATH_REQ_OUTER(cheapest_partial_path) == NULL);
if (!pathkeys_contained_in(root->group_pathkeys,
cheapest_partial_path->pathkeys))
{
@@ -7400,6 +7405,9 @@ gather_grouping_paths(PlannerInfo *root, RelOptInfo *rel)
if (presorted_keys == 0)
continue;
+ /* By grouping time we shouldn't have any lateral dependencies. */
+ Assert(PATH_REQ_OUTER(path) == NULL);
+
path = (Path *) create_incremental_sort_path(root,
rel,
path,
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index aa584848cf..4a3a0489a8 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -89,7 +89,6 @@ typedef struct
{
char max_hazard; /* worst proparallel hazard found so far */
char max_interesting; /* worst proparallel hazard of interest */
- List *safe_param_ids; /* PARAM_EXEC Param IDs to treat as safe */
} max_parallel_hazard_context;
static bool contain_agg_clause_walker(Node *node, void *context);
@@ -618,7 +617,6 @@ max_parallel_hazard(Query *parse)
context.max_hazard = PROPARALLEL_SAFE;
context.max_interesting = PROPARALLEL_UNSAFE;
- context.safe_param_ids = NIL;
(void) max_parallel_hazard_walker((Node *) parse, &context);
return context.max_hazard;
}
@@ -629,43 +627,24 @@ max_parallel_hazard(Query *parse)
*
* root->glob->maxParallelHazard must previously have been set to the
* result of max_parallel_hazard() on the whole query.
+ *
+ * The caller is responsible for verifying that PARAM_EXEC Params are generated
+ * at the current plan level.
*/
bool
is_parallel_safe(PlannerInfo *root, Node *node)
{
max_parallel_hazard_context context;
- PlannerInfo *proot;
- ListCell *l;
/*
- * Even if the original querytree contained nothing unsafe, we need to
- * search the expression if we have generated any PARAM_EXEC Params while
- * planning, because those are parallel-restricted and there might be one
- * in this expression. But otherwise we don't need to look.
+ * If we've already checked the querytree don't burn cycles doing it again.
*/
- if (root->glob->maxParallelHazard == PROPARALLEL_SAFE &&
- root->glob->paramExecTypes == NIL)
+ if (root->glob->maxParallelHazard == PROPARALLEL_SAFE)
return true;
+
/* Else use max_parallel_hazard's search logic, but stop on RESTRICTED */
context.max_hazard = PROPARALLEL_SAFE;
context.max_interesting = PROPARALLEL_RESTRICTED;
- context.safe_param_ids = NIL;
-
- /*
- * The params that refer to the same or parent query level are considered
- * parallel-safe. The idea is that we compute such params at Gather or
- * Gather Merge node and pass their value to workers.
- */
- for (proot = root; proot != NULL; proot = proot->parent_root)
- {
- foreach(l, proot->init_plans)
- {
- SubPlan *initsubplan = (SubPlan *) lfirst(l);
-
- context.safe_param_ids = list_concat(context.safe_param_ids,
- initsubplan->setParam);
- }
- }
return !max_parallel_hazard_walker(node, &context);
}
@@ -775,39 +754,34 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
}
/*
- * Only parallel-safe SubPlans can be sent to workers. Within the
- * testexpr of the SubPlan, Params representing the output columns of the
- * subplan can be treated as parallel-safe, so temporarily add their IDs
- * to the safe_param_ids list while examining the testexpr.
+ * Only parallel-safe SubPlans can be sent to workers.
*/
else if (IsA(node, SubPlan))
{
SubPlan *subplan = (SubPlan *) node;
- List *save_safe_param_ids;
if (!subplan->parallel_safe &&
max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
return true;
- save_safe_param_ids = context->safe_param_ids;
- context->safe_param_ids = list_concat_copy(context->safe_param_ids,
- subplan->paramIds);
+
if (max_parallel_hazard_walker(subplan->testexpr, context))
- return true; /* no need to restore safe_param_ids */
- list_free(context->safe_param_ids);
- context->safe_param_ids = save_safe_param_ids;
- /* we must also check args, but no special Param treatment there */
+ return true;
+
if (max_parallel_hazard_walker((Node *) subplan->args, context))
return true;
+
/* don't want to recurse normally, so we're done */
return false;
}
/*
- * We can't pass Params to workers at the moment either, so they are also
- * parallel-restricted, unless they are PARAM_EXTERN Params or are
- * PARAM_EXEC Params listed in safe_param_ids, meaning they could be
- * either generated within workers or can be computed by the leader and
- * then their value can be passed to workers.
+ * We can't pass all types of Params to workers at the moment either.
+ * PARAM_EXTERN Params are always allowed. PARAM_EXEC Params are parallel-
+ * safe when they can be computed by the leader and their value passed to
+ * workers or are generated within a worker. However we don't always know
+ * whether a param will be generated within a worker when we are parsing a
+ * querytree. In that case we leave it to the consumer to verify that the
+ * current plan level provides these params.
*/
else if (IsA(node, Param))
{
@@ -816,12 +790,12 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
if (param->paramkind == PARAM_EXTERN)
return false;
- if (param->paramkind != PARAM_EXEC ||
- !list_member_int(context->safe_param_ids, param->paramid))
- {
- if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
- return true;
- }
+ if (param->paramkind == PARAM_EXEC)
+ return false;
+
+ if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
+ return true;
+
return false; /* nothing to recurse to */
}
diff --git a/src/test/regress/expected/incremental_sort.out b/src/test/regress/expected/incremental_sort.out
index 0c3433f8e5..2b52b0ca3c 100644
--- a/src/test/regress/expected/incremental_sort.out
+++ b/src/test/regress/expected/incremental_sort.out
@@ -1597,20 +1597,21 @@ explain (costs off) select distinct
unique1,
(select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
from tenk1 t, generate_series(1, 1000);
- QUERY PLAN
----------------------------------------------------------------------------------
+ QUERY PLAN
+---------------------------------------------------------------------------------------
Unique
- -> Sort
- Sort Key: t.unique1, ((SubPlan 1))
- -> Gather
- Workers Planned: 2
- -> Nested Loop
- -> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
- -> Function Scan on generate_series
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on tenk1
- Index Cond: (unique1 = t.unique1)
-(11 rows)
+ -> Gather Merge
+ Workers Planned: 2
+ -> Unique
+ -> Sort
+ Sort Key: t.unique1, ((SubPlan 1))
+ -> Nested Loop
+ -> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
+ -> Function Scan on generate_series
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on tenk1
+ Index Cond: (unique1 = t.unique1)
+(12 rows)
explain (costs off) select
unique1,
@@ -1619,16 +1620,16 @@ from tenk1 t, generate_series(1, 1000)
order by 1, 2;
QUERY PLAN
---------------------------------------------------------------------------
- Sort
- Sort Key: t.unique1, ((SubPlan 1))
- -> Gather
- Workers Planned: 2
+ Gather Merge
+ Workers Planned: 2
+ -> Sort
+ Sort Key: t.unique1, ((SubPlan 1))
-> Nested Loop
-> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
-> Function Scan on generate_series
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on tenk1
- Index Cond: (unique1 = t.unique1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on tenk1
+ Index Cond: (unique1 = t.unique1)
(10 rows)
-- Parallel sort but with expression not available until the upper rel.
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index 7555764c77..5c45f9c0a5 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -1284,60 +1284,64 @@ EXPLAIN (VERBOSE, COSTS OFF) SELECT * FROM part p(x) ORDER BY x;
--
-- pruning won't work for mc3p, because some keys are Params
explain (costs off) select * from mc2p t1, lateral (select count(*) from mc3p t2 where t2.a = t1.b and abs(t2.b) = 1 and t2.c = 1) s where t1.a = 1;
- QUERY PLAN
------------------------------------------------------------------------
- Nested Loop
- -> Append
- -> Seq Scan on mc2p1 t1_1
- Filter: (a = 1)
- -> Seq Scan on mc2p2 t1_2
- Filter: (a = 1)
- -> Seq Scan on mc2p_default t1_3
- Filter: (a = 1)
- -> Aggregate
- -> Append
- -> Seq Scan on mc3p0 t2_1
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p1 t2_2
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p2 t2_3
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p3 t2_4
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p4 t2_5
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p5 t2_6
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p6 t2_7
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p7 t2_8
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p_default t2_9
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
-(28 rows)
+ QUERY PLAN
+-----------------------------------------------------------------------------
+ Gather
+ Workers Planned: 2
+ -> Nested Loop
+ -> Parallel Append
+ -> Parallel Seq Scan on mc2p1 t1_1
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p2 t1_2
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p_default t1_3
+ Filter: (a = 1)
+ -> Aggregate
+ -> Append
+ -> Seq Scan on mc3p0 t2_1
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p1 t2_2
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p2 t2_3
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p3 t2_4
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p4 t2_5
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p5 t2_6
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p6 t2_7
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p7 t2_8
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p_default t2_9
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+(30 rows)
-- pruning should work fine, because values for a prefix of keys (a, b) are
-- available
explain (costs off) select * from mc2p t1, lateral (select count(*) from mc3p t2 where t2.c = t1.b and abs(t2.b) = 1 and t2.a = 1) s where t1.a = 1;
- QUERY PLAN
------------------------------------------------------------------------
- Nested Loop
- -> Append
- -> Seq Scan on mc2p1 t1_1
- Filter: (a = 1)
- -> Seq Scan on mc2p2 t1_2
- Filter: (a = 1)
- -> Seq Scan on mc2p_default t1_3
- Filter: (a = 1)
- -> Aggregate
- -> Append
- -> Seq Scan on mc3p0 t2_1
- Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p1 t2_2
- Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p_default t2_3
- Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
-(16 rows)
+ QUERY PLAN
+-----------------------------------------------------------------------------
+ Gather
+ Workers Planned: 2
+ -> Nested Loop
+ -> Parallel Append
+ -> Parallel Seq Scan on mc2p1 t1_1
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p2 t1_2
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p_default t1_3
+ Filter: (a = 1)
+ -> Aggregate
+ -> Append
+ -> Seq Scan on mc3p0 t2_1
+ Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p1 t2_2
+ Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p_default t2_3
+ Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
+(18 rows)
-- also here, because values for all keys are provided
explain (costs off) select * from mc2p t1, lateral (select count(*) from mc3p t2 where t2.a = 1 and abs(t2.b) = 1 and t2.c = 1) s where t1.a = 1;
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 9b4d7dd44a..01443e2ffb 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -137,8 +137,8 @@ create table part_pa_test_p2 partition of part_pa_test for values from (0) to (m
explain (costs off)
select (select max((select pa1.b from part_pa_test pa1 where pa1.a = pa2.a)))
from part_pa_test pa2;
- QUERY PLAN
---------------------------------------------------------------
+ QUERY PLAN
+----------------------------------------------------------------
Aggregate
-> Gather
Workers Planned: 3
@@ -148,12 +148,14 @@ explain (costs off)
SubPlan 2
-> Result
SubPlan 1
- -> Append
- -> Seq Scan on part_pa_test_p1 pa1_1
- Filter: (a = pa2.a)
- -> Seq Scan on part_pa_test_p2 pa1_2
- Filter: (a = pa2.a)
-(14 rows)
+ -> Gather
+ Workers Planned: 3
+ -> Parallel Append
+ -> Parallel Seq Scan on part_pa_test_p1 pa1_1
+ Filter: (a = pa2.a)
+ -> Parallel Seq Scan on part_pa_test_p2 pa1_2
+ Filter: (a = pa2.a)
+(16 rows)
drop table part_pa_test;
-- test with leader participation disabled
@@ -320,19 +322,19 @@ explain (costs off, verbose) select
QUERY PLAN
----------------------------------------------------------------------------
Gather
- Output: (SubPlan 1)
+ Output: ((SubPlan 1))
Workers Planned: 4
-> Nested Loop
- Output: t.unique1
+ Output: (SubPlan 1)
-> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
Output: t.unique1
-> Function Scan on pg_catalog.generate_series
Output: generate_series.generate_series
Function Call: generate_series(1, 10)
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(14 rows)
explain (costs off, verbose) select
@@ -341,63 +343,69 @@ explain (costs off, verbose) select
QUERY PLAN
----------------------------------------------------------------------
Gather
- Output: (SubPlan 1)
+ Output: ((SubPlan 1))
Workers Planned: 4
-> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
- Output: t.unique1
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ Output: (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(9 rows)
explain (costs off, verbose) select
(select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
from tenk1 t
limit 1;
- QUERY PLAN
--------------------------------------------------------------------
+ QUERY PLAN
+----------------------------------------------------------------------------
Limit
Output: ((SubPlan 1))
- -> Seq Scan on public.tenk1 t
- Output: (SubPlan 1)
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
-(8 rows)
+ -> Gather
+ Output: ((SubPlan 1))
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(11 rows)
explain (costs off, verbose) select t.unique1
from tenk1 t
where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
- QUERY PLAN
--------------------------------------------------------------
- Seq Scan on public.tenk1 t
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
Output: t.unique1
- Filter: (t.unique1 = (SubPlan 1))
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
-(7 rows)
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ Filter: (t.unique1 = (SubPlan 1))
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(10 rows)
explain (costs off, verbose) select *
from tenk1 t
order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
- QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
- Sort
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Gather Merge
Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
- Sort Key: ((SubPlan 1))
- -> Gather
- Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, (SubPlan 1)
- Workers Planned: 4
+ Workers Planned: 4
+ -> Sort
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
+ Sort Key: ((SubPlan 1))
-> Parallel Seq Scan on public.tenk1 t
- Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(12 rows)
-- test subplan in join/lateral join
@@ -409,14 +417,14 @@ explain (costs off, verbose, timing off) select t.unique1, l.*
QUERY PLAN
----------------------------------------------------------------------
Gather
- Output: t.unique1, (SubPlan 1)
+ Output: t.unique1, ((SubPlan 1))
Workers Planned: 4
-> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
- Output: t.unique1
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ Output: t.unique1, (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(9 rows)
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
@@ -1322,8 +1330,10 @@ SELECT 1 FROM tenk1_vw_sec
-> Parallel Index Only Scan using tenk1_unique1 on tenk1
SubPlan 1
-> Aggregate
- -> Seq Scan on int4_tbl
- Filter: (f1 < tenk1_vw_sec.unique1)
-(9 rows)
+ -> Gather
+ Workers Planned: 1
+ -> Parallel Seq Scan on int4_tbl
+ Filter: (f1 < tenk1_vw_sec.unique1)
+(11 rows)
rollback;
--
2.32.1 (Apple Git-133)
Re: Parallelize correlated subqueries that execute within each worker
James Coleman <jtc331@gmail.com> wrote:
On Mon, Feb 6, 2023 at 11:39 AM Antonin Houska <ah@cybertec.at> wrote:
Attached is v9.
ok, I've changed the status to RfC
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
Re: Parallelize correlated subqueries that execute within each worker
On Mon, Jan 23, 2023 at 10:00 PM James Coleman <jtc331@gmail.com> wrote:
Which this patch we do in fact now see (as expected) rels with
non-empty lateral_relids showing up in generate_[useful_]gather_paths.
And the partial paths can now have non-empty required outer rels. I'm
not able to come up with a plan that would actually be caught by those
checks; I theorize that because of the few places we actually call
generate_[useful_]gather_paths we are in practice already excluding
those, but for now I've left these as a conditional rather than an
assertion because it seems like the kind of guard we'd want to ensure
those methods are safe.
I'm trying to understand this part. AFAICS we will not create partial
paths for a rel, base or join, if it has lateral references. So it
seems to me that in generate_[useful_]gather_paths after we've checked
that there are partial paths, the checks for lateral_relids are not
necessary because lateral_relids should always be empty in this case.
Maybe I'm missing something.
And while trying the v9 patch I came across a crash with the query
below.
set min_parallel_table_scan_size to 0;
set parallel_setup_cost to 0;
set parallel_tuple_cost to 0;
explain (costs off)
select * from pg_description t1 where objoid in
(select objoid from pg_description t2 where t2.description =
t1.description);
QUERY PLAN
--------------------------------------------------------
Seq Scan on pg_description t1
Filter: (SubPlan 1)
SubPlan 1
-> Gather
Workers Planned: 2
-> Parallel Seq Scan on pg_description t2
Filter: (description = t1.description)
(7 rows)
select * from pg_description t1 where objoid in
(select objoid from pg_description t2 where t2.description =
t1.description);
WARNING: terminating connection because of crash of another server process
Seems something is wrong when extracting the argument from the Param in
parallel worker.
BTW another rebase is needed as it no longer applies to HEAD.
Thanks
Richard
Re: Parallelize correlated subqueries that execute within each worker
On Tue, Jun 6, 2023 at 4:36 AM Richard Guo <guofenglinux@gmail.com> wrote:
On Mon, Jan 23, 2023 at 10:00 PM James Coleman <jtc331@gmail.com> wrote:
Which this patch we do in fact now see (as expected) rels with
non-empty lateral_relids showing up in generate_[useful_]gather_paths.
And the partial paths can now have non-empty required outer rels. I'm
not able to come up with a plan that would actually be caught by those
checks; I theorize that because of the few places we actually call
generate_[useful_]gather_paths we are in practice already excluding
those, but for now I've left these as a conditional rather than an
assertion because it seems like the kind of guard we'd want to ensure
those methods are safe.I'm trying to understand this part. AFAICS we will not create partial
paths for a rel, base or join, if it has lateral references. So it
seems to me that in generate_[useful_]gather_paths after we've checked
that there are partial paths, the checks for lateral_relids are not
necessary because lateral_relids should always be empty in this case.
Maybe I'm missing something.
At first I was thinking "isn't the point of the patch to generate
partial paths for rels with lateral references" given what I'd written
back in January, but I added "Assert(bms_is_empty(required_outer));"
to both of those functions and the assertion never fails running the
tests (including my newly parallelizable queries). I'm almost positive
I'd checked this back in January (not only had I'd explicitly written
that I'd confirmed we had non-empty lateral_relids there, but also it
was the entire based of the alternate approach to the patch), but...I
can't go back to 5 months ago and remember what I'd done.
Ah! Your comment about "after we've checked that there are partial
paths" triggered a thought. I think originally I'd had the
"bms_is_subset(required_outer, rel->relids)" check first in these
functions. And indeed if I run the tests with that the assertion moved
to above the partial paths check, I get failures in
generate_useful_gather_paths specifically. Mystery solved!
And while trying the v9 patch I came across a crash with the query
below.set min_parallel_table_scan_size to 0;
set parallel_setup_cost to 0;
set parallel_tuple_cost to 0;explain (costs off)
select * from pg_description t1 where objoid in
(select objoid from pg_description t2 where t2.description = t1.description);
QUERY PLAN
--------------------------------------------------------
Seq Scan on pg_description t1
Filter: (SubPlan 1)
SubPlan 1
-> Gather
Workers Planned: 2
-> Parallel Seq Scan on pg_description t2
Filter: (description = t1.description)
(7 rows)select * from pg_description t1 where objoid in
(select objoid from pg_description t2 where t2.description = t1.description);
WARNING: terminating connection because of crash of another server processSeems something is wrong when extracting the argument from the Param in
parallel worker.
With what I'm trying to change I don't think this plan should ever be
generated since it means we'd have to pass a param from the outer seq
scan into the parallel subplan, which we can't do (currently).
I've attached the full backtrace to the email, but as you hinted at
the parallel worker is trying to get the param (in this case
detoasting it), but the param doesn't exist on the worker, so it seg
faults. Looking at this further I think there's an existing test case
that exposes the misplanning here (the one right under the comment
"Parallel Append is not to be used when the subpath depends on the
outer param" in select_parallel.sql), but it doesn't seg fault because
the param is an integer, doesn't need to be detoasted, and therefore
(I think) we skate by (but probably with wrong results in depending on
the dataset).
Interestingly this is one of the existing test queries my original
patch approach didn't change, so this gives me something specific to
work with improving the path. Thanks for testing this and bringing
this to my attention!
BTW are you by any chance testing on ARM macOS? I reproduced the issue
there, but for some reason I did not reproduce the error (and the plan
wasn't parallelized) when I tested this on linux. Perhaps I missed
setting something up; it seems odd.
BTW another rebase is needed as it no longer applies to HEAD.
Apologies; I'd rebased, but hadn't updated the thread. See attached
for an updated series (albeit still broken on your test query).
Thanks,
James
Attachments:
v10-0002-Parallelize-correlated-subqueries.patchapplication/octet-stream; name=v10-0002-Parallelize-correlated-subqueries.patchDownload
From 8cd41d370c96e58f097a651b525fd585408f618a Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Wed, 18 Jan 2023 20:43:26 -0500
Subject: [PATCH v10 2/2] Parallelize correlated subqueries
When params are provided at the current query level (i.e., are generated
within a single worker and not shared across workers) we can safely
execute these in parallel.
Alternative approach using just relids subset check
---
doc/src/sgml/parallel.sgml | 3 +-
src/backend/optimizer/path/allpaths.c | 46 +++++--
src/backend/optimizer/path/joinpath.c | 10 +-
src/backend/optimizer/plan/planner.c | 8 ++
src/backend/optimizer/util/clauses.c | 74 ++++------
.../regress/expected/incremental_sort.out | 41 +++---
src/test/regress/expected/partition_prune.out | 104 +++++++-------
src/test/regress/expected/select_parallel.out | 128 ++++++++++--------
8 files changed, 222 insertions(+), 192 deletions(-)
diff --git a/doc/src/sgml/parallel.sgml b/doc/src/sgml/parallel.sgml
index 5acc9537d6..fd32572ec8 100644
--- a/doc/src/sgml/parallel.sgml
+++ b/doc/src/sgml/parallel.sgml
@@ -518,7 +518,8 @@ EXPLAIN SELECT * FROM pgbench_accounts WHERE filler LIKE '%x%';
<listitem>
<para>
- Plan nodes that reference a correlated <literal>SubPlan</literal>.
+ Plan nodes that reference a correlated <literal>SubPlan</literal> where
+ the result is shared between workers.
</para>
</listitem>
</itemizedlist>
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 9bdc70c702..c9ccb508b4 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -3064,11 +3064,19 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
ListCell *lc;
double rows;
double *rowsp = NULL;
+ Relids required_outer = rel->lateral_relids;
/* If there are no partial paths, there's nothing to do here. */
if (rel->partial_pathlist == NIL)
return;
+ /*
+ * Delay gather path creation until the level in the join tree where all
+ * params used in a worker are generated within that worker.
+ */
+ if (!bms_is_subset(required_outer, rel->relids))
+ return;
+
/* Should we override the rel's rowcount estimate? */
if (override_rows)
rowsp = &rows;
@@ -3079,12 +3087,17 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
* of partial_pathlist because of the way add_partial_path works.
*/
cheapest_partial_path = linitial(rel->partial_pathlist);
- rows =
- cheapest_partial_path->rows * cheapest_partial_path->parallel_workers;
- simple_gather_path = (Path *)
- create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
- NULL, rowsp);
- add_path(rel, simple_gather_path);
+
+ /* We can't pass params to workers. */
+ if (bms_is_subset(PATH_REQ_OUTER(cheapest_partial_path), rel->relids))
+ {
+ rows =
+ cheapest_partial_path->rows * cheapest_partial_path->parallel_workers;
+ simple_gather_path = (Path *)
+ create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
+ PATH_REQ_OUTER(cheapest_partial_path), rowsp);
+ add_path(rel, simple_gather_path);
+ }
/*
* For each useful ordering, we can consider an order-preserving Gather
@@ -3098,9 +3111,14 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
if (subpath->pathkeys == NIL)
continue;
+ /* We can't pass params to workers. */
+ if (!bms_is_subset(PATH_REQ_OUTER(subpath), rel->relids))
+ continue;
+
rows = subpath->rows * subpath->parallel_workers;
path = create_gather_merge_path(root, rel, subpath, rel->reltarget,
- subpath->pathkeys, NULL, rowsp);
+ subpath->pathkeys,
+ PATH_REQ_OUTER(subpath), rowsp);
add_path(rel, &path->path);
}
}
@@ -3202,11 +3220,19 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
double *rowsp = NULL;
List *useful_pathkeys_list = NIL;
Path *cheapest_partial_path = NULL;
+ Relids required_outer = rel->lateral_relids;
/* If there are no partial paths, there's nothing to do here. */
if (rel->partial_pathlist == NIL)
return;
+ /*
+ * Delay gather path creation until the level in the join tree where all
+ * params used in a worker are generated within that worker.
+ */
+ if (!bms_is_subset(required_outer, rel->relids))
+ return;
+
/* Should we override the rel's rowcount estimate? */
if (override_rows)
rowsp = &rows;
@@ -3236,6 +3262,10 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
Path *subpath = (Path *) lfirst(lc2);
GatherMergePath *path;
+ /* We can't pass params to workers. */
+ if (!bms_is_subset(PATH_REQ_OUTER(subpath), rel->relids))
+ continue;
+
is_sorted = pathkeys_count_contained_in(useful_pathkeys,
subpath->pathkeys,
&presorted_keys);
@@ -3295,7 +3325,7 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
subpath,
rel->reltarget,
subpath->pathkeys,
- NULL,
+ PATH_REQ_OUTER(subpath),
rowsp);
add_path(rel, &path->path);
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index cd80e61fd7..66d133d0ea 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -1850,10 +1850,12 @@ match_unsorted_outer(PlannerInfo *root,
* Consider partial nestloop and mergejoin plan if outerrel has any
* partial path and the joinrel is parallel-safe. However, we can't
* handle JOIN_UNIQUE_OUTER, because the outer path will be partial, and
- * therefore we won't be able to properly guarantee uniqueness. Nor can
- * we handle joins needing lateral rels, since partial paths must not be
- * parameterized. Similarly, we can't handle JOIN_FULL, JOIN_RIGHT and
- * JOIN_RIGHT_ANTI, because they can produce false null extended rows.
+ * therefore we won't be able to properly guarantee uniqueness. Similarly,
+ * we can't handle JOIN_FULL JOIN_RIGHT, and JOIN_RIGHT_ANTI, because they
+ * can produce false null extended rows.
+ *
+ * While partial paths may now be parameterized so long as all of the params
+ * can be generated wholly within a worker we punt on supporting that here.
*/
if (joinrel->consider_parallel &&
save_jointype != JOIN_UNIQUE_OUTER &&
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 1e4dd27dba..2ca363df61 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -7353,11 +7353,16 @@ gather_grouping_paths(PlannerInfo *root, RelOptInfo *rel)
ListCell *lc;
Path *cheapest_partial_path;
+ /* By grouping time we shouldn't have any lateral dependencies. */
+ Assert(rel->lateral_relids == NULL);
+
/* Try Gather for unordered paths and Gather Merge for ordered ones. */
generate_useful_gather_paths(root, rel, true);
/* Try cheapest partial path + explicit Sort + Gather Merge. */
cheapest_partial_path = linitial(rel->partial_pathlist);
+ /* By grouping time we shouldn't have any lateral dependencies. */
+ Assert(PATH_REQ_OUTER(cheapest_partial_path) == NULL);
if (!pathkeys_contained_in(root->group_pathkeys,
cheapest_partial_path->pathkeys))
{
@@ -7409,6 +7414,9 @@ gather_grouping_paths(PlannerInfo *root, RelOptInfo *rel)
if (presorted_keys == 0)
continue;
+ /* By grouping time we shouldn't have any lateral dependencies. */
+ Assert(PATH_REQ_OUTER(path) == NULL);
+
path = (Path *) create_incremental_sort_path(root,
rel,
path,
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 7f453b04f8..894b93b8c2 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -91,7 +91,6 @@ typedef struct
{
char max_hazard; /* worst proparallel hazard found so far */
char max_interesting; /* worst proparallel hazard of interest */
- List *safe_param_ids; /* PARAM_EXEC Param IDs to treat as safe */
} max_parallel_hazard_context;
static bool contain_agg_clause_walker(Node *node, void *context);
@@ -654,7 +653,6 @@ max_parallel_hazard(Query *parse)
context.max_hazard = PROPARALLEL_SAFE;
context.max_interesting = PROPARALLEL_UNSAFE;
- context.safe_param_ids = NIL;
(void) max_parallel_hazard_walker((Node *) parse, &context);
return context.max_hazard;
}
@@ -665,43 +663,24 @@ max_parallel_hazard(Query *parse)
*
* root->glob->maxParallelHazard must previously have been set to the
* result of max_parallel_hazard() on the whole query.
+ *
+ * The caller is responsible for verifying that PARAM_EXEC Params are generated
+ * at the current plan level.
*/
bool
is_parallel_safe(PlannerInfo *root, Node *node)
{
max_parallel_hazard_context context;
- PlannerInfo *proot;
- ListCell *l;
/*
- * Even if the original querytree contained nothing unsafe, we need to
- * search the expression if we have generated any PARAM_EXEC Params while
- * planning, because those are parallel-restricted and there might be one
- * in this expression. But otherwise we don't need to look.
+ * If we've already checked the querytree don't burn cycles doing it again.
*/
- if (root->glob->maxParallelHazard == PROPARALLEL_SAFE &&
- root->glob->paramExecTypes == NIL)
+ if (root->glob->maxParallelHazard == PROPARALLEL_SAFE)
return true;
+
/* Else use max_parallel_hazard's search logic, but stop on RESTRICTED */
context.max_hazard = PROPARALLEL_SAFE;
context.max_interesting = PROPARALLEL_RESTRICTED;
- context.safe_param_ids = NIL;
-
- /*
- * The params that refer to the same or parent query level are considered
- * parallel-safe. The idea is that we compute such params at Gather or
- * Gather Merge node and pass their value to workers.
- */
- for (proot = root; proot != NULL; proot = proot->parent_root)
- {
- foreach(l, proot->init_plans)
- {
- SubPlan *initsubplan = (SubPlan *) lfirst(l);
-
- context.safe_param_ids = list_concat(context.safe_param_ids,
- initsubplan->setParam);
- }
- }
return !max_parallel_hazard_walker(node, &context);
}
@@ -811,39 +790,34 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
}
/*
- * Only parallel-safe SubPlans can be sent to workers. Within the
- * testexpr of the SubPlan, Params representing the output columns of the
- * subplan can be treated as parallel-safe, so temporarily add their IDs
- * to the safe_param_ids list while examining the testexpr.
+ * Only parallel-safe SubPlans can be sent to workers.
*/
else if (IsA(node, SubPlan))
{
SubPlan *subplan = (SubPlan *) node;
- List *save_safe_param_ids;
if (!subplan->parallel_safe &&
max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
return true;
- save_safe_param_ids = context->safe_param_ids;
- context->safe_param_ids = list_concat_copy(context->safe_param_ids,
- subplan->paramIds);
+
if (max_parallel_hazard_walker(subplan->testexpr, context))
- return true; /* no need to restore safe_param_ids */
- list_free(context->safe_param_ids);
- context->safe_param_ids = save_safe_param_ids;
- /* we must also check args, but no special Param treatment there */
+ return true;
+
if (max_parallel_hazard_walker((Node *) subplan->args, context))
return true;
+
/* don't want to recurse normally, so we're done */
return false;
}
/*
- * We can't pass Params to workers at the moment either, so they are also
- * parallel-restricted, unless they are PARAM_EXTERN Params or are
- * PARAM_EXEC Params listed in safe_param_ids, meaning they could be
- * either generated within workers or can be computed by the leader and
- * then their value can be passed to workers.
+ * We can't pass all types of Params to workers at the moment either.
+ * PARAM_EXTERN Params are always allowed. PARAM_EXEC Params are parallel-
+ * safe when they can be computed by the leader and their value passed to
+ * workers or are generated within a worker. However we don't always know
+ * whether a param will be generated within a worker when we are parsing a
+ * querytree. In that case we leave it to the consumer to verify that the
+ * current plan level provides these params.
*/
else if (IsA(node, Param))
{
@@ -852,12 +826,12 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
if (param->paramkind == PARAM_EXTERN)
return false;
- if (param->paramkind != PARAM_EXEC ||
- !list_member_int(context->safe_param_ids, param->paramid))
- {
- if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
- return true;
- }
+ if (param->paramkind == PARAM_EXEC)
+ return false;
+
+ if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
+ return true;
+
return false; /* nothing to recurse to */
}
diff --git a/src/test/regress/expected/incremental_sort.out b/src/test/regress/expected/incremental_sort.out
index 0c3433f8e5..2b52b0ca3c 100644
--- a/src/test/regress/expected/incremental_sort.out
+++ b/src/test/regress/expected/incremental_sort.out
@@ -1597,20 +1597,21 @@ explain (costs off) select distinct
unique1,
(select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
from tenk1 t, generate_series(1, 1000);
- QUERY PLAN
----------------------------------------------------------------------------------
+ QUERY PLAN
+---------------------------------------------------------------------------------------
Unique
- -> Sort
- Sort Key: t.unique1, ((SubPlan 1))
- -> Gather
- Workers Planned: 2
- -> Nested Loop
- -> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
- -> Function Scan on generate_series
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on tenk1
- Index Cond: (unique1 = t.unique1)
-(11 rows)
+ -> Gather Merge
+ Workers Planned: 2
+ -> Unique
+ -> Sort
+ Sort Key: t.unique1, ((SubPlan 1))
+ -> Nested Loop
+ -> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
+ -> Function Scan on generate_series
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on tenk1
+ Index Cond: (unique1 = t.unique1)
+(12 rows)
explain (costs off) select
unique1,
@@ -1619,16 +1620,16 @@ from tenk1 t, generate_series(1, 1000)
order by 1, 2;
QUERY PLAN
---------------------------------------------------------------------------
- Sort
- Sort Key: t.unique1, ((SubPlan 1))
- -> Gather
- Workers Planned: 2
+ Gather Merge
+ Workers Planned: 2
+ -> Sort
+ Sort Key: t.unique1, ((SubPlan 1))
-> Nested Loop
-> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
-> Function Scan on generate_series
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on tenk1
- Index Cond: (unique1 = t.unique1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on tenk1
+ Index Cond: (unique1 = t.unique1)
(10 rows)
-- Parallel sort but with expression not available until the upper rel.
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index 2abf759385..4566260fc4 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -1489,60 +1489,64 @@ EXPLAIN (VERBOSE, COSTS OFF) SELECT * FROM part p(x) ORDER BY x;
--
-- pruning won't work for mc3p, because some keys are Params
explain (costs off) select * from mc2p t1, lateral (select count(*) from mc3p t2 where t2.a = t1.b and abs(t2.b) = 1 and t2.c = 1) s where t1.a = 1;
- QUERY PLAN
------------------------------------------------------------------------
- Nested Loop
- -> Append
- -> Seq Scan on mc2p1 t1_1
- Filter: (a = 1)
- -> Seq Scan on mc2p2 t1_2
- Filter: (a = 1)
- -> Seq Scan on mc2p_default t1_3
- Filter: (a = 1)
- -> Aggregate
- -> Append
- -> Seq Scan on mc3p0 t2_1
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p1 t2_2
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p2 t2_3
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p3 t2_4
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p4 t2_5
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p5 t2_6
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p6 t2_7
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p7 t2_8
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p_default t2_9
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
-(28 rows)
+ QUERY PLAN
+-----------------------------------------------------------------------------
+ Gather
+ Workers Planned: 2
+ -> Nested Loop
+ -> Parallel Append
+ -> Parallel Seq Scan on mc2p1 t1_1
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p2 t1_2
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p_default t1_3
+ Filter: (a = 1)
+ -> Aggregate
+ -> Append
+ -> Seq Scan on mc3p0 t2_1
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p1 t2_2
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p2 t2_3
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p3 t2_4
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p4 t2_5
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p5 t2_6
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p6 t2_7
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p7 t2_8
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p_default t2_9
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+(30 rows)
-- pruning should work fine, because values for a prefix of keys (a, b) are
-- available
explain (costs off) select * from mc2p t1, lateral (select count(*) from mc3p t2 where t2.c = t1.b and abs(t2.b) = 1 and t2.a = 1) s where t1.a = 1;
- QUERY PLAN
------------------------------------------------------------------------
- Nested Loop
- -> Append
- -> Seq Scan on mc2p1 t1_1
- Filter: (a = 1)
- -> Seq Scan on mc2p2 t1_2
- Filter: (a = 1)
- -> Seq Scan on mc2p_default t1_3
- Filter: (a = 1)
- -> Aggregate
- -> Append
- -> Seq Scan on mc3p0 t2_1
- Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p1 t2_2
- Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p_default t2_3
- Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
-(16 rows)
+ QUERY PLAN
+-----------------------------------------------------------------------------
+ Gather
+ Workers Planned: 2
+ -> Nested Loop
+ -> Parallel Append
+ -> Parallel Seq Scan on mc2p1 t1_1
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p2 t1_2
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p_default t1_3
+ Filter: (a = 1)
+ -> Aggregate
+ -> Append
+ -> Seq Scan on mc3p0 t2_1
+ Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p1 t2_2
+ Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p_default t2_3
+ Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
+(18 rows)
-- also here, because values for all keys are provided
explain (costs off) select * from mc2p t1, lateral (select count(*) from mc3p t2 where t2.a = 1 and abs(t2.b) = 1 and t2.c = 1) s where t1.a = 1;
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 04810d2756..eb59b9437c 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -137,8 +137,8 @@ create table part_pa_test_p2 partition of part_pa_test for values from (0) to (m
explain (costs off)
select (select max((select pa1.b from part_pa_test pa1 where pa1.a = pa2.a)))
from part_pa_test pa2;
- QUERY PLAN
---------------------------------------------------------------
+ QUERY PLAN
+----------------------------------------------------------------
Aggregate
-> Gather
Workers Planned: 3
@@ -148,12 +148,14 @@ explain (costs off)
SubPlan 2
-> Result
SubPlan 1
- -> Append
- -> Seq Scan on part_pa_test_p1 pa1_1
- Filter: (a = pa2.a)
- -> Seq Scan on part_pa_test_p2 pa1_2
- Filter: (a = pa2.a)
-(14 rows)
+ -> Gather
+ Workers Planned: 3
+ -> Parallel Append
+ -> Parallel Seq Scan on part_pa_test_p1 pa1_1
+ Filter: (a = pa2.a)
+ -> Parallel Seq Scan on part_pa_test_p2 pa1_2
+ Filter: (a = pa2.a)
+(16 rows)
drop table part_pa_test;
-- test with leader participation disabled
@@ -320,19 +322,19 @@ explain (costs off, verbose) select
QUERY PLAN
----------------------------------------------------------------------------
Gather
- Output: (SubPlan 1)
+ Output: ((SubPlan 1))
Workers Planned: 4
-> Nested Loop
- Output: t.unique1
+ Output: (SubPlan 1)
-> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
Output: t.unique1
-> Function Scan on pg_catalog.generate_series
Output: generate_series.generate_series
Function Call: generate_series(1, 10)
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(14 rows)
explain (costs off, verbose) select
@@ -341,63 +343,69 @@ explain (costs off, verbose) select
QUERY PLAN
----------------------------------------------------------------------
Gather
- Output: (SubPlan 1)
+ Output: ((SubPlan 1))
Workers Planned: 4
-> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
- Output: t.unique1
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ Output: (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(9 rows)
explain (costs off, verbose) select
(select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
from tenk1 t
limit 1;
- QUERY PLAN
--------------------------------------------------------------------
+ QUERY PLAN
+----------------------------------------------------------------------------
Limit
Output: ((SubPlan 1))
- -> Seq Scan on public.tenk1 t
- Output: (SubPlan 1)
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
-(8 rows)
+ -> Gather
+ Output: ((SubPlan 1))
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(11 rows)
explain (costs off, verbose) select t.unique1
from tenk1 t
where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
- QUERY PLAN
--------------------------------------------------------------
- Seq Scan on public.tenk1 t
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
Output: t.unique1
- Filter: (t.unique1 = (SubPlan 1))
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
-(7 rows)
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ Filter: (t.unique1 = (SubPlan 1))
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(10 rows)
explain (costs off, verbose) select *
from tenk1 t
order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
- QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
- Sort
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Gather Merge
Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
- Sort Key: ((SubPlan 1))
- -> Gather
- Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, (SubPlan 1)
- Workers Planned: 4
+ Workers Planned: 4
+ -> Sort
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
+ Sort Key: ((SubPlan 1))
-> Parallel Seq Scan on public.tenk1 t
- Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(12 rows)
-- test subplan in join/lateral join
@@ -409,14 +417,14 @@ explain (costs off, verbose, timing off) select t.unique1, l.*
QUERY PLAN
----------------------------------------------------------------------
Gather
- Output: t.unique1, (SubPlan 1)
+ Output: t.unique1, ((SubPlan 1))
Workers Planned: 4
-> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
- Output: t.unique1
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ Output: t.unique1, (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(9 rows)
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
@@ -1322,8 +1330,10 @@ SELECT 1 FROM tenk1_vw_sec
-> Parallel Index Only Scan using tenk1_unique1 on tenk1
SubPlan 1
-> Aggregate
- -> Seq Scan on int4_tbl
- Filter: (f1 < tenk1_vw_sec.unique1)
-(9 rows)
+ -> Gather
+ Workers Planned: 1
+ -> Parallel Seq Scan on int4_tbl
+ Filter: (f1 < tenk1_vw_sec.unique1)
+(11 rows)
rollback;
--
2.39.2 (Apple Git-143)
v10-0001-Add-tests-before-change.patchapplication/octet-stream; name=v10-0001-Add-tests-before-change.patchDownload
From 59c67d670e024173e8417867cd15d7edc257f012 Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Mon, 26 Sep 2022 20:30:23 -0400
Subject: [PATCH v10 1/2] Add tests before change
---
src/test/regress/expected/select_parallel.out | 108 ++++++++++++++++++
src/test/regress/sql/select_parallel.sql | 25 ++++
2 files changed, 133 insertions(+)
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index d88353d496..04810d2756 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -311,6 +311,114 @@ select count(*) from tenk1 where (two, four) not in
10000
(1 row)
+-- test parallel plans for queries containing correlated subplans
+-- where the subplan only needs params available from the current
+-- worker's scan.
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t, generate_series(1, 10);
+ QUERY PLAN
+----------------------------------------------------------------------------
+ Gather
+ Output: (SubPlan 1)
+ Workers Planned: 4
+ -> Nested Loop
+ Output: t.unique1
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ -> Function Scan on pg_catalog.generate_series
+ Output: generate_series.generate_series
+ Function Call: generate_series(1, 10)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(14 rows)
+
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t;
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: (SubPlan 1)
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(9 rows)
+
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t
+ limit 1;
+ QUERY PLAN
+-------------------------------------------------------------------
+ Limit
+ Output: ((SubPlan 1))
+ -> Seq Scan on public.tenk1 t
+ Output: (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(8 rows)
+
+explain (costs off, verbose) select t.unique1
+ from tenk1 t
+ where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+ QUERY PLAN
+-------------------------------------------------------------
+ Seq Scan on public.tenk1 t
+ Output: t.unique1
+ Filter: (t.unique1 = (SubPlan 1))
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(7 rows)
+
+explain (costs off, verbose) select *
+ from tenk1 t
+ order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Sort
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
+ Sort Key: ((SubPlan 1))
+ -> Gather
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, (SubPlan 1)
+ Workers Planned: 4
+ -> Parallel Seq Scan on public.tenk1 t
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(12 rows)
+
+-- test subplan in join/lateral join
+explain (costs off, verbose, timing off) select t.unique1, l.*
+ from tenk1 t
+ join lateral (
+ select (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1 offset 0)
+ ) l on true;
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: t.unique1, (SubPlan 1)
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(9 rows)
+
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
explain (costs off)
select * from tenk1 where (unique1 + random())::integer not in
diff --git a/src/test/regress/sql/select_parallel.sql b/src/test/regress/sql/select_parallel.sql
index 80c914dc02..fd968285c1 100644
--- a/src/test/regress/sql/select_parallel.sql
+++ b/src/test/regress/sql/select_parallel.sql
@@ -111,6 +111,31 @@ explain (costs off)
(select hundred, thousand from tenk2 where thousand > 100);
select count(*) from tenk1 where (two, four) not in
(select hundred, thousand from tenk2 where thousand > 100);
+-- test parallel plans for queries containing correlated subplans
+-- where the subplan only needs params available from the current
+-- worker's scan.
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t, generate_series(1, 10);
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t;
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t
+ limit 1;
+explain (costs off, verbose) select t.unique1
+ from tenk1 t
+ where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+explain (costs off, verbose) select *
+ from tenk1 t
+ order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+-- test subplan in join/lateral join
+explain (costs off, verbose, timing off) select t.unique1, l.*
+ from tenk1 t
+ join lateral (
+ select (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1 offset 0)
+ ) l on true;
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
explain (costs off)
select * from tenk1 where (unique1 + random())::integer not in
--
2.39.2 (Apple Git-143)
Re: Parallelize correlated subqueries that execute within each worker
On Mon, Jun 12, 2023 at 10:23 AM James Coleman <jtc331@gmail.com> wrote:
BTW are you by any chance testing on ARM macOS? I reproduced the issue
there, but for some reason I did not reproduce the error (and the plan
wasn't parallelized) when I tested this on linux. Perhaps I missed
setting something up; it seems odd.
Hmm, that's weird. I was also testing that query on linux. But please
note that several GUC settings are needed to generate parallel plan for
that query.
set min_parallel_table_scan_size to 0;
set parallel_setup_cost to 0;
set parallel_tuple_cost to 0;
Thanks
Richard
Re: Parallelize correlated subqueries that execute within each worker
On Sun, Jun 11, 2023 at 10:23 PM James Coleman <jtc331@gmail.com> wrote:
...
And while trying the v9 patch I came across a crash with the query
below.set min_parallel_table_scan_size to 0;
set parallel_setup_cost to 0;
set parallel_tuple_cost to 0;explain (costs off)
select * from pg_description t1 where objoid in
(select objoid from pg_description t2 where t2.description = t1.description);
QUERY PLAN
--------------------------------------------------------
Seq Scan on pg_description t1
Filter: (SubPlan 1)
SubPlan 1
-> Gather
Workers Planned: 2
-> Parallel Seq Scan on pg_description t2
Filter: (description = t1.description)
(7 rows)select * from pg_description t1 where objoid in
(select objoid from pg_description t2 where t2.description = t1.description);
WARNING: terminating connection because of crash of another server processSeems something is wrong when extracting the argument from the Param in
parallel worker.With what I'm trying to change I don't think this plan should ever be
generated since it means we'd have to pass a param from the outer seq
scan into the parallel subplan, which we can't do (currently).I've attached the full backtrace to the email, but as you hinted at
the parallel worker is trying to get the param (in this case
detoasting it), but the param doesn't exist on the worker, so it seg
faults. Looking at this further I think there's an existing test case
that exposes the misplanning here (the one right under the comment
"Parallel Append is not to be used when the subpath depends on the
outer param" in select_parallel.sql), but it doesn't seg fault because
the param is an integer, doesn't need to be detoasted, and therefore
(I think) we skate by (but probably with wrong results in depending on
the dataset).Interestingly this is one of the existing test queries my original
patch approach didn't change, so this gives me something specific to
work with improving the path. Thanks for testing this and bringing
this to my attention!
Here's what I've found debugging this:
There's only a single gather path ever created when planning this
query, making it easy to know which one is the problem. That gather
path is created with this stacktrace:
frame #0: 0x0000000105291590
postgres`create_gather_path(root=0x000000013081ae78,
rel=0x000000013080c8e8, subpath=0x000000013081c080,
target=0x000000013081c8c0, required_outer=0x0000000000000000,
rows=0x0000000000000000) at pathnode.c:1971:2
frame #1: 0x0000000105208e54
postgres`generate_gather_paths(root=0x000000013081ae78,
rel=0x000000013080c8e8, override_rows=false) at allpaths.c:3097:4
frame #2: 0x00000001052090ec
postgres`generate_useful_gather_paths(root=0x000000013081ae78,
rel=0x000000013080c8e8, override_rows=false) at allpaths.c:3241:2
frame #3: 0x0000000105258754
postgres`apply_scanjoin_target_to_paths(root=0x000000013081ae78,
rel=0x000000013080c8e8, scanjoin_targets=0x000000013081c978,
scanjoin_targets_contain_srfs=0x0000000000000000,
scanjoin_target_parallel_safe=true, tlist_same_exprs=true) at
planner.c:7696:3
frame #4: 0x00000001052533cc
postgres`grouping_planner(root=0x000000013081ae78, tuple_fraction=0.5)
at planner.c:1611:3
frame #5: 0x0000000105251e9c
postgres`subquery_planner(glob=0x00000001308188d8,
parse=0x000000013080caf8, parent_root=0x000000013080cc38,
hasRecursion=false, tuple_fraction=0.5) at planner.c:1062:2
frame #6: 0x000000010526b134
postgres`make_subplan(root=0x000000013080cc38,
orig_subquery=0x000000013080ff58, subLinkType=ANY_SUBLINK,
subLinkId=0, testexpr=0x000000013080d848, isTopQual=true) at
subselect.c:221:12
frame #7: 0x0000000105268b8c
postgres`process_sublinks_mutator(node=0x000000013080d6d8,
context=0x000000016b0998f8) at subselect.c:1950:10
frame #8: 0x0000000105268ad8
postgres`SS_process_sublinks(root=0x000000013080cc38,
expr=0x000000013080d6d8, isQual=true) at subselect.c:1923:9
frame #9: 0x00000001052527b8
postgres`preprocess_expression(root=0x000000013080cc38,
expr=0x000000013080d6d8, kind=0) at planner.c:1169:10
frame #10: 0x0000000105252954
postgres`preprocess_qual_conditions(root=0x000000013080cc38,
jtnode=0x000000013080d108) at planner.c:1214:14
frame #11: 0x0000000105251580
postgres`subquery_planner(glob=0x00000001308188d8,
parse=0x0000000137010d68, parent_root=0x0000000000000000,
hasRecursion=false, tuple_fraction=0) at planner.c:832:2
frame #12: 0x000000010525042c
postgres`standard_planner(parse=0x0000000137010d68,
query_string="explain (costs off)\nselect * from pg_description t1
where objoid in\n (select objoid from pg_description t2 where
t2.description = t1.description);", cursorOptions=2048,
boundParams=0x0000000000000000) at planner.c:411:9
There aren't any lateral markings on the rels. Additionally the
partial path has param_info=null (I found out from Tom in a separate
thread [1] that this is only set for outer relations from the same
query level).
The only param that I could easily find at first was a single param of
type PARAM_EXTERN in root->plan_params in make_subplan().
I spent a lot of time trying to figure out where we could find the
PARAM_EXEC param that's being fed into the subplan, but it doesn't
seem like we have access to any of these things at the point in the
path creation process that it's interesting to us when inserting the
gather nodes.
Given all of that I settled on this approach:
1. Modify is_parallel_safe() to by default ignore PARAM_EXEC params.
2. Add is_parallel_safe_with_params() that checks for the existence of
such params.
3. Store the required params in a bitmapset on each base rel.
4. Union the bitmapset on join rels.
5. Only insert a gather node if that bitmapset is empty.
I have an intuition that there's some spot (e.g. joins) that we should
be removing params from this set (e.g., when we've satisfied them),
but I haven't been able to come up with such a scenario as yet.
The attached v11 fixes the issue you reported.
Thanks,
James Coleman
Attachments:
v11-0001-Add-tests-before-change.patchapplication/octet-stream; name=v11-0001-Add-tests-before-change.patchDownload
From 931e64c7e2e2f819f35b5c7878d1c04ae14d9853 Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Mon, 26 Sep 2022 20:30:23 -0400
Subject: [PATCH v11 1/2] Add tests before change
---
src/test/regress/expected/select_parallel.out | 136 ++++++++++++++++++
src/test/regress/sql/select_parallel.sql | 32 +++++
2 files changed, 168 insertions(+)
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index d88353d496..0ddcf8e0b1 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -311,6 +311,142 @@ select count(*) from tenk1 where (two, four) not in
10000
(1 row)
+-- test parallel plans for queries containing correlated subplans
+-- where the subplan only needs params available from the current
+-- worker's scan.
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t, generate_series(1, 10);
+ QUERY PLAN
+----------------------------------------------------------------------------
+ Gather
+ Output: (SubPlan 1)
+ Workers Planned: 4
+ -> Nested Loop
+ Output: t.unique1
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ -> Function Scan on pg_catalog.generate_series
+ Output: generate_series.generate_series
+ Function Call: generate_series(1, 10)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(14 rows)
+
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t;
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: (SubPlan 1)
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(9 rows)
+
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t
+ limit 1;
+ QUERY PLAN
+-------------------------------------------------------------------
+ Limit
+ Output: ((SubPlan 1))
+ -> Seq Scan on public.tenk1 t
+ Output: (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(8 rows)
+
+explain (costs off, verbose) select t.unique1
+ from tenk1 t
+ where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+ QUERY PLAN
+-------------------------------------------------------------
+ Seq Scan on public.tenk1 t
+ Output: t.unique1
+ Filter: (t.unique1 = (SubPlan 1))
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(7 rows)
+
+explain (costs off, verbose) select *
+ from tenk1 t
+ order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Sort
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
+ Sort Key: ((SubPlan 1))
+ -> Gather
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, (SubPlan 1)
+ Workers Planned: 4
+ -> Parallel Seq Scan on public.tenk1 t
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(12 rows)
+
+-- test subplan in join/lateral join
+explain (costs off, verbose, timing off) select t.unique1, l.*
+ from tenk1 t
+ join lateral (
+ select (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1 offset 0)
+ ) l on true;
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: t.unique1, (SubPlan 1)
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(9 rows)
+
+-- can't put a gather at the top of a subplan that takes a param
+explain (costs off, verbose) select * from tenk1 t where t.two in (
+ select t.two
+ from tenk1
+ join tenk1 t3 on t3.stringu1 = tenk1.stringu1
+ where tenk1.four = t.four
+);
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Seq Scan on public.tenk1 t
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4
+ Filter: (SubPlan 1)
+ SubPlan 1
+ -> Hash Join
+ Output: t.two
+ Hash Cond: (tenk1.stringu1 = t3.stringu1)
+ -> Seq Scan on public.tenk1
+ Output: tenk1.unique1, tenk1.unique2, tenk1.two, tenk1.four, tenk1.ten, tenk1.twenty, tenk1.hundred, tenk1.thousand, tenk1.twothousand, tenk1.fivethous, tenk1.tenthous, tenk1.odd, tenk1.even, tenk1.stringu1, tenk1.stringu2, tenk1.string4
+ Filter: (tenk1.four = t.four)
+ -> Hash
+ Output: t3.stringu1
+ -> Gather
+ Output: t3.stringu1
+ Workers Planned: 4
+ -> Parallel Seq Scan on public.tenk1 t3
+ Output: t3.stringu1
+(17 rows)
+
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
explain (costs off)
select * from tenk1 where (unique1 + random())::integer not in
diff --git a/src/test/regress/sql/select_parallel.sql b/src/test/regress/sql/select_parallel.sql
index 80c914dc02..c6908cb919 100644
--- a/src/test/regress/sql/select_parallel.sql
+++ b/src/test/regress/sql/select_parallel.sql
@@ -111,6 +111,38 @@ explain (costs off)
(select hundred, thousand from tenk2 where thousand > 100);
select count(*) from tenk1 where (two, four) not in
(select hundred, thousand from tenk2 where thousand > 100);
+-- test parallel plans for queries containing correlated subplans
+-- where the subplan only needs params available from the current
+-- worker's scan.
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t, generate_series(1, 10);
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t;
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t
+ limit 1;
+explain (costs off, verbose) select t.unique1
+ from tenk1 t
+ where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+explain (costs off, verbose) select *
+ from tenk1 t
+ order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+-- test subplan in join/lateral join
+explain (costs off, verbose, timing off) select t.unique1, l.*
+ from tenk1 t
+ join lateral (
+ select (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1 offset 0)
+ ) l on true;
+-- can't put a gather at the top of a subplan that takes a param
+explain (costs off, verbose) select * from tenk1 t where t.two in (
+ select t.two
+ from tenk1
+ join tenk1 t3 on t3.stringu1 = tenk1.stringu1
+ where tenk1.four = t.four
+);
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
explain (costs off)
select * from tenk1 where (unique1 + random())::integer not in
--
2.39.2 (Apple Git-143)
v11-0002-Parallelize-correlated-subqueries.patchapplication/octet-stream; name=v11-0002-Parallelize-correlated-subqueries.patchDownload
From 7c19d7f498375cd3ee6b676226c1a9b627298c5f Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Sun, 2 Jul 2023 15:23:49 -0400
Subject: [PATCH v11 2/2] Parallelize correlated subqueries
When params are provided at the current query level (i.e., are generated
within a single worker and not shared across workers) we can safely
execute these in parallel.
We accomplish this by tracking the PARAM_EXEC params that are needed for
a rel to be safely executed in parallel and only inserting a Gather node
if that set is empty.
---
doc/src/sgml/parallel.sgml | 3 +-
src/backend/optimizer/path/allpaths.c | 24 +++-
src/backend/optimizer/util/clauses.c | 73 +++++++++--
src/backend/optimizer/util/relnode.c | 1 +
src/include/nodes/pathnodes.h | 6 +
src/include/optimizer/clauses.h | 1 +
.../regress/expected/incremental_sort.out | 41 +++----
src/test/regress/expected/partition_prune.out | 104 ++++++++--------
src/test/regress/expected/select_parallel.out | 113 ++++++++++--------
9 files changed, 227 insertions(+), 139 deletions(-)
diff --git a/doc/src/sgml/parallel.sgml b/doc/src/sgml/parallel.sgml
index 5acc9537d6..fd32572ec8 100644
--- a/doc/src/sgml/parallel.sgml
+++ b/doc/src/sgml/parallel.sgml
@@ -518,7 +518,8 @@ EXPLAIN SELECT * FROM pgbench_accounts WHERE filler LIKE '%x%';
<listitem>
<para>
- Plan nodes that reference a correlated <literal>SubPlan</literal>.
+ Plan nodes that reference a correlated <literal>SubPlan</literal> where
+ the result is shared between workers.
</para>
</listitem>
</itemizedlist>
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 9bdc70c702..cebad18319 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -632,7 +632,7 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
if (proparallel != PROPARALLEL_SAFE)
return;
- if (!is_parallel_safe(root, (Node *) rte->tablesample->args))
+ if (!is_parallel_safe_with_params(root, (Node *) rte->tablesample->args, &rel->params_req_for_parallel))
return;
}
@@ -698,7 +698,7 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
case RTE_FUNCTION:
/* Check for parallel-restricted functions. */
- if (!is_parallel_safe(root, (Node *) rte->functions))
+ if (!is_parallel_safe_with_params(root, (Node *) rte->functions, &rel->params_req_for_parallel))
return;
break;
@@ -708,7 +708,7 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
case RTE_VALUES:
/* Check for parallel-restricted functions. */
- if (!is_parallel_safe(root, (Node *) rte->values_lists))
+ if (!is_parallel_safe_with_params(root, (Node *) rte->values_lists, &rel->params_req_for_parallel))
return;
break;
@@ -745,14 +745,14 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
* outer join clauses work correctly. It would likely break equivalence
* classes, too.
*/
- if (!is_parallel_safe(root, (Node *) rel->baserestrictinfo))
+ if (!is_parallel_safe_with_params(root, (Node *) rel->baserestrictinfo, &rel->params_req_for_parallel))
return;
/*
* Likewise, if the relation's outputs are not parallel-safe, give up.
* (Usually, they're just Vars, but sometimes they're not.)
*/
- if (!is_parallel_safe(root, (Node *) rel->reltarget->exprs))
+ if (!is_parallel_safe_with_params(root, (Node *) rel->reltarget->exprs, &rel->params_req_for_parallel))
return;
/* We have a winner. */
@@ -3069,6 +3069,13 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
if (rel->partial_pathlist == NIL)
return;
+ /*
+ * Wait to insert Gather nodes until all PARAM_EXEC params are provided
+ * within the current rel since we can't pass them to workers.
+ */
+ if (!bms_is_empty(rel->params_req_for_parallel))
+ return;
+
/* Should we override the rel's rowcount estimate? */
if (override_rows)
rowsp = &rows;
@@ -3207,6 +3214,13 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
if (rel->partial_pathlist == NIL)
return;
+ /*
+ * Wait to insert Gather nodes until all PARAM_EXEC params are provided
+ * within the current rel since we can't pass them to workers.
+ */
+ if (!bms_is_empty(rel->params_req_for_parallel))
+ return;
+
/* Should we override the rel's rowcount estimate? */
if (override_rows)
rowsp = &rows;
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 7f453b04f8..a82a7835c4 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -91,6 +91,8 @@ typedef struct
{
char max_hazard; /* worst proparallel hazard found so far */
char max_interesting; /* worst proparallel hazard of interest */
+ bool check_params;
+ Bitmapset **required_params;
List *safe_param_ids; /* PARAM_EXEC Param IDs to treat as safe */
} max_parallel_hazard_context;
@@ -654,6 +656,7 @@ max_parallel_hazard(Query *parse)
context.max_hazard = PROPARALLEL_SAFE;
context.max_interesting = PROPARALLEL_UNSAFE;
+ context.check_params = true;
context.safe_param_ids = NIL;
(void) max_parallel_hazard_walker((Node *) parse, &context);
return context.max_hazard;
@@ -670,8 +673,6 @@ bool
is_parallel_safe(PlannerInfo *root, Node *node)
{
max_parallel_hazard_context context;
- PlannerInfo *proot;
- ListCell *l;
/*
* Even if the original querytree contained nothing unsafe, we need to
@@ -685,6 +686,43 @@ is_parallel_safe(PlannerInfo *root, Node *node)
/* Else use max_parallel_hazard's search logic, but stop on RESTRICTED */
context.max_hazard = PROPARALLEL_SAFE;
context.max_interesting = PROPARALLEL_RESTRICTED;
+ context.check_params = false;
+ context.required_params = NULL;
+
+ return !max_parallel_hazard_walker(node, &context);
+}
+
+/*
+ * is_parallel_safe_with_params
+ * As above, but additionally tracking what PARAM_EXEC params required to
+ * be provided within a worker for a gather to be inserted at this level
+ * of the query. Those required params are passed to the caller through
+ * the required_params argument.
+ *
+ * Note: required_params is only valid if node is otherwise parallel safe.
+ */
+bool
+is_parallel_safe_with_params(PlannerInfo *root, Node *node, Bitmapset **required_params)
+{
+ max_parallel_hazard_context context;
+ PlannerInfo *proot;
+ ListCell *l;
+
+ /*
+ * Even if the original querytree contained nothing unsafe, we need to
+ * search the expression if we have generated any PARAM_EXEC Params while
+ * planning, because those will have to be provided for the expression to
+ * remain parallel safe and there might be one in this expression. But
+ * otherwise we don't need to look.
+ */
+ if (root->glob->maxParallelHazard == PROPARALLEL_SAFE &&
+ root->glob->paramExecTypes == NIL)
+ return true;
+ /* Else use max_parallel_hazard's search logic, but stop on RESTRICTED */
+ context.max_hazard = PROPARALLEL_SAFE;
+ context.max_interesting = PROPARALLEL_RESTRICTED;
+ context.check_params = false;
+ context.required_params = required_params;
context.safe_param_ids = NIL;
/*
@@ -824,13 +862,22 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
if (!subplan->parallel_safe &&
max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
return true;
- save_safe_param_ids = context->safe_param_ids;
- context->safe_param_ids = list_concat_copy(context->safe_param_ids,
- subplan->paramIds);
+
+ if (context->check_params || context->required_params != NULL)
+ {
+ save_safe_param_ids = context->safe_param_ids;
+ context->safe_param_ids = list_concat_copy(context->safe_param_ids,
+ subplan->paramIds);
+ }
if (max_parallel_hazard_walker(subplan->testexpr, context))
return true; /* no need to restore safe_param_ids */
- list_free(context->safe_param_ids);
- context->safe_param_ids = save_safe_param_ids;
+
+ if (context->check_params || context->required_params != NULL)
+ {
+ list_free(context->safe_param_ids);
+ context->safe_param_ids = save_safe_param_ids;
+ }
+
/* we must also check args, but no special Param treatment there */
if (max_parallel_hazard_walker((Node *) subplan->args, context))
return true;
@@ -849,14 +896,18 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
{
Param *param = (Param *) node;
- if (param->paramkind == PARAM_EXTERN)
+ if (param->paramkind != PARAM_EXEC || !(context->check_params || context->required_params != NULL))
return false;
- if (param->paramkind != PARAM_EXEC ||
- !list_member_int(context->safe_param_ids, param->paramid))
+ if (!list_member_int(context->safe_param_ids, param->paramid))
{
if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
- return true;
+ {
+ if (context->required_params != NULL)
+ *context->required_params = bms_add_member(*context->required_params, param->paramid);
+ if (context->check_params)
+ return true;
+ }
}
return false; /* nothing to recurse to */
}
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 15e3910b79..2e12a61ceb 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -687,6 +687,7 @@ build_join_rel(PlannerInfo *root,
joinrel->consider_startup = (root->tuple_fraction > 0);
joinrel->consider_param_startup = false;
joinrel->consider_parallel = false;
+ joinrel->params_req_for_parallel = bms_union(outer_rel->params_req_for_parallel, inner_rel->params_req_for_parallel);
joinrel->reltarget = create_empty_pathtarget();
joinrel->pathlist = NIL;
joinrel->ppilist = NIL;
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index c17b53f7ad..4966010fd0 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -871,6 +871,12 @@ typedef struct RelOptInfo
/* consider parallel paths? */
bool consider_parallel;
+ /*
+ * Params, if any, required to be provided when consider_parallel is true.
+ * Note: if consider_parallel is false then this is not meaningful.
+ */
+ Bitmapset *params_req_for_parallel;
+
/*
* default result targetlist for Paths scanning this relation; list of
* Vars/Exprs, cost, width
diff --git a/src/include/optimizer/clauses.h b/src/include/optimizer/clauses.h
index cbe0607e85..31cf484d0f 100644
--- a/src/include/optimizer/clauses.h
+++ b/src/include/optimizer/clauses.h
@@ -34,6 +34,7 @@ extern bool contain_subplans(Node *clause);
extern char max_parallel_hazard(Query *parse);
extern bool is_parallel_safe(PlannerInfo *root, Node *node);
+extern bool is_parallel_safe_with_params(PlannerInfo *root, Node *node, Bitmapset **required_params);
extern bool contain_nonstrict_functions(Node *clause);
extern bool contain_exec_param(Node *clause, List *param_ids);
extern bool contain_leaked_vars(Node *clause);
diff --git a/src/test/regress/expected/incremental_sort.out b/src/test/regress/expected/incremental_sort.out
index 0c3433f8e5..2b52b0ca3c 100644
--- a/src/test/regress/expected/incremental_sort.out
+++ b/src/test/regress/expected/incremental_sort.out
@@ -1597,20 +1597,21 @@ explain (costs off) select distinct
unique1,
(select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
from tenk1 t, generate_series(1, 1000);
- QUERY PLAN
----------------------------------------------------------------------------------
+ QUERY PLAN
+---------------------------------------------------------------------------------------
Unique
- -> Sort
- Sort Key: t.unique1, ((SubPlan 1))
- -> Gather
- Workers Planned: 2
- -> Nested Loop
- -> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
- -> Function Scan on generate_series
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on tenk1
- Index Cond: (unique1 = t.unique1)
-(11 rows)
+ -> Gather Merge
+ Workers Planned: 2
+ -> Unique
+ -> Sort
+ Sort Key: t.unique1, ((SubPlan 1))
+ -> Nested Loop
+ -> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
+ -> Function Scan on generate_series
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on tenk1
+ Index Cond: (unique1 = t.unique1)
+(12 rows)
explain (costs off) select
unique1,
@@ -1619,16 +1620,16 @@ from tenk1 t, generate_series(1, 1000)
order by 1, 2;
QUERY PLAN
---------------------------------------------------------------------------
- Sort
- Sort Key: t.unique1, ((SubPlan 1))
- -> Gather
- Workers Planned: 2
+ Gather Merge
+ Workers Planned: 2
+ -> Sort
+ Sort Key: t.unique1, ((SubPlan 1))
-> Nested Loop
-> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
-> Function Scan on generate_series
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on tenk1
- Index Cond: (unique1 = t.unique1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on tenk1
+ Index Cond: (unique1 = t.unique1)
(10 rows)
-- Parallel sort but with expression not available until the upper rel.
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index 2abf759385..4566260fc4 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -1489,60 +1489,64 @@ EXPLAIN (VERBOSE, COSTS OFF) SELECT * FROM part p(x) ORDER BY x;
--
-- pruning won't work for mc3p, because some keys are Params
explain (costs off) select * from mc2p t1, lateral (select count(*) from mc3p t2 where t2.a = t1.b and abs(t2.b) = 1 and t2.c = 1) s where t1.a = 1;
- QUERY PLAN
------------------------------------------------------------------------
- Nested Loop
- -> Append
- -> Seq Scan on mc2p1 t1_1
- Filter: (a = 1)
- -> Seq Scan on mc2p2 t1_2
- Filter: (a = 1)
- -> Seq Scan on mc2p_default t1_3
- Filter: (a = 1)
- -> Aggregate
- -> Append
- -> Seq Scan on mc3p0 t2_1
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p1 t2_2
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p2 t2_3
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p3 t2_4
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p4 t2_5
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p5 t2_6
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p6 t2_7
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p7 t2_8
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p_default t2_9
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
-(28 rows)
+ QUERY PLAN
+-----------------------------------------------------------------------------
+ Gather
+ Workers Planned: 2
+ -> Nested Loop
+ -> Parallel Append
+ -> Parallel Seq Scan on mc2p1 t1_1
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p2 t1_2
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p_default t1_3
+ Filter: (a = 1)
+ -> Aggregate
+ -> Append
+ -> Seq Scan on mc3p0 t2_1
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p1 t2_2
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p2 t2_3
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p3 t2_4
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p4 t2_5
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p5 t2_6
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p6 t2_7
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p7 t2_8
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p_default t2_9
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+(30 rows)
-- pruning should work fine, because values for a prefix of keys (a, b) are
-- available
explain (costs off) select * from mc2p t1, lateral (select count(*) from mc3p t2 where t2.c = t1.b and abs(t2.b) = 1 and t2.a = 1) s where t1.a = 1;
- QUERY PLAN
------------------------------------------------------------------------
- Nested Loop
- -> Append
- -> Seq Scan on mc2p1 t1_1
- Filter: (a = 1)
- -> Seq Scan on mc2p2 t1_2
- Filter: (a = 1)
- -> Seq Scan on mc2p_default t1_3
- Filter: (a = 1)
- -> Aggregate
- -> Append
- -> Seq Scan on mc3p0 t2_1
- Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p1 t2_2
- Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p_default t2_3
- Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
-(16 rows)
+ QUERY PLAN
+-----------------------------------------------------------------------------
+ Gather
+ Workers Planned: 2
+ -> Nested Loop
+ -> Parallel Append
+ -> Parallel Seq Scan on mc2p1 t1_1
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p2 t1_2
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p_default t1_3
+ Filter: (a = 1)
+ -> Aggregate
+ -> Append
+ -> Seq Scan on mc3p0 t2_1
+ Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p1 t2_2
+ Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p_default t2_3
+ Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
+(18 rows)
-- also here, because values for all keys are provided
explain (costs off) select * from mc2p t1, lateral (select count(*) from mc3p t2 where t2.a = 1 and abs(t2.b) = 1 and t2.c = 1) s where t1.a = 1;
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 0ddcf8e0b1..e422854574 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -320,19 +320,19 @@ explain (costs off, verbose) select
QUERY PLAN
----------------------------------------------------------------------------
Gather
- Output: (SubPlan 1)
+ Output: ((SubPlan 1))
Workers Planned: 4
-> Nested Loop
- Output: t.unique1
+ Output: (SubPlan 1)
-> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
Output: t.unique1
-> Function Scan on pg_catalog.generate_series
Output: generate_series.generate_series
Function Call: generate_series(1, 10)
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(14 rows)
explain (costs off, verbose) select
@@ -341,63 +341,69 @@ explain (costs off, verbose) select
QUERY PLAN
----------------------------------------------------------------------
Gather
- Output: (SubPlan 1)
+ Output: ((SubPlan 1))
Workers Planned: 4
-> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
- Output: t.unique1
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ Output: (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(9 rows)
explain (costs off, verbose) select
(select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
from tenk1 t
limit 1;
- QUERY PLAN
--------------------------------------------------------------------
+ QUERY PLAN
+----------------------------------------------------------------------------
Limit
Output: ((SubPlan 1))
- -> Seq Scan on public.tenk1 t
- Output: (SubPlan 1)
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
-(8 rows)
+ -> Gather
+ Output: ((SubPlan 1))
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(11 rows)
explain (costs off, verbose) select t.unique1
from tenk1 t
where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
- QUERY PLAN
--------------------------------------------------------------
- Seq Scan on public.tenk1 t
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
Output: t.unique1
- Filter: (t.unique1 = (SubPlan 1))
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
-(7 rows)
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ Filter: (t.unique1 = (SubPlan 1))
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(10 rows)
explain (costs off, verbose) select *
from tenk1 t
order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
- QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
- Sort
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Gather Merge
Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
- Sort Key: ((SubPlan 1))
- -> Gather
- Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, (SubPlan 1)
- Workers Planned: 4
+ Workers Planned: 4
+ -> Sort
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
+ Sort Key: ((SubPlan 1))
-> Parallel Seq Scan on public.tenk1 t
- Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(12 rows)
-- test subplan in join/lateral join
@@ -409,14 +415,14 @@ explain (costs off, verbose, timing off) select t.unique1, l.*
QUERY PLAN
----------------------------------------------------------------------
Gather
- Output: t.unique1, (SubPlan 1)
+ Output: t.unique1, ((SubPlan 1))
Workers Planned: 4
-> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
- Output: t.unique1
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ Output: t.unique1, (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(9 rows)
-- can't put a gather at the top of a subplan that takes a param
@@ -1349,9 +1355,12 @@ SELECT 1 FROM tenk1_vw_sec
Workers Planned: 4
-> Parallel Index Only Scan using tenk1_unique1 on tenk1
SubPlan 1
- -> Aggregate
- -> Seq Scan on int4_tbl
- Filter: (f1 < tenk1_vw_sec.unique1)
-(9 rows)
+ -> Finalize Aggregate
+ -> Gather
+ Workers Planned: 1
+ -> Partial Aggregate
+ -> Parallel Seq Scan on int4_tbl
+ Filter: (f1 < tenk1_vw_sec.unique1)
+(12 rows)
rollback;
--
2.39.2 (Apple Git-143)
Re: Parallelize correlated subqueries that execute within each worker
On Tue, 4 Jul 2023 at 06:56, James Coleman <jtc331@gmail.com> wrote:
On Sun, Jun 11, 2023 at 10:23 PM James Coleman <jtc331@gmail.com> wrote:
...
And while trying the v9 patch I came across a crash with the query
below.set min_parallel_table_scan_size to 0;
set parallel_setup_cost to 0;
set parallel_tuple_cost to 0;explain (costs off)
select * from pg_description t1 where objoid in
(select objoid from pg_description t2 where t2.description = t1.description);
QUERY PLAN
--------------------------------------------------------
Seq Scan on pg_description t1
Filter: (SubPlan 1)
SubPlan 1
-> Gather
Workers Planned: 2
-> Parallel Seq Scan on pg_description t2
Filter: (description = t1.description)
(7 rows)select * from pg_description t1 where objoid in
(select objoid from pg_description t2 where t2.description = t1.description);
WARNING: terminating connection because of crash of another server processSeems something is wrong when extracting the argument from the Param in
parallel worker.With what I'm trying to change I don't think this plan should ever be
generated since it means we'd have to pass a param from the outer seq
scan into the parallel subplan, which we can't do (currently).I've attached the full backtrace to the email, but as you hinted at
the parallel worker is trying to get the param (in this case
detoasting it), but the param doesn't exist on the worker, so it seg
faults. Looking at this further I think there's an existing test case
that exposes the misplanning here (the one right under the comment
"Parallel Append is not to be used when the subpath depends on the
outer param" in select_parallel.sql), but it doesn't seg fault because
the param is an integer, doesn't need to be detoasted, and therefore
(I think) we skate by (but probably with wrong results in depending on
the dataset).Interestingly this is one of the existing test queries my original
patch approach didn't change, so this gives me something specific to
work with improving the path. Thanks for testing this and bringing
this to my attention!Here's what I've found debugging this:
There's only a single gather path ever created when planning this
query, making it easy to know which one is the problem. That gather
path is created with this stacktrace:frame #0: 0x0000000105291590
postgres`create_gather_path(root=0x000000013081ae78,
rel=0x000000013080c8e8, subpath=0x000000013081c080,
target=0x000000013081c8c0, required_outer=0x0000000000000000,
rows=0x0000000000000000) at pathnode.c:1971:2
frame #1: 0x0000000105208e54
postgres`generate_gather_paths(root=0x000000013081ae78,
rel=0x000000013080c8e8, override_rows=false) at allpaths.c:3097:4
frame #2: 0x00000001052090ec
postgres`generate_useful_gather_paths(root=0x000000013081ae78,
rel=0x000000013080c8e8, override_rows=false) at allpaths.c:3241:2
frame #3: 0x0000000105258754
postgres`apply_scanjoin_target_to_paths(root=0x000000013081ae78,
rel=0x000000013080c8e8, scanjoin_targets=0x000000013081c978,
scanjoin_targets_contain_srfs=0x0000000000000000,
scanjoin_target_parallel_safe=true, tlist_same_exprs=true) at
planner.c:7696:3
frame #4: 0x00000001052533cc
postgres`grouping_planner(root=0x000000013081ae78, tuple_fraction=0.5)
at planner.c:1611:3
frame #5: 0x0000000105251e9c
postgres`subquery_planner(glob=0x00000001308188d8,
parse=0x000000013080caf8, parent_root=0x000000013080cc38,
hasRecursion=false, tuple_fraction=0.5) at planner.c:1062:2
frame #6: 0x000000010526b134
postgres`make_subplan(root=0x000000013080cc38,
orig_subquery=0x000000013080ff58, subLinkType=ANY_SUBLINK,
subLinkId=0, testexpr=0x000000013080d848, isTopQual=true) at
subselect.c:221:12
frame #7: 0x0000000105268b8c
postgres`process_sublinks_mutator(node=0x000000013080d6d8,
context=0x000000016b0998f8) at subselect.c:1950:10
frame #8: 0x0000000105268ad8
postgres`SS_process_sublinks(root=0x000000013080cc38,
expr=0x000000013080d6d8, isQual=true) at subselect.c:1923:9
frame #9: 0x00000001052527b8
postgres`preprocess_expression(root=0x000000013080cc38,
expr=0x000000013080d6d8, kind=0) at planner.c:1169:10
frame #10: 0x0000000105252954
postgres`preprocess_qual_conditions(root=0x000000013080cc38,
jtnode=0x000000013080d108) at planner.c:1214:14
frame #11: 0x0000000105251580
postgres`subquery_planner(glob=0x00000001308188d8,
parse=0x0000000137010d68, parent_root=0x0000000000000000,
hasRecursion=false, tuple_fraction=0) at planner.c:832:2
frame #12: 0x000000010525042c
postgres`standard_planner(parse=0x0000000137010d68,
query_string="explain (costs off)\nselect * from pg_description t1
where objoid in\n (select objoid from pg_description t2 where
t2.description = t1.description);", cursorOptions=2048,
boundParams=0x0000000000000000) at planner.c:411:9There aren't any lateral markings on the rels. Additionally the
partial path has param_info=null (I found out from Tom in a separate
thread [1] that this is only set for outer relations from the same
query level).The only param that I could easily find at first was a single param of
type PARAM_EXTERN in root->plan_params in make_subplan().I spent a lot of time trying to figure out where we could find the
PARAM_EXEC param that's being fed into the subplan, but it doesn't
seem like we have access to any of these things at the point in the
path creation process that it's interesting to us when inserting the
gather nodes.Given all of that I settled on this approach:
1. Modify is_parallel_safe() to by default ignore PARAM_EXEC params.
2. Add is_parallel_safe_with_params() that checks for the existence of
such params.
3. Store the required params in a bitmapset on each base rel.
4. Union the bitmapset on join rels.
5. Only insert a gather node if that bitmapset is empty.I have an intuition that there's some spot (e.g. joins) that we should
be removing params from this set (e.g., when we've satisfied them),
but I haven't been able to come up with such a scenario as yet.The attached v11 fixes the issue you reported.
One of the tests has failed in CFBot at [1] with:
+++ /tmp/cirrus-ci-build/build/testrun/pg_upgrade/002_pg_upgrade/data/results/select_parallel.out
2023-12-20 20:08:42.480004000 +0000
@@ -137,23 +137,24 @@
explain (costs off)
select (select max((select pa1.b from part_pa_test pa1 where pa1.a = pa2.a)))
from part_pa_test pa2;
- QUERY PLAN
---------------------------------------------------------------
- Aggregate
+ QUERY PLAN
+--------------------------------------------------------------------
+ Finalize Aggregate
-> Gather
Workers Planned: 3
- -> Parallel Append
- -> Parallel Seq Scan on part_pa_test_p1 pa2_1
- -> Parallel Seq Scan on part_pa_test_p2 pa2_2
+ -> Partial Aggregate
+ -> Parallel Append
+ -> Parallel Seq Scan on part_pa_test_p1 pa2_1
+ -> Parallel Seq Scan on part_pa_test_p2 pa2_2
+ SubPlan 1
+ -> Append
+ -> Seq Scan on part_pa_test_p1 pa1_1
+ Filter: (a = pa2.a)
+ -> Seq Scan on part_pa_test_p2 pa1_2
+ Filter: (a = pa2.a)
SubPlan 2
-> Result
- SubPlan 1
- -> Append
- -> Seq Scan on part_pa_test_p1 pa1_1
- Filter: (a = pa2.a)
- -> Seq Scan on part_pa_test_p2 pa1_2
- Filter: (a = pa2.a)
-(14 rows)
+(15 rows)More details of the failure is available at [2]https://api.cirrus-ci.com/v1/artifact/task/5685696451575808/testrun/build/testrun/pg_upgrade/002_pg_upgrade/log/regress_log_002_pg_upgrade.
[1]: https://cirrus-ci.com/task/5685696451575808
[2]: https://api.cirrus-ci.com/v1/artifact/task/5685696451575808/testrun/build/testrun/pg_upgrade/002_pg_upgrade/log/regress_log_002_pg_upgrade
Regards,
Vignesh
Re: Parallelize correlated subqueries that execute within each worker
Hello!
I was going through the previous conversations for this particular patch and it seems that this patch failed some tests previously?
Imo we should move it to the next CF so that the remaining issues can be resolved accordingly.
Re: Parallelize correlated subqueries that execute within each worker
On Tue, Jan 23, 2024 at 9:34 AM Akshat Jaimini <destrex271@gmail.com> wrote:
Hello!
I was going through the previous conversations for this particular patch and it seems that this patch failed some tests previously?
Imo we should move it to the next CF so that the remaining issues can be resolved accordingly.
So I guess the question here is whether this is thought to be ready
for serious review or whether it is still thought to need work. If the
latter, it should be marked RwF until that happens -- if the former,
then we should try to review it rather than letting it languish
forever.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: Parallelize correlated subqueries that execute within each worker
I think we should move this patch to the next CF as I believe that work is still going on resolving the last reported bug.
Re: Parallelize correlated subqueries that execute within each worker
On Tue, Jan 30, 2024 at 11:17 AM Akshat Jaimini <destrex271@gmail.com> wrote:
I think we should move this patch to the next CF as I believe that work is still going on resolving the last reported bug.
We shouldn't just keep pushing this forward to the next CF. It's been
idle since July. If it needs more work, mark it RwF and it can be
reopened when there's something for a reviewer to do.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: Parallelize correlated subqueries that execute within each worker
On Tue, Jan 9, 2024 at 2:09 AM vignesh C <vignesh21@gmail.com> wrote:
...
Given all of that I settled on this approach:
1. Modify is_parallel_safe() to by default ignore PARAM_EXEC params.
2. Add is_parallel_safe_with_params() that checks for the existence of
such params.
3. Store the required params in a bitmapset on each base rel.
4. Union the bitmapset on join rels.
5. Only insert a gather node if that bitmapset is empty.I have an intuition that there's some spot (e.g. joins) that we should
be removing params from this set (e.g., when we've satisfied them),
but I haven't been able to come up with such a scenario as yet.The attached v11 fixes the issue you reported.
One of the tests has failed in CFBot at [1] with: +++ /tmp/cirrus-ci-build/build/testrun/pg_upgrade/002_pg_upgrade/data/results/select_parallel.out 2023-12-20 20:08:42.480004000 +0000 @@ -137,23 +137,24 @@ explain (costs off) select (select max((select pa1.b from part_pa_test pa1 where pa1.a = pa2.a))) from part_pa_test pa2; - QUERY PLAN --------------------------------------------------------------- - Aggregate + QUERY PLAN +-------------------------------------------------------------------- + Finalize Aggregate -> Gather Workers Planned: 3 - -> Parallel Append - -> Parallel Seq Scan on part_pa_test_p1 pa2_1 - -> Parallel Seq Scan on part_pa_test_p2 pa2_2 + -> Partial Aggregate + -> Parallel Append + -> Parallel Seq Scan on part_pa_test_p1 pa2_1 + -> Parallel Seq Scan on part_pa_test_p2 pa2_2 + SubPlan 1 + -> Append + -> Seq Scan on part_pa_test_p1 pa1_1 + Filter: (a = pa2.a) + -> Seq Scan on part_pa_test_p2 pa1_2 + Filter: (a = pa2.a) SubPlan 2 -> Result - SubPlan 1 - -> Append - -> Seq Scan on part_pa_test_p1 pa1_1 - Filter: (a = pa2.a) - -> Seq Scan on part_pa_test_p2 pa1_2 - Filter: (a = pa2.a) -(14 rows) +(15 rows)More details of the failure is available at [2].
[1] - https://cirrus-ci.com/task/5685696451575808
[2] - https://api.cirrus-ci.com/v1/artifact/task/5685696451575808/testrun/build/testrun/pg_upgrade/002_pg_upgrade/log/regress_log_002_pg_upgrade
Thanks for noting this here.
I've finally had a chance to look at this, and I don't believe there's
any real failure here, merely drift of how the planner works on master
resulting in this query now being eligible for a different plan shape.
I was a bit wary at first because the changing test query is one I'd
previously referenced in [1] as likely exposing the bug I'd fixed
where params where being used across worker boundaries. However
looking at the diff in the patch at that point (v10) that particular
test query formed a different plan shape (there were two gather nodes
being created, and params crossing between them).
But in the current revision of master with the current patch applied
that's no longer true: we have a Gather node, and the Subplan using
the param is properly under that Gather node, and the param should be
being both generated and consumed within the same worker process.
So I've updated the patch to show that plan change as part of the diff.
See attached v12
Regards,
James Coleman
1: /messages/by-id/CAAaqYe-_TObm5KwmZLYXBJ3BJGh4cUZWM0v1mY1gWTMkRNQXDQ@mail.gmail.com
Attachments:
v12-0002-Parallelize-correlated-subqueries.patchapplication/octet-stream; name=v12-0002-Parallelize-correlated-subqueries.patchDownload
From ac42a4080e002627af2c221591d7c872612a7ce7 Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Sun, 2 Jul 2023 15:23:49 -0400
Subject: [PATCH v12 2/2] Parallelize correlated subqueries
When params are provided at the current query level (i.e., are generated
within a single worker and not shared across workers) we can safely
execute these in parallel.
We accomplish this by tracking the PARAM_EXEC params that are needed for
a rel to be safely executed in parallel and only inserting a Gather node
if that set is empty.
---
doc/src/sgml/parallel.sgml | 3 +-
src/backend/optimizer/path/allpaths.c | 24 ++-
src/backend/optimizer/util/clauses.c | 73 ++++++--
src/backend/optimizer/util/relnode.c | 1 +
src/include/nodes/pathnodes.h | 6 +
src/include/optimizer/clauses.h | 1 +
.../regress/expected/incremental_sort.out | 41 ++---
src/test/regress/expected/partition_prune.out | 104 +++++------
src/test/regress/expected/select_parallel.out | 161 ++++++++++--------
src/test/regress/sql/select_parallel.sql | 8 +-
10 files changed, 266 insertions(+), 156 deletions(-)
diff --git a/doc/src/sgml/parallel.sgml b/doc/src/sgml/parallel.sgml
index 5acc9537d6..fd32572ec8 100644
--- a/doc/src/sgml/parallel.sgml
+++ b/doc/src/sgml/parallel.sgml
@@ -518,7 +518,8 @@ EXPLAIN SELECT * FROM pgbench_accounts WHERE filler LIKE '%x%';
<listitem>
<para>
- Plan nodes that reference a correlated <literal>SubPlan</literal>.
+ Plan nodes that reference a correlated <literal>SubPlan</literal> where
+ the result is shared between workers.
</para>
</listitem>
</itemizedlist>
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 84c4de488a..37c44d393b 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -632,7 +632,7 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
if (proparallel != PROPARALLEL_SAFE)
return;
- if (!is_parallel_safe(root, (Node *) rte->tablesample->args))
+ if (!is_parallel_safe_with_params(root, (Node *) rte->tablesample->args, &rel->params_req_for_parallel))
return;
}
@@ -698,7 +698,7 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
case RTE_FUNCTION:
/* Check for parallel-restricted functions. */
- if (!is_parallel_safe(root, (Node *) rte->functions))
+ if (!is_parallel_safe_with_params(root, (Node *) rte->functions, &rel->params_req_for_parallel))
return;
break;
@@ -708,7 +708,7 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
case RTE_VALUES:
/* Check for parallel-restricted functions. */
- if (!is_parallel_safe(root, (Node *) rte->values_lists))
+ if (!is_parallel_safe_with_params(root, (Node *) rte->values_lists, &rel->params_req_for_parallel))
return;
break;
@@ -745,14 +745,14 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
* outer join clauses work correctly. It would likely break equivalence
* classes, too.
*/
- if (!is_parallel_safe(root, (Node *) rel->baserestrictinfo))
+ if (!is_parallel_safe_with_params(root, (Node *) rel->baserestrictinfo, &rel->params_req_for_parallel))
return;
/*
* Likewise, if the relation's outputs are not parallel-safe, give up.
* (Usually, they're just Vars, but sometimes they're not.)
*/
- if (!is_parallel_safe(root, (Node *) rel->reltarget->exprs))
+ if (!is_parallel_safe_with_params(root, (Node *) rel->reltarget->exprs, &rel->params_req_for_parallel))
return;
/* We have a winner. */
@@ -3071,6 +3071,13 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
if (rel->partial_pathlist == NIL)
return;
+ /*
+ * Wait to insert Gather nodes until all PARAM_EXEC params are provided
+ * within the current rel since we can't pass them to workers.
+ */
+ if (!bms_is_empty(rel->params_req_for_parallel))
+ return;
+
/* Should we override the rel's rowcount estimate? */
if (override_rows)
rowsp = &rows;
@@ -3209,6 +3216,13 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
if (rel->partial_pathlist == NIL)
return;
+ /*
+ * Wait to insert Gather nodes until all PARAM_EXEC params are provided
+ * within the current rel since we can't pass them to workers.
+ */
+ if (!bms_is_empty(rel->params_req_for_parallel))
+ return;
+
/* Should we override the rel's rowcount estimate? */
if (override_rows)
rowsp = &rows;
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 94eb56a1e7..9acad18154 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -91,6 +91,8 @@ typedef struct
{
char max_hazard; /* worst proparallel hazard found so far */
char max_interesting; /* worst proparallel hazard of interest */
+ bool check_params;
+ Bitmapset **required_params;
List *safe_param_ids; /* PARAM_EXEC Param IDs to treat as safe */
} max_parallel_hazard_context;
@@ -720,6 +722,7 @@ max_parallel_hazard(Query *parse)
context.max_hazard = PROPARALLEL_SAFE;
context.max_interesting = PROPARALLEL_UNSAFE;
+ context.check_params = true;
context.safe_param_ids = NIL;
(void) max_parallel_hazard_walker((Node *) parse, &context);
return context.max_hazard;
@@ -736,8 +739,6 @@ bool
is_parallel_safe(PlannerInfo *root, Node *node)
{
max_parallel_hazard_context context;
- PlannerInfo *proot;
- ListCell *l;
/*
* Even if the original querytree contained nothing unsafe, we need to
@@ -751,6 +752,43 @@ is_parallel_safe(PlannerInfo *root, Node *node)
/* Else use max_parallel_hazard's search logic, but stop on RESTRICTED */
context.max_hazard = PROPARALLEL_SAFE;
context.max_interesting = PROPARALLEL_RESTRICTED;
+ context.check_params = false;
+ context.required_params = NULL;
+
+ return !max_parallel_hazard_walker(node, &context);
+}
+
+/*
+ * is_parallel_safe_with_params
+ * As above, but additionally tracking what PARAM_EXEC params required to
+ * be provided within a worker for a gather to be inserted at this level
+ * of the query. Those required params are passed to the caller through
+ * the required_params argument.
+ *
+ * Note: required_params is only valid if node is otherwise parallel safe.
+ */
+bool
+is_parallel_safe_with_params(PlannerInfo *root, Node *node, Bitmapset **required_params)
+{
+ max_parallel_hazard_context context;
+ PlannerInfo *proot;
+ ListCell *l;
+
+ /*
+ * Even if the original querytree contained nothing unsafe, we need to
+ * search the expression if we have generated any PARAM_EXEC Params while
+ * planning, because those will have to be provided for the expression to
+ * remain parallel safe and there might be one in this expression. But
+ * otherwise we don't need to look.
+ */
+ if (root->glob->maxParallelHazard == PROPARALLEL_SAFE &&
+ root->glob->paramExecTypes == NIL)
+ return true;
+ /* Else use max_parallel_hazard's search logic, but stop on RESTRICTED */
+ context.max_hazard = PROPARALLEL_SAFE;
+ context.max_interesting = PROPARALLEL_RESTRICTED;
+ context.check_params = false;
+ context.required_params = required_params;
context.safe_param_ids = NIL;
/*
@@ -890,13 +928,22 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
if (!subplan->parallel_safe &&
max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
return true;
- save_safe_param_ids = context->safe_param_ids;
- context->safe_param_ids = list_concat_copy(context->safe_param_ids,
- subplan->paramIds);
+
+ if (context->check_params || context->required_params != NULL)
+ {
+ save_safe_param_ids = context->safe_param_ids;
+ context->safe_param_ids = list_concat_copy(context->safe_param_ids,
+ subplan->paramIds);
+ }
if (max_parallel_hazard_walker(subplan->testexpr, context))
return true; /* no need to restore safe_param_ids */
- list_free(context->safe_param_ids);
- context->safe_param_ids = save_safe_param_ids;
+
+ if (context->check_params || context->required_params != NULL)
+ {
+ list_free(context->safe_param_ids);
+ context->safe_param_ids = save_safe_param_ids;
+ }
+
/* we must also check args, but no special Param treatment there */
if (max_parallel_hazard_walker((Node *) subplan->args, context))
return true;
@@ -915,14 +962,18 @@ max_parallel_hazard_walker(Node *node, max_parallel_hazard_context *context)
{
Param *param = (Param *) node;
- if (param->paramkind == PARAM_EXTERN)
+ if (param->paramkind != PARAM_EXEC || !(context->check_params || context->required_params != NULL))
return false;
- if (param->paramkind != PARAM_EXEC ||
- !list_member_int(context->safe_param_ids, param->paramid))
+ if (!list_member_int(context->safe_param_ids, param->paramid))
{
if (max_parallel_hazard_test(PROPARALLEL_RESTRICTED, context))
- return true;
+ {
+ if (context->required_params != NULL)
+ *context->required_params = bms_add_member(*context->required_params, param->paramid);
+ if (context->check_params)
+ return true;
+ }
}
return false; /* nothing to recurse to */
}
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 22d01cef5b..58228ff269 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -699,6 +699,7 @@ build_join_rel(PlannerInfo *root,
joinrel->consider_startup = (root->tuple_fraction > 0);
joinrel->consider_param_startup = false;
joinrel->consider_parallel = false;
+ joinrel->params_req_for_parallel = bms_union(outer_rel->params_req_for_parallel, inner_rel->params_req_for_parallel);
joinrel->reltarget = create_empty_pathtarget();
joinrel->pathlist = NIL;
joinrel->ppilist = NIL;
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 137da178dc..cb5721963d 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -871,6 +871,12 @@ typedef struct RelOptInfo
/* consider parallel paths? */
bool consider_parallel;
+ /*
+ * Params, if any, required to be provided when consider_parallel is true.
+ * Note: if consider_parallel is false then this is not meaningful.
+ */
+ Bitmapset *params_req_for_parallel;
+
/*
* default result targetlist for Paths scanning this relation; list of
* Vars/Exprs, cost, width
diff --git a/src/include/optimizer/clauses.h b/src/include/optimizer/clauses.h
index 34b301e537..eb4334f565 100644
--- a/src/include/optimizer/clauses.h
+++ b/src/include/optimizer/clauses.h
@@ -34,6 +34,7 @@ extern bool contain_subplans(Node *clause);
extern char max_parallel_hazard(Query *parse);
extern bool is_parallel_safe(PlannerInfo *root, Node *node);
+extern bool is_parallel_safe_with_params(PlannerInfo *root, Node *node, Bitmapset **required_params);
extern bool contain_nonstrict_functions(Node *clause);
extern bool contain_exec_param(Node *clause, List *param_ids);
extern bool contain_leaked_vars(Node *clause);
diff --git a/src/test/regress/expected/incremental_sort.out b/src/test/regress/expected/incremental_sort.out
index 7fdb685313..df1c52414e 100644
--- a/src/test/regress/expected/incremental_sort.out
+++ b/src/test/regress/expected/incremental_sort.out
@@ -1597,20 +1597,21 @@ explain (costs off) select distinct
unique1,
(select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
from tenk1 t, generate_series(1, 1000);
- QUERY PLAN
----------------------------------------------------------------------------------
+ QUERY PLAN
+---------------------------------------------------------------------------------------
Unique
- -> Sort
- Sort Key: t.unique1, ((SubPlan 1))
- -> Gather
- Workers Planned: 2
- -> Nested Loop
- -> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
- -> Function Scan on generate_series
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on tenk1
- Index Cond: (unique1 = t.unique1)
-(11 rows)
+ -> Gather Merge
+ Workers Planned: 2
+ -> Unique
+ -> Sort
+ Sort Key: t.unique1, ((SubPlan 1))
+ -> Nested Loop
+ -> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
+ -> Function Scan on generate_series
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on tenk1
+ Index Cond: (unique1 = t.unique1)
+(12 rows)
explain (costs off) select
unique1,
@@ -1619,16 +1620,16 @@ from tenk1 t, generate_series(1, 1000)
order by 1, 2;
QUERY PLAN
---------------------------------------------------------------------------
- Sort
- Sort Key: t.unique1, ((SubPlan 1))
- -> Gather
- Workers Planned: 2
+ Gather Merge
+ Workers Planned: 2
+ -> Sort
+ Sort Key: t.unique1, ((SubPlan 1))
-> Nested Loop
-> Parallel Index Only Scan using tenk1_unique1 on tenk1 t
-> Function Scan on generate_series
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on tenk1
- Index Cond: (unique1 = t.unique1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on tenk1
+ Index Cond: (unique1 = t.unique1)
(10 rows)
-- Parallel sort but with expression not available until the upper rel.
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index 9a4c48c055..c45590fdfe 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -1489,60 +1489,64 @@ EXPLAIN (VERBOSE, COSTS OFF) SELECT * FROM part p(x) ORDER BY x;
--
-- pruning won't work for mc3p, because some keys are Params
explain (costs off) select * from mc2p t1, lateral (select count(*) from mc3p t2 where t2.a = t1.b and abs(t2.b) = 1 and t2.c = 1) s where t1.a = 1;
- QUERY PLAN
------------------------------------------------------------------------
- Nested Loop
- -> Append
- -> Seq Scan on mc2p1 t1_1
- Filter: (a = 1)
- -> Seq Scan on mc2p2 t1_2
- Filter: (a = 1)
- -> Seq Scan on mc2p_default t1_3
- Filter: (a = 1)
- -> Aggregate
- -> Append
- -> Seq Scan on mc3p0 t2_1
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p1 t2_2
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p2 t2_3
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p3 t2_4
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p4 t2_5
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p5 t2_6
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p6 t2_7
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p7 t2_8
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p_default t2_9
- Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
-(28 rows)
+ QUERY PLAN
+-----------------------------------------------------------------------------
+ Gather
+ Workers Planned: 2
+ -> Nested Loop
+ -> Parallel Append
+ -> Parallel Seq Scan on mc2p1 t1_1
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p2 t1_2
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p_default t1_3
+ Filter: (a = 1)
+ -> Aggregate
+ -> Append
+ -> Seq Scan on mc3p0 t2_1
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p1 t2_2
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p2 t2_3
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p3 t2_4
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p4 t2_5
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p5 t2_6
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p6 t2_7
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p7 t2_8
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p_default t2_9
+ Filter: ((a = t1.b) AND (c = 1) AND (abs(b) = 1))
+(30 rows)
-- pruning should work fine, because values for a prefix of keys (a, b) are
-- available
explain (costs off) select * from mc2p t1, lateral (select count(*) from mc3p t2 where t2.c = t1.b and abs(t2.b) = 1 and t2.a = 1) s where t1.a = 1;
- QUERY PLAN
------------------------------------------------------------------------
- Nested Loop
- -> Append
- -> Seq Scan on mc2p1 t1_1
- Filter: (a = 1)
- -> Seq Scan on mc2p2 t1_2
- Filter: (a = 1)
- -> Seq Scan on mc2p_default t1_3
- Filter: (a = 1)
- -> Aggregate
- -> Append
- -> Seq Scan on mc3p0 t2_1
- Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p1 t2_2
- Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
- -> Seq Scan on mc3p_default t2_3
- Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
-(16 rows)
+ QUERY PLAN
+-----------------------------------------------------------------------------
+ Gather
+ Workers Planned: 2
+ -> Nested Loop
+ -> Parallel Append
+ -> Parallel Seq Scan on mc2p1 t1_1
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p2 t1_2
+ Filter: (a = 1)
+ -> Parallel Seq Scan on mc2p_default t1_3
+ Filter: (a = 1)
+ -> Aggregate
+ -> Append
+ -> Seq Scan on mc3p0 t2_1
+ Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p1 t2_2
+ Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
+ -> Seq Scan on mc3p_default t2_3
+ Filter: ((c = t1.b) AND (a = 1) AND (abs(b) = 1))
+(18 rows)
-- also here, because values for all keys are provided
explain (costs off) select * from mc2p t1, lateral (select count(*) from mc3p t2 where t2.a = 1 and abs(t2.b) = 1 and t2.c = 1) s where t1.a = 1;
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index b76132ffe4..a6c4d6bf3f 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -130,30 +130,48 @@ select sp_test_func() order by 1;
foo
(2 rows)
--- Parallel Append is not to be used when the subpath depends on the outer param
+-- Parallel Append is can be used when the subpath depends on the outer params
+-- when those params are consumed within the worker that generates them.
create table part_pa_test(a int, b int) partition by range(a);
create table part_pa_test_p1 partition of part_pa_test for values from (minvalue) to (0);
create table part_pa_test_p2 partition of part_pa_test for values from (0) to (maxvalue);
-explain (costs off)
+explain (costs off, verbose)
select (select max((select pa1.b from part_pa_test pa1 where pa1.a = pa2.a)))
from part_pa_test pa2;
- QUERY PLAN
---------------------------------------------------------------
- Aggregate
+ QUERY PLAN
+---------------------------------------------------------------------------
+ Finalize Aggregate
+ Output: (SubPlan 2)
-> Gather
+ Output: (PARTIAL max((SubPlan 1)))
Workers Planned: 3
- -> Parallel Append
- -> Parallel Seq Scan on part_pa_test_p1 pa2_1
- -> Parallel Seq Scan on part_pa_test_p2 pa2_2
+ -> Partial Aggregate
+ Output: PARTIAL max((SubPlan 1))
+ -> Parallel Append
+ -> Parallel Seq Scan on public.part_pa_test_p1 pa2_1
+ Output: pa2_1.a
+ -> Parallel Seq Scan on public.part_pa_test_p2 pa2_2
+ Output: pa2_2.a
+ SubPlan 1
+ -> Append
+ -> Seq Scan on public.part_pa_test_p1 pa1_1
+ Output: pa1_1.b
+ Filter: (pa1_1.a = pa2.a)
+ -> Seq Scan on public.part_pa_test_p2 pa1_2
+ Output: pa1_2.b
+ Filter: (pa1_2.a = pa2.a)
SubPlan 2
-> Result
- SubPlan 1
- -> Append
- -> Seq Scan on part_pa_test_p1 pa1_1
- Filter: (a = pa2.a)
- -> Seq Scan on part_pa_test_p2 pa1_2
- Filter: (a = pa2.a)
-(14 rows)
+ Output: max((SubPlan 1))
+(23 rows)
+
+insert into part_pa_test(a, b) values (-1, 1), (1, 3);
+select (select max((select pa1.b from part_pa_test pa1 where pa1.a = pa2.a)))
+from part_pa_test pa2;
+ max
+-----
+ 3
+(1 row)
drop table part_pa_test;
-- test with leader participation disabled
@@ -320,19 +338,19 @@ explain (costs off, verbose) select
QUERY PLAN
----------------------------------------------------------------------------
Gather
- Output: (SubPlan 1)
+ Output: ((SubPlan 1))
Workers Planned: 4
-> Nested Loop
- Output: t.unique1
+ Output: (SubPlan 1)
-> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
Output: t.unique1
-> Function Scan on pg_catalog.generate_series
Output: generate_series.generate_series
Function Call: generate_series(1, 10)
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(14 rows)
explain (costs off, verbose) select
@@ -341,63 +359,69 @@ explain (costs off, verbose) select
QUERY PLAN
----------------------------------------------------------------------
Gather
- Output: (SubPlan 1)
+ Output: ((SubPlan 1))
Workers Planned: 4
-> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
- Output: t.unique1
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ Output: (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(9 rows)
explain (costs off, verbose) select
(select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
from tenk1 t
limit 1;
- QUERY PLAN
--------------------------------------------------------------------
+ QUERY PLAN
+----------------------------------------------------------------------------
Limit
Output: ((SubPlan 1))
- -> Seq Scan on public.tenk1 t
- Output: (SubPlan 1)
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
-(8 rows)
+ -> Gather
+ Output: ((SubPlan 1))
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(11 rows)
explain (costs off, verbose) select t.unique1
from tenk1 t
where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
- QUERY PLAN
--------------------------------------------------------------
- Seq Scan on public.tenk1 t
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
Output: t.unique1
- Filter: (t.unique1 = (SubPlan 1))
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
-(7 rows)
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ Filter: (t.unique1 = (SubPlan 1))
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(10 rows)
explain (costs off, verbose) select *
from tenk1 t
order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
- QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
- Sort
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Gather Merge
Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
- Sort Key: ((SubPlan 1))
- -> Gather
- Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, (SubPlan 1)
- Workers Planned: 4
+ Workers Planned: 4
+ -> Sort
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
+ Sort Key: ((SubPlan 1))
-> Parallel Seq Scan on public.tenk1 t
- Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(12 rows)
-- test subplan in join/lateral join
@@ -409,14 +433,14 @@ explain (costs off, verbose, timing off) select t.unique1, l.*
QUERY PLAN
----------------------------------------------------------------------
Gather
- Output: t.unique1, (SubPlan 1)
+ Output: t.unique1, ((SubPlan 1))
Workers Planned: 4
-> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
- Output: t.unique1
- SubPlan 1
- -> Index Only Scan using tenk1_unique1 on public.tenk1
- Output: t.unique1
- Index Cond: (tenk1.unique1 = t.unique1)
+ Output: t.unique1, (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
(9 rows)
-- can't put a gather at the top of a subplan that takes a param
@@ -1350,9 +1374,12 @@ SELECT 1 FROM tenk1_vw_sec
Workers Planned: 4
-> Parallel Index Only Scan using tenk1_unique1 on tenk1
SubPlan 1
- -> Aggregate
- -> Seq Scan on int4_tbl
- Filter: (f1 < tenk1_vw_sec.unique1)
-(9 rows)
+ -> Finalize Aggregate
+ -> Gather
+ Workers Planned: 1
+ -> Partial Aggregate
+ -> Parallel Seq Scan on int4_tbl
+ Filter: (f1 < tenk1_vw_sec.unique1)
+(12 rows)
rollback;
diff --git a/src/test/regress/sql/select_parallel.sql b/src/test/regress/sql/select_parallel.sql
index cc752a7d45..3fc24b2b94 100644
--- a/src/test/regress/sql/select_parallel.sql
+++ b/src/test/regress/sql/select_parallel.sql
@@ -53,13 +53,17 @@ $$ select 'foo'::varchar union all select 'bar'::varchar $$
language sql stable;
select sp_test_func() order by 1;
--- Parallel Append is not to be used when the subpath depends on the outer param
+-- Parallel Append is can be used when the subpath depends on the outer params
+-- when those params are consumed within the worker that generates them.
create table part_pa_test(a int, b int) partition by range(a);
create table part_pa_test_p1 partition of part_pa_test for values from (minvalue) to (0);
create table part_pa_test_p2 partition of part_pa_test for values from (0) to (maxvalue);
-explain (costs off)
+explain (costs off, verbose)
select (select max((select pa1.b from part_pa_test pa1 where pa1.a = pa2.a)))
from part_pa_test pa2;
+insert into part_pa_test(a, b) values (-1, 1), (1, 3);
+select (select max((select pa1.b from part_pa_test pa1 where pa1.a = pa2.a)))
+from part_pa_test pa2;
drop table part_pa_test;
-- test with leader participation disabled
--
2.39.3 (Apple Git-145)
v12-0001-Add-tests-before-change.patchapplication/octet-stream; name=v12-0001-Add-tests-before-change.patchDownload
From 3ba7b7b5a638cd74b5b8d546722514409d9e3fd3 Mon Sep 17 00:00:00 2001
From: jcoleman <jtc331@gmail.com>
Date: Mon, 26 Sep 2022 20:30:23 -0400
Subject: [PATCH v12 1/2] Add tests before change
---
src/test/regress/expected/select_parallel.out | 137 ++++++++++++++++++
src/test/regress/sql/select_parallel.sql | 33 +++++
2 files changed, 170 insertions(+)
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index d88353d496..b76132ffe4 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -311,6 +311,143 @@ select count(*) from tenk1 where (two, four) not in
10000
(1 row)
+-- test parallel plans for queries containing correlated subplans
+-- where the subplan only needs params available from the current
+-- worker's scan.
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t, generate_series(1, 10);
+ QUERY PLAN
+----------------------------------------------------------------------------
+ Gather
+ Output: (SubPlan 1)
+ Workers Planned: 4
+ -> Nested Loop
+ Output: t.unique1
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ -> Function Scan on pg_catalog.generate_series
+ Output: generate_series.generate_series
+ Function Call: generate_series(1, 10)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(14 rows)
+
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t;
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: (SubPlan 1)
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(9 rows)
+
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t
+ limit 1;
+ QUERY PLAN
+-------------------------------------------------------------------
+ Limit
+ Output: ((SubPlan 1))
+ -> Seq Scan on public.tenk1 t
+ Output: (SubPlan 1)
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(8 rows)
+
+explain (costs off, verbose) select t.unique1
+ from tenk1 t
+ where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+ QUERY PLAN
+-------------------------------------------------------------
+ Seq Scan on public.tenk1 t
+ Output: t.unique1
+ Filter: (t.unique1 = (SubPlan 1))
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(7 rows)
+
+explain (costs off, verbose) select *
+ from tenk1 t
+ order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Sort
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, ((SubPlan 1))
+ Sort Key: ((SubPlan 1))
+ -> Gather
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4, (SubPlan 1)
+ Workers Planned: 4
+ -> Parallel Seq Scan on public.tenk1 t
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(12 rows)
+
+-- test subplan in join/lateral join
+explain (costs off, verbose, timing off) select t.unique1, l.*
+ from tenk1 t
+ join lateral (
+ select (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1 offset 0)
+ ) l on true;
+ QUERY PLAN
+----------------------------------------------------------------------
+ Gather
+ Output: t.unique1, (SubPlan 1)
+ Workers Planned: 4
+ -> Parallel Index Only Scan using tenk1_unique1 on public.tenk1 t
+ Output: t.unique1
+ SubPlan 1
+ -> Index Only Scan using tenk1_unique1 on public.tenk1
+ Output: t.unique1
+ Index Cond: (tenk1.unique1 = t.unique1)
+(9 rows)
+
+-- can't put a gather at the top of a subplan that takes a param
+-- (use a join in the subplan for the harder case)
+explain (costs off, verbose) select * from tenk1 t where t.two in (
+ select t.two
+ from tenk1
+ join tenk1 t3 on t3.stringu1 = tenk1.stringu1
+ where tenk1.four = t.four
+);
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Seq Scan on public.tenk1 t
+ Output: t.unique1, t.unique2, t.two, t.four, t.ten, t.twenty, t.hundred, t.thousand, t.twothousand, t.fivethous, t.tenthous, t.odd, t.even, t.stringu1, t.stringu2, t.string4
+ Filter: (SubPlan 1)
+ SubPlan 1
+ -> Hash Join
+ Output: t.two
+ Hash Cond: (tenk1.stringu1 = t3.stringu1)
+ -> Seq Scan on public.tenk1
+ Output: tenk1.unique1, tenk1.unique2, tenk1.two, tenk1.four, tenk1.ten, tenk1.twenty, tenk1.hundred, tenk1.thousand, tenk1.twothousand, tenk1.fivethous, tenk1.tenthous, tenk1.odd, tenk1.even, tenk1.stringu1, tenk1.stringu2, tenk1.string4
+ Filter: (tenk1.four = t.four)
+ -> Hash
+ Output: t3.stringu1
+ -> Gather
+ Output: t3.stringu1
+ Workers Planned: 4
+ -> Parallel Seq Scan on public.tenk1 t3
+ Output: t3.stringu1
+(17 rows)
+
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
explain (costs off)
select * from tenk1 where (unique1 + random())::integer not in
diff --git a/src/test/regress/sql/select_parallel.sql b/src/test/regress/sql/select_parallel.sql
index 80c914dc02..cc752a7d45 100644
--- a/src/test/regress/sql/select_parallel.sql
+++ b/src/test/regress/sql/select_parallel.sql
@@ -111,6 +111,39 @@ explain (costs off)
(select hundred, thousand from tenk2 where thousand > 100);
select count(*) from tenk1 where (two, four) not in
(select hundred, thousand from tenk2 where thousand > 100);
+-- test parallel plans for queries containing correlated subplans
+-- where the subplan only needs params available from the current
+-- worker's scan.
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t, generate_series(1, 10);
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t;
+explain (costs off, verbose) select
+ (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
+ from tenk1 t
+ limit 1;
+explain (costs off, verbose) select t.unique1
+ from tenk1 t
+ where t.unique1 = (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+explain (costs off, verbose) select *
+ from tenk1 t
+ order by (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1);
+-- test subplan in join/lateral join
+explain (costs off, verbose, timing off) select t.unique1, l.*
+ from tenk1 t
+ join lateral (
+ select (select t.unique1 from tenk1 where tenk1.unique1 = t.unique1 offset 0)
+ ) l on true;
+-- can't put a gather at the top of a subplan that takes a param
+-- (use a join in the subplan for the harder case)
+explain (costs off, verbose) select * from tenk1 t where t.two in (
+ select t.two
+ from tenk1
+ join tenk1 t3 on t3.stringu1 = tenk1.stringu1
+ where tenk1.four = t.four
+);
-- this is not parallel-safe due to use of random() within SubLink's testexpr:
explain (costs off)
select * from tenk1 where (unique1 + random())::integer not in
--
2.39.3 (Apple Git-145)
Re: Parallelize correlated subqueries that execute within each worker
On Tue, Jan 30, 2024 at 11:54 AM Robert Haas <robertmhaas@gmail.com> wrote:
On Tue, Jan 30, 2024 at 11:17 AM Akshat Jaimini <destrex271@gmail.com> wrote:
I think we should move this patch to the next CF as I believe that work is still going on resolving the last reported bug.
We shouldn't just keep pushing this forward to the next CF. It's been
idle since July. If it needs more work, mark it RwF and it can be
reopened when there's something for a reviewer to do.
I don't follow the "Idle since July" since it just hasn't received
review since then, so there's been nothing to reply to.
That being said, Vignesh's note in January about a now-failing test is
relevant activity, and I've just today responded to that, so I'm
changing the status back from Waiting on Author to Needs Review.
Regards,
James Coleman
Re: Parallelize correlated subqueries that execute within each worker
James Coleman <jtc331@gmail.com> writes:
I've finally had a chance to look at this, and I don't believe there's
any real failure here, merely drift of how the planner works on master
resulting in this query now being eligible for a different plan shape.
I was a bit wary at first because the changing test query is one I'd
previously referenced in [1] as likely exposing the bug I'd fixed
where params where being used across worker boundaries. However
looking at the diff in the patch at that point (v10) that particular
test query formed a different plan shape (there were two gather nodes
being created, and params crossing between them).
But in the current revision of master with the current patch applied
that's no longer true: we have a Gather node, and the Subplan using
the param is properly under that Gather node, and the param should be
being both generated and consumed within the same worker process.
Hmm ... so the question this raises for me is: was that test intended
to verify behavior of params being passed across workers? If so,
haven't you broken the point of the test? This doesn't mean that
your code change is wrong; but I think maybe you need to find a way
to modify that test case so that it still tests what it's meant to.
This is a common hazard when changing the planner's behavior.
regards, tom lane
Re: Parallelize correlated subqueries that execute within each worker
On Tue, Jan 30, 2024 at 10:34 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
James Coleman <jtc331@gmail.com> writes:
I've finally had a chance to look at this, and I don't believe there's
any real failure here, merely drift of how the planner works on master
resulting in this query now being eligible for a different plan shape.I was a bit wary at first because the changing test query is one I'd
previously referenced in [1] as likely exposing the bug I'd fixed
where params where being used across worker boundaries. However
looking at the diff in the patch at that point (v10) that particular
test query formed a different plan shape (there were two gather nodes
being created, and params crossing between them).But in the current revision of master with the current patch applied
that's no longer true: we have a Gather node, and the Subplan using
the param is properly under that Gather node, and the param should be
being both generated and consumed within the same worker process.Hmm ... so the question this raises for me is: was that test intended
to verify behavior of params being passed across workers? If so,
haven't you broken the point of the test? This doesn't mean that
your code change is wrong; but I think maybe you need to find a way
to modify that test case so that it still tests what it's meant to.
This is a common hazard when changing the planner's behavior.
I'd been thinking it was covered by another test I'd added in 0001,
but looking at it again that test doesn't exercise parallel append
(though it does exercise a different case of cross-worker param
usage), so I'll add another test for the parallel append behavior.
Regards,
James Coleman
Re: Parallelize correlated subqueries that execute within each worker
On Tue, Jan 30, 2024 at 9:56 PM James Coleman <jtc331@gmail.com> wrote:
I don't follow the "Idle since July" since it just hasn't received
review since then, so there's been nothing to reply to.
It wasn't clear to me if you thought that the patch was ready for
review since July, or if it was waiting on you since July. Those are
quite different, IMV.
That being said, Vignesh's note in January about a now-failing test is
relevant activity, and I've just today responded to that, so I'm
changing the status back from Waiting on Author to Needs Review.
Sounds good.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: Parallelize correlated subqueries that execute within each worker
On Wed, Jan 31, 2024 at 3:18 PM Robert Haas <robertmhaas@gmail.com> wrote:
On Tue, Jan 30, 2024 at 9:56 PM James Coleman <jtc331@gmail.com> wrote:
I don't follow the "Idle since July" since it just hasn't received
review since then, so there's been nothing to reply to.It wasn't clear to me if you thought that the patch was ready for
review since July, or if it was waiting on you since July. Those are
quite different, IMV.
Agreed they're very different. I'd thought it was actually in "Needs
review" and with no outstanding questions on the thread since July,
but maybe I'm missing something -- I've definitely misunderstood CF
app status before, but usually that's been in the other direction
(forgetting to mark it back to Needs Review after responding to a
Waiting on Author.
Regards,
James Coleman
Re: Parallelize correlated subqueries that execute within each worker
Hi,
I was going through the "needs review" patches with no recent messages,
trying to figure out what is needed to move them forward, and this one
caught my eye because I commented on it before. And it's also a bit sad
example, because it started in 2021 and is moving at glacier speed :-(
I read through the thread, to understand how the design changed over
time, and I like the current approach (proposed by Robert) much more
than the initial idea of adding new flag next to parallel_safe etc.
And in general the patch looks reasonably simple and clean, but my
knowledge of PARAM intricacies is pretty much nil, so I'm hesitant to
claim the patch is correct. And I'm not sure what exactly needs to
happen to validate the approach :-(
The regression tests currently fail, due to a different plan for one of
the new queries in select_parallel. I guess it's due to some committed
patch, and it looks like a sensible change, but I haven't looked closely.
Also, I do get this warning when building with GCC 12.2.0 on Debian:
clauses.c: In function ‘max_parallel_hazard_walker’:
clauses.c:961:49: warning: ‘save_safe_param_ids’ may be used
uninitialized [-Wmaybe-uninitialized]
961 | context->safe_param_ids =
save_safe_param_ids;
|
~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~
clauses.c:943:29: note: ‘save_safe_param_ids’ was declared here
943 | List *save_safe_param_ids;
| ^~~~~~~~~~~~~~~~~~~
It's harmless, the compiler simply does not realize the two blocks
(where save_safe_param_ids is set and used) have exactly the same if
conditions, but it's a bit confusing for people too.
I was wondering if this could affect some queries in TPC-H, but the only
query affected seems to be Q2 - where it helps, cutting the time in
half, but Q2 is generally pretty fast / the expensive part was already
parallelized quite well (i.e. the correlated subquery is fairly cheap).
However, it's not difficult to construct a query where this helps a lot.
If the correlated subquery does something expensive (e.g. aggregation of
non-trivial amounts of data), this would help. So I wonder if e.g.
TPC-DS would benefit from this more ...
A couple review comments about the code:
1) new fields in max_parallel_hazard_context should have comments:
+ bool check_params;
+ Bitmapset **required_params;
2) Do we need both is_parallel_safe and is_parallel_safe_with_params?
ISTM the main difference is the for() loop, so why not add an extra
parameter to is_parallel_safe() and skip that loop if it's null? Or, if
we worry about external code, keep is_parallel_safe_with_params() and
define is_parallel_safe() as is_parallel_safe_with_params(...,NULL)?
3) Isn't it a bit weird that is_parallel_safe_with_params() actually
sets check_params=false, which seems like it doesn't actually check
parameters? I'm a bit confused / unsure if this is a bug or how it
actually checks parameters. If it's correct, it may need a comment.
4) The only place setting check_params is max_parallel_hazard, which is
called only for the whole Query from standard_planner(). But it does not
set required_params - can't this be an issue if the pointer happens to
be random garbage?
5) It probably needs a pgindent run, there's a bunch of rather long
lines and those are likely to get wrapped and look weird.
6) Is the comment in max_parallel_hazard_walker() still accurate? It
talks about PARAM_EXTERN and PARAM_EXEC, but the patch removes the
PARAM_EXTERN check entirely. So maybe the comment needs updating?
7) I don't like the very complex if condition very much, it's hard to
understand. I'd split that into two separate conditions, and add a short
comment for each of them. I.e. instead of:
if (param->paramkind != PARAM_EXEC || !(context->check_params ||
context->required_params != NULL))
return false;
I'd do
/* ... comment ... */
if (param->paramkind != PARAM_EXEC)
return false;
/* ... comment ... */
if (!(context->check_params || context->required_params != NULL))
return false;
or something like that.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: Parallelize correlated subqueries that execute within each worker
Hi James:
Very nice to find this topic, I am recently working on this topic [1]/messages/by-id/871pqzm5wj.fsf@163.com as
well without finding this topic has been discussed before. I just go
through this thread and find it goes with a different direction with mine.
would you mind to check my soluation to see is there any case I can't
cover? I suggested this because my soluation should be much easier than
yours. But I'm not suprised to know I'm miss some obvious keypoint.
One of the queries in in incremental_sort changed plans a little bit:
explain (costs off) select distinct
unique1,
(select t.unique1 from tenk1 where tenk1.unique1 = t.unique1)
from tenk1 t, generate_series(1, 1000);switched from
Unique (cost=18582710.41..18747375.21 rows=10000 width=8)
-> Gather Merge (cost=18582710.41..18697375.21 rows=10000000 ...)
Workers Planned: 2
-> Sort (cost=18582710.39..18593127.06 rows=4166667 ...)
Sort Key: t.unique1, ((SubPlan 1))
...to
Unique (cost=18582710.41..18614268.91 rows=10000 ...)
-> Gather Merge (cost=18582710.41..18614168.91 rows=20000 ...)
Workers Planned: 2
-> Unique (cost=18582710.39..18613960.39 rows=10000 ...)
-> Sort (cost=18582710.39..18593127.06 ...)
Sort Key: t.unique1, ((SubPlan 1))
...which probably makes sense, as the cost estimate decreases a bit.
Off the cuff that seems fine. I'll read it over again when I send the
updated series.
I had a detailed explaination for this plan change in [1]/messages/by-id/871pqzm5wj.fsf@163.com and I think
this could be amazing gain no matter which way we go finally.
[1]: /messages/by-id/871pqzm5wj.fsf@163.com
--
Best Regards
Andy Fan