allowing extensions to control planner behavior
I'm somewhat expecting to be flamed to a well-done crisp for saying
this, but I think we need better ways for extensions to control the
behavior of PostgreSQL's query planner. I know of two major reasons
why somebody might want to do this. First, you might want to do
something like what pg_hint_plan does, where it essentially implements
Oracle-style hints that can be either inline or stored in a side table
and automatically applied to queries.[1]https://github.com/ossc-db/pg_hint_plan In addition to supporting
Oracle-style hints, it also supports some other kinds of hints so that
you can, for example, try to fix broken cardinality estimates. Second,
you might want to convince the planner to keep producing the same kind
of plan that it produced previously. I believe this is what Amazon's
query plan management feature[2]https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraPostgreSQL.Optimize.html does, although since it is closed
source and I don't work at Amazon maybe it's actually implemented
completely differently. Regardless of what Amazon did in this case,
plan stability is a feature people want. Just trying to keep using the
same plan data structure forever doesn't seem like a good strategy,
because for example it would be fragile in the case of any DDL
changes, like dropping and recreating an index, or dropping or adding
a column. But you might want conceptually the same plan. Although it's
not frequently admitted on this mailing list, unexpected plan changes
are a frequent cause of sudden database outages, and wanting to
prevent that is a legitimate thing for a user to try to do. Naturally,
there is a risk that you might in so doing also prevent plan changes
that would have dramatically improved performance, or stick with a
plan long after you've outgrown it, but that doesn't stop people from
wanting it, or other databases (or proprietary forks of this database)
from offering it, and I don't think it should.
We have some hooks right now that offer a few options in this area,
but there are problems. The hook that I believe to be closest to the
right thing is this one:
/*
* Allow a plugin to editorialize on the set of Paths for this base
* relation. It could add new paths (such as CustomPaths) by calling
* add_path(), or add_partial_path() if parallel aware. It could also
* delete or modify paths added by the core code.
*/
if (set_rel_pathlist_hook)
(*set_rel_pathlist_hook) (root, rel, rti, rte);
Unfortunately, the part about the hook having the freedom to delete
paths isn't really true. Perhaps technically you can delete a path
that you don't want to be chosen, but any paths that were dominated by
the path you deleted have already been thrown away and it's too late
to get them back. You can modify paths if you don't want to change
their costs, but if you change their costs then you have the same
problem: the contents of the pathlist at the time that you see it are
determined by the costs that each path had when it was initially
added, and it's really too late to editorialize on that. So all you
can really do here in practice is add new paths.
set_join_pathlist_hook, which applies to joinrels, is similarly
limited. appendrels don't even have an equivalent of this hook.
So, how could we do better?
I think there are two basic approaches that are possible here. If
someone sees a third option, let me know. First, we could allow users
to hook add_path() and add_partial_path(). That certainly provides the
flexibility on paper to accept or reject whatever paths you do or do
not want. However, I don't find this approach very appealing. One
problem is that it's likely to be fairly expensive, because add_path()
gets called A LOT. A second problem is that you don't actually get an
awful lot of context: I think anybody writing a hook would have to
write code to basically analyze each proposed path and figure out why
it was getting added and then decide what to do. In some cases that
might be fine, because for example accepting or rejecting paths based
on path type seems fairly straightforward with this approach, but as
soon as you want to do anything more complicated than that it starts
to seem difficult. If, for example, you want relation R1 to be the
driving table for the whole query plan, you're going to have to
determine whether or not that is the case for every single candidate
(partial) path that someone hands you, so you're going to end up
making that decision a whole lot of times. It doesn't sound
particularly fun. Third, even if you are doing something really simple
like trying to reject mergejoins, you've already lost the opportunity
to skip a bunch of work. If you had known when you started planning
the joinrel that you didn't care about mergejoins, you could have
skipped looking for merge-joinable clauses. Overall, while I wouldn't
be completely against further exploration of this option, I suspect
it's pretty hard to do anything useful with it.
The other possible approach is to allow extensions to feed some
information into the planner before path generation and let that
influence which paths are generated. This is essentially what
pg_hint_plan is doing: it implements plan type hints by arranging to
flip the various enable_* GUCs on and off during the planning of
various rels. That's clever but ugly, and it ends up duplicating
substantial chunks of planner code due to the inadequacy of the
existing hooks. With some refactoring and some additional hooks, we
could make this much less ugly. But that gets at what I believe to be
the core difficulty of this approach, which is that the core planner
code needs to be somewhat aware of and on board with what the user or
the extension is trying to do. If an extension wants to force the join
order, that is likely to require different scaffolding than if it
wants to force the join methods which is again different from if a
hook wants to bias the query planner towards or against particular
indexes. Putting in hooks or other infrastructure that allows an
extension to control a particular aspect of planner behavior is to
some extent an endorsement of controlling the planner behavior in that
particular way. Since any amount of allowing the user to control the
planner tends to be controversial around here, that opens up the
spectre of putting a whole lot of effort into arguing about which
things extensions should be allowed to do, getting most of the patches
rejected, and ending up with nothing that's actually useful.
But on the other hand, it's not like we have to design everything in a
greenfield. Other database systems have provided in-core, user-facing
features to control the planner for decades, and we can look at those
offerings -- and existing offerings in the PG space -- as we try to
judge whether a particular use case is totally insane. I am not here
to argue that everything that every system has done is completely
perfect and without notable flaws, but our own system has its own
share of flaws, and the fact that you can do very little when a
previously unproblematic query starts suddenly producing a bad plan is
definitely one of them. I believe we are long past the point where we
can simply hold our breath and pretend like there's no issue here. At
the very least, allowing extensions to control scan methods (including
choice of indexes), join methods, and join order (including which
table ends up on which side of a given join) and similar things for
aggregates and appendrels seems to me like it ought to be table
stakes. And those extensions shouldn't have to duplicate large chunks
of code or resort to gross hacks to do it. Eventually, maybe we'll
even want to have directly user-facing features to do some of this
stuff (in query hints, out of query hints, or whatever) but I think
opening the door up to extensions doing it is a good first step,
because (1) that allows different extensions to do different things
without taking a position on what the One Right Thing To Do is and (2)
if it becomes clear that something improvident has been done, it is a
lot easier to back out a hook or some C API change than it is to
back-out a user-visible feature. Or maybe we'll never want to expose a
user-visible feature here, but it can still be useful to enable
extensions.
The attached patch, briefly mentioned above, essentially converts the
enable_* GUCs into RelOptInfo properties where the defaults are set by
the corresponding GUCs. The idea is that a hook could then change this
on a per-RelOptInfo basis before path generation happens. For
baserels, I believe that could be done from get_relation_info_hook for
baserels, and we could introduce something similar for other kinds of
rels. I don't think this is in any way the perfect approach. On the
one hand, it doesn't give you all the kinds of control over path
generation that you might want. On the other hand, the more I look at
what our enable_* GUCs actually do, the less impressed I am. IMHO,
things like enable_hashjoin make a lot of sense, but enable_sort seems
like it just controls an absolutely random smattering of behaviors in
a way that seems to me to have very little to recommend it, and I've
complained elsewhere about how enable_indexscan and
enable_indexonlyscan are really quite odd when you look at how they're
implemented. Still, this seemed like a fairly easy thing to do as a
way of demonstrating the kind of thing that we could do to provide
extensions with more control over planner behavior, and I believe it
would be concretely useful to pg_hint_plan in particular. But all that
said, as much as anything, I want to get some feedback on what
approaches and trade-offs people think might be acceptable here,
because there's not much point in me spending a bunch of time writing
code that everyone (or a critical mass of people) are going to hate.
Thanks,
--
Robert Haas
EDB: http://www.enterprisedb.com
[1]: https://github.com/ossc-db/pg_hint_plan
[2]: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraPostgreSQL.Optimize.html
Attachments:
v1-0001-Convert-enable_-GUCs-into-per-RelOptInfo-values-w.patchapplication/octet-stream; name=v1-0001-Convert-enable_-GUCs-into-per-RelOptInfo-values-w.patchDownload
From baee6cd9e575728650813152af0d4d2d9c96674f Mon Sep 17 00:00:00 2001
From: Robert Haas <rhaas@postgresql.org>
Date: Mon, 26 Aug 2024 12:27:08 -0400
Subject: [PATCH v1] Convert enable_* GUCs into per-RelOptInfo values with GUCs
setting defaults.
---
contrib/postgres_fdw/postgres_fdw.c | 5 +-
src/backend/optimizer/path/allpaths.c | 15 ++--
src/backend/optimizer/path/costsize.c | 107 +++++++++++++++++-------
src/backend/optimizer/path/indxpath.c | 2 +-
src/backend/optimizer/path/joinpath.c | 53 ++++++------
src/backend/optimizer/path/pathkeys.c | 3 +-
src/backend/optimizer/path/tidpath.c | 11 +--
src/backend/optimizer/plan/createplan.c | 26 +++---
src/backend/optimizer/plan/planner.c | 58 ++++++++-----
src/backend/optimizer/plan/subselect.c | 9 +-
src/backend/optimizer/prep/prepunion.c | 20 +++--
src/backend/optimizer/util/pathnode.c | 21 ++---
src/backend/optimizer/util/plancat.c | 3 +-
src/backend/optimizer/util/relnode.c | 6 +-
src/backend/utils/misc/guc_tables.c | 37 ++++----
src/include/nodes/pathnodes.h | 29 +++++++
src/include/optimizer/cost.h | 29 ++++++-
src/include/optimizer/planmain.h | 2 +-
18 files changed, 288 insertions(+), 148 deletions(-)
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index adc62576d1..1df4ddf268 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -439,6 +439,7 @@ static void get_remote_estimate(const char *sql,
Cost *startup_cost,
Cost *total_cost);
static void adjust_foreign_grouping_path_cost(PlannerInfo *root,
+ RelOptInfo *rel,
List *pathkeys,
double retrieved_rows,
double width,
@@ -3489,7 +3490,7 @@ estimate_path_cost_size(PlannerInfo *root,
{
Assert(foreignrel->reloptkind == RELOPT_UPPER_REL &&
fpinfo->stage == UPPERREL_GROUP_AGG);
- adjust_foreign_grouping_path_cost(root, pathkeys,
+ adjust_foreign_grouping_path_cost(root, foreignrel, pathkeys,
retrieved_rows, width,
fpextra->limit_tuples,
&disabled_nodes,
@@ -3644,6 +3645,7 @@ get_remote_estimate(const char *sql, PGconn *conn,
*/
static void
adjust_foreign_grouping_path_cost(PlannerInfo *root,
+ RelOptInfo *rel,
List *pathkeys,
double retrieved_rows,
double width,
@@ -3667,6 +3669,7 @@ adjust_foreign_grouping_path_cost(PlannerInfo *root,
cost_sort(&sort_path,
root,
+ rel,
pathkeys,
0,
*p_startup_cost + *p_run_cost,
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 057b4b79eb..8645244f84 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -970,7 +970,7 @@ set_append_rel_size(PlannerInfo *root, RelOptInfo *rel,
* flag; currently, we only consider partitionwise joins with the baserel
* if its targetlist doesn't contain a whole-row Var.
*/
- if (enable_partitionwise_join &&
+ if (REL_CAN_USE_PATH(rel, PartitionwiseJoin) &&
rel->reloptkind == RELOPT_BASEREL &&
rte->relkind == RELKIND_PARTITIONED_TABLE &&
bms_is_empty(rel->attr_needed[InvalidAttrNumber - rel->min_attr]))
@@ -1325,7 +1325,8 @@ add_paths_to_append_rel(PlannerInfo *root, RelOptInfo *rel,
double partial_rows = -1;
/* If appropriate, consider parallel append */
- pa_subpaths_valid = enable_parallel_append && rel->consider_parallel;
+ pa_subpaths_valid = REL_CAN_USE_PATH(rel, ParallelAppend)
+ && rel->consider_parallel;
/*
* For every non-dummy child, remember the cheapest path. Also, identify
@@ -1535,7 +1536,7 @@ add_paths_to_append_rel(PlannerInfo *root, RelOptInfo *rel,
* partitions vs. an unpartitioned table with the same data, so the
* use of some kind of log-scaling here seems to make some sense.
*/
- if (enable_parallel_append)
+ if (REL_CAN_USE_PATH(rel, ParallelAppend))
{
parallel_workers = Max(parallel_workers,
pg_leftmost_one_pos32(list_length(live_childrels)) + 1);
@@ -1547,7 +1548,7 @@ add_paths_to_append_rel(PlannerInfo *root, RelOptInfo *rel,
/* Generate a partial append path. */
appendpath = create_append_path(root, rel, NIL, partial_subpaths,
NIL, NULL, parallel_workers,
- enable_parallel_append,
+ REL_CAN_USE_PATH(rel, ParallelAppend),
-1);
/*
@@ -3259,7 +3260,8 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
* input path).
*/
if (subpath != cheapest_partial_path &&
- (presorted_keys == 0 || !enable_incremental_sort))
+ (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(rel, IncrementalSort)))
continue;
/*
@@ -3274,7 +3276,8 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
* output. Here we add an explicit sort to match the useful
* ordering.
*/
- if (presorted_keys == 0 || !enable_incremental_sort)
+ if (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(rel, IncrementalSort))
{
subpath = (Path *) create_sort_path(root,
rel,
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index e1523d15df..3dfa22fdcd 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -163,6 +163,7 @@ bool enable_parallel_hash = true;
bool enable_partition_pruning = true;
bool enable_presorted_aggregate = true;
bool enable_async_append = true;
+uint32 default_path_type_mask = PathTypeMaskAll;
typedef struct
{
@@ -283,6 +284,38 @@ clamp_cardinality_to_long(Cardinality x)
return (x < (double) LONG_MAX) ? (long) x : LONG_MAX;
}
+/*
+ * Define assign hooks for each enable_<whatever> GUC that affects
+ * default_path_type_mask. These are all basically identical, so we use
+ * a templating macro to define them.
+ */
+#define define_assign_hook(gucname, type) \
+ void \
+ gucname ## _assign_hook(bool newval, void *extra) \
+ { \
+ if (newval) \
+ default_path_type_mask |= PathType ## type; \
+ else \
+ default_path_type_mask &= ~(PathType ## type); \
+ }
+define_assign_hook(enable_bitmapscan, BitmapScan)
+define_assign_hook(enable_gathermerge, GatherMerge)
+define_assign_hook(enable_hashagg, HashAgg)
+define_assign_hook(enable_hashjoin, HashJoin)
+define_assign_hook(enable_incremental_sort, IncrementalSort)
+define_assign_hook(enable_indexscan, IndexScan)
+define_assign_hook(enable_indexonlyscan, IndexOnlyScan)
+define_assign_hook(enable_material, Material)
+define_assign_hook(enable_memoize, Memoize)
+define_assign_hook(enable_mergejoin, MergeJoin)
+define_assign_hook(enable_nestloop, NestLoop)
+define_assign_hook(enable_parallel_append, ParallelAppend)
+define_assign_hook(enable_parallel_hash, ParallelHash)
+define_assign_hook(enable_partitionwise_join, PartitionwiseJoin)
+define_assign_hook(enable_partitionwise_aggregate, PartitionwiseAggregate)
+define_assign_hook(enable_seqscan, SeqScan)
+define_assign_hook(enable_sort, Sort)
+define_assign_hook(enable_tidscan, TIDScan)
/*
* cost_seqscan
@@ -354,7 +387,7 @@ cost_seqscan(Path *path, PlannerInfo *root,
path->rows = clamp_row_est(path->rows / parallel_divisor);
}
- path->disabled_nodes = enable_seqscan ? 0 : 1;
+ path->disabled_nodes = REL_CAN_USE_PATH(baserel, SeqScan) ? 0 : 1;
path->startup_cost = startup_cost;
path->total_cost = startup_cost + cpu_run_cost + disk_run_cost;
}
@@ -533,7 +566,7 @@ cost_gather_merge(GatherMergePath *path, PlannerInfo *root,
run_cost += parallel_tuple_cost * path->path.rows * 1.05;
path->path.disabled_nodes = input_disabled_nodes
- + (enable_gathermerge ? 0 : 1);
+ + (REL_CAN_USE_PATH(rel, GatherMerge) ? 0 : 1);
path->path.startup_cost = startup_cost + input_startup_cost;
path->path.total_cost = (startup_cost + run_cost + input_total_cost);
}
@@ -615,7 +648,8 @@ cost_index(IndexPath *path, PlannerInfo *root, double loop_count,
}
/* we don't need to check enable_indexonlyscan; indxpath.c does that */
- path->path.disabled_nodes = enable_indexscan ? 0 : 1;
+ path->path.disabled_nodes =
+ REL_CAN_USE_PATH(baserel, IndexScan) ? 0 : 1;
/*
* Call index-access-method-specific code to estimate the processing cost
@@ -1109,7 +1143,7 @@ cost_bitmap_heap_scan(Path *path, PlannerInfo *root, RelOptInfo *baserel,
startup_cost += path->pathtarget->cost.startup;
run_cost += path->pathtarget->cost.per_tuple * path->rows;
- path->disabled_nodes = enable_bitmapscan ? 0 : 1;
+ path->disabled_nodes = REL_CAN_USE_PATH(baserel, BitmapScan) ? 0 : 1;
path->startup_cost = startup_cost;
path->total_cost = startup_cost + run_cost;
}
@@ -1287,10 +1321,11 @@ cost_tidscan(Path *path, PlannerInfo *root,
/*
* We must use a TID scan for CurrentOfExpr; in any other case, we
- * should be generating a TID scan only if enable_tidscan=true. Also,
- * if CurrentOfExpr is the qual, there should be only one.
+ * should be generating a TID scan only if this TID scans are enabled
+ * for this rel. Also, if CurrentOfExpr is the qual, there should be
+ * only one.
*/
- Assert(enable_tidscan || IsA(qual, CurrentOfExpr));
+ Assert(REL_CAN_USE_PATH(baserel, TIDScan) || IsA(qual, CurrentOfExpr));
Assert(list_length(tidquals) == 1 || !IsA(qual, CurrentOfExpr));
if (IsA(qual, ScalarArrayOpExpr))
@@ -1342,8 +1377,8 @@ cost_tidscan(Path *path, PlannerInfo *root,
/*
* There are assertions above verifying that we only reach this function
- * either when enable_tidscan=true or when the TID scan is the only legal
- * path, so it's safe to set disabled_nodes to zero here.
+ * either when TID scans are enabled for this rel or when a TID scan is
+ * the only legal path, so it's safe to set disabled_nodes to zero here.
*/
path->disabled_nodes = 0;
path->startup_cost = startup_cost;
@@ -1438,8 +1473,8 @@ cost_tidrangescan(Path *path, PlannerInfo *root,
startup_cost += path->pathtarget->cost.startup;
run_cost += path->pathtarget->cost.per_tuple * path->rows;
- /* we should not generate this path type when enable_tidscan=false */
- Assert(enable_tidscan);
+ /* we should not generate this path type when TID scans are disabled */
+ Assert(REL_CAN_USE_PATH(baserel, TIDScan));
path->disabled_nodes = 0;
path->startup_cost = startup_cost;
path->total_cost = startup_cost + run_cost;
@@ -2120,8 +2155,11 @@ cost_incremental_sort(Path *path,
path->rows = input_tuples;
- /* should not generate these paths when enable_incremental_sort=false */
- Assert(enable_incremental_sort);
+ /*
+ * If incremental sort is not enabled here, we should not have generated a
+ * path of this type.
+ */
+ Assert(REL_CAN_USE_PATH(path->parent, IncrementalSort));
path->disabled_nodes = input_disabled_nodes;
path->startup_cost = startup_cost;
@@ -2141,7 +2179,7 @@ cost_incremental_sort(Path *path,
* of sort keys, which all callers *could* supply.)
*/
void
-cost_sort(Path *path, PlannerInfo *root,
+cost_sort(Path *path, PlannerInfo *root, RelOptInfo *rel,
List *pathkeys, int input_disabled_nodes,
Cost input_cost, double tuples, int width,
Cost comparison_cost, int sort_mem,
@@ -2159,7 +2197,8 @@ cost_sort(Path *path, PlannerInfo *root,
startup_cost += input_cost;
path->rows = tuples;
- path->disabled_nodes = input_disabled_nodes + (enable_sort ? 0 : 1);
+ path->disabled_nodes = input_disabled_nodes +
+ (REL_CAN_USE_PATH(rel, Sort) ? 0 : 1);
path->startup_cost = startup_cost;
path->total_cost = startup_cost + run_cost;
}
@@ -2321,6 +2360,7 @@ cost_append(AppendPath *apath)
*/
cost_sort(&sort_path,
NULL, /* doesn't currently need root */
+ apath->path.parent,
pathkeys,
subpath->disabled_nodes,
subpath->total_cost,
@@ -2480,7 +2520,7 @@ cost_merge_append(Path *path, PlannerInfo *root,
* occur only on rescan, which is estimated in cost_rescan.
*/
void
-cost_material(Path *path,
+cost_material(Path *path, RelOptInfo *rel,
int input_disabled_nodes,
Cost input_startup_cost, Cost input_total_cost,
double tuples, int width)
@@ -2519,7 +2559,8 @@ cost_material(Path *path,
run_cost += seq_page_cost * npages;
}
- path->disabled_nodes = input_disabled_nodes + (enable_material ? 0 : 1);
+ path->disabled_nodes = input_disabled_nodes +
+ (REL_CAN_USE_PATH(rel, Material) ? 0 : 1);
path->startup_cost = startup_cost;
path->total_cost = startup_cost + run_cost;
}
@@ -2679,7 +2720,7 @@ cost_memoize_rescan(PlannerInfo *root, MemoizePath *mpath,
* are for appropriately-sorted input.
*/
void
-cost_agg(Path *path, PlannerInfo *root,
+cost_agg(Path *path, PlannerInfo *root, RelOptInfo *rel,
AggStrategy aggstrategy, const AggClauseCosts *aggcosts,
int numGroupCols, double numGroups,
List *quals,
@@ -2738,7 +2779,7 @@ cost_agg(Path *path, PlannerInfo *root,
/* Here we are able to deliver output on-the-fly */
startup_cost = input_startup_cost;
total_cost = input_total_cost;
- if (aggstrategy == AGG_MIXED && !enable_hashagg)
+ if (aggstrategy == AGG_MIXED && !REL_CAN_USE_PATH(rel, HashAgg))
++disabled_nodes;
/* calcs phrased this way to match HASHED case, see note above */
total_cost += aggcosts->transCost.startup;
@@ -2753,7 +2794,7 @@ cost_agg(Path *path, PlannerInfo *root,
{
/* must be AGG_HASHED */
startup_cost = input_total_cost;
- if (!enable_hashagg)
+ if (!REL_CAN_USE_PATH(rel, HashAgg))
++disabled_nodes;
startup_cost += aggcosts->transCost.startup;
startup_cost += aggcosts->transCost.per_tuple * input_tuples;
@@ -3266,7 +3307,7 @@ cost_group(Path *path, PlannerInfo *root,
*/
void
initial_cost_nestloop(PlannerInfo *root, JoinCostWorkspace *workspace,
- JoinType jointype,
+ RelOptInfo *joinrel, JoinType jointype,
Path *outer_path, Path *inner_path,
JoinPathExtraData *extra)
{
@@ -3280,7 +3321,7 @@ initial_cost_nestloop(PlannerInfo *root, JoinCostWorkspace *workspace,
Cost inner_rescan_run_cost;
/* Count up disabled nodes. */
- disabled_nodes = enable_nestloop ? 0 : 1;
+ disabled_nodes = REL_CAN_USE_PATH(joinrel, NestLoop) ? 0 : 1;
disabled_nodes += inner_path->disabled_nodes;
disabled_nodes += outer_path->disabled_nodes;
@@ -3549,7 +3590,7 @@ final_cost_nestloop(PlannerInfo *root, NestPath *path,
*/
void
initial_cost_mergejoin(PlannerInfo *root, JoinCostWorkspace *workspace,
- JoinType jointype,
+ RelOptInfo *joinrel, JoinType jointype,
List *mergeclauses,
Path *outer_path, Path *inner_path,
List *outersortkeys, List *innersortkeys,
@@ -3676,7 +3717,7 @@ initial_cost_mergejoin(PlannerInfo *root, JoinCostWorkspace *workspace,
Assert(outerstartsel <= outerendsel);
Assert(innerstartsel <= innerendsel);
- disabled_nodes = enable_mergejoin ? 0 : 1;
+ disabled_nodes = REL_CAN_USE_PATH(joinrel, MergeJoin) ? 0 : 1;
/* cost of source data */
@@ -3684,6 +3725,7 @@ initial_cost_mergejoin(PlannerInfo *root, JoinCostWorkspace *workspace,
{
cost_sort(&sort_path,
root,
+ outer_path->parent,
outersortkeys,
outer_path->disabled_nodes,
outer_path->total_cost,
@@ -3713,6 +3755,7 @@ initial_cost_mergejoin(PlannerInfo *root, JoinCostWorkspace *workspace,
{
cost_sort(&sort_path,
root,
+ inner_path->parent,
innersortkeys,
inner_path->disabled_nodes,
inner_path->total_cost,
@@ -3793,6 +3836,7 @@ final_cost_mergejoin(PlannerInfo *root, MergePath *path,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra)
{
+ RelOptInfo *joinrel = path->jpath.path.parent;
Path *outer_path = path->jpath.outerjoinpath;
Path *inner_path = path->jpath.innerjoinpath;
double inner_path_rows = inner_path->rows;
@@ -3946,7 +3990,8 @@ final_cost_mergejoin(PlannerInfo *root, MergePath *path,
* Prefer materializing if it looks cheaper, unless the user has asked to
* suppress materialization.
*/
- else if (enable_material && mat_inner_cost < bare_inner_cost)
+ else if (REL_CAN_USE_PATH(joinrel, Material) &&
+ mat_inner_cost < bare_inner_cost)
path->materialize_inner = true;
/*
@@ -3961,7 +4006,7 @@ final_cost_mergejoin(PlannerInfo *root, MergePath *path,
* selected as the input of a mergejoin, and they don't support
* mark/restore at present.
*
- * We don't test the value of enable_material here, because
+ * We don't test whether a MaterialPath is allowed here, because
* materialization is required for correctness in this case, and turning
* it off does not entitle us to deliver an invalid plan.
*/
@@ -3977,10 +4022,10 @@ final_cost_mergejoin(PlannerInfo *root, MergePath *path,
* though.
*
* Since materialization is a performance optimization in this case,
- * rather than necessary for correctness, we skip it if enable_material is
- * off.
+ * rather than necessary for correctness, we skip it if MaterialPath is
+ * not allowed here.
*/
- else if (enable_material && innersortkeys != NIL &&
+ else if (REL_CAN_USE_PATH(joinrel, Material) && innersortkeys != NIL &&
relation_byte_size(inner_path_rows,
inner_path->pathtarget->width) >
(work_mem * 1024L))
@@ -4113,7 +4158,7 @@ cached_scansel(PlannerInfo *root, RestrictInfo *rinfo, PathKey *pathkey)
*/
void
initial_cost_hashjoin(PlannerInfo *root, JoinCostWorkspace *workspace,
- JoinType jointype,
+ RelOptInfo *joinrel, JoinType jointype,
List *hashclauses,
Path *outer_path, Path *inner_path,
JoinPathExtraData *extra,
@@ -4132,7 +4177,7 @@ initial_cost_hashjoin(PlannerInfo *root, JoinCostWorkspace *workspace,
size_t space_allowed; /* unused */
/* Count up disabled nodes. */
- disabled_nodes = enable_hashjoin ? 0 : 1;
+ disabled_nodes = REL_CAN_USE_PATH(joinrel, HashJoin) ? 0 : 1;
disabled_nodes += inner_path->disabled_nodes;
disabled_nodes += outer_path->disabled_nodes;
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
index c0fcc7d78d..be183db4ce 100644
--- a/src/backend/optimizer/path/indxpath.c
+++ b/src/backend/optimizer/path/indxpath.c
@@ -1736,7 +1736,7 @@ check_index_only(RelOptInfo *rel, IndexOptInfo *index)
int i;
/* Index-only scans must be enabled */
- if (!enable_indexonlyscan)
+ if (!REL_CAN_USE_PATH(rel, IndexOnlyScan))
return false;
/*
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index b0e8c94dfc..5f0040cd1b 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -205,10 +205,10 @@ add_paths_to_joinrel(PlannerInfo *root,
/*
* Find potential mergejoin clauses. We can skip this if we are not
* interested in doing a mergejoin. However, mergejoin may be our only
- * way of implementing a full outer join, so override enable_mergejoin if
- * it's a full join.
+ * way of implementing a full outer join, so disregard the result of
+ * REL_CAN_USE_PATH() if it's a full join.
*/
- if (enable_mergejoin || jointype == JOIN_FULL)
+ if (REL_CAN_USE_PATH(joinrel, MergeJoin) || jointype == JOIN_FULL)
extra.mergeclause_list = select_mergejoin_clauses(root,
joinrel,
outerrel,
@@ -316,10 +316,11 @@ add_paths_to_joinrel(PlannerInfo *root,
/*
* 4. Consider paths where both outer and inner relations must be hashed
- * before being joined. As above, disregard enable_hashjoin for full
- * joins, because there may be no other alternative.
+ * before being joined. As above, the result of REL_CAN_USE_PATH()
+ * doesn't matter for full joins, because there may be no other
+ * alternative.
*/
- if (enable_hashjoin || jointype == JOIN_FULL)
+ if (REL_CAN_USE_PATH(joinrel, HashJoin) || jointype == JOIN_FULL)
hash_inner_and_outer(root, joinrel, outerrel, innerrel,
jointype, &extra);
@@ -672,7 +673,7 @@ extract_lateral_vars_from_PHVs(PlannerInfo *root, Relids innerrelids)
* we do not have a way to extract cache keys from joinrels.
*/
static Path *
-get_memoize_path(PlannerInfo *root, RelOptInfo *innerrel,
+get_memoize_path(PlannerInfo *root, RelOptInfo *joinrel, RelOptInfo *innerrel,
RelOptInfo *outerrel, Path *inner_path,
Path *outer_path, JoinType jointype,
JoinPathExtraData *extra)
@@ -684,7 +685,7 @@ get_memoize_path(PlannerInfo *root, RelOptInfo *innerrel,
List *ph_lateral_vars;
/* Obviously not if it's disabled */
- if (!enable_memoize)
+ if (!REL_CAN_USE_PATH(joinrel, Memoize))
return NULL;
/*
@@ -912,7 +913,7 @@ try_nestloop_path(PlannerInfo *root,
* The latter two steps are expensive enough to make this two-phase
* methodology worthwhile.
*/
- initial_cost_nestloop(root, &workspace, jointype,
+ initial_cost_nestloop(root, &workspace, joinrel, jointype,
outer_path, inner_path, extra);
if (add_path_precheck(joinrel, workspace.disabled_nodes,
@@ -997,7 +998,7 @@ try_partial_nestloop_path(PlannerInfo *root,
* Before creating a path, get a quick lower bound on what it is likely to
* cost. Bail out right away if it looks terrible.
*/
- initial_cost_nestloop(root, &workspace, jointype,
+ initial_cost_nestloop(root, &workspace, joinrel, jointype,
outer_path, inner_path, extra);
if (!add_partial_path_precheck(joinrel, workspace.disabled_nodes,
workspace.total_cost, pathkeys))
@@ -1092,7 +1093,7 @@ try_mergejoin_path(PlannerInfo *root,
/*
* See comments in try_nestloop_path().
*/
- initial_cost_mergejoin(root, &workspace, jointype, mergeclauses,
+ initial_cost_mergejoin(root, &workspace, joinrel, jointype, mergeclauses,
outer_path, inner_path,
outersortkeys, innersortkeys,
extra);
@@ -1164,7 +1165,7 @@ try_partial_mergejoin_path(PlannerInfo *root,
/*
* See comments in try_partial_nestloop_path().
*/
- initial_cost_mergejoin(root, &workspace, jointype, mergeclauses,
+ initial_cost_mergejoin(root, &workspace, joinrel, jointype, mergeclauses,
outer_path, inner_path,
outersortkeys, innersortkeys,
extra);
@@ -1236,7 +1237,7 @@ try_hashjoin_path(PlannerInfo *root,
* See comments in try_nestloop_path(). Also note that hashjoin paths
* never have any output pathkeys, per comments in create_hashjoin_path.
*/
- initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
+ initial_cost_hashjoin(root, &workspace, joinrel, jointype, hashclauses,
outer_path, inner_path, extra, false);
if (add_path_precheck(joinrel, workspace.disabled_nodes,
@@ -1298,7 +1299,7 @@ try_partial_hashjoin_path(PlannerInfo *root,
* Before creating a path, get a quick lower bound on what it is likely to
* cost. Bail out right away if it looks terrible.
*/
- initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
+ initial_cost_hashjoin(root, &workspace, joinrel, jointype, hashclauses,
outer_path, inner_path, extra, parallel_hash);
if (!add_partial_path_precheck(joinrel, workspace.disabled_nodes,
workspace.total_cost, NIL))
@@ -1899,10 +1900,11 @@ match_unsorted_outer(PlannerInfo *root,
{
/*
* Consider materializing the cheapest inner path, unless
- * enable_material is off or the path in question materializes its
- * output anyway.
+ * materialization is disabled or the path in question materializes
+ * its output anyway.
*/
- if (enable_material && inner_cheapest_total != NULL &&
+ if (REL_CAN_USE_PATH(innerrel, Material) &&
+ inner_cheapest_total != NULL &&
!ExecMaterializesOutput(inner_cheapest_total->pathtype))
matpath = (Path *)
create_material_path(innerrel, inner_cheapest_total);
@@ -1982,7 +1984,7 @@ match_unsorted_outer(PlannerInfo *root,
* Try generating a memoize path and see if that makes the
* nested loop any cheaper.
*/
- mpath = get_memoize_path(root, innerrel, outerrel,
+ mpath = get_memoize_path(root, joinrel, innerrel, outerrel,
innerpath, outerpath, jointype,
extra);
if (mpath != NULL)
@@ -2134,13 +2136,14 @@ consider_parallel_nestloop(PlannerInfo *root,
/*
* Consider materializing the cheapest inner path, unless: 1) we're doing
* JOIN_UNIQUE_INNER, because in this case we have to unique-ify the
- * cheapest inner path, 2) enable_material is off, 3) the cheapest inner
- * path is not parallel-safe, 4) the cheapest inner path is parameterized
- * by the outer rel, or 5) the cheapest inner path materializes its output
- * anyway.
+ * cheapest inner path, 2) MaterialPath is allowed for this rel, 3) the
+ * cheapest inner path is not parallel-safe, 4) the cheapest inner path is
+ * parameterized by the outer rel, or 5) the cheapest inner path
+ * materializes its output anyway.
*/
if (save_jointype != JOIN_UNIQUE_INNER &&
- enable_material && inner_cheapest_total->parallel_safe &&
+ REL_CAN_USE_PATH(joinrel, Material) &&
+ inner_cheapest_total->parallel_safe &&
!PATH_PARAM_BY_REL(inner_cheapest_total, outerrel) &&
!ExecMaterializesOutput(inner_cheapest_total->pathtype))
{
@@ -2198,7 +2201,7 @@ consider_parallel_nestloop(PlannerInfo *root,
* Try generating a memoize path and see if that makes the nested
* loop any cheaper.
*/
- mpath = get_memoize_path(root, innerrel, outerrel,
+ mpath = get_memoize_path(root, joinrel, innerrel, outerrel,
innerpath, outerpath, jointype,
extra);
if (mpath != NULL)
@@ -2416,7 +2419,7 @@ hash_inner_and_outer(PlannerInfo *root,
*/
if (innerrel->partial_pathlist != NIL &&
save_jointype != JOIN_UNIQUE_INNER &&
- enable_parallel_hash)
+ REL_CAN_USE_PATH(joinrel, ParallelHash))
{
cheapest_partial_inner =
(Path *) linitial(innerrel->partial_pathlist);
diff --git a/src/backend/optimizer/path/pathkeys.c b/src/backend/optimizer/path/pathkeys.c
index e25798972f..26a0f5c0c3 100644
--- a/src/backend/optimizer/path/pathkeys.c
+++ b/src/backend/optimizer/path/pathkeys.c
@@ -505,7 +505,8 @@ get_useful_group_keys_orderings(PlannerInfo *root, Path *path)
root->num_groupby_pathkeys);
if (n > 0 &&
- (enable_incremental_sort || n == root->num_groupby_pathkeys) &&
+ (REL_CAN_USE_PATH(path->parent, IncrementalSort) ||
+ n == root->num_groupby_pathkeys) &&
compare_pathkeys(pathkeys, root->group_pathkeys) != PATHKEYS_EQUAL)
{
info = makeNode(GroupByOrdering);
diff --git a/src/backend/optimizer/path/tidpath.c b/src/backend/optimizer/path/tidpath.c
index b0323b26ec..ed447916ec 100644
--- a/src/backend/optimizer/path/tidpath.c
+++ b/src/backend/optimizer/path/tidpath.c
@@ -505,13 +505,14 @@ create_tidscan_paths(PlannerInfo *root, RelOptInfo *rel)
* If any suitable quals exist in the rel's baserestrict list, generate a
* plain (unparameterized) TidPath with them.
*
- * We skip this when enable_tidscan = false, except when the qual is
- * CurrentOfExpr. In that case, a TID scan is the only correct path.
+ * We skip this when TID scans are disabled for this rel, except when
+ * the qual is CurrentOfExpr. In that case, a TID scan is the only
+ * correct path.
*/
tidquals = TidQualFromRestrictInfoList(root, rel->baserestrictinfo, rel,
&isCurrentOf);
- if (tidquals != NIL && (enable_tidscan || isCurrentOf))
+ if (tidquals != NIL && (REL_CAN_USE_PATH(rel, TIDScan) || isCurrentOf))
{
/*
* This path uses no join clauses, but it could still have required
@@ -532,8 +533,8 @@ create_tidscan_paths(PlannerInfo *root, RelOptInfo *rel)
return true;
}
- /* Skip the rest if TID scans are disabled. */
- if (!enable_tidscan)
+ /* Skip the rest if TID scans are disabled for this rel. */
+ if (!REL_CAN_USE_PATH(rel, TIDScan))
return false;
/*
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 8e0e5977a9..3f3770bfa7 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -177,8 +177,8 @@ static List *get_switched_clauses(List *clauses, Relids outerrelids);
static List *order_qual_clauses(PlannerInfo *root, List *clauses);
static void copy_generic_path_info(Plan *dest, Path *src);
static void copy_plan_costsize(Plan *dest, Plan *src);
-static void label_sort_with_costsize(PlannerInfo *root, Sort *plan,
- double limit_tuples);
+static void label_sort_with_costsize(PlannerInfo *root, RelOptInfo *rel,
+ Sort *plan, double limit_tuples);
static SeqScan *make_seqscan(List *qptlist, List *qpqual, Index scanrelid);
static SampleScan *make_samplescan(List *qptlist, List *qpqual, Index scanrelid,
TableSampleClause *tsc);
@@ -1361,7 +1361,8 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
sortColIdx, sortOperators,
collations, nullsFirst);
- label_sort_with_costsize(root, sort, best_path->limit_tuples);
+ label_sort_with_costsize(root, rel, sort,
+ best_path->limit_tuples);
subplan = (Plan *) sort;
}
}
@@ -1533,7 +1534,7 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
sortColIdx, sortOperators,
collations, nullsFirst);
- label_sort_with_costsize(root, sort, best_path->limit_tuples);
+ label_sort_with_costsize(root, rel, sort, best_path->limit_tuples);
subplan = (Plan *) sort;
}
@@ -1900,7 +1901,7 @@ create_unique_plan(PlannerInfo *root, UniquePath *best_path, int flags)
groupColPos++;
}
sort = make_sort_from_sortclauses(sortList, subplan);
- label_sort_with_costsize(root, sort, -1.0);
+ label_sort_with_costsize(root, best_path->path.parent, sort, -1.0);
plan = (Plan *) make_unique_from_sortclauses((Plan *) sort, sortList);
}
@@ -4527,7 +4528,8 @@ create_mergejoin_plan(PlannerInfo *root,
best_path->outersortkeys,
outer_relids);
- label_sort_with_costsize(root, sort, -1.0);
+ label_sort_with_costsize(root, best_path->jpath.path.parent,
+ sort, -1.0);
outer_plan = (Plan *) sort;
outerpathkeys = best_path->outersortkeys;
}
@@ -4541,7 +4543,8 @@ create_mergejoin_plan(PlannerInfo *root,
best_path->innersortkeys,
inner_relids);
- label_sort_with_costsize(root, sort, -1.0);
+ label_sort_with_costsize(root, best_path->jpath.path.parent,
+ sort, -1.0);
inner_plan = (Plan *) sort;
innerpathkeys = best_path->innersortkeys;
}
@@ -5442,7 +5445,8 @@ copy_plan_costsize(Plan *dest, Plan *src)
* limit_tuples is as for cost_sort (in particular, pass -1 if no limit)
*/
static void
-label_sort_with_costsize(PlannerInfo *root, Sort *plan, double limit_tuples)
+label_sort_with_costsize(PlannerInfo *root, RelOptInfo *rel, Sort *plan,
+ double limit_tuples)
{
Plan *lefttree = plan->plan.lefttree;
Path sort_path; /* dummy for result of cost_sort */
@@ -5453,7 +5457,7 @@ label_sort_with_costsize(PlannerInfo *root, Sort *plan, double limit_tuples)
*/
Assert(IsA(plan, Sort));
- cost_sort(&sort_path, root, NIL,
+ cost_sort(&sort_path, root, rel, NIL,
lefttree->total_cost,
plan->plan.disabled_nodes,
lefttree->plan_rows,
@@ -6524,7 +6528,7 @@ make_material(Plan *lefttree)
* Path representation, but it's not worth the trouble yet.
*/
Plan *
-materialize_finished_plan(Plan *subplan)
+materialize_finished_plan(Plan *subplan, RelOptInfo *rel)
{
Plan *matplan;
Path matpath; /* dummy for result of cost_material */
@@ -6549,7 +6553,7 @@ materialize_finished_plan(Plan *subplan)
subplan->total_cost -= initplan_cost;
/* Set cost data */
- cost_material(&matpath,
+ cost_material(&matpath, rel,
subplan->disabled_nodes,
subplan->startup_cost,
subplan->total_cost,
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index b5827d3980..c0f431cf0d 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -427,7 +427,7 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions,
if (cursorOptions & CURSOR_OPT_SCROLL)
{
if (!ExecSupportsBackwardScan(top_plan))
- top_plan = materialize_finished_plan(top_plan);
+ top_plan = materialize_finished_plan(top_plan, final_rel);
}
/*
@@ -3831,7 +3831,8 @@ create_grouping_paths(PlannerInfo *root,
* support grouping sets. create_ordinary_grouping_paths() will check
* additional conditions, such as whether input_rel is partitioned.
*/
- if (enable_partitionwise_aggregate && !parse->groupingSets)
+ if (REL_CAN_USE_PATH(grouped_rel, PartitionwiseAggregate) &&
+ !parse->groupingSets)
extra.patype = PARTITIONWISE_AGGREGATE_FULL;
else
extra.patype = PARTITIONWISE_AGGREGATE_NONE;
@@ -4634,7 +4635,8 @@ create_one_window_path(PlannerInfo *root,
* No presorted keys or incremental sort disabled, just perform a
* complete sort.
*/
- if (presorted_keys == 0 || !enable_incremental_sort)
+ if (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(window_rel, IncrementalSort))
path = (Path *) create_sort_path(root, window_rel,
path,
window_pathkeys,
@@ -4893,7 +4895,8 @@ create_partial_distinct_paths(PlannerInfo *root, RelOptInfo *input_rel,
* cheapest partial path).
*/
if (input_path != cheapest_partial_path &&
- (presorted_keys == 0 || !enable_incremental_sort))
+ (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(partial_distinct_rel, IncrementalSort)))
continue;
/*
@@ -4901,7 +4904,8 @@ create_partial_distinct_paths(PlannerInfo *root, RelOptInfo *input_rel,
* We'll just do a sort if there are no presorted keys and an
* incremental sort when there are presorted keys.
*/
- if (presorted_keys == 0 || !enable_incremental_sort)
+ if (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(partial_distinct_rel, IncrementalSort))
sorted_path = (Path *) create_sort_path(root,
partial_distinct_rel,
input_path,
@@ -4961,10 +4965,12 @@ create_partial_distinct_paths(PlannerInfo *root, RelOptInfo *input_rel,
/*
* Now try hash aggregate paths, if enabled and hashing is possible. Since
* we're not on the hook to ensure we do our best to create at least one
- * path here, we treat enable_hashagg as a hard off-switch rather than the
- * slightly softer variant in create_final_distinct_paths.
+ * path here, we treat completely skip this if hash aggregation is not
+ * enabled. (In contrast, create_final_distinct_paths sometimes considers
+ * hash aggregation even when it's disabled, to avoid failing completely.)
*/
- if (enable_hashagg && grouping_is_hashable(root->processed_distinctClause))
+ if (REL_CAN_USE_PATH(partial_distinct_rel, HashAgg) &&
+ grouping_is_hashable(root->processed_distinctClause))
{
add_partial_path(partial_distinct_rel, (Path *)
create_agg_path(root,
@@ -5105,7 +5111,8 @@ create_final_distinct_paths(PlannerInfo *root, RelOptInfo *input_rel,
* cheapest input path).
*/
if (input_path != cheapest_input_path &&
- (presorted_keys == 0 || !enable_incremental_sort))
+ (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(distinct_rel, IncrementalSort)))
continue;
/*
@@ -5113,7 +5120,8 @@ create_final_distinct_paths(PlannerInfo *root, RelOptInfo *input_rel,
* We'll just do a sort if there are no presorted keys and an
* incremental sort when there are presorted keys.
*/
- if (presorted_keys == 0 || !enable_incremental_sort)
+ if (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(distinct_rel, IncrementalSort))
sorted_path = (Path *) create_sort_path(root,
distinct_rel,
input_path,
@@ -5177,14 +5185,14 @@ create_final_distinct_paths(PlannerInfo *root, RelOptInfo *input_rel,
* die trying. If we do have other choices, there are two things that
* should prevent selection of hashing: if the query uses DISTINCT ON
* (because it won't really have the expected behavior if we hash), or if
- * enable_hashagg is off.
+ * hash aggregation is disabled.
*
* Note: grouping_is_hashable() is much more expensive to check than the
* other gating conditions, so we want to do it last.
*/
if (distinct_rel->pathlist == NIL)
allow_hash = true; /* we have no alternatives */
- else if (parse->hasDistinctOn || !enable_hashagg)
+ else if (parse->hasDistinctOn || !REL_CAN_USE_PATH(distinct_rel, HashAgg))
allow_hash = false; /* policy-based decision not to hash */
else
allow_hash = true; /* default */
@@ -5277,7 +5285,8 @@ create_ordered_paths(PlannerInfo *root,
* input path).
*/
if (input_path != cheapest_input_path &&
- (presorted_keys == 0 || !enable_incremental_sort))
+ (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(ordered_rel, IncrementalSort)))
continue;
/*
@@ -5285,7 +5294,8 @@ create_ordered_paths(PlannerInfo *root,
* We'll just do a sort if there are no presorted keys and an
* incremental sort when there are presorted keys.
*/
- if (presorted_keys == 0 || !enable_incremental_sort)
+ if (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(ordered_rel, IncrementalSort))
sorted_path = (Path *) create_sort_path(root,
ordered_rel,
input_path,
@@ -5349,7 +5359,8 @@ create_ordered_paths(PlannerInfo *root,
* partial path).
*/
if (input_path != cheapest_partial_path &&
- (presorted_keys == 0 || !enable_incremental_sort))
+ (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(ordered_rel, IncrementalSort)))
continue;
/*
@@ -5357,7 +5368,8 @@ create_ordered_paths(PlannerInfo *root,
* We'll just do a sort if there are no presorted keys and an
* incremental sort when there are presorted keys.
*/
- if (presorted_keys == 0 || !enable_incremental_sort)
+ if (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(ordered_rel, IncrementalSort))
sorted_path = (Path *) create_sort_path(root,
ordered_rel,
input_path,
@@ -6747,7 +6759,7 @@ plan_cluster_use_sort(Oid tableOid, Oid indexOid)
/* Estimate the cost of seq scan + sort */
seqScanPath = create_seqscan_path(root, rel, NULL, 0);
- cost_sort(&seqScanAndSortPath, root, NIL,
+ cost_sort(&seqScanAndSortPath, root, rel, NIL,
seqScanPath->disabled_nodes,
seqScanPath->total_cost, rel->tuples, rel->reltarget->width,
comparisonCost, maintenance_work_mem, -1.0);
@@ -6931,7 +6943,8 @@ make_ordered_path(PlannerInfo *root, RelOptInfo *rel, Path *path,
* disabled unless it's the cheapest input path).
*/
if (path != cheapest_path &&
- (presorted_keys == 0 || !enable_incremental_sort))
+ (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(rel, IncrementalSort)))
return NULL;
/*
@@ -6939,7 +6952,8 @@ make_ordered_path(PlannerInfo *root, RelOptInfo *rel, Path *path,
* just do a sort if there are no presorted keys and an incremental
* sort when there are presorted keys.
*/
- if (presorted_keys == 0 || !enable_incremental_sort)
+ if (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(rel, IncrementalSort))
path = (Path *) create_sort_path(root,
rel,
path,
@@ -7540,7 +7554,8 @@ gather_grouping_paths(PlannerInfo *root, RelOptInfo *rel)
* disabled unless it's the cheapest input path).
*/
if (path != cheapest_partial_path &&
- (presorted_keys == 0 || !enable_incremental_sort))
+ (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(rel, IncrementalSort)))
continue;
/*
@@ -7548,7 +7563,8 @@ gather_grouping_paths(PlannerInfo *root, RelOptInfo *rel)
* just do a sort if there are no presorted keys and an incremental
* sort when there are presorted keys.
*/
- if (presorted_keys == 0 || !enable_incremental_sort)
+ if (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(rel, IncrementalSort))
path = (Path *) create_sort_path(root, rel, path,
groupby_pathkeys,
-1.0);
diff --git a/src/backend/optimizer/plan/subselect.c b/src/backend/optimizer/plan/subselect.c
index 6d003cc8e5..6dbcf6e0aa 100644
--- a/src/backend/optimizer/plan/subselect.c
+++ b/src/backend/optimizer/plan/subselect.c
@@ -525,13 +525,14 @@ build_subplan(PlannerInfo *root, Plan *plan, Path *path,
* is pointless for a direct-correlated subplan, since we'd have to
* recompute its results each time anyway. For uncorrelated/undirect
* correlated subplans, we add Material unless the subplan's top plan
- * node would materialize its output anyway. Also, if enable_material
- * is false, then the user does not want us to materialize anything
+ * node would materialize its output anyway. Also, if Materialize is
+ * dissabled, then the user does not want us to materialize anything
* unnecessarily, so we don't.
*/
- else if (splan->parParam == NIL && enable_material &&
+ else if (splan->parParam == NIL &&
+ REL_CAN_USE_PATH(path->parent, Material) &&
!ExecMaterializesOutput(nodeTag(plan)))
- plan = materialize_finished_plan(plan);
+ plan = materialize_finished_plan(plan, path->parent);
result = (Node *) splan;
isInitPlan = false;
diff --git a/src/backend/optimizer/prep/prepunion.c b/src/backend/optimizer/prep/prepunion.c
index a0baf6d4a1..f8fd5db66b 100644
--- a/src/backend/optimizer/prep/prepunion.c
+++ b/src/backend/optimizer/prep/prepunion.c
@@ -587,7 +587,8 @@ build_setop_child_paths(PlannerInfo *root, RelOptInfo *rel,
* input path).
*/
if (subpath != cheapest_input_path &&
- (presorted_keys == 0 || !enable_incremental_sort))
+ (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(final_rel, IncrementalSort)))
continue;
/*
@@ -595,7 +596,8 @@ build_setop_child_paths(PlannerInfo *root, RelOptInfo *rel,
* We'll just do a sort if there are no presorted keys and an
* incremental sort when there are presorted keys.
*/
- if (presorted_keys == 0 || !enable_incremental_sort)
+ if (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(final_rel, IncrementalSort))
subpath = (Path *) create_sort_path(rel->subroot,
final_rel,
subpath,
@@ -867,7 +869,7 @@ generate_union_paths(SetOperationStmt *op, PlannerInfo *root,
* the children. The precise formula is just a guess; see
* add_paths_to_append_rel.
*/
- if (enable_parallel_append)
+ if (REL_CAN_USE_PATH(result_rel, ParallelAppend))
{
parallel_workers = Max(parallel_workers,
pg_leftmost_one_pos32(list_length(partial_pathlist)) + 1);
@@ -879,7 +881,8 @@ generate_union_paths(SetOperationStmt *op, PlannerInfo *root,
papath = (Path *)
create_append_path(root, result_rel, NIL, partial_pathlist,
NIL, NULL, parallel_workers,
- enable_parallel_append, -1);
+ REL_CAN_USE_PATH(result_rel, ParallelAppend),
+ -1);
gpath = (Path *)
create_gather_path(root, result_rel, papath,
result_rel->reltarget, NULL, NULL);
@@ -1319,8 +1322,8 @@ choose_hashed_setop(PlannerInfo *root, List *groupClauses,
errmsg("could not implement %s", construct),
errdetail("Some of the datatypes only support hashing, while others only support sorting.")));
- /* Prefer sorting when enable_hashagg is off */
- if (!enable_hashagg)
+ /* Prefer sorting when hash aggregation is disabled */
+ if (!REL_CAN_USE_PATH(input_path->parent, HashAgg))
return false;
/*
@@ -1343,7 +1346,7 @@ choose_hashed_setop(PlannerInfo *root, List *groupClauses,
* These path variables are dummies that just hold cost fields; we don't
* make actual Paths for these steps.
*/
- cost_agg(&hashed_p, root, AGG_HASHED, NULL,
+ cost_agg(&hashed_p, root, input_path->parent, AGG_HASHED, NULL,
numGroupCols, dNumGroups,
NIL,
input_path->disabled_nodes,
@@ -1358,7 +1361,8 @@ choose_hashed_setop(PlannerInfo *root, List *groupClauses,
sorted_p.startup_cost = input_path->startup_cost;
sorted_p.total_cost = input_path->total_cost;
/* XXX cost_sort doesn't actually look at pathkeys, so just pass NIL */
- cost_sort(&sorted_p, root, NIL, sorted_p.disabled_nodes,
+ cost_sort(&sorted_p, root, input_path->parent, NIL,
+ sorted_p.disabled_nodes,
sorted_p.total_cost,
input_path->rows, input_path->pathtarget->width,
0.0, work_mem, -1.0);
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index fc97bf6ee2..77ed747437 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -1537,6 +1537,7 @@ create_merge_append_path(PlannerInfo *root,
cost_sort(&sort_path,
root,
+ rel,
pathkeys,
subpath->disabled_nodes,
subpath->total_cost,
@@ -1649,7 +1650,7 @@ create_material_path(RelOptInfo *rel, Path *subpath)
pathnode->subpath = subpath;
- cost_material(&pathnode->path,
+ cost_material(&pathnode->path, rel,
subpath->disabled_nodes,
subpath->startup_cost,
subpath->total_cost,
@@ -1698,7 +1699,7 @@ create_memoize_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
pathnode->est_entries = 0;
/* we should not generate this path type when enable_memoize=false */
- Assert(enable_memoize);
+ Assert(REL_CAN_USE_PATH(rel, Memoize));
pathnode->path.disabled_nodes = subpath->disabled_nodes;
/*
@@ -1866,7 +1867,7 @@ create_unique_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
/*
* Estimate cost for sort+unique implementation
*/
- cost_sort(&sort_path, root, NIL,
+ cost_sort(&sort_path, root, rel, NIL,
subpath->disabled_nodes,
subpath->total_cost,
rel->rows,
@@ -1901,7 +1902,7 @@ create_unique_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
sjinfo->semi_can_hash = false;
}
else
- cost_agg(&agg_path, root,
+ cost_agg(&agg_path, root, rel,
AGG_HASHED, NULL,
numCols, pathnode->path.rows,
NIL,
@@ -3101,7 +3102,7 @@ create_sort_path(PlannerInfo *root,
pathnode->subpath = subpath;
- cost_sort(&pathnode->path, root, pathkeys,
+ cost_sort(&pathnode->path, root, rel, pathkeys,
subpath->disabled_nodes,
subpath->total_cost,
subpath->rows,
@@ -3288,7 +3289,7 @@ create_agg_path(PlannerInfo *root,
pathnode->groupClause = groupClause;
pathnode->qual = qual;
- cost_agg(&pathnode->path, root,
+ cost_agg(&pathnode->path, root, rel,
aggstrategy, aggcosts,
list_length(groupClause), numGroups,
qual,
@@ -3395,7 +3396,7 @@ create_groupingsets_path(PlannerInfo *root,
*/
if (is_first)
{
- cost_agg(&pathnode->path, root,
+ cost_agg(&pathnode->path, root, rel,
aggstrategy,
agg_costs,
numGroupCols,
@@ -3421,7 +3422,7 @@ create_groupingsets_path(PlannerInfo *root,
* Account for cost of aggregation, but don't charge input
* cost again
*/
- cost_agg(&agg_path, root,
+ cost_agg(&agg_path, root, rel,
rollup->is_hashed ? AGG_HASHED : AGG_SORTED,
agg_costs,
numGroupCols,
@@ -3436,7 +3437,7 @@ create_groupingsets_path(PlannerInfo *root,
else
{
/* Account for cost of sort, but don't charge input cost again */
- cost_sort(&sort_path, root, NIL, 0,
+ cost_sort(&sort_path, root, rel, NIL, 0,
0.0,
subpath->rows,
subpath->pathtarget->width,
@@ -3446,7 +3447,7 @@ create_groupingsets_path(PlannerInfo *root,
/* Account for cost of aggregation */
- cost_agg(&agg_path, root,
+ cost_agg(&agg_path, root, rel,
AGG_SORTED,
agg_costs,
numGroupCols,
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 78a3cfafde..3b9a3746ee 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -570,7 +570,8 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
/*
* Allow a plugin to editorialize on the info we obtained from the
* catalogs. Actions might include altering the assumed relation size,
- * removing an index, or adding a hypothetical index to the indexlist.
+ * removing an index, adding a hypothetical index to the indexlist, or
+ * changing the path type mask.
*/
if (get_relation_info_hook)
(*get_relation_info_hook) (root, relationObjectId, inhparent, rel);
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index d7266e4cdb..88e468795a 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -211,6 +211,7 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
rel->consider_startup = (root->tuple_fraction > 0);
rel->consider_param_startup = false; /* might get changed later */
rel->consider_parallel = false; /* might get changed later */
+ rel->path_type_mask = default_path_type_mask;
rel->reltarget = create_empty_pathtarget();
rel->pathlist = NIL;
rel->ppilist = NIL;
@@ -707,6 +708,7 @@ build_join_rel(PlannerInfo *root,
joinrel->consider_startup = (root->tuple_fraction > 0);
joinrel->consider_param_startup = false;
joinrel->consider_parallel = false;
+ joinrel->path_type_mask = default_path_type_mask;
joinrel->reltarget = create_empty_pathtarget();
joinrel->pathlist = NIL;
joinrel->ppilist = NIL;
@@ -900,6 +902,7 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->consider_startup = (root->tuple_fraction > 0);
joinrel->consider_param_startup = false;
joinrel->consider_parallel = false;
+ joinrel->path_type_mask = default_path_type_mask;
joinrel->reltarget = create_empty_pathtarget();
joinrel->pathlist = NIL;
joinrel->ppilist = NIL;
@@ -1484,6 +1487,7 @@ fetch_upper_rel(PlannerInfo *root, UpperRelationKind kind, Relids relids)
upperrel->consider_startup = (root->tuple_fraction > 0);
upperrel->consider_param_startup = false;
upperrel->consider_parallel = false; /* might get changed later */
+ upperrel->path_type_mask = default_path_type_mask;
upperrel->reltarget = create_empty_pathtarget();
upperrel->pathlist = NIL;
upperrel->cheapest_startup_path = NULL;
@@ -2010,7 +2014,7 @@ build_joinrel_partition_info(PlannerInfo *root,
PartitionScheme part_scheme;
/* Nothing to do if partitionwise join technique is disabled. */
- if (!enable_partitionwise_join)
+ if (!REL_CAN_USE_PATH(joinrel, PartitionwiseJoin))
{
Assert(!IS_PARTITIONED_REL(joinrel));
return;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index af227b1f24..1bd21bbe34 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -55,6 +55,7 @@
#include "optimizer/geqo.h"
#include "optimizer/optimizer.h"
#include "optimizer/paths.h"
+#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "parser/parse_expr.h"
#include "parser/parser.h"
@@ -777,7 +778,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_seqscan,
true,
- NULL, NULL, NULL
+ NULL, enable_seqscan_assign_hook, NULL
},
{
{"enable_indexscan", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -787,7 +788,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_indexscan,
true,
- NULL, NULL, NULL
+ NULL, enable_indexscan_assign_hook, NULL
},
{
{"enable_indexonlyscan", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -797,7 +798,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_indexonlyscan,
true,
- NULL, NULL, NULL
+ NULL, enable_indexonlyscan_assign_hook, NULL
},
{
{"enable_bitmapscan", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -807,7 +808,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_bitmapscan,
true,
- NULL, NULL, NULL
+ NULL, enable_bitmapscan_assign_hook, NULL
},
{
{"enable_tidscan", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -817,7 +818,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_tidscan,
true,
- NULL, NULL, NULL
+ NULL, enable_tidscan_assign_hook, NULL
},
{
{"enable_sort", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -827,7 +828,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_sort,
true,
- NULL, NULL, NULL
+ NULL, enable_sort_assign_hook, NULL
},
{
{"enable_incremental_sort", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -837,7 +838,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_incremental_sort,
true,
- NULL, NULL, NULL
+ NULL, enable_incremental_sort_assign_hook, NULL
},
{
{"enable_hashagg", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -847,7 +848,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_hashagg,
true,
- NULL, NULL, NULL
+ NULL, enable_hashagg_assign_hook, NULL
},
{
{"enable_material", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -857,7 +858,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_material,
true,
- NULL, NULL, NULL
+ NULL, enable_material_assign_hook, NULL
},
{
{"enable_memoize", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -867,7 +868,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_memoize,
true,
- NULL, NULL, NULL
+ NULL, enable_memoize_assign_hook, NULL
},
{
{"enable_nestloop", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -877,7 +878,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_nestloop,
true,
- NULL, NULL, NULL
+ NULL, enable_nestloop_assign_hook, NULL
},
{
{"enable_mergejoin", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -887,7 +888,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_mergejoin,
true,
- NULL, NULL, NULL
+ NULL, enable_mergejoin_assign_hook, NULL
},
{
{"enable_hashjoin", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -897,7 +898,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_hashjoin,
true,
- NULL, NULL, NULL
+ NULL, enable_hashjoin_assign_hook, NULL
},
{
{"enable_gathermerge", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -907,7 +908,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_gathermerge,
true,
- NULL, NULL, NULL
+ NULL, enable_gathermerge_assign_hook, NULL
},
{
{"enable_partitionwise_join", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -917,7 +918,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_partitionwise_join,
false,
- NULL, NULL, NULL
+ NULL, enable_partitionwise_join_assign_hook, NULL
},
{
{"enable_partitionwise_aggregate", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -927,7 +928,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_partitionwise_aggregate,
false,
- NULL, NULL, NULL
+ NULL, enable_partitionwise_aggregate_assign_hook, NULL
},
{
{"enable_parallel_append", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -937,7 +938,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_parallel_append,
true,
- NULL, NULL, NULL
+ NULL, enable_parallel_append_assign_hook, NULL
},
{
{"enable_parallel_hash", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -947,7 +948,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_parallel_hash,
true,
- NULL, NULL, NULL
+ NULL, enable_parallel_hash_assign_hook, NULL
},
{
{"enable_partition_pruning", PGC_USERSET, QUERY_TUNING_METHOD,
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 540d021592..bbff482cec 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -80,6 +80,26 @@ typedef enum UpperRelationKind
/* NB: UPPERREL_FINAL must be last enum entry; it's used to size arrays */
} UpperRelationKind;
+#define PathTypeBitmapScan 0x00000001
+#define PathTypeGatherMerge 0x00000002
+#define PathTypeHashAgg 0x00000004
+#define PathTypeHashJoin 0x00000008
+#define PathTypeIncrementalSort 0x00000010
+#define PathTypeIndexScan 0x00000020
+#define PathTypeIndexOnlyScan 0x00000040
+#define PathTypeMaterial 0x00000080
+#define PathTypeMemoize 0x00000100
+#define PathTypeMergeJoin 0x00000200
+#define PathTypeNestLoop 0x00000400
+#define PathTypeParallelAppend 0x00000800
+#define PathTypeParallelHash 0x00001000
+#define PathTypePartitionwiseJoin 0x00002000
+#define PathTypePartitionwiseAggregate 0x00004000
+#define PathTypeSeqScan 0x00008000
+#define PathTypeSort 0x00010000
+#define PathTypeTIDScan 0x00020000
+#define PathTypeMaskAll 0x0003FFFF
+
/*----------
* PlannerGlobal
* Global information for planning/optimization
@@ -879,6 +899,8 @@ typedef struct RelOptInfo
bool consider_param_startup;
/* consider parallel paths? */
bool consider_parallel;
+ /* path type mask for this rel */
+ uint32 path_type_mask;
/*
* default result targetlist for Paths scanning this relation; list of
@@ -1065,6 +1087,13 @@ typedef struct RelOptInfo
((rel)->part_scheme && (rel)->boundinfo && (rel)->nparts > 0 && \
(rel)->part_rels && (rel)->partexprs && (rel)->nullable_partexprs)
+/*
+ * Convenience macro to test for whether a certain a PathTypeXXX bit is
+ * set in a relation's path_type_mask.
+ */
+#define REL_CAN_USE_PATH(rel, type) \
+ (((rel)->path_type_mask & PathType##type) != 0)
+
/*
* IndexOptInfo
* Per-index information for planning/optimization
diff --git a/src/include/optimizer/cost.h b/src/include/optimizer/cost.h
index 854a782944..2cd1d8c34c 100644
--- a/src/include/optimizer/cost.h
+++ b/src/include/optimizer/cost.h
@@ -71,6 +71,26 @@ extern PGDLLIMPORT bool enable_partition_pruning;
extern PGDLLIMPORT bool enable_presorted_aggregate;
extern PGDLLIMPORT bool enable_async_append;
extern PGDLLIMPORT int constraint_exclusion;
+extern PGDLLIMPORT uint32 default_path_type_mask;
+
+extern void enable_bitmapscan_assign_hook(bool newval, void *extra);
+extern void enable_gathermerge_assign_hook(bool newval, void *extra);
+extern void enable_hashagg_assign_hook(bool newval, void *extra);
+extern void enable_hashjoin_assign_hook(bool newval, void *extra);
+extern void enable_incremental_sort_assign_hook(bool newval, void *extra);
+extern void enable_indexscan_assign_hook(bool newval, void *extra);
+extern void enable_indexonlyscan_assign_hook(bool newval, void *extra);
+extern void enable_material_assign_hook(bool newval, void *extra);
+extern void enable_memoize_assign_hook(bool newval, void *extra);
+extern void enable_mergejoin_assign_hook(bool newval, void *extra);
+extern void enable_nestloop_assign_hook(bool newval, void *extra);
+extern void enable_parallel_append_assign_hook(bool newval, void *extra);
+extern void enable_parallel_hash_assign_hook(bool newval, void *extra);
+extern void enable_partitionwise_join_assign_hook(bool newval, void *extra);
+extern void enable_partitionwise_aggregate_assign_hook(bool newval, void *extra);
+extern void enable_seqscan_assign_hook(bool newval, void *extra);
+extern void enable_sort_assign_hook(bool newval, void *extra);
+extern void enable_tidscan_assign_hook(bool newval, void *extra);
extern double index_pages_fetched(double tuples_fetched, BlockNumber pages,
double index_pages, PlannerInfo *root);
@@ -107,7 +127,7 @@ extern void cost_namedtuplestorescan(Path *path, PlannerInfo *root,
extern void cost_resultscan(Path *path, PlannerInfo *root,
RelOptInfo *baserel, ParamPathInfo *param_info);
extern void cost_recursive_union(Path *runion, Path *nrterm, Path *rterm);
-extern void cost_sort(Path *path, PlannerInfo *root,
+extern void cost_sort(Path *path, PlannerInfo *root, RelOptInfo *rel,
List *pathkeys, int disabled_nodes,
Cost input_cost, double tuples, int width,
Cost comparison_cost, int sort_mem,
@@ -124,11 +144,11 @@ extern void cost_merge_append(Path *path, PlannerInfo *root,
int input_disabled_nodes,
Cost input_startup_cost, Cost input_total_cost,
double tuples);
-extern void cost_material(Path *path,
+extern void cost_material(Path *path, RelOptInfo *rel,
int input_disabled_nodes,
Cost input_startup_cost, Cost input_total_cost,
double tuples, int width);
-extern void cost_agg(Path *path, PlannerInfo *root,
+extern void cost_agg(Path *path, PlannerInfo *root, RelOptInfo *rel,
AggStrategy aggstrategy, const AggClauseCosts *aggcosts,
int numGroupCols, double numGroups,
List *quals,
@@ -148,6 +168,7 @@ extern void cost_group(Path *path, PlannerInfo *root,
double input_tuples);
extern void initial_cost_nestloop(PlannerInfo *root,
JoinCostWorkspace *workspace,
+ RelOptInfo *joinrel,
JoinType jointype,
Path *outer_path, Path *inner_path,
JoinPathExtraData *extra);
@@ -156,6 +177,7 @@ extern void final_cost_nestloop(PlannerInfo *root, NestPath *path,
JoinPathExtraData *extra);
extern void initial_cost_mergejoin(PlannerInfo *root,
JoinCostWorkspace *workspace,
+ RelOptInfo *joinrel,
JoinType jointype,
List *mergeclauses,
Path *outer_path, Path *inner_path,
@@ -166,6 +188,7 @@ extern void final_cost_mergejoin(PlannerInfo *root, MergePath *path,
JoinPathExtraData *extra);
extern void initial_cost_hashjoin(PlannerInfo *root,
JoinCostWorkspace *workspace,
+ RelOptInfo *joinrel,
JoinType jointype,
List *hashclauses,
Path *outer_path, Path *inner_path,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index aafc173792..07623eff79 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -45,7 +45,7 @@ extern ForeignScan *make_foreignscan(List *qptlist, List *qpqual,
Plan *outer_plan);
extern Plan *change_plan_targetlist(Plan *subplan, List *tlist,
bool tlist_parallel_safe);
-extern Plan *materialize_finished_plan(Plan *subplan);
+extern Plan *materialize_finished_plan(Plan *subplan, RelOptInfo *rel);
extern bool is_projection_capable_path(Path *path);
extern bool is_projection_capable_plan(Plan *plan);
--
2.39.3 (Apple Git-145)
Robert Haas <robertmhaas@gmail.com> writes:
I'm somewhat expecting to be flamed to a well-done crisp for saying
this, but I think we need better ways for extensions to control the
behavior of PostgreSQL's query planner.
Nah, I won't flame you for that, it's a reasonable thing to think
about. However, the devil is in the details, and ...
The attached patch, briefly mentioned above, essentially converts the
enable_* GUCs into RelOptInfo properties where the defaults are set by
the corresponding GUCs.
... this doesn't seem like it's moving the football very far at all.
The enable_XXX GUCs are certainly blunt instruments, but I'm not sure
how much better it is if they're per-rel. For example, I don't see
how this gets us any closer to letting an extension fix a poor choice
of join order. Or, if your problem is that the planner wants to scan
index A but you want it to scan index B, enable_indexscan won't help.
... On the other hand, the more I look at
what our enable_* GUCs actually do, the less impressed I am. IMHO,
things like enable_hashjoin make a lot of sense, but enable_sort seems
like it just controls an absolutely random smattering of behaviors in
a way that seems to me to have very little to recommend it, and I've
complained elsewhere about how enable_indexscan and
enable_indexonlyscan are really quite odd when you look at how they're
implemented.
Yeah, these sorts of questions aren't made better this way either.
If anything, having extensions manipulating these variables will
make it even harder to rethink what they do.
You mentioned that there is prior art out there, but this proposal
doesn't seem like it's drawing on any such thing. What ideas should
we be stealing?
regards, tom lane
On 26/8/2024 18:32, Robert Haas wrote:
I'm somewhat expecting to be flamed to a well-done crisp for saying
this, but I think we need better ways for extensions to control the
behavior of PostgreSQL's query planner. I know of two major reasons
It is the change I have been waiting for a long time. Remember how many
kludge codes in pg_hint_plan, aqo, citus, timescale, etc., are written
for only the reason of a small number of hooks - I guess many other
people could cheer such work.
why somebody might want to do this. First, you might want to do
something like what pg_hint_plan does, where it essentially implements
Oracle-style hints that can be either inline or stored in a side table
and automatically applied to queries.[1] In addition to supporting
Oracle-style hints, it also supports some other kinds of hints so that
you can, for example, try to fix broken cardinality estimates. Second,
My personal most wanted list:
- Selectivity list estimation hook
- Groups number estimation hook
- hooks on memory estimations, involving work_mem
- add_path() hook
- Hook on final RelOptInfo pathlist
- a custom list of nodes in RelOptinfo, PlannerStmt, Plan and Query
structures
- Extensibility of extended and plain statistics
- Hook on portal error processing
- Canonicalise expressions hook
you might want to convince the planner to keep producing the same kind
of plan that it produced previously. I believe this is what Amazon's
query plan management feature[2] does, although since it is closed
source and I don't work at Amazon maybe it's actually implemented
completely differently. Regardless of what Amazon did in this case,
plan stability is a feature people want. Just trying to keep using the
same plan data structure forever doesn't seem like a good strategy,
because for example it would be fragile in the case of any DDL
changes, like dropping and recreating an index, or dropping or adding
As a designer of plan freezing feature [1]https://postgrespro.com/docs/enterprise/16/sr-plan I can say it utilises
plancache and, being under its invalidation callbacks it doesn't afraid
DDL or any other stuff altering database objects.
Unfortunately, the part about the hook having the freedom to delete
paths isn't really true. Perhaps technically you can delete a path
that you don't want to be chosen, but any paths that were dominated by
the path you deleted have already been thrown away and it's too late
to get them back. You can modify paths if you don't want to change
their costs, but if you change their costs then you have the same
problem: the contents of the pathlist at the time that you see it are
determined by the costs that each path had when it was initially
added, and it's really too late to editorialize on that. So all you
can really do here in practice is add new paths.
From my standpoint, it is enough to export routines creating paths and
calculating costs.
set_join_pathlist_hook, which applies to joinrels, is similarly
limited. appendrels don't even have an equivalent of this hook.So, how could we do better?
I think there are two basic approaches that are possible here. If
someone sees a third option, let me know. First, we could allow users
to hook add_path() and add_partial_path(). That certainly provides the
flexibility on paper to accept or reject whatever paths you do or do
+1
The attached patch, briefly mentioned above, essentially converts the
enable_* GUCs into RelOptInfo properties where the defaults are set by
the corresponding GUCs. The idea is that a hook could then change this
on a per-RelOptInfo basis before path generation happens. For
IMO, it is better not to switch on/off algorithms, but allow extensions
to change their cost multipliers, modifying costs balance. 10E9 looks
like a disable, but multiplier == 10 for a cost node just provide more
freedom for hashing strategies.
[1]: https://postgrespro.com/docs/enterprise/16/sr-plan
--
regards, Andrei Lepikhov
On Mon, Aug 26, 2024 at 1:37 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Robert Haas <robertmhaas@gmail.com> writes:
I'm somewhat expecting to be flamed to a well-done crisp for saying
this, but I think we need better ways for extensions to control the
behavior of PostgreSQL's query planner.Nah, I won't flame you for that, it's a reasonable thing to think
about. However, the devil is in the details, and ...
Thank you. Not being flamed is one of my favorite things. :-)
The attached patch, briefly mentioned above, essentially converts the
enable_* GUCs into RelOptInfo properties where the defaults are set by
the corresponding GUCs.... this doesn't seem like it's moving the football very far at all.
The enable_XXX GUCs are certainly blunt instruments, but I'm not sure
how much better it is if they're per-rel. For example, I don't see
how this gets us any closer to letting an extension fix a poor choice
of join order. Or, if your problem is that the planner wants to scan
index A but you want it to scan index B, enable_indexscan won't help.
Well, I agree that this doesn't address everything you might want to
do, and I thought I said so, admittedly in the middle of a long wall
of text. This would JUST be a step toward letting an extension control
the scan and join methods, not the join order or the choice of index
or whatever else there is. But the fact that it doesn't do everything
is not a strike against it unless there's some competing design that
lets you take care of everything with a single mechanism, which I do
not see as realistic. If this proposal -- or really any proposal in
this area -- gets through, I will very happily propose more things to
address the other problems that I know about, but it doesn't make
sense to do a huge amount of work to craft a comprehensive solution
before we've had any discussion here.
Yeah, these sorts of questions aren't made better this way either.
If anything, having extensions manipulating these variables will
make it even harder to rethink what they do.
Correct, but my proposal to make enable_indexscan behave like
enable_indexonlyscan, which I thought was a slam-dunk, just provoked a
lot of grumbling. There's a kind of chicken and egg problem here. If
the existing GUCs were better designed, then using them here would
make sense. And the patch that I attached to my previous email were in
master, then cleaning up the design of the GUCs would have more value.
But if I can't make any progress with either problem because the other
problem also exists, then I'm kind of boxed into a corner. I could
also propose something here that is diverges from the enable_*
behavior, but then people will complain that the two shouldn't be
inconsistent, which I agree with, BTW. I thought maybe doing this
first would make sense, and then we could refine afterwards.
You mentioned that there is prior art out there, but this proposal
doesn't seem like it's drawing on any such thing. What ideas should
we be stealing?
Depends what you mean. As far as PostgreSQL-related things, the two
things that I mentioned in my opening paragraph and for which I
provided links seem to be me to the best examples we have. It's pretty
easy to see how to make pg_hint_plan require less kludgery, and I
think we can just iterate through the various problems there and solve
them pretty easily by adding a few hooks here and there and a few
extension-settable structure members here and there. I am engaging in
some serious hand-waving here, but this is not rocket science. I am
confident that if you made it your top priority to get into PG 18
stuff which would thoroughly un-hackify pg_hint_plan, you could be
done in months, possibly weeks. It will take me longer, but if we have
an agreement in principal that it is worth doing, I just can't see it
as being particularly difficult.
Amazon's query plan management stuff is a much tougher lift. For that,
you're asking the planner to try to create a new plan which is like
some old plan that you got before. So in a perfect world, you want to
control every planner decision. That's hard just because there are a
lot of them. If for example you want to get the same index scan that
you got before, you need not only to get the same type of index scan
(index, index-only, bitmap) and the same index, but also things like
the same non-native saop treatment, which seems like it would be
asking an awful lot of a hook system. On the other hand, maybe you can
cheat. If your regurgitate-the-same-plan system could force the same
join order, join methods, scan methods, choice of indexes, and
probably some stuff about aggregate and appendrel strategy, it might
be close enough to giving you the same plan you had before that nobody
would really care if the non-native saop treatment was different. I'm
almost positive it's better than not having a feature, which is where
are today. And although allowing control over just the major decisions
in query planning doesn't seem like something we can do in one patch,
I don't think it takes 100 patches either. Maybe five or ten.
If we step outside of the PostgreSQL ecosystem, I think we should look
at Oracle as one example. I have never been a real believer in hints
like SeqScan(foo), because if you don't fix the cardinality estimate
for table foo, then the rest of your plan is going to suck, too. On
the other hand, "hint everything" for some people in some situations
is a way to address that. It's stupid in a sense, but if you have an
automated way to do it, especially one that allows applying hints
out-of-query, it's not THAT stupid. Also, Oracle offers some other
pretty useful hints. In particular, using the LEADING hint to set the
driving table for the query plan does not seem dumb to me at all.
Hinting that things should be parallel or not, and with what degree of
parallelism, also seem quite reasonable. They've also got ALL_ROWS and
FIRST_ROWS(n) hints, which let you say whether you want fast-start
behavior or not, and it hardly needs to be said how often we get that
wrong or how badly. pg_hint_plan, which copies a lot of stuff that
Oracle does, innovates by allowing you to hint that a certain join
will return X number of rows or that the number or rows that the
planner thinks should be returned should be corrected by multiplying,
adding, or subtracting some constant. I'm not sure how useful this is
really because I feel like a lot of times you'd just pick some join
order where that particular join is no longer used e.g. if. A JOIN B
JOIN C and I hint the AB join, perhaps the planner will just start by
joining C to either A or B, and then that join will never occur.
However, that can be avoided by also using LEADING, or maybe in some
other cleverer way, like making an AB selectivity hint apply at
whatever point in the plan you join something that includes A to
something that includes B.
There's some details on SQL server's hinting here:
https://learn.microsoft.com/en-us/sql/t-sql/queries/hints-transact-sql?view=sql-server-ver16
It looks pretty complicated, but some of the basic concepts that you'd
expect are also present here: force the join method, rule in or out,
force the use of an index or of no index, force the join order. Those
seem to be the major things that "everyone" supports. I think we'd
want to expand a bit on that to allow forcing aggregate strategy and
perhaps some PostgreSQL-specific things e.g. other systems won't have
a hint to force a TIDRangeScan or not because that's a
PostgreSQL-specific concept, but it would be silly to make a system
that lets an extension control sequential scans and index scans but
not other, more rarely-used ways of scanning a relation, so probably
we want to do something.
I don't know if that helps, in terms of context. If it doesn't, let me
know what would help. And just to be clear, I *absolutely* think we
need to take a light touch here. If we install a ton of new
highly-opinionated infrastructure we will make a lot of people mad and
that's definitely not where I want to end up. I just think we need to
grow beyond "the planner is a black box and you shall not presume to
direct it." If every other system provides a way to control, say, the
join order, then it seems reasonable to suppose that a PostgreSQL
extension should be able to control the join order too. A lot of
details might be different but if multiple other systems have the
concept then the concept itself probably isn't ridiculous.
--
Robert Haas
EDB: http://www.enterprisedb.com
On Mon, Aug 26, 2024 at 2:00 PM Andrei Lepikhov <lepihov@gmail.com> wrote:
It is the change I have been waiting for a long time. Remember how many
kludge codes in pg_hint_plan, aqo, citus, timescale, etc., are written
for only the reason of a small number of hooks - I guess many other
people could cheer such work.
I think so, too. I know there are going to be people who hate this,
but I think the cat is already out of the bag. It's not a question any
more of whether it will happen, it's just a question of whether we
want to collaborate with extension developers or try to make their
life difficult.
My personal most wanted list:
- Selectivity list estimation hook
- Groups number estimation hook
- hooks on memory estimations, involving work_mem
- add_path() hook
- Hook on final RelOptInfo pathlist
- a custom list of nodes in RelOptinfo, PlannerStmt, Plan and Query
structures
- Extensibility of extended and plain statistics
- Hook on portal error processing
- Canonicalise expressions hook
One of my chronic complaints about hooks is that people propose hooks
that are just in any random spot in the code where they happen to want
to change something. If we accept that, we end up with a lot of hooks
where nobody can say how the hook can be used usefully and maybe it
can't actually be used usefully even by the original author, or only
them and nobody else. So these kinds of proposals need detailed,
case-by-case scrutiny. It's unacceptable for the planner to get filled
up with a bunch of poorly-designed hooks just as it is for any other
part of the system, but well-designed hooks whose usefulness can
clearly be seen should be just as welcome here as anywhere else.
IMO, it is better not to switch on/off algorithms, but allow extensions
to change their cost multipliers, modifying costs balance. 10E9 looks
like a disable, but multiplier == 10 for a cost node just provide more
freedom for hashing strategies.
That may be a valid use case, but I do not think it is a typical use
case. In my experience, when people want to force the planner to do
something, they really mean it. They don't mean "please do it this way
unless you really, really don't feel like it." They mean "please do it
this way, period." And that is also what other systems provide. Oracle
could provide a hint MERGE_COST(foo,10) meaning make merge joins look
ten times as expensive but in fact they only provide MERGE and
NO_MERGE. And a "reproduce this previous plan" feature really demands
infrastructure that truly forces the planner to do what it's told,
rather than just nicely suggesting that it might want to do as it's
told. I wouldn't be sad at all if we happen to end up with a system
that's powerful enough for an extension to implement "make merge joins
ten times as expensive"; in fact, I think that would be pretty cool.
But I don't think it should be the design center for what we
implement, because it looks nothing like what existing PG or non-PG
systems do, at least in my experience.
--
Robert Haas
EDB: http://www.enterprisedb.com
On 26/8/2024 21:44, Robert Haas wrote:
On Mon, Aug 26, 2024 at 2:00 PM Andrei Lepikhov <lepihov@gmail.com> wrote:
My personal most wanted list:
- Selectivity list estimation hook
- Groups number estimation hook
- hooks on memory estimations, involving work_mem
- add_path() hook
- Hook on final RelOptInfo pathlist
- a custom list of nodes in RelOptinfo, PlannerStmt, Plan and Query
structures
- Extensibility of extended and plain statistics
- Hook on portal error processing
- Canonicalise expressions hookOne of my chronic complaints about hooks is that people propose hooks
that are just in any random spot in the code where they happen to want
to change something. If we accept that, we end up with a lot of hooks
where nobody can say how the hook can be used usefully and maybe it
can't actually be used usefully even by the original author, or only
them and nobody else. So these kinds of proposals need detailed,
case-by-case scrutiny. It's unacceptable for the planner to get filled
up with a bunch of poorly-designed hooks just as it is for any other
part of the system, but well-designed hooks whose usefulness can
clearly be seen should be just as welcome here as anywhere else.
Definitely so. Think about that as a sketch proposal on the roadmap.
Right now, I know about only one hook - selectivity hook - which we
already discussed and have Tomas Vondra's patch on the table. But even
this is a big deal, because multi-clause estimations are a huge pain for
users that can't be resolved with extensions for now without core patches.
IMO, it is better not to switch on/off algorithms, but allow extensions
to change their cost multipliers, modifying costs balance. 10E9 looks
like a disable, but multiplier == 10 for a cost node just provide more
freedom for hashing strategies.That may be a valid use case, but I do not think it is a typical use
case. In my experience, when people want to force the planner to do
something, they really mean it. They don't mean "please do it this way
unless you really, really don't feel like it." They mean "please do it
this way, period." And that is also what other systems provide. Oracle
could provide a hint MERGE_COST(foo,10) meaning make merge joins look
ten times as expensive but in fact they only provide MERGE and
NO_MERGE. And a "reproduce this previous plan" feature really demands
infrastructure that truly forces the planner to do what it's told,
rather than just nicely suggesting that it might want to do as it's
told. I wouldn't be sad at all if we happen to end up with a system
that's powerful enough for an extension to implement "make merge joins
ten times as expensive"; in fact, I think that would be pretty cool.
But I don't think it should be the design center for what we
implement, because it looks nothing like what existing PG or non-PG
systems do, at least in my experience.
Heh, I meant not manual usage, but automatical one, provided by extensions.
--
regards, Andrei Lepikhov
At 2024-08-27 00:32:53, "Robert Haas" <robertmhaas@gmail.com> wrote:
I'm somewhat expecting to be flamed to a well-done crisp for saying
this, but I think we need better ways for extensions to control the
behavior of PostgreSQL's query planner. I know of two major reasons
why somebody might want to do this. First, you might want to do
something like what pg_hint_plan does, where it essentially implements
Oracle-style hints that can be either inline or stored in a side table
and automatically applied to queries.[1] In addition to supporting
Oracle-style hints, it also supports some other kinds of hints so that
you can, for example, try to fix broken cardinality estimates. Second,
you might want to convince the planner to keep producing the same kind
of plan that it produced previously. I believe this is what Amazon's
query plan management feature[2] does, although since it is closed
source and I don't work at Amazon maybe it's actually implemented
completely differently. Regardless of what Amazon did in this case,
plan stability is a feature people want. Just trying to keep using the
same plan data structure forever doesn't seem like a good strategy,
because for example it would be fragile in the case of any DDL
changes, like dropping and recreating an index, or dropping or adding
a column. But you might want conceptually the same plan. Although it's
not frequently admitted on this mailing list, unexpected plan changes
are a frequent cause of sudden database outages, and wanting to
prevent that is a legitimate thing for a user to try to do. Naturally,
there is a risk that you might in so doing also prevent plan changes
that would have dramatically improved performance, or stick with a
plan long after you've outgrown it, but that doesn't stop people from
wanting it, or other databases (or proprietary forks of this database)
from offering it, and I don't think it should.We have some hooks right now that offer a few options in this area,
but there are problems. The hook that I believe to be closest to the
right thing is this one:/*
* Allow a plugin to editorialize on the set of Paths for this base
* relation. It could add new paths (such as CustomPaths) by calling
* add_path(), or add_partial_path() if parallel aware. It could also
* delete or modify paths added by the core code.
*/
if (set_rel_pathlist_hook)
(*set_rel_pathlist_hook) (root, rel, rti, rte);Unfortunately, the part about the hook having the freedom to delete
paths isn't really true. Perhaps technically you can delete a path
that you don't want to be chosen, but any paths that were dominated by
the path you deleted have already been thrown away and it's too late
to get them back. You can modify paths if you don't want to change
their costs, but if you change their costs then you have the same
problem: the contents of the pathlist at the time that you see it are
determined by the costs that each path had when it was initially
added, and it's really too late to editorialize on that. So all you
can really do here in practice is add new paths.
set_join_pathlist_hook, which applies to joinrels, is similarly
limited. appendrels don't even have an equivalent of this hook.So, how could we do better?
I think there are two basic approaches that are possible here. If
someone sees a third option, let me know. First, we could allow users
to hook add_path() and add_partial_path(). That certainly provides the
flexibility on paper to accept or reject whatever paths you do or do
not want. However, I don't find this approach very appealing. One
problem is that it's likely to be fairly expensive, because add_path()
gets called A LOT. A second problem is that you don't actually get an
awful lot of context: I think anybody writing a hook would have to
write code to basically analyze each proposed path and figure out why
it was getting added and then decide what to do. In some cases that
might be fine, because for example accepting or rejecting paths based
on path type seems fairly straightforward with this approach, but as
soon as you want to do anything more complicated than that it starts
to seem difficult. If, for example, you want relation R1 to be the
driving table for the whole query plan, you're going to have to
determine whether or not that is the case for every single candidate
(partial) path that someone hands you, so you're going to end up
making that decision a whole lot of times. It doesn't sound
particularly fun. Third, even if you are doing something really simple
like trying to reject mergejoins, you've already lost the opportunity
to skip a bunch of work. If you had known when you started planning
the joinrel that you didn't care about mergejoins, you could have
skipped looking for merge-joinable clauses. Overall, while I wouldn't
be completely against further exploration of this option, I suspect
it's pretty hard to do anything useful with it.The other possible approach is to allow extensions to feed some
information into the planner before path generation and let that
influence which paths are generated. This is essentially what
pg_hint_plan is doing: it implements plan type hints by arranging to
flip the various enable_* GUCs on and off during the planning of
various rels. That's clever but ugly, and it ends up duplicating
substantial chunks of planner code due to the inadequacy of the
existing hooks. With some refactoring and some additional hooks, we
could make this much less ugly. But that gets at what I believe to be
the core difficulty of this approach, which is that the core planner
code needs to be somewhat aware of and on board with what the user or
the extension is trying to do. If an extension wants to force the join
order, that is likely to require different scaffolding than if it
wants to force the join methods which is again different from if a
hook wants to bias the query planner towards or against particular
indexes. Putting in hooks or other infrastructure that allows an
extension to control a particular aspect of planner behavior is to
some extent an endorsement of controlling the planner behavior in that
particular way. Since any amount of allowing the user to control the
planner tends to be controversial around here, that opens up the
spectre of putting a whole lot of effort into arguing about which
things extensions should be allowed to do, getting most of the patches
rejected, and ending up with nothing that's actually useful.But on the other hand, it's not like we have to design everything in a
greenfield. Other database systems have provided in-core, user-facing
features to control the planner for decades, and we can look at those
offerings -- and existing offerings in the PG space -- as we try to
judge whether a particular use case is totally insane. I am not here
to argue that everything that every system has done is completely
perfect and without notable flaws, but our own system has its own
share of flaws, and the fact that you can do very little when a
previously unproblematic query starts suddenly producing a bad plan is
definitely one of them. I believe we are long past the point where we
can simply hold our breath and pretend like there's no issue here. At
the very least, allowing extensions to control scan methods (including
choice of indexes), join methods, and join order (including which
table ends up on which side of a given join) and similar things for
aggregates and appendrels seems to me like it ought to be table
stakes. And those extensions shouldn't have to duplicate large chunks
of code or resort to gross hacks to do it. Eventually, maybe we'll
even want to have directly user-facing features to do some of this
stuff (in query hints, out of query hints, or whatever) but I think
opening the door up to extensions doing it is a good first step,
because (1) that allows different extensions to do different things
without taking a position on what the One Right Thing To Do is and (2)
if it becomes clear that something improvident has been done, it is a
lot easier to back out a hook or some C API change than it is to
back-out a user-visible feature. Or maybe we'll never want to expose a
user-visible feature here, but it can still be useful to enable
extensions.The attached patch, briefly mentioned above, essentially converts the
enable_* GUCs into RelOptInfo properties where the defaults are set by
the corresponding GUCs. The idea is that a hook could then change this
on a per-RelOptInfo basis before path generation happens. For
baserels, I believe that could be done from get_relation_info_hook for
baserels, and we could introduce something similar for other kinds of
rels. I don't think this is in any way the perfect approach. On the
one hand, it doesn't give you all the kinds of control over path
generation that you might want. On the other hand, the more I look at
what our enable_* GUCs actually do, the less impressed I am. IMHO,
things like enable_hashjoin make a lot of sense, but enable_sort seems
like it just controls an absolutely random smattering of behaviors in
a way that seems to me to have very little to recommend it, and I've
complained elsewhere about how enable_indexscan and
enable_indexonlyscan are really quite odd when you look at how they're
implemented. Still, this seemed like a fairly easy thing to do as a
way of demonstrating the kind of thing that we could do to provide
extensions with more control over planner behavior, and I believe it
would be concretely useful to pg_hint_plan in particular. But all that
said, as much as anything, I want to get some feedback on what
approaches and trade-offs people think might be acceptable here,
because there's not much point in me spending a bunch of time writing
code that everyone (or a critical mass of people) are going to hate.Thanks,
--
Robert Haas
EDB: http://www.enterprisedb.com
[2] https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraPostgreSQL.Optimize.html
I really admire this idea.
here is my confusion: Isn't the core of this idea whether to turn the planner into a framework? Personally, I think that under PostgreSQL's heap table storage, the optimizer might be better off focusing on optimizing the generation of execution plans. It’s possible that in some specific scenarios, developers might want to intervene in the generation of execution plans by extensions. I'm not sure if these scenarios usually occur when the storage structure is also extended by developers. If so, could existing solutions like "planner_hook" potentially solve the problem?
On Mon, Aug 26, 2024 at 3:28 PM Robert Haas <robertmhaas@gmail.com> wrote:
Well, I agree that this doesn't address everything you might want to
do, ... I will very happily propose more things to
address the other problems that I know about ...
In that vein, here's a new patch set where I've added a second patch
that allows extensions to control choice of index. It's 3 lines of new
code, plus 7 lines of comments and whitespace. Feeling inspired, I
also included a contrib module, initial_vowels_are_evil, to
demonstrate how this can be used by an extension that wants to disable
certain indexes but not others. This is obviously quite silly and we
might (or might not) want a more serious example in contrib, but it
demonstrates how easy this can be with just a tiny bit of core
infrastructure:
robert.haas=# load 'initial_vowels_are_evil';
LOAD
robert.haas=# explain select count(*) from pgbench_accounts;
QUERY PLAN
-----------------------------------------------------------------------------------------------------------------
Aggregate (cost=2854.29..2854.30 rows=1 width=8)
-> Index Only Scan using pgbench_accounts_pkey on pgbench_accounts
(cost=0.29..2604.29 rows=100000 width=0)
(2 rows)
robert.haas=# alter index pgbench_accounts_pkey rename to
evil_pgbench_accounts_pkey;
ALTER INDEX
robert.haas=# explain select count(*) from pgbench_accounts;
QUERY PLAN
------------------------------------------------------------------------------
Aggregate (cost=2890.00..2890.01 rows=1 width=8)
-> Seq Scan on pgbench_accounts (cost=0.00..2640.00 rows=100000 width=0)
(2 rows)
robert.haas=#
--
Robert Haas
EDB: http://www.enterprisedb.com
Attachments:
v2-0002-Allow-extensions-to-mark-an-individual-IndexOptIn.patchapplication/octet-stream; name=v2-0002-Allow-extensions-to-mark-an-individual-IndexOptIn.patchDownload
From b1371ddcd3d869919785025d08a76c22b43a98b5 Mon Sep 17 00:00:00 2001
From: Robert Haas <rhaas@postgresql.org>
Date: Tue, 27 Aug 2024 11:30:39 -0400
Subject: [PATCH v2 2/2] Allow extensions to mark an individual IndexOptInfo as
disabled.
As a demonstration that we definitely shouldn't commit to the real tree,
includes an example contrib module, initial_vowels_are_evil.
---
contrib/Makefile | 1 +
contrib/initial_vowels_are_evil/Makefile | 17 +++++++
.../initial_vowels_are_evil.c | 45 +++++++++++++++++++
contrib/initial_vowels_are_evil/meson.build | 12 +++++
contrib/meson.build | 1 +
src/backend/optimizer/util/pathnode.c | 8 ++++
src/include/nodes/pathnodes.h | 2 +
7 files changed, 86 insertions(+)
create mode 100644 contrib/initial_vowels_are_evil/Makefile
create mode 100644 contrib/initial_vowels_are_evil/initial_vowels_are_evil.c
create mode 100644 contrib/initial_vowels_are_evil/meson.build
diff --git a/contrib/Makefile b/contrib/Makefile
index abd780f277..5478b19832 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -24,6 +24,7 @@ SUBDIRS = \
hstore \
intagg \
intarray \
+ initial_vowels_are_evil \
isn \
lo \
ltree \
diff --git a/contrib/initial_vowels_are_evil/Makefile b/contrib/initial_vowels_are_evil/Makefile
new file mode 100644
index 0000000000..14955651cb
--- /dev/null
+++ b/contrib/initial_vowels_are_evil/Makefile
@@ -0,0 +1,17 @@
+# contrib/initial_vowels_are_evil/Makefile
+
+MODULE_big = initial_vowels_are_evil
+OBJS = \
+ $(WIN32RES) \
+ initial_vowels_are_evil.o
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/initial_vowels_are_evil
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/initial_vowels_are_evil/initial_vowels_are_evil.c b/contrib/initial_vowels_are_evil/initial_vowels_are_evil.c
new file mode 100644
index 0000000000..80ccce030a
--- /dev/null
+++ b/contrib/initial_vowels_are_evil/initial_vowels_are_evil.c
@@ -0,0 +1,45 @@
+/*-------------------------------------------------------------------------
+ *
+ * initial_vowels_are_evil.c
+ * disable indexes whose names start with a vowel
+ *
+ * Copyright (c) 2016-2024, PostgreSQL Global Development Group
+ *
+ * contrib/initial_vowels_are_evil/initial_vowels_are_evil.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "fmgr.h"
+#include "nodes/pathnodes.h"
+#include "optimizer/plancat.h"
+#include "utils/lsyscache.h"
+
+static void ivae_get_relation_info(PlannerInfo *root,
+ Oid relationObjectId,
+ bool inhparent,
+ RelOptInfo *rel);
+
+static get_relation_info_hook_type prev_get_relation_info_hook = NULL;
+
+PG_MODULE_MAGIC;
+
+void
+_PG_init(void)
+{
+ prev_get_relation_info_hook = get_relation_info_hook;
+ get_relation_info_hook = ivae_get_relation_info;
+}
+
+static void
+ivae_get_relation_info(PlannerInfo *root, Oid relationObjectId,
+ bool inhparent, RelOptInfo *rel)
+{
+ foreach_node(IndexOptInfo, index, rel->indexlist)
+ {
+ char *name = get_rel_name(index->indexoid);
+
+ if (name != NULL && strchr("aeiouAEIOU", name[0]) != NULL)
+ index->disabled = true;
+ }
+}
diff --git a/contrib/initial_vowels_are_evil/meson.build b/contrib/initial_vowels_are_evil/meson.build
new file mode 100644
index 0000000000..7f7fa72b53
--- /dev/null
+++ b/contrib/initial_vowels_are_evil/meson.build
@@ -0,0 +1,12 @@
+# Copyright (c) 2022-2024, PostgreSQL Global Development Group
+
+initial_vowels_are_evil_sources = files(
+ 'initial_vowels_are_evil.c',
+)
+
+initial_vowels_are_evil = shared_module('initial_vowels_are_evil',
+ initial_vowels_are_evil_sources,
+ kwargs: contrib_mod_args,
+)
+
+contrib_targets += initial_vowels_are_evil
diff --git a/contrib/meson.build b/contrib/meson.build
index 14a8906865..5f87711c2d 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -32,6 +32,7 @@ subdir('fuzzystrmatch')
subdir('hstore')
subdir('hstore_plperl')
subdir('hstore_plpython')
+subdir('initial_vowels_are_evil')
subdir('intagg')
subdir('intarray')
subdir('isn')
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 77ed747437..e85b4e5e73 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -1079,6 +1079,14 @@ create_index_path(PlannerInfo *root,
cost_index(pathnode, root, loop_count, partial_path);
+ /*
+ * cost_index will set disabled_nodes to 1 if this rel is not allowed to
+ * use index scans in general, but it doesn't have the IndexOptInfo to know
+ * whether this specific index has been disabled.
+ */
+ if (index->disabled)
+ pathnode->path.disabled_nodes = 1;
+
return pathnode;
}
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index bbff482cec..ece8ce76ca 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -1209,6 +1209,8 @@ struct IndexOptInfo
bool unique;
/* is uniqueness enforced immediately? */
bool immediate;
+ /* true if paths using this index should be marked disabled */
+ bool disabled;
/* true if index doesn't really exist */
bool hypothetical;
--
2.39.3 (Apple Git-145)
v2-0001-Convert-enable_-GUCs-into-per-RelOptInfo-values-w.patchapplication/octet-stream; name=v2-0001-Convert-enable_-GUCs-into-per-RelOptInfo-values-w.patchDownload
From baee6cd9e575728650813152af0d4d2d9c96674f Mon Sep 17 00:00:00 2001
From: Robert Haas <rhaas@postgresql.org>
Date: Mon, 26 Aug 2024 12:27:08 -0400
Subject: [PATCH v2 1/2] Convert enable_* GUCs into per-RelOptInfo values with
GUCs setting defaults.
---
contrib/postgres_fdw/postgres_fdw.c | 5 +-
src/backend/optimizer/path/allpaths.c | 15 ++--
src/backend/optimizer/path/costsize.c | 107 +++++++++++++++++-------
src/backend/optimizer/path/indxpath.c | 2 +-
src/backend/optimizer/path/joinpath.c | 53 ++++++------
src/backend/optimizer/path/pathkeys.c | 3 +-
src/backend/optimizer/path/tidpath.c | 11 +--
src/backend/optimizer/plan/createplan.c | 26 +++---
src/backend/optimizer/plan/planner.c | 58 ++++++++-----
src/backend/optimizer/plan/subselect.c | 9 +-
src/backend/optimizer/prep/prepunion.c | 20 +++--
src/backend/optimizer/util/pathnode.c | 21 ++---
src/backend/optimizer/util/plancat.c | 3 +-
src/backend/optimizer/util/relnode.c | 6 +-
src/backend/utils/misc/guc_tables.c | 37 ++++----
src/include/nodes/pathnodes.h | 29 +++++++
src/include/optimizer/cost.h | 29 ++++++-
src/include/optimizer/planmain.h | 2 +-
18 files changed, 288 insertions(+), 148 deletions(-)
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index adc62576d1..1df4ddf268 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -439,6 +439,7 @@ static void get_remote_estimate(const char *sql,
Cost *startup_cost,
Cost *total_cost);
static void adjust_foreign_grouping_path_cost(PlannerInfo *root,
+ RelOptInfo *rel,
List *pathkeys,
double retrieved_rows,
double width,
@@ -3489,7 +3490,7 @@ estimate_path_cost_size(PlannerInfo *root,
{
Assert(foreignrel->reloptkind == RELOPT_UPPER_REL &&
fpinfo->stage == UPPERREL_GROUP_AGG);
- adjust_foreign_grouping_path_cost(root, pathkeys,
+ adjust_foreign_grouping_path_cost(root, foreignrel, pathkeys,
retrieved_rows, width,
fpextra->limit_tuples,
&disabled_nodes,
@@ -3644,6 +3645,7 @@ get_remote_estimate(const char *sql, PGconn *conn,
*/
static void
adjust_foreign_grouping_path_cost(PlannerInfo *root,
+ RelOptInfo *rel,
List *pathkeys,
double retrieved_rows,
double width,
@@ -3667,6 +3669,7 @@ adjust_foreign_grouping_path_cost(PlannerInfo *root,
cost_sort(&sort_path,
root,
+ rel,
pathkeys,
0,
*p_startup_cost + *p_run_cost,
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 057b4b79eb..8645244f84 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -970,7 +970,7 @@ set_append_rel_size(PlannerInfo *root, RelOptInfo *rel,
* flag; currently, we only consider partitionwise joins with the baserel
* if its targetlist doesn't contain a whole-row Var.
*/
- if (enable_partitionwise_join &&
+ if (REL_CAN_USE_PATH(rel, PartitionwiseJoin) &&
rel->reloptkind == RELOPT_BASEREL &&
rte->relkind == RELKIND_PARTITIONED_TABLE &&
bms_is_empty(rel->attr_needed[InvalidAttrNumber - rel->min_attr]))
@@ -1325,7 +1325,8 @@ add_paths_to_append_rel(PlannerInfo *root, RelOptInfo *rel,
double partial_rows = -1;
/* If appropriate, consider parallel append */
- pa_subpaths_valid = enable_parallel_append && rel->consider_parallel;
+ pa_subpaths_valid = REL_CAN_USE_PATH(rel, ParallelAppend)
+ && rel->consider_parallel;
/*
* For every non-dummy child, remember the cheapest path. Also, identify
@@ -1535,7 +1536,7 @@ add_paths_to_append_rel(PlannerInfo *root, RelOptInfo *rel,
* partitions vs. an unpartitioned table with the same data, so the
* use of some kind of log-scaling here seems to make some sense.
*/
- if (enable_parallel_append)
+ if (REL_CAN_USE_PATH(rel, ParallelAppend))
{
parallel_workers = Max(parallel_workers,
pg_leftmost_one_pos32(list_length(live_childrels)) + 1);
@@ -1547,7 +1548,7 @@ add_paths_to_append_rel(PlannerInfo *root, RelOptInfo *rel,
/* Generate a partial append path. */
appendpath = create_append_path(root, rel, NIL, partial_subpaths,
NIL, NULL, parallel_workers,
- enable_parallel_append,
+ REL_CAN_USE_PATH(rel, ParallelAppend),
-1);
/*
@@ -3259,7 +3260,8 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
* input path).
*/
if (subpath != cheapest_partial_path &&
- (presorted_keys == 0 || !enable_incremental_sort))
+ (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(rel, IncrementalSort)))
continue;
/*
@@ -3274,7 +3276,8 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
* output. Here we add an explicit sort to match the useful
* ordering.
*/
- if (presorted_keys == 0 || !enable_incremental_sort)
+ if (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(rel, IncrementalSort))
{
subpath = (Path *) create_sort_path(root,
rel,
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index e1523d15df..3dfa22fdcd 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -163,6 +163,7 @@ bool enable_parallel_hash = true;
bool enable_partition_pruning = true;
bool enable_presorted_aggregate = true;
bool enable_async_append = true;
+uint32 default_path_type_mask = PathTypeMaskAll;
typedef struct
{
@@ -283,6 +284,38 @@ clamp_cardinality_to_long(Cardinality x)
return (x < (double) LONG_MAX) ? (long) x : LONG_MAX;
}
+/*
+ * Define assign hooks for each enable_<whatever> GUC that affects
+ * default_path_type_mask. These are all basically identical, so we use
+ * a templating macro to define them.
+ */
+#define define_assign_hook(gucname, type) \
+ void \
+ gucname ## _assign_hook(bool newval, void *extra) \
+ { \
+ if (newval) \
+ default_path_type_mask |= PathType ## type; \
+ else \
+ default_path_type_mask &= ~(PathType ## type); \
+ }
+define_assign_hook(enable_bitmapscan, BitmapScan)
+define_assign_hook(enable_gathermerge, GatherMerge)
+define_assign_hook(enable_hashagg, HashAgg)
+define_assign_hook(enable_hashjoin, HashJoin)
+define_assign_hook(enable_incremental_sort, IncrementalSort)
+define_assign_hook(enable_indexscan, IndexScan)
+define_assign_hook(enable_indexonlyscan, IndexOnlyScan)
+define_assign_hook(enable_material, Material)
+define_assign_hook(enable_memoize, Memoize)
+define_assign_hook(enable_mergejoin, MergeJoin)
+define_assign_hook(enable_nestloop, NestLoop)
+define_assign_hook(enable_parallel_append, ParallelAppend)
+define_assign_hook(enable_parallel_hash, ParallelHash)
+define_assign_hook(enable_partitionwise_join, PartitionwiseJoin)
+define_assign_hook(enable_partitionwise_aggregate, PartitionwiseAggregate)
+define_assign_hook(enable_seqscan, SeqScan)
+define_assign_hook(enable_sort, Sort)
+define_assign_hook(enable_tidscan, TIDScan)
/*
* cost_seqscan
@@ -354,7 +387,7 @@ cost_seqscan(Path *path, PlannerInfo *root,
path->rows = clamp_row_est(path->rows / parallel_divisor);
}
- path->disabled_nodes = enable_seqscan ? 0 : 1;
+ path->disabled_nodes = REL_CAN_USE_PATH(baserel, SeqScan) ? 0 : 1;
path->startup_cost = startup_cost;
path->total_cost = startup_cost + cpu_run_cost + disk_run_cost;
}
@@ -533,7 +566,7 @@ cost_gather_merge(GatherMergePath *path, PlannerInfo *root,
run_cost += parallel_tuple_cost * path->path.rows * 1.05;
path->path.disabled_nodes = input_disabled_nodes
- + (enable_gathermerge ? 0 : 1);
+ + (REL_CAN_USE_PATH(rel, GatherMerge) ? 0 : 1);
path->path.startup_cost = startup_cost + input_startup_cost;
path->path.total_cost = (startup_cost + run_cost + input_total_cost);
}
@@ -615,7 +648,8 @@ cost_index(IndexPath *path, PlannerInfo *root, double loop_count,
}
/* we don't need to check enable_indexonlyscan; indxpath.c does that */
- path->path.disabled_nodes = enable_indexscan ? 0 : 1;
+ path->path.disabled_nodes =
+ REL_CAN_USE_PATH(baserel, IndexScan) ? 0 : 1;
/*
* Call index-access-method-specific code to estimate the processing cost
@@ -1109,7 +1143,7 @@ cost_bitmap_heap_scan(Path *path, PlannerInfo *root, RelOptInfo *baserel,
startup_cost += path->pathtarget->cost.startup;
run_cost += path->pathtarget->cost.per_tuple * path->rows;
- path->disabled_nodes = enable_bitmapscan ? 0 : 1;
+ path->disabled_nodes = REL_CAN_USE_PATH(baserel, BitmapScan) ? 0 : 1;
path->startup_cost = startup_cost;
path->total_cost = startup_cost + run_cost;
}
@@ -1287,10 +1321,11 @@ cost_tidscan(Path *path, PlannerInfo *root,
/*
* We must use a TID scan for CurrentOfExpr; in any other case, we
- * should be generating a TID scan only if enable_tidscan=true. Also,
- * if CurrentOfExpr is the qual, there should be only one.
+ * should be generating a TID scan only if this TID scans are enabled
+ * for this rel. Also, if CurrentOfExpr is the qual, there should be
+ * only one.
*/
- Assert(enable_tidscan || IsA(qual, CurrentOfExpr));
+ Assert(REL_CAN_USE_PATH(baserel, TIDScan) || IsA(qual, CurrentOfExpr));
Assert(list_length(tidquals) == 1 || !IsA(qual, CurrentOfExpr));
if (IsA(qual, ScalarArrayOpExpr))
@@ -1342,8 +1377,8 @@ cost_tidscan(Path *path, PlannerInfo *root,
/*
* There are assertions above verifying that we only reach this function
- * either when enable_tidscan=true or when the TID scan is the only legal
- * path, so it's safe to set disabled_nodes to zero here.
+ * either when TID scans are enabled for this rel or when a TID scan is
+ * the only legal path, so it's safe to set disabled_nodes to zero here.
*/
path->disabled_nodes = 0;
path->startup_cost = startup_cost;
@@ -1438,8 +1473,8 @@ cost_tidrangescan(Path *path, PlannerInfo *root,
startup_cost += path->pathtarget->cost.startup;
run_cost += path->pathtarget->cost.per_tuple * path->rows;
- /* we should not generate this path type when enable_tidscan=false */
- Assert(enable_tidscan);
+ /* we should not generate this path type when TID scans are disabled */
+ Assert(REL_CAN_USE_PATH(baserel, TIDScan));
path->disabled_nodes = 0;
path->startup_cost = startup_cost;
path->total_cost = startup_cost + run_cost;
@@ -2120,8 +2155,11 @@ cost_incremental_sort(Path *path,
path->rows = input_tuples;
- /* should not generate these paths when enable_incremental_sort=false */
- Assert(enable_incremental_sort);
+ /*
+ * If incremental sort is not enabled here, we should not have generated a
+ * path of this type.
+ */
+ Assert(REL_CAN_USE_PATH(path->parent, IncrementalSort));
path->disabled_nodes = input_disabled_nodes;
path->startup_cost = startup_cost;
@@ -2141,7 +2179,7 @@ cost_incremental_sort(Path *path,
* of sort keys, which all callers *could* supply.)
*/
void
-cost_sort(Path *path, PlannerInfo *root,
+cost_sort(Path *path, PlannerInfo *root, RelOptInfo *rel,
List *pathkeys, int input_disabled_nodes,
Cost input_cost, double tuples, int width,
Cost comparison_cost, int sort_mem,
@@ -2159,7 +2197,8 @@ cost_sort(Path *path, PlannerInfo *root,
startup_cost += input_cost;
path->rows = tuples;
- path->disabled_nodes = input_disabled_nodes + (enable_sort ? 0 : 1);
+ path->disabled_nodes = input_disabled_nodes +
+ (REL_CAN_USE_PATH(rel, Sort) ? 0 : 1);
path->startup_cost = startup_cost;
path->total_cost = startup_cost + run_cost;
}
@@ -2321,6 +2360,7 @@ cost_append(AppendPath *apath)
*/
cost_sort(&sort_path,
NULL, /* doesn't currently need root */
+ apath->path.parent,
pathkeys,
subpath->disabled_nodes,
subpath->total_cost,
@@ -2480,7 +2520,7 @@ cost_merge_append(Path *path, PlannerInfo *root,
* occur only on rescan, which is estimated in cost_rescan.
*/
void
-cost_material(Path *path,
+cost_material(Path *path, RelOptInfo *rel,
int input_disabled_nodes,
Cost input_startup_cost, Cost input_total_cost,
double tuples, int width)
@@ -2519,7 +2559,8 @@ cost_material(Path *path,
run_cost += seq_page_cost * npages;
}
- path->disabled_nodes = input_disabled_nodes + (enable_material ? 0 : 1);
+ path->disabled_nodes = input_disabled_nodes +
+ (REL_CAN_USE_PATH(rel, Material) ? 0 : 1);
path->startup_cost = startup_cost;
path->total_cost = startup_cost + run_cost;
}
@@ -2679,7 +2720,7 @@ cost_memoize_rescan(PlannerInfo *root, MemoizePath *mpath,
* are for appropriately-sorted input.
*/
void
-cost_agg(Path *path, PlannerInfo *root,
+cost_agg(Path *path, PlannerInfo *root, RelOptInfo *rel,
AggStrategy aggstrategy, const AggClauseCosts *aggcosts,
int numGroupCols, double numGroups,
List *quals,
@@ -2738,7 +2779,7 @@ cost_agg(Path *path, PlannerInfo *root,
/* Here we are able to deliver output on-the-fly */
startup_cost = input_startup_cost;
total_cost = input_total_cost;
- if (aggstrategy == AGG_MIXED && !enable_hashagg)
+ if (aggstrategy == AGG_MIXED && !REL_CAN_USE_PATH(rel, HashAgg))
++disabled_nodes;
/* calcs phrased this way to match HASHED case, see note above */
total_cost += aggcosts->transCost.startup;
@@ -2753,7 +2794,7 @@ cost_agg(Path *path, PlannerInfo *root,
{
/* must be AGG_HASHED */
startup_cost = input_total_cost;
- if (!enable_hashagg)
+ if (!REL_CAN_USE_PATH(rel, HashAgg))
++disabled_nodes;
startup_cost += aggcosts->transCost.startup;
startup_cost += aggcosts->transCost.per_tuple * input_tuples;
@@ -3266,7 +3307,7 @@ cost_group(Path *path, PlannerInfo *root,
*/
void
initial_cost_nestloop(PlannerInfo *root, JoinCostWorkspace *workspace,
- JoinType jointype,
+ RelOptInfo *joinrel, JoinType jointype,
Path *outer_path, Path *inner_path,
JoinPathExtraData *extra)
{
@@ -3280,7 +3321,7 @@ initial_cost_nestloop(PlannerInfo *root, JoinCostWorkspace *workspace,
Cost inner_rescan_run_cost;
/* Count up disabled nodes. */
- disabled_nodes = enable_nestloop ? 0 : 1;
+ disabled_nodes = REL_CAN_USE_PATH(joinrel, NestLoop) ? 0 : 1;
disabled_nodes += inner_path->disabled_nodes;
disabled_nodes += outer_path->disabled_nodes;
@@ -3549,7 +3590,7 @@ final_cost_nestloop(PlannerInfo *root, NestPath *path,
*/
void
initial_cost_mergejoin(PlannerInfo *root, JoinCostWorkspace *workspace,
- JoinType jointype,
+ RelOptInfo *joinrel, JoinType jointype,
List *mergeclauses,
Path *outer_path, Path *inner_path,
List *outersortkeys, List *innersortkeys,
@@ -3676,7 +3717,7 @@ initial_cost_mergejoin(PlannerInfo *root, JoinCostWorkspace *workspace,
Assert(outerstartsel <= outerendsel);
Assert(innerstartsel <= innerendsel);
- disabled_nodes = enable_mergejoin ? 0 : 1;
+ disabled_nodes = REL_CAN_USE_PATH(joinrel, MergeJoin) ? 0 : 1;
/* cost of source data */
@@ -3684,6 +3725,7 @@ initial_cost_mergejoin(PlannerInfo *root, JoinCostWorkspace *workspace,
{
cost_sort(&sort_path,
root,
+ outer_path->parent,
outersortkeys,
outer_path->disabled_nodes,
outer_path->total_cost,
@@ -3713,6 +3755,7 @@ initial_cost_mergejoin(PlannerInfo *root, JoinCostWorkspace *workspace,
{
cost_sort(&sort_path,
root,
+ inner_path->parent,
innersortkeys,
inner_path->disabled_nodes,
inner_path->total_cost,
@@ -3793,6 +3836,7 @@ final_cost_mergejoin(PlannerInfo *root, MergePath *path,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra)
{
+ RelOptInfo *joinrel = path->jpath.path.parent;
Path *outer_path = path->jpath.outerjoinpath;
Path *inner_path = path->jpath.innerjoinpath;
double inner_path_rows = inner_path->rows;
@@ -3946,7 +3990,8 @@ final_cost_mergejoin(PlannerInfo *root, MergePath *path,
* Prefer materializing if it looks cheaper, unless the user has asked to
* suppress materialization.
*/
- else if (enable_material && mat_inner_cost < bare_inner_cost)
+ else if (REL_CAN_USE_PATH(joinrel, Material) &&
+ mat_inner_cost < bare_inner_cost)
path->materialize_inner = true;
/*
@@ -3961,7 +4006,7 @@ final_cost_mergejoin(PlannerInfo *root, MergePath *path,
* selected as the input of a mergejoin, and they don't support
* mark/restore at present.
*
- * We don't test the value of enable_material here, because
+ * We don't test whether a MaterialPath is allowed here, because
* materialization is required for correctness in this case, and turning
* it off does not entitle us to deliver an invalid plan.
*/
@@ -3977,10 +4022,10 @@ final_cost_mergejoin(PlannerInfo *root, MergePath *path,
* though.
*
* Since materialization is a performance optimization in this case,
- * rather than necessary for correctness, we skip it if enable_material is
- * off.
+ * rather than necessary for correctness, we skip it if MaterialPath is
+ * not allowed here.
*/
- else if (enable_material && innersortkeys != NIL &&
+ else if (REL_CAN_USE_PATH(joinrel, Material) && innersortkeys != NIL &&
relation_byte_size(inner_path_rows,
inner_path->pathtarget->width) >
(work_mem * 1024L))
@@ -4113,7 +4158,7 @@ cached_scansel(PlannerInfo *root, RestrictInfo *rinfo, PathKey *pathkey)
*/
void
initial_cost_hashjoin(PlannerInfo *root, JoinCostWorkspace *workspace,
- JoinType jointype,
+ RelOptInfo *joinrel, JoinType jointype,
List *hashclauses,
Path *outer_path, Path *inner_path,
JoinPathExtraData *extra,
@@ -4132,7 +4177,7 @@ initial_cost_hashjoin(PlannerInfo *root, JoinCostWorkspace *workspace,
size_t space_allowed; /* unused */
/* Count up disabled nodes. */
- disabled_nodes = enable_hashjoin ? 0 : 1;
+ disabled_nodes = REL_CAN_USE_PATH(joinrel, HashJoin) ? 0 : 1;
disabled_nodes += inner_path->disabled_nodes;
disabled_nodes += outer_path->disabled_nodes;
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
index c0fcc7d78d..be183db4ce 100644
--- a/src/backend/optimizer/path/indxpath.c
+++ b/src/backend/optimizer/path/indxpath.c
@@ -1736,7 +1736,7 @@ check_index_only(RelOptInfo *rel, IndexOptInfo *index)
int i;
/* Index-only scans must be enabled */
- if (!enable_indexonlyscan)
+ if (!REL_CAN_USE_PATH(rel, IndexOnlyScan))
return false;
/*
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index b0e8c94dfc..5f0040cd1b 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -205,10 +205,10 @@ add_paths_to_joinrel(PlannerInfo *root,
/*
* Find potential mergejoin clauses. We can skip this if we are not
* interested in doing a mergejoin. However, mergejoin may be our only
- * way of implementing a full outer join, so override enable_mergejoin if
- * it's a full join.
+ * way of implementing a full outer join, so disregard the result of
+ * REL_CAN_USE_PATH() if it's a full join.
*/
- if (enable_mergejoin || jointype == JOIN_FULL)
+ if (REL_CAN_USE_PATH(joinrel, MergeJoin) || jointype == JOIN_FULL)
extra.mergeclause_list = select_mergejoin_clauses(root,
joinrel,
outerrel,
@@ -316,10 +316,11 @@ add_paths_to_joinrel(PlannerInfo *root,
/*
* 4. Consider paths where both outer and inner relations must be hashed
- * before being joined. As above, disregard enable_hashjoin for full
- * joins, because there may be no other alternative.
+ * before being joined. As above, the result of REL_CAN_USE_PATH()
+ * doesn't matter for full joins, because there may be no other
+ * alternative.
*/
- if (enable_hashjoin || jointype == JOIN_FULL)
+ if (REL_CAN_USE_PATH(joinrel, HashJoin) || jointype == JOIN_FULL)
hash_inner_and_outer(root, joinrel, outerrel, innerrel,
jointype, &extra);
@@ -672,7 +673,7 @@ extract_lateral_vars_from_PHVs(PlannerInfo *root, Relids innerrelids)
* we do not have a way to extract cache keys from joinrels.
*/
static Path *
-get_memoize_path(PlannerInfo *root, RelOptInfo *innerrel,
+get_memoize_path(PlannerInfo *root, RelOptInfo *joinrel, RelOptInfo *innerrel,
RelOptInfo *outerrel, Path *inner_path,
Path *outer_path, JoinType jointype,
JoinPathExtraData *extra)
@@ -684,7 +685,7 @@ get_memoize_path(PlannerInfo *root, RelOptInfo *innerrel,
List *ph_lateral_vars;
/* Obviously not if it's disabled */
- if (!enable_memoize)
+ if (!REL_CAN_USE_PATH(joinrel, Memoize))
return NULL;
/*
@@ -912,7 +913,7 @@ try_nestloop_path(PlannerInfo *root,
* The latter two steps are expensive enough to make this two-phase
* methodology worthwhile.
*/
- initial_cost_nestloop(root, &workspace, jointype,
+ initial_cost_nestloop(root, &workspace, joinrel, jointype,
outer_path, inner_path, extra);
if (add_path_precheck(joinrel, workspace.disabled_nodes,
@@ -997,7 +998,7 @@ try_partial_nestloop_path(PlannerInfo *root,
* Before creating a path, get a quick lower bound on what it is likely to
* cost. Bail out right away if it looks terrible.
*/
- initial_cost_nestloop(root, &workspace, jointype,
+ initial_cost_nestloop(root, &workspace, joinrel, jointype,
outer_path, inner_path, extra);
if (!add_partial_path_precheck(joinrel, workspace.disabled_nodes,
workspace.total_cost, pathkeys))
@@ -1092,7 +1093,7 @@ try_mergejoin_path(PlannerInfo *root,
/*
* See comments in try_nestloop_path().
*/
- initial_cost_mergejoin(root, &workspace, jointype, mergeclauses,
+ initial_cost_mergejoin(root, &workspace, joinrel, jointype, mergeclauses,
outer_path, inner_path,
outersortkeys, innersortkeys,
extra);
@@ -1164,7 +1165,7 @@ try_partial_mergejoin_path(PlannerInfo *root,
/*
* See comments in try_partial_nestloop_path().
*/
- initial_cost_mergejoin(root, &workspace, jointype, mergeclauses,
+ initial_cost_mergejoin(root, &workspace, joinrel, jointype, mergeclauses,
outer_path, inner_path,
outersortkeys, innersortkeys,
extra);
@@ -1236,7 +1237,7 @@ try_hashjoin_path(PlannerInfo *root,
* See comments in try_nestloop_path(). Also note that hashjoin paths
* never have any output pathkeys, per comments in create_hashjoin_path.
*/
- initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
+ initial_cost_hashjoin(root, &workspace, joinrel, jointype, hashclauses,
outer_path, inner_path, extra, false);
if (add_path_precheck(joinrel, workspace.disabled_nodes,
@@ -1298,7 +1299,7 @@ try_partial_hashjoin_path(PlannerInfo *root,
* Before creating a path, get a quick lower bound on what it is likely to
* cost. Bail out right away if it looks terrible.
*/
- initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
+ initial_cost_hashjoin(root, &workspace, joinrel, jointype, hashclauses,
outer_path, inner_path, extra, parallel_hash);
if (!add_partial_path_precheck(joinrel, workspace.disabled_nodes,
workspace.total_cost, NIL))
@@ -1899,10 +1900,11 @@ match_unsorted_outer(PlannerInfo *root,
{
/*
* Consider materializing the cheapest inner path, unless
- * enable_material is off or the path in question materializes its
- * output anyway.
+ * materialization is disabled or the path in question materializes
+ * its output anyway.
*/
- if (enable_material && inner_cheapest_total != NULL &&
+ if (REL_CAN_USE_PATH(innerrel, Material) &&
+ inner_cheapest_total != NULL &&
!ExecMaterializesOutput(inner_cheapest_total->pathtype))
matpath = (Path *)
create_material_path(innerrel, inner_cheapest_total);
@@ -1982,7 +1984,7 @@ match_unsorted_outer(PlannerInfo *root,
* Try generating a memoize path and see if that makes the
* nested loop any cheaper.
*/
- mpath = get_memoize_path(root, innerrel, outerrel,
+ mpath = get_memoize_path(root, joinrel, innerrel, outerrel,
innerpath, outerpath, jointype,
extra);
if (mpath != NULL)
@@ -2134,13 +2136,14 @@ consider_parallel_nestloop(PlannerInfo *root,
/*
* Consider materializing the cheapest inner path, unless: 1) we're doing
* JOIN_UNIQUE_INNER, because in this case we have to unique-ify the
- * cheapest inner path, 2) enable_material is off, 3) the cheapest inner
- * path is not parallel-safe, 4) the cheapest inner path is parameterized
- * by the outer rel, or 5) the cheapest inner path materializes its output
- * anyway.
+ * cheapest inner path, 2) MaterialPath is allowed for this rel, 3) the
+ * cheapest inner path is not parallel-safe, 4) the cheapest inner path is
+ * parameterized by the outer rel, or 5) the cheapest inner path
+ * materializes its output anyway.
*/
if (save_jointype != JOIN_UNIQUE_INNER &&
- enable_material && inner_cheapest_total->parallel_safe &&
+ REL_CAN_USE_PATH(joinrel, Material) &&
+ inner_cheapest_total->parallel_safe &&
!PATH_PARAM_BY_REL(inner_cheapest_total, outerrel) &&
!ExecMaterializesOutput(inner_cheapest_total->pathtype))
{
@@ -2198,7 +2201,7 @@ consider_parallel_nestloop(PlannerInfo *root,
* Try generating a memoize path and see if that makes the nested
* loop any cheaper.
*/
- mpath = get_memoize_path(root, innerrel, outerrel,
+ mpath = get_memoize_path(root, joinrel, innerrel, outerrel,
innerpath, outerpath, jointype,
extra);
if (mpath != NULL)
@@ -2416,7 +2419,7 @@ hash_inner_and_outer(PlannerInfo *root,
*/
if (innerrel->partial_pathlist != NIL &&
save_jointype != JOIN_UNIQUE_INNER &&
- enable_parallel_hash)
+ REL_CAN_USE_PATH(joinrel, ParallelHash))
{
cheapest_partial_inner =
(Path *) linitial(innerrel->partial_pathlist);
diff --git a/src/backend/optimizer/path/pathkeys.c b/src/backend/optimizer/path/pathkeys.c
index e25798972f..26a0f5c0c3 100644
--- a/src/backend/optimizer/path/pathkeys.c
+++ b/src/backend/optimizer/path/pathkeys.c
@@ -505,7 +505,8 @@ get_useful_group_keys_orderings(PlannerInfo *root, Path *path)
root->num_groupby_pathkeys);
if (n > 0 &&
- (enable_incremental_sort || n == root->num_groupby_pathkeys) &&
+ (REL_CAN_USE_PATH(path->parent, IncrementalSort) ||
+ n == root->num_groupby_pathkeys) &&
compare_pathkeys(pathkeys, root->group_pathkeys) != PATHKEYS_EQUAL)
{
info = makeNode(GroupByOrdering);
diff --git a/src/backend/optimizer/path/tidpath.c b/src/backend/optimizer/path/tidpath.c
index b0323b26ec..ed447916ec 100644
--- a/src/backend/optimizer/path/tidpath.c
+++ b/src/backend/optimizer/path/tidpath.c
@@ -505,13 +505,14 @@ create_tidscan_paths(PlannerInfo *root, RelOptInfo *rel)
* If any suitable quals exist in the rel's baserestrict list, generate a
* plain (unparameterized) TidPath with them.
*
- * We skip this when enable_tidscan = false, except when the qual is
- * CurrentOfExpr. In that case, a TID scan is the only correct path.
+ * We skip this when TID scans are disabled for this rel, except when
+ * the qual is CurrentOfExpr. In that case, a TID scan is the only
+ * correct path.
*/
tidquals = TidQualFromRestrictInfoList(root, rel->baserestrictinfo, rel,
&isCurrentOf);
- if (tidquals != NIL && (enable_tidscan || isCurrentOf))
+ if (tidquals != NIL && (REL_CAN_USE_PATH(rel, TIDScan) || isCurrentOf))
{
/*
* This path uses no join clauses, but it could still have required
@@ -532,8 +533,8 @@ create_tidscan_paths(PlannerInfo *root, RelOptInfo *rel)
return true;
}
- /* Skip the rest if TID scans are disabled. */
- if (!enable_tidscan)
+ /* Skip the rest if TID scans are disabled for this rel. */
+ if (!REL_CAN_USE_PATH(rel, TIDScan))
return false;
/*
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 8e0e5977a9..3f3770bfa7 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -177,8 +177,8 @@ static List *get_switched_clauses(List *clauses, Relids outerrelids);
static List *order_qual_clauses(PlannerInfo *root, List *clauses);
static void copy_generic_path_info(Plan *dest, Path *src);
static void copy_plan_costsize(Plan *dest, Plan *src);
-static void label_sort_with_costsize(PlannerInfo *root, Sort *plan,
- double limit_tuples);
+static void label_sort_with_costsize(PlannerInfo *root, RelOptInfo *rel,
+ Sort *plan, double limit_tuples);
static SeqScan *make_seqscan(List *qptlist, List *qpqual, Index scanrelid);
static SampleScan *make_samplescan(List *qptlist, List *qpqual, Index scanrelid,
TableSampleClause *tsc);
@@ -1361,7 +1361,8 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
sortColIdx, sortOperators,
collations, nullsFirst);
- label_sort_with_costsize(root, sort, best_path->limit_tuples);
+ label_sort_with_costsize(root, rel, sort,
+ best_path->limit_tuples);
subplan = (Plan *) sort;
}
}
@@ -1533,7 +1534,7 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
sortColIdx, sortOperators,
collations, nullsFirst);
- label_sort_with_costsize(root, sort, best_path->limit_tuples);
+ label_sort_with_costsize(root, rel, sort, best_path->limit_tuples);
subplan = (Plan *) sort;
}
@@ -1900,7 +1901,7 @@ create_unique_plan(PlannerInfo *root, UniquePath *best_path, int flags)
groupColPos++;
}
sort = make_sort_from_sortclauses(sortList, subplan);
- label_sort_with_costsize(root, sort, -1.0);
+ label_sort_with_costsize(root, best_path->path.parent, sort, -1.0);
plan = (Plan *) make_unique_from_sortclauses((Plan *) sort, sortList);
}
@@ -4527,7 +4528,8 @@ create_mergejoin_plan(PlannerInfo *root,
best_path->outersortkeys,
outer_relids);
- label_sort_with_costsize(root, sort, -1.0);
+ label_sort_with_costsize(root, best_path->jpath.path.parent,
+ sort, -1.0);
outer_plan = (Plan *) sort;
outerpathkeys = best_path->outersortkeys;
}
@@ -4541,7 +4543,8 @@ create_mergejoin_plan(PlannerInfo *root,
best_path->innersortkeys,
inner_relids);
- label_sort_with_costsize(root, sort, -1.0);
+ label_sort_with_costsize(root, best_path->jpath.path.parent,
+ sort, -1.0);
inner_plan = (Plan *) sort;
innerpathkeys = best_path->innersortkeys;
}
@@ -5442,7 +5445,8 @@ copy_plan_costsize(Plan *dest, Plan *src)
* limit_tuples is as for cost_sort (in particular, pass -1 if no limit)
*/
static void
-label_sort_with_costsize(PlannerInfo *root, Sort *plan, double limit_tuples)
+label_sort_with_costsize(PlannerInfo *root, RelOptInfo *rel, Sort *plan,
+ double limit_tuples)
{
Plan *lefttree = plan->plan.lefttree;
Path sort_path; /* dummy for result of cost_sort */
@@ -5453,7 +5457,7 @@ label_sort_with_costsize(PlannerInfo *root, Sort *plan, double limit_tuples)
*/
Assert(IsA(plan, Sort));
- cost_sort(&sort_path, root, NIL,
+ cost_sort(&sort_path, root, rel, NIL,
lefttree->total_cost,
plan->plan.disabled_nodes,
lefttree->plan_rows,
@@ -6524,7 +6528,7 @@ make_material(Plan *lefttree)
* Path representation, but it's not worth the trouble yet.
*/
Plan *
-materialize_finished_plan(Plan *subplan)
+materialize_finished_plan(Plan *subplan, RelOptInfo *rel)
{
Plan *matplan;
Path matpath; /* dummy for result of cost_material */
@@ -6549,7 +6553,7 @@ materialize_finished_plan(Plan *subplan)
subplan->total_cost -= initplan_cost;
/* Set cost data */
- cost_material(&matpath,
+ cost_material(&matpath, rel,
subplan->disabled_nodes,
subplan->startup_cost,
subplan->total_cost,
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index b5827d3980..c0f431cf0d 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -427,7 +427,7 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions,
if (cursorOptions & CURSOR_OPT_SCROLL)
{
if (!ExecSupportsBackwardScan(top_plan))
- top_plan = materialize_finished_plan(top_plan);
+ top_plan = materialize_finished_plan(top_plan, final_rel);
}
/*
@@ -3831,7 +3831,8 @@ create_grouping_paths(PlannerInfo *root,
* support grouping sets. create_ordinary_grouping_paths() will check
* additional conditions, such as whether input_rel is partitioned.
*/
- if (enable_partitionwise_aggregate && !parse->groupingSets)
+ if (REL_CAN_USE_PATH(grouped_rel, PartitionwiseAggregate) &&
+ !parse->groupingSets)
extra.patype = PARTITIONWISE_AGGREGATE_FULL;
else
extra.patype = PARTITIONWISE_AGGREGATE_NONE;
@@ -4634,7 +4635,8 @@ create_one_window_path(PlannerInfo *root,
* No presorted keys or incremental sort disabled, just perform a
* complete sort.
*/
- if (presorted_keys == 0 || !enable_incremental_sort)
+ if (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(window_rel, IncrementalSort))
path = (Path *) create_sort_path(root, window_rel,
path,
window_pathkeys,
@@ -4893,7 +4895,8 @@ create_partial_distinct_paths(PlannerInfo *root, RelOptInfo *input_rel,
* cheapest partial path).
*/
if (input_path != cheapest_partial_path &&
- (presorted_keys == 0 || !enable_incremental_sort))
+ (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(partial_distinct_rel, IncrementalSort)))
continue;
/*
@@ -4901,7 +4904,8 @@ create_partial_distinct_paths(PlannerInfo *root, RelOptInfo *input_rel,
* We'll just do a sort if there are no presorted keys and an
* incremental sort when there are presorted keys.
*/
- if (presorted_keys == 0 || !enable_incremental_sort)
+ if (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(partial_distinct_rel, IncrementalSort))
sorted_path = (Path *) create_sort_path(root,
partial_distinct_rel,
input_path,
@@ -4961,10 +4965,12 @@ create_partial_distinct_paths(PlannerInfo *root, RelOptInfo *input_rel,
/*
* Now try hash aggregate paths, if enabled and hashing is possible. Since
* we're not on the hook to ensure we do our best to create at least one
- * path here, we treat enable_hashagg as a hard off-switch rather than the
- * slightly softer variant in create_final_distinct_paths.
+ * path here, we treat completely skip this if hash aggregation is not
+ * enabled. (In contrast, create_final_distinct_paths sometimes considers
+ * hash aggregation even when it's disabled, to avoid failing completely.)
*/
- if (enable_hashagg && grouping_is_hashable(root->processed_distinctClause))
+ if (REL_CAN_USE_PATH(partial_distinct_rel, HashAgg) &&
+ grouping_is_hashable(root->processed_distinctClause))
{
add_partial_path(partial_distinct_rel, (Path *)
create_agg_path(root,
@@ -5105,7 +5111,8 @@ create_final_distinct_paths(PlannerInfo *root, RelOptInfo *input_rel,
* cheapest input path).
*/
if (input_path != cheapest_input_path &&
- (presorted_keys == 0 || !enable_incremental_sort))
+ (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(distinct_rel, IncrementalSort)))
continue;
/*
@@ -5113,7 +5120,8 @@ create_final_distinct_paths(PlannerInfo *root, RelOptInfo *input_rel,
* We'll just do a sort if there are no presorted keys and an
* incremental sort when there are presorted keys.
*/
- if (presorted_keys == 0 || !enable_incremental_sort)
+ if (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(distinct_rel, IncrementalSort))
sorted_path = (Path *) create_sort_path(root,
distinct_rel,
input_path,
@@ -5177,14 +5185,14 @@ create_final_distinct_paths(PlannerInfo *root, RelOptInfo *input_rel,
* die trying. If we do have other choices, there are two things that
* should prevent selection of hashing: if the query uses DISTINCT ON
* (because it won't really have the expected behavior if we hash), or if
- * enable_hashagg is off.
+ * hash aggregation is disabled.
*
* Note: grouping_is_hashable() is much more expensive to check than the
* other gating conditions, so we want to do it last.
*/
if (distinct_rel->pathlist == NIL)
allow_hash = true; /* we have no alternatives */
- else if (parse->hasDistinctOn || !enable_hashagg)
+ else if (parse->hasDistinctOn || !REL_CAN_USE_PATH(distinct_rel, HashAgg))
allow_hash = false; /* policy-based decision not to hash */
else
allow_hash = true; /* default */
@@ -5277,7 +5285,8 @@ create_ordered_paths(PlannerInfo *root,
* input path).
*/
if (input_path != cheapest_input_path &&
- (presorted_keys == 0 || !enable_incremental_sort))
+ (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(ordered_rel, IncrementalSort)))
continue;
/*
@@ -5285,7 +5294,8 @@ create_ordered_paths(PlannerInfo *root,
* We'll just do a sort if there are no presorted keys and an
* incremental sort when there are presorted keys.
*/
- if (presorted_keys == 0 || !enable_incremental_sort)
+ if (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(ordered_rel, IncrementalSort))
sorted_path = (Path *) create_sort_path(root,
ordered_rel,
input_path,
@@ -5349,7 +5359,8 @@ create_ordered_paths(PlannerInfo *root,
* partial path).
*/
if (input_path != cheapest_partial_path &&
- (presorted_keys == 0 || !enable_incremental_sort))
+ (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(ordered_rel, IncrementalSort)))
continue;
/*
@@ -5357,7 +5368,8 @@ create_ordered_paths(PlannerInfo *root,
* We'll just do a sort if there are no presorted keys and an
* incremental sort when there are presorted keys.
*/
- if (presorted_keys == 0 || !enable_incremental_sort)
+ if (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(ordered_rel, IncrementalSort))
sorted_path = (Path *) create_sort_path(root,
ordered_rel,
input_path,
@@ -6747,7 +6759,7 @@ plan_cluster_use_sort(Oid tableOid, Oid indexOid)
/* Estimate the cost of seq scan + sort */
seqScanPath = create_seqscan_path(root, rel, NULL, 0);
- cost_sort(&seqScanAndSortPath, root, NIL,
+ cost_sort(&seqScanAndSortPath, root, rel, NIL,
seqScanPath->disabled_nodes,
seqScanPath->total_cost, rel->tuples, rel->reltarget->width,
comparisonCost, maintenance_work_mem, -1.0);
@@ -6931,7 +6943,8 @@ make_ordered_path(PlannerInfo *root, RelOptInfo *rel, Path *path,
* disabled unless it's the cheapest input path).
*/
if (path != cheapest_path &&
- (presorted_keys == 0 || !enable_incremental_sort))
+ (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(rel, IncrementalSort)))
return NULL;
/*
@@ -6939,7 +6952,8 @@ make_ordered_path(PlannerInfo *root, RelOptInfo *rel, Path *path,
* just do a sort if there are no presorted keys and an incremental
* sort when there are presorted keys.
*/
- if (presorted_keys == 0 || !enable_incremental_sort)
+ if (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(rel, IncrementalSort))
path = (Path *) create_sort_path(root,
rel,
path,
@@ -7540,7 +7554,8 @@ gather_grouping_paths(PlannerInfo *root, RelOptInfo *rel)
* disabled unless it's the cheapest input path).
*/
if (path != cheapest_partial_path &&
- (presorted_keys == 0 || !enable_incremental_sort))
+ (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(rel, IncrementalSort)))
continue;
/*
@@ -7548,7 +7563,8 @@ gather_grouping_paths(PlannerInfo *root, RelOptInfo *rel)
* just do a sort if there are no presorted keys and an incremental
* sort when there are presorted keys.
*/
- if (presorted_keys == 0 || !enable_incremental_sort)
+ if (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(rel, IncrementalSort))
path = (Path *) create_sort_path(root, rel, path,
groupby_pathkeys,
-1.0);
diff --git a/src/backend/optimizer/plan/subselect.c b/src/backend/optimizer/plan/subselect.c
index 6d003cc8e5..6dbcf6e0aa 100644
--- a/src/backend/optimizer/plan/subselect.c
+++ b/src/backend/optimizer/plan/subselect.c
@@ -525,13 +525,14 @@ build_subplan(PlannerInfo *root, Plan *plan, Path *path,
* is pointless for a direct-correlated subplan, since we'd have to
* recompute its results each time anyway. For uncorrelated/undirect
* correlated subplans, we add Material unless the subplan's top plan
- * node would materialize its output anyway. Also, if enable_material
- * is false, then the user does not want us to materialize anything
+ * node would materialize its output anyway. Also, if Materialize is
+ * dissabled, then the user does not want us to materialize anything
* unnecessarily, so we don't.
*/
- else if (splan->parParam == NIL && enable_material &&
+ else if (splan->parParam == NIL &&
+ REL_CAN_USE_PATH(path->parent, Material) &&
!ExecMaterializesOutput(nodeTag(plan)))
- plan = materialize_finished_plan(plan);
+ plan = materialize_finished_plan(plan, path->parent);
result = (Node *) splan;
isInitPlan = false;
diff --git a/src/backend/optimizer/prep/prepunion.c b/src/backend/optimizer/prep/prepunion.c
index a0baf6d4a1..f8fd5db66b 100644
--- a/src/backend/optimizer/prep/prepunion.c
+++ b/src/backend/optimizer/prep/prepunion.c
@@ -587,7 +587,8 @@ build_setop_child_paths(PlannerInfo *root, RelOptInfo *rel,
* input path).
*/
if (subpath != cheapest_input_path &&
- (presorted_keys == 0 || !enable_incremental_sort))
+ (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(final_rel, IncrementalSort)))
continue;
/*
@@ -595,7 +596,8 @@ build_setop_child_paths(PlannerInfo *root, RelOptInfo *rel,
* We'll just do a sort if there are no presorted keys and an
* incremental sort when there are presorted keys.
*/
- if (presorted_keys == 0 || !enable_incremental_sort)
+ if (presorted_keys == 0 ||
+ !REL_CAN_USE_PATH(final_rel, IncrementalSort))
subpath = (Path *) create_sort_path(rel->subroot,
final_rel,
subpath,
@@ -867,7 +869,7 @@ generate_union_paths(SetOperationStmt *op, PlannerInfo *root,
* the children. The precise formula is just a guess; see
* add_paths_to_append_rel.
*/
- if (enable_parallel_append)
+ if (REL_CAN_USE_PATH(result_rel, ParallelAppend))
{
parallel_workers = Max(parallel_workers,
pg_leftmost_one_pos32(list_length(partial_pathlist)) + 1);
@@ -879,7 +881,8 @@ generate_union_paths(SetOperationStmt *op, PlannerInfo *root,
papath = (Path *)
create_append_path(root, result_rel, NIL, partial_pathlist,
NIL, NULL, parallel_workers,
- enable_parallel_append, -1);
+ REL_CAN_USE_PATH(result_rel, ParallelAppend),
+ -1);
gpath = (Path *)
create_gather_path(root, result_rel, papath,
result_rel->reltarget, NULL, NULL);
@@ -1319,8 +1322,8 @@ choose_hashed_setop(PlannerInfo *root, List *groupClauses,
errmsg("could not implement %s", construct),
errdetail("Some of the datatypes only support hashing, while others only support sorting.")));
- /* Prefer sorting when enable_hashagg is off */
- if (!enable_hashagg)
+ /* Prefer sorting when hash aggregation is disabled */
+ if (!REL_CAN_USE_PATH(input_path->parent, HashAgg))
return false;
/*
@@ -1343,7 +1346,7 @@ choose_hashed_setop(PlannerInfo *root, List *groupClauses,
* These path variables are dummies that just hold cost fields; we don't
* make actual Paths for these steps.
*/
- cost_agg(&hashed_p, root, AGG_HASHED, NULL,
+ cost_agg(&hashed_p, root, input_path->parent, AGG_HASHED, NULL,
numGroupCols, dNumGroups,
NIL,
input_path->disabled_nodes,
@@ -1358,7 +1361,8 @@ choose_hashed_setop(PlannerInfo *root, List *groupClauses,
sorted_p.startup_cost = input_path->startup_cost;
sorted_p.total_cost = input_path->total_cost;
/* XXX cost_sort doesn't actually look at pathkeys, so just pass NIL */
- cost_sort(&sorted_p, root, NIL, sorted_p.disabled_nodes,
+ cost_sort(&sorted_p, root, input_path->parent, NIL,
+ sorted_p.disabled_nodes,
sorted_p.total_cost,
input_path->rows, input_path->pathtarget->width,
0.0, work_mem, -1.0);
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index fc97bf6ee2..77ed747437 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -1537,6 +1537,7 @@ create_merge_append_path(PlannerInfo *root,
cost_sort(&sort_path,
root,
+ rel,
pathkeys,
subpath->disabled_nodes,
subpath->total_cost,
@@ -1649,7 +1650,7 @@ create_material_path(RelOptInfo *rel, Path *subpath)
pathnode->subpath = subpath;
- cost_material(&pathnode->path,
+ cost_material(&pathnode->path, rel,
subpath->disabled_nodes,
subpath->startup_cost,
subpath->total_cost,
@@ -1698,7 +1699,7 @@ create_memoize_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
pathnode->est_entries = 0;
/* we should not generate this path type when enable_memoize=false */
- Assert(enable_memoize);
+ Assert(REL_CAN_USE_PATH(rel, Memoize));
pathnode->path.disabled_nodes = subpath->disabled_nodes;
/*
@@ -1866,7 +1867,7 @@ create_unique_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
/*
* Estimate cost for sort+unique implementation
*/
- cost_sort(&sort_path, root, NIL,
+ cost_sort(&sort_path, root, rel, NIL,
subpath->disabled_nodes,
subpath->total_cost,
rel->rows,
@@ -1901,7 +1902,7 @@ create_unique_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
sjinfo->semi_can_hash = false;
}
else
- cost_agg(&agg_path, root,
+ cost_agg(&agg_path, root, rel,
AGG_HASHED, NULL,
numCols, pathnode->path.rows,
NIL,
@@ -3101,7 +3102,7 @@ create_sort_path(PlannerInfo *root,
pathnode->subpath = subpath;
- cost_sort(&pathnode->path, root, pathkeys,
+ cost_sort(&pathnode->path, root, rel, pathkeys,
subpath->disabled_nodes,
subpath->total_cost,
subpath->rows,
@@ -3288,7 +3289,7 @@ create_agg_path(PlannerInfo *root,
pathnode->groupClause = groupClause;
pathnode->qual = qual;
- cost_agg(&pathnode->path, root,
+ cost_agg(&pathnode->path, root, rel,
aggstrategy, aggcosts,
list_length(groupClause), numGroups,
qual,
@@ -3395,7 +3396,7 @@ create_groupingsets_path(PlannerInfo *root,
*/
if (is_first)
{
- cost_agg(&pathnode->path, root,
+ cost_agg(&pathnode->path, root, rel,
aggstrategy,
agg_costs,
numGroupCols,
@@ -3421,7 +3422,7 @@ create_groupingsets_path(PlannerInfo *root,
* Account for cost of aggregation, but don't charge input
* cost again
*/
- cost_agg(&agg_path, root,
+ cost_agg(&agg_path, root, rel,
rollup->is_hashed ? AGG_HASHED : AGG_SORTED,
agg_costs,
numGroupCols,
@@ -3436,7 +3437,7 @@ create_groupingsets_path(PlannerInfo *root,
else
{
/* Account for cost of sort, but don't charge input cost again */
- cost_sort(&sort_path, root, NIL, 0,
+ cost_sort(&sort_path, root, rel, NIL, 0,
0.0,
subpath->rows,
subpath->pathtarget->width,
@@ -3446,7 +3447,7 @@ create_groupingsets_path(PlannerInfo *root,
/* Account for cost of aggregation */
- cost_agg(&agg_path, root,
+ cost_agg(&agg_path, root, rel,
AGG_SORTED,
agg_costs,
numGroupCols,
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 78a3cfafde..3b9a3746ee 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -570,7 +570,8 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
/*
* Allow a plugin to editorialize on the info we obtained from the
* catalogs. Actions might include altering the assumed relation size,
- * removing an index, or adding a hypothetical index to the indexlist.
+ * removing an index, adding a hypothetical index to the indexlist, or
+ * changing the path type mask.
*/
if (get_relation_info_hook)
(*get_relation_info_hook) (root, relationObjectId, inhparent, rel);
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index d7266e4cdb..88e468795a 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -211,6 +211,7 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
rel->consider_startup = (root->tuple_fraction > 0);
rel->consider_param_startup = false; /* might get changed later */
rel->consider_parallel = false; /* might get changed later */
+ rel->path_type_mask = default_path_type_mask;
rel->reltarget = create_empty_pathtarget();
rel->pathlist = NIL;
rel->ppilist = NIL;
@@ -707,6 +708,7 @@ build_join_rel(PlannerInfo *root,
joinrel->consider_startup = (root->tuple_fraction > 0);
joinrel->consider_param_startup = false;
joinrel->consider_parallel = false;
+ joinrel->path_type_mask = default_path_type_mask;
joinrel->reltarget = create_empty_pathtarget();
joinrel->pathlist = NIL;
joinrel->ppilist = NIL;
@@ -900,6 +902,7 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
joinrel->consider_startup = (root->tuple_fraction > 0);
joinrel->consider_param_startup = false;
joinrel->consider_parallel = false;
+ joinrel->path_type_mask = default_path_type_mask;
joinrel->reltarget = create_empty_pathtarget();
joinrel->pathlist = NIL;
joinrel->ppilist = NIL;
@@ -1484,6 +1487,7 @@ fetch_upper_rel(PlannerInfo *root, UpperRelationKind kind, Relids relids)
upperrel->consider_startup = (root->tuple_fraction > 0);
upperrel->consider_param_startup = false;
upperrel->consider_parallel = false; /* might get changed later */
+ upperrel->path_type_mask = default_path_type_mask;
upperrel->reltarget = create_empty_pathtarget();
upperrel->pathlist = NIL;
upperrel->cheapest_startup_path = NULL;
@@ -2010,7 +2014,7 @@ build_joinrel_partition_info(PlannerInfo *root,
PartitionScheme part_scheme;
/* Nothing to do if partitionwise join technique is disabled. */
- if (!enable_partitionwise_join)
+ if (!REL_CAN_USE_PATH(joinrel, PartitionwiseJoin))
{
Assert(!IS_PARTITIONED_REL(joinrel));
return;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index af227b1f24..1bd21bbe34 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -55,6 +55,7 @@
#include "optimizer/geqo.h"
#include "optimizer/optimizer.h"
#include "optimizer/paths.h"
+#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "parser/parse_expr.h"
#include "parser/parser.h"
@@ -777,7 +778,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_seqscan,
true,
- NULL, NULL, NULL
+ NULL, enable_seqscan_assign_hook, NULL
},
{
{"enable_indexscan", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -787,7 +788,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_indexscan,
true,
- NULL, NULL, NULL
+ NULL, enable_indexscan_assign_hook, NULL
},
{
{"enable_indexonlyscan", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -797,7 +798,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_indexonlyscan,
true,
- NULL, NULL, NULL
+ NULL, enable_indexonlyscan_assign_hook, NULL
},
{
{"enable_bitmapscan", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -807,7 +808,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_bitmapscan,
true,
- NULL, NULL, NULL
+ NULL, enable_bitmapscan_assign_hook, NULL
},
{
{"enable_tidscan", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -817,7 +818,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_tidscan,
true,
- NULL, NULL, NULL
+ NULL, enable_tidscan_assign_hook, NULL
},
{
{"enable_sort", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -827,7 +828,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_sort,
true,
- NULL, NULL, NULL
+ NULL, enable_sort_assign_hook, NULL
},
{
{"enable_incremental_sort", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -837,7 +838,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_incremental_sort,
true,
- NULL, NULL, NULL
+ NULL, enable_incremental_sort_assign_hook, NULL
},
{
{"enable_hashagg", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -847,7 +848,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_hashagg,
true,
- NULL, NULL, NULL
+ NULL, enable_hashagg_assign_hook, NULL
},
{
{"enable_material", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -857,7 +858,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_material,
true,
- NULL, NULL, NULL
+ NULL, enable_material_assign_hook, NULL
},
{
{"enable_memoize", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -867,7 +868,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_memoize,
true,
- NULL, NULL, NULL
+ NULL, enable_memoize_assign_hook, NULL
},
{
{"enable_nestloop", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -877,7 +878,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_nestloop,
true,
- NULL, NULL, NULL
+ NULL, enable_nestloop_assign_hook, NULL
},
{
{"enable_mergejoin", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -887,7 +888,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_mergejoin,
true,
- NULL, NULL, NULL
+ NULL, enable_mergejoin_assign_hook, NULL
},
{
{"enable_hashjoin", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -897,7 +898,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_hashjoin,
true,
- NULL, NULL, NULL
+ NULL, enable_hashjoin_assign_hook, NULL
},
{
{"enable_gathermerge", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -907,7 +908,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_gathermerge,
true,
- NULL, NULL, NULL
+ NULL, enable_gathermerge_assign_hook, NULL
},
{
{"enable_partitionwise_join", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -917,7 +918,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_partitionwise_join,
false,
- NULL, NULL, NULL
+ NULL, enable_partitionwise_join_assign_hook, NULL
},
{
{"enable_partitionwise_aggregate", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -927,7 +928,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_partitionwise_aggregate,
false,
- NULL, NULL, NULL
+ NULL, enable_partitionwise_aggregate_assign_hook, NULL
},
{
{"enable_parallel_append", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -937,7 +938,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_parallel_append,
true,
- NULL, NULL, NULL
+ NULL, enable_parallel_append_assign_hook, NULL
},
{
{"enable_parallel_hash", PGC_USERSET, QUERY_TUNING_METHOD,
@@ -947,7 +948,7 @@ struct config_bool ConfigureNamesBool[] =
},
&enable_parallel_hash,
true,
- NULL, NULL, NULL
+ NULL, enable_parallel_hash_assign_hook, NULL
},
{
{"enable_partition_pruning", PGC_USERSET, QUERY_TUNING_METHOD,
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 540d021592..bbff482cec 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -80,6 +80,26 @@ typedef enum UpperRelationKind
/* NB: UPPERREL_FINAL must be last enum entry; it's used to size arrays */
} UpperRelationKind;
+#define PathTypeBitmapScan 0x00000001
+#define PathTypeGatherMerge 0x00000002
+#define PathTypeHashAgg 0x00000004
+#define PathTypeHashJoin 0x00000008
+#define PathTypeIncrementalSort 0x00000010
+#define PathTypeIndexScan 0x00000020
+#define PathTypeIndexOnlyScan 0x00000040
+#define PathTypeMaterial 0x00000080
+#define PathTypeMemoize 0x00000100
+#define PathTypeMergeJoin 0x00000200
+#define PathTypeNestLoop 0x00000400
+#define PathTypeParallelAppend 0x00000800
+#define PathTypeParallelHash 0x00001000
+#define PathTypePartitionwiseJoin 0x00002000
+#define PathTypePartitionwiseAggregate 0x00004000
+#define PathTypeSeqScan 0x00008000
+#define PathTypeSort 0x00010000
+#define PathTypeTIDScan 0x00020000
+#define PathTypeMaskAll 0x0003FFFF
+
/*----------
* PlannerGlobal
* Global information for planning/optimization
@@ -879,6 +899,8 @@ typedef struct RelOptInfo
bool consider_param_startup;
/* consider parallel paths? */
bool consider_parallel;
+ /* path type mask for this rel */
+ uint32 path_type_mask;
/*
* default result targetlist for Paths scanning this relation; list of
@@ -1065,6 +1087,13 @@ typedef struct RelOptInfo
((rel)->part_scheme && (rel)->boundinfo && (rel)->nparts > 0 && \
(rel)->part_rels && (rel)->partexprs && (rel)->nullable_partexprs)
+/*
+ * Convenience macro to test for whether a certain a PathTypeXXX bit is
+ * set in a relation's path_type_mask.
+ */
+#define REL_CAN_USE_PATH(rel, type) \
+ (((rel)->path_type_mask & PathType##type) != 0)
+
/*
* IndexOptInfo
* Per-index information for planning/optimization
diff --git a/src/include/optimizer/cost.h b/src/include/optimizer/cost.h
index 854a782944..2cd1d8c34c 100644
--- a/src/include/optimizer/cost.h
+++ b/src/include/optimizer/cost.h
@@ -71,6 +71,26 @@ extern PGDLLIMPORT bool enable_partition_pruning;
extern PGDLLIMPORT bool enable_presorted_aggregate;
extern PGDLLIMPORT bool enable_async_append;
extern PGDLLIMPORT int constraint_exclusion;
+extern PGDLLIMPORT uint32 default_path_type_mask;
+
+extern void enable_bitmapscan_assign_hook(bool newval, void *extra);
+extern void enable_gathermerge_assign_hook(bool newval, void *extra);
+extern void enable_hashagg_assign_hook(bool newval, void *extra);
+extern void enable_hashjoin_assign_hook(bool newval, void *extra);
+extern void enable_incremental_sort_assign_hook(bool newval, void *extra);
+extern void enable_indexscan_assign_hook(bool newval, void *extra);
+extern void enable_indexonlyscan_assign_hook(bool newval, void *extra);
+extern void enable_material_assign_hook(bool newval, void *extra);
+extern void enable_memoize_assign_hook(bool newval, void *extra);
+extern void enable_mergejoin_assign_hook(bool newval, void *extra);
+extern void enable_nestloop_assign_hook(bool newval, void *extra);
+extern void enable_parallel_append_assign_hook(bool newval, void *extra);
+extern void enable_parallel_hash_assign_hook(bool newval, void *extra);
+extern void enable_partitionwise_join_assign_hook(bool newval, void *extra);
+extern void enable_partitionwise_aggregate_assign_hook(bool newval, void *extra);
+extern void enable_seqscan_assign_hook(bool newval, void *extra);
+extern void enable_sort_assign_hook(bool newval, void *extra);
+extern void enable_tidscan_assign_hook(bool newval, void *extra);
extern double index_pages_fetched(double tuples_fetched, BlockNumber pages,
double index_pages, PlannerInfo *root);
@@ -107,7 +127,7 @@ extern void cost_namedtuplestorescan(Path *path, PlannerInfo *root,
extern void cost_resultscan(Path *path, PlannerInfo *root,
RelOptInfo *baserel, ParamPathInfo *param_info);
extern void cost_recursive_union(Path *runion, Path *nrterm, Path *rterm);
-extern void cost_sort(Path *path, PlannerInfo *root,
+extern void cost_sort(Path *path, PlannerInfo *root, RelOptInfo *rel,
List *pathkeys, int disabled_nodes,
Cost input_cost, double tuples, int width,
Cost comparison_cost, int sort_mem,
@@ -124,11 +144,11 @@ extern void cost_merge_append(Path *path, PlannerInfo *root,
int input_disabled_nodes,
Cost input_startup_cost, Cost input_total_cost,
double tuples);
-extern void cost_material(Path *path,
+extern void cost_material(Path *path, RelOptInfo *rel,
int input_disabled_nodes,
Cost input_startup_cost, Cost input_total_cost,
double tuples, int width);
-extern void cost_agg(Path *path, PlannerInfo *root,
+extern void cost_agg(Path *path, PlannerInfo *root, RelOptInfo *rel,
AggStrategy aggstrategy, const AggClauseCosts *aggcosts,
int numGroupCols, double numGroups,
List *quals,
@@ -148,6 +168,7 @@ extern void cost_group(Path *path, PlannerInfo *root,
double input_tuples);
extern void initial_cost_nestloop(PlannerInfo *root,
JoinCostWorkspace *workspace,
+ RelOptInfo *joinrel,
JoinType jointype,
Path *outer_path, Path *inner_path,
JoinPathExtraData *extra);
@@ -156,6 +177,7 @@ extern void final_cost_nestloop(PlannerInfo *root, NestPath *path,
JoinPathExtraData *extra);
extern void initial_cost_mergejoin(PlannerInfo *root,
JoinCostWorkspace *workspace,
+ RelOptInfo *joinrel,
JoinType jointype,
List *mergeclauses,
Path *outer_path, Path *inner_path,
@@ -166,6 +188,7 @@ extern void final_cost_mergejoin(PlannerInfo *root, MergePath *path,
JoinPathExtraData *extra);
extern void initial_cost_hashjoin(PlannerInfo *root,
JoinCostWorkspace *workspace,
+ RelOptInfo *joinrel,
JoinType jointype,
List *hashclauses,
Path *outer_path, Path *inner_path,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index aafc173792..07623eff79 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -45,7 +45,7 @@ extern ForeignScan *make_foreignscan(List *qptlist, List *qpqual,
Plan *outer_plan);
extern Plan *change_plan_targetlist(Plan *subplan, List *tlist,
bool tlist_parallel_safe);
-extern Plan *materialize_finished_plan(Plan *subplan);
+extern Plan *materialize_finished_plan(Plan *subplan, RelOptInfo *rel);
extern bool is_projection_capable_path(Path *path);
extern bool is_projection_capable_plan(Plan *plan);
--
2.39.3 (Apple Git-145)
On 8/27/24 11:45, Robert Haas wrote:
On Mon, Aug 26, 2024 at 3:28 PM Robert Haas <robertmhaas@gmail.com> wrote:
Well, I agree that this doesn't address everything you might want to
do, ... I will very happily propose more things to
address the other problems that I know about ...In that vein, here's a new patch set where I've added a second patch
that allows extensions to control choice of index. It's 3 lines of new
code, plus 7 lines of comments and whitespace. Feeling inspired, I
also included a contrib module, initial_vowels_are_evil, to
demonstrate how this can be used by an extension that wants to disable
certain indexes but not others. This is obviously quite silly and we
might (or might not) want a more serious example in contrib, but it
demonstrates how easy this can be with just a tiny bit of core
infrastructure:robert.haas=# load 'initial_vowels_are_evil';
LOAD
robert.haas=# explain select count(*) from pgbench_accounts;
QUERY PLAN
-----------------------------------------------------------------------------------------------------------------
Aggregate (cost=2854.29..2854.30 rows=1 width=8)
-> Index Only Scan using pgbench_accounts_pkey on pgbench_accounts
(cost=0.29..2604.29 rows=100000 width=0)
(2 rows)
robert.haas=# alter index pgbench_accounts_pkey rename to
evil_pgbench_accounts_pkey;
ALTER INDEX
robert.haas=# explain select count(*) from pgbench_accounts;
QUERY PLAN
------------------------------------------------------------------------------
Aggregate (cost=2890.00..2890.01 rows=1 width=8)
-> Seq Scan on pgbench_accounts (cost=0.00..2640.00 rows=100000 width=0)
(2 rows)
robert.haas=#
Nice!
On the one hand, excluding indexes by initial vowels is definitely
silly. On the other, I can see how it might be useful for an extension
to exclude indexes based on a regex match of the index name or something
similar, at least for testing.
--
Joe Conway
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
On Tue, Aug 27, 2024 at 2:44 AM chungui.wcg <wcg2008zl@126.com> wrote:
I really admire this idea.
Thanks.
here is my confusion: Isn't the core of this idea whether to turn the planner into a framework? Personally, I think that under PostgreSQL's heap table storage, the optimizer might be better off focusing on optimizing the generation of execution plans. It’s possible that in some specific scenarios, developers might want to intervene in the generation of execution plans by extensions. I'm not sure if these scenarios usually occur when the storage structure is also extended by developers. If so, could existing solutions like "planner_hook" potentially solve the problem?
You could use planner_hook if you wanted to replace the entire planner
with your own planner. However, that doesn't seem like something
practical, as the planner code is very large. The real use of the hook
is to allow running some extra code when the planner is invoked, as
demonstrated by the pg_stat_statements contrib module. To get some
meaningful control over the planner, you need something more
fine-grained. You need to be able to run code at specific points in
the planner, as we already allow with, for example,
get_relation_info_hook or set_rel_pathlist_hook.
Whether or not that constitutes "turning the planner into a framework"
is, I suppose, a question of opinion. Perhaps a more positive way to
phrase it would be "allowing for some code reuse". Right now, if you
mostly like the behavior of the planner but want a few things to be
different, you've got to duplicate a lot of code and then hack it up.
That's not very nice. I think it's better to set things up so that you
can keep most of the planner behavior but override it in a few
specific cases without a lot of difficulty.
Cases where the data is stored in some different way are really a
separate issue from what I'm talking about here. In that case, you
don't want to override the planner behavior for all tables everywhere,
so planner_hook still isn't a good solution. You only want to change
the behavior for the specific table AM that implements the new
storage. You would probably want there to be an option where
cost_seqscan() calls a tableam-specific function instead of just doing
the same thing for every AM; and maybe something similar for indexes,
although that is less clear. The details aren't quite clear, which is
probably part of why we haven't done anything yet.
But this patch set is really more about enabling use cases where the
user wants an extension to take control of the plan more explicitly,
say to avoid some bad plan that they got before and that they don't
want to get again.
--
Robert Haas
EDB: http://www.enterprisedb.com
On Tue, Aug 27, 2024 at 11:57 AM Joe Conway <mail@joeconway.com> wrote:
On the one hand, excluding indexes by initial vowels is definitely
silly. On the other, I can see how it might be useful for an extension
to exclude indexes based on a regex match of the index name or something
similar, at least for testing.
Right. I deliberately picked a contrib module that implemented a silly
policy, because what I wanted to demonstrate with it is that this
little bit of infrastructure provides enough mechanism to implement
whatever policy you want. And I think it demonstrates it quite well,
because the whole contrib module to implement this has just 6 lines of
executable code. If you wanted a policy that would be more
realistically useful, you'd need more code, but only however much is
needed to implement your policy. All you need do is replace this
strchr call with something else:
if (name != NULL && strchr("aeiouAEIOU", name[0]) != NULL)
index->disabled = true;
--
Robert Haas
EDB: http://www.enterprisedb.com
Robert Haas <robertmhaas@gmail.com> writes:
In that vein, here's a new patch set where I've added a second patch
that allows extensions to control choice of index.
I'm minus-several on this bit, because that is a solved problem and
we really don't need to introduce More Than One Way To Do It. The
intention has always been that get_relation_info_hook can editorialize
on the rel's indexlist by removing entries (or adding fake ones,
in the case of hypothetical-index extensions). For that matter,
if you really want to do it by modifying the IndexInfo rather than
deleting it from the list, that's already possible: just set
indexinfo->hypothetical = true.
regards, tom lane
On Tue, Aug 27, 2024 at 12:56 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Robert Haas <robertmhaas@gmail.com> writes:
In that vein, here's a new patch set where I've added a second patch
that allows extensions to control choice of index.I'm minus-several on this bit, because that is a solved problem and
we really don't need to introduce More Than One Way To Do It. The
intention has always been that get_relation_info_hook can editorialize
on the rel's indexlist by removing entries (or adding fake ones,
in the case of hypothetical-index extensions). For that matter,
if you really want to do it by modifying the IndexInfo rather than
deleting it from the list, that's already possible: just set
indexinfo->hypothetical = true.
Well, now I'm confused. Just yesterday, in response to the 0001 patch
that allows extensions to exert control over the join strategy, you
complained that "Or, if your problem is that the planner wants to scan
index A but you want it to scan index B, enable_indexscan won't help."
So today, I produced a patch demonstrating how we could address that
issue, and your response is "well, actually we don't need to do
anything about it because that problem is already solved." But if that
is true, then the fact that yesterday's patch did nothing about it was
a feature, not a bug.
Am I missing something here?
--
Robert Haas
EDB: http://www.enterprisedb.com
Robert Haas <robertmhaas@gmail.com> writes:
Well, now I'm confused. Just yesterday, in response to the 0001 patch
that allows extensions to exert control over the join strategy, you
complained that "Or, if your problem is that the planner wants to scan
index A but you want it to scan index B, enable_indexscan won't help."
I was just using that to illustrate that making the enable_XXX GUCs
relation-local covers only a small part of the planner-control problem.
You had not, at that point, been very clear that you intended that
patch as only a small part of a solution.
I do think that index selection is pretty well under control already,
thanks to stuff that we put in ages ago at the urging of people who
wanted to write "index advisor" extensions. (The fact that that
area seems a bit moribund is disheartening, though. Is it a lack
of documentation?)
regards, tom lane
On Tue, Aug 27, 2024 at 2:24 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
I was just using that to illustrate that making the enable_XXX GUCs
relation-local covers only a small part of the planner-control problem.
You had not, at that point, been very clear that you intended that
patch as only a small part of a solution.
Ah, OK, apologies for the lack of clarity. I actually think it's a
medium part of the solution. I believe the minimum viable product here
is probably something like:
- control over scan methods
- control over index selection
- control over join methods
- control over join order
It gets a lot better if we also have:
- control over aggregation methods
- something that I'm not quite sure about for appendrels
- control over whether parallelism is used and the degree of parallelism
If control over index selection is already adequate, then the proposed
patch is one way to get about 1/3 of the way to the MVP, which isn't
nothing. Maybe I'm underestimating the amount of stuff that people are
going to want here, but if you look at pg_hint_plan, it isn't doing a
whole lot more than this.
I do think that index selection is pretty well under control already,
thanks to stuff that we put in ages ago at the urging of people who
wanted to write "index advisor" extensions. (The fact that that
area seems a bit moribund is disheartening, though. Is it a lack
of documentation?)
So a couple of things about this.
First, EDB maintains closed-source index advisor code that uses this
machinery. In fact, if I'm not mistaken, we now have two extensions
that use it. So it's not dead from that point of view, but of course
anything closed-source can't be promoted through community channels.
There's open-source code around too; to my knowledge,
https://github.com/HypoPG/hypopg is the leading open-source
implementation, but my knowledge may very well be incomplete.
Second, I do think that the lack of documentation poses somewhat of a
challenge, and our exchange about whether an IndexOptInfo needs a
disabled flag is perhaps an example of that. To be fair, now that I
look at it, the comment where get_relation_info_hook does say that you
can remove indexes from the index list, so maybe I should have
realized that the problem can be solved that way, but on the other
hand, the comment for set_rel_pathlist_hook claims you can delete
paths from the pathlist, which AFAICS is completely non-viable, so one
can't necessarily rely too much on the comments in this area to learn
what actually does and does not work. Having some in-core examples
showing how to use this stuff correctly and demonstrating its full
power would also be really helpful. Right now, I often find myself
looking at out-of-core code which is sometimes poorly written and
frequently resorts to nasty hacks. It can be hard to determine whether
those nasty hacks are there because they're the only way to implement
some bit of functionality or because the author missed an opportunity
to do better.
Third, I think there's simply a lack of critical mass in terms of our
planner hooks. While the ability to add hypothetical indexes has some
use, the ability to remove indexes from consideration is probably
significantly more useful. But not if it's the only technique for
fixing a bad plan that you have available. Nobody gets excited about a
toolbox that contains just one tool. That's why I'm keen to expand
what can be done cleanly via hooks, and I think if we do that and also
provide either some very good documentation or some well-written
example implementations, we'll get more traction here.
--
Robert Haas
EDB: http://www.enterprisedb.com
On Mon, Aug 26, 2024 at 1:37 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
For example, I don't see
how this gets us any closer to letting an extension fix a poor choice
of join order.
Thinking more about this particular sub-problem, let's say we're
joining four tables A, B, C, and D. An extension wants to compel join
order A-B-C-D. Let's suppose, however, that it wants to do this in a
way where planning won't fail if that's impossible, so it wants to use
disabled_nodes rather than skipping path generation entirely.
When we're planning the baserels, we don't need to do anything
special. When we plan 2-way joins, we need to mark all paths disabled
except those originating from the A-B join. When we plan 3-way joins,
we need to mark all paths disabled except those originating from an
(A-B)-C join. When we plan the final 4-way join, we don't really need
to do anything extra: the only way to end up with a non-disabled path
at the top level is to pick a path from the (A-B)-C join and a path
from D.
There's a bit of nuance here, though. Suppose that when planning the
A-B join, the planner generates HashJoin(SeqScan(B),Hash(A)). Does
that path need to be disabled? If you think that join order A-B-C-D
means that table A should be the driving table, then the answer is
yes, because that path will lead to a join order beginning with B-A,
not one beginning with A-B. But you might also have a mental model
where it doesn't matter which side of the table is on which side of
the join, and as long as you start by joining A and B in some way,
that's good enough to qualify as an A-B join order. I believe actual
implementations vary in which approach they take.
I think that the beginning of add_paths_to_joinrel() looks like a
useful spot to get control. You could, for example, have a hook there
which returns a bitmask indicating which of merge-join, nested-loop,
and hash join will be allowable for this call; that hook would then
allow for control over the join method and the join order, and the
join order control is strong enough that you can implement either of
the two interpretations above. This idea theorizes that 0001 was wrong
to make the path mask a per-RelOptInfo value, because there could be
many calls to add_paths_to_joinrel() for a single RelOptInfo and, in
this idea, every one of those can enforce a different mask.
Potentially, such a hook could return additional information, either
by using more bits of the bitmask or by returning other information
via some other data type. For instance, I still believe that
distinguishing between parameterized-nestloops and
unparameterized-nestloops would be real darn useful, so we could have
separate bits for each; or you could have a bit to control whether
foreign-join paths get disabled (or are considered at all), or you
could have separate bits for merge joins that involve 0, 1, or 2
sorts. Whether we need or want any or all of that is certainly
debatable, but the point is that if you did want some of that, or
something else, it doesn't look difficult to feed that information
through to the places where you would need it to be available.
Thoughts?
--
Robert Haas
EDB: http://www.enterprisedb.com
Hi,
On Tue, Aug 27, 2024 at 03:11:15PM -0400, Robert Haas wrote:
Third, I think there's simply a lack of critical mass in terms of our
planner hooks. While the ability to add hypothetical indexes has some
use, the ability to remove indexes from consideration is probably
significantly more useful.
JFTR, hypopg can also mask away/hide indexes since version 1.4.0:
https://github.com/HypoPG/hypopg/commit/351f14a79daae8ab57339d2367d7f2fc639041f7
I haven't looked closely at the implementation though, and maybe you
meant something else in the above entirely.
Michael
Robert Haas <robertmhaas@gmail.com> writes:
I believe the minimum viable product here
is probably something like:
- control over scan methods
- control over index selection
- control over join methods
- control over join order
Seems reasonable. It might be possible to say that our answer
to "control over join order" is to provide a hook that can modify
the "joinlist" before it's passed to make_one_rel. If you want
to force a particular join order you can rearrange that
list-of-lists-of-range-table-indexes to do so. The thing this
would not give you is control over which rel is picked as outer
in any given join step. Not sure how critical that bit is.
regards, tom lane
On Tue, Aug 27, 2024 at 4:15 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Seems reasonable. It might be possible to say that our answer
to "control over join order" is to provide a hook that can modify
the "joinlist" before it's passed to make_one_rel. If you want
to force a particular join order you can rearrange that
list-of-lists-of-range-table-indexes to do so. The thing this
would not give you is control over which rel is picked as outer
in any given join step. Not sure how critical that bit is.
This has a big advantage over what I proposed yesterday in that it's
basically declarative. With one call to the hook, you get all the
information about the join order that you could ever want. That's
really nice. However, I don't really think it quite works, partly
because of what you mention here about not being able to control which
rel ends up on which side of the join, which I do think is important,
and also because if the join order isn't possible, planning will fail,
rather than falling back to some other plan shape. If you have an idea
how we could address those things within this same general framework,
I'd be keen to hear it.
It has occurred to me more than once that it might be really useful if
we could attempt to plan under a set of constraints and then, if we
don't end up finding a plan, retry without the constraints. But I
don't quite see how to make it work. When I tried to do that as a
solution to the disable_cost problem, it ended up meaning that once
you couldn't satisfy every constraint perfectly, you gave up on even
trying. I wasn't immediately certain that such behavior was
unacceptable, but I didn't have to look any further than our own
regression test suites to see that it was going to cause a lot of
unhappiness. In this case, if we could attempt join planning with the
user-prescribed join order and then try it again if we fail to find a
path, that would be really cool. Or if we could do all of planning
without generating disabled paths *at all* and then go back and
restart if it becomes clear that's not working out, that would be
slick. But, unless you have a clever idea, that all seems like
advanced magic that should wait until we have basic things working.
Right now, I think we should focus on getting something in place where
we still try all the paths but an extension can arrange for some of
them to be disabled. Then all the right things will happen naturally;
we'll only be leaving some CPU cycles on the table. Which isn't
amazing, but I don't think it's a critical defect either, and we can
try to improve things later if we want to.
--
Robert Haas
EDB: http://www.enterprisedb.com
Hi Robert,
On Mon, Aug 26, 2024 at 6:33 PM Robert Haas <robertmhaas@gmail.com> wrote:
I'm somewhat expecting to be flamed to a well-done crisp for saying
this, but I think we need better ways for extensions to control the
behavior of PostgreSQL's query planner.
[..]
[..] But all that
said, as much as anything, I want to get some feedback on what
approaches and trade-offs people think might be acceptable here,
because there's not much point in me spending a bunch of time writing
code that everyone (or a critical mass of people) are going to hate.
As a somewhat tiny culprit of the self-flaming done by Robert (due to
nagging him about this in the past on various occasions), I'm of
course obligated to +1 to any work related to giving end-users/DBA the
ability to cage the plans generated by the optimizer.
When dealing with issues like those, I have a feeling we have 2
classes of most frequent issues being reported (that's my subjective
experience):
a. cardinality misestimate leading usually to nest loop plans (e.g.
JOIN estimates thread [1]/messages/by-id/c8c0ff31-3a8a-7562-bbd3-78b2ec65f16c@enterprisedb.com could also somehow help and it also has nice
reproducers)
b. issues after upgrades
So the "selectivity estimation hook(s)" mentioned by Andrei seems to
be a must, but also the ability not to just guess & tweak (shape) the
plan, but a way to export all SQL plans before upgrade with capability
to import and override(lock) specific SQL query to specific plan from
before upgrade.
I'm not into the internals of optimizer at all, but here are other
random thoughts/questions:
- I do think that "hints" words have bad connotations and should not
be used. It might be because of embedding them in SQL query text of
the application itself. On one front they are localized to the SQL
(good), but the PG operator has no realistic way of altering that once
it's embedded in binary (bad), as the application team is usually
separate if not from an external company (very bad situation, but
happens almost always).
- Would stacking of such extensions, each overriding the same planner
hooks, be allowed or not in the long run ? Technically there's nothing
preventing it and I think I could imagine someone attempting to run
multiple planner hook hotfixes for multiple issues, all at once?
- Shouldn't EXPLAIN contain additional information that for that
specific plan the optimizer hooks changed at least 1 thing ? (e.g.
"Plan was tainted" or something like that). Maybe extension could mark
it politely that it did by setting a certain flag, or maybe there
should be routines exposed by the core to do that ?
- the "b" (upgrade) seems like a much more heavy duty issue, as that
would require transfer of version-independent and textualized dump of
SQL plan that could be back-filled into a newer version of optimizer.
Is such a big thing realistic at all and it's better to just
concentrate on the hooks approach?
-J.
[1]: /messages/by-id/c8c0ff31-3a8a-7562-bbd3-78b2ec65f16c@enterprisedb.com
On Wed, Aug 28, 2024 at 9:46 AM Jakub Wartak
<jakub.wartak@enterprisedb.com> wrote:
As a somewhat tiny culprit of the self-flaming done by Robert (due to
nagging him about this in the past on various occasions), I'm of
course obligated to +1 to any work related to giving end-users/DBA the
ability to cage the plans generated by the optimizer.
Thanks.
When dealing with issues like those, I have a feeling we have 2
classes of most frequent issues being reported (that's my subjective
experience):
a. cardinality misestimate leading usually to nest loop plans (e.g.
JOIN estimates thread [1] could also somehow help and it also has nice
reproducers)
b. issues after upgradesSo the "selectivity estimation hook(s)" mentioned by Andrei seems to
be a must, but also the ability not to just guess & tweak (shape) the
plan, but a way to export all SQL plans before upgrade with capability
to import and override(lock) specific SQL query to specific plan from
before upgrade.
I'm not against some kind of selectivity estimation hook in principle,
but I don't know what the proposal is specifically, and I think it's
separate from the topic of this thread. On the other hand, being able
to force the same plans after an upgrade that you were getting before
an upgrade is the kind of thing that I'd like to enable with the
infrastructure proposed here. I do not propose to put something like
that in core, at least not any time soon, but I'd like to have the
infrastructure be good enough that people can try to do it in an
extension module and learn from how it turns out.
Ever since I read
https://15721.courses.cs.cmu.edu/spring2020/papers/22-costmodels/p204-leis.pdf
I have believed that the cardinality misestimate leading to nested
loop plans is just because we're doing something dumb. They write:
"When looking at the queries that did not finish in a reasonable time
using the estimates, we found that most have one thing in common:
PostgreSQL’s optimizer decides to introduce a nestedloop join (without
an index lookup) because of a very low cardinality estimate, whereas
in reality the true cardinality is larger. As we saw in the previous
section, systematic underestimation happens very frequently, which
occasionally results in the introduction of nested-loop joins. [...]
if the cost estimate is 1,000,000 with the nested-loop join algorithm
and 1,000,001 with a hash join, PostgreSQL will always prefer the
nested-loop algorithm even if there is a equality join predicate,
which allows one to use hashing. [...] given the fact that
underestimates are quite frequent, this decision is extremely risky.
And even if the estimates happen to be correct, any potential
performance advantage of a nested-loop join in comparison with a hash
join is very small, so taking this high risk can only result in a very
small payoff. Therefore, we disabled nested-loop joins (but not
index-nestedloop joins) in all following experiments."
We don't even have an option to turn off that kind of join, and we
could probably avoid a lot of pain if we did. This, too, is mostly
separate from the topic of this thread, but I just can't believe we've
chosen to do literally nothing about this given that we all know this
specific thing hoses everybody, everywhere, all the time.
I'm not into the internals of optimizer at all, but here are other
random thoughts/questions:
- I do think that "hints" words have bad connotations and should not
be used. It might be because of embedding them in SQL query text of
the application itself. On one front they are localized to the SQL
(good), but the PG operator has no realistic way of altering that once
it's embedded in binary (bad), as the application team is usually
separate if not from an external company (very bad situation, but
happens almost always).
I haven't quite figured out whether the problem is that hints are
actually bad or whether it's more that we just hate saying the word
hints. The reason I'm talking about hints here is mostly because
that's how other systems let users control the query planner. If we
want to let extensions control the query planner, we need to know in
what ways it needs to be controllable, and looking to hints in other
systems is one way to understand what would be useful. As far as
having hints in PostgreSQL, which admittedly is not really the topic
of this thread either, one objection is that we should just make the
query planner instead, and I used to believe that, but I no longer do,
because I've been doing this PostgreSQL thing for 15+ years and we're
not much closer to a perfect query planner that never makes any
mistakes than we were when I started. It's not really clear that
perfection is possible, but it's extremely clear that we're not
getting there any time soon. Another objection to hints is that they
require modifying the query text, which does indeed suck but it
doesn't mean they're useless either. There are also schemes that put
them out of line, including pg_hint_plan's optional use of a hint
table. Yet another objection is that you should fix cardinalities
instead of controlling the plan manually, and I agree that's often a
better solution, but it again does not mean that using a hint is never
defensible in any situation. I think we've become so negative about
hints that we rarely have a rational discussion about them. I'm no
more keen to see every PostgreSQL query in the universe decorated with
a bunch of hints than anyone else here, but I also don't enjoy telling
a customer "hey, I know this query started misbehaving in the middle
of the night on Christmas, but hints are bad and we shouldn't ever
have them so you'd better get started on redesigning your schema or
alternatively you can just have your web site be down for the next 20
years while we try to improve the optimizer." I don't know what the
right solution(s) are exactly but it's insane not to have some kind of
pressure relief valve that can be used in case of emergency.
- Would stacking of such extensions, each overriding the same planner
hooks, be allowed or not in the long run ? Technically there's nothing
preventing it and I think I could imagine someone attempting to run
multiple planner hook hotfixes for multiple issues, all at once?
I suspect this would tend to work poorly in practice, but there might
be specific cases where it works OK. It's usually best if only one
person is steering a given vehicle at a time, and so here. But there's
no intrinsic reason you couldn't use multiple extensions at once if
you happen to have multiple extensions that use the hooks in mutually
compatible ways.
- Shouldn't EXPLAIN contain additional information that for that
specific plan the optimizer hooks changed at least 1 thing ? (e.g.
"Plan was tainted" or something like that). Maybe extension could mark
it politely that it did by setting a certain flag, or maybe there
should be routines exposed by the core to do that ?
This could be really useful when things go wrong and someone is trying
to figure out from an EXPLAIN ANALYZE output what in the world
happened. I'm not sure exactly what makes sense to do here but I think
we should come back to this topic after we've settled some of the
basics.
--
Robert Haas
EDB: http://www.enterprisedb.com
On Wed, Aug 28, 2024 at 8:37 AM Robert Haas <robertmhaas@gmail.com> wrote:
This has a big advantage over what I proposed yesterday in that it's
basically declarative. With one call to the hook, you get all the
information about the join order that you could ever want. That's
really nice.
Hmm. On further thought, I suppose that another disadvantage of this
kind of declarative approach is that there are some kinds of
constraints that you could conceivably want that you just can't
declare, especially negative constraints. For example, imagine we're
joining tables A1 ... A10 and we don't want A1 to be joined directly
to A2. Or suppose you want to insist on a non-bushy plan. I don't
think there's a way to express those types of requirements by frobbing
the joinlist.
I'm not quite sure whether those kinds of gaps are sufficiently
serious that we should worry about them. After all, there's a lot of
things that you can express perfectly clearly with this kind of
scheme. I don't think I know of something that you can do to control
the join order in an existing hinting system that cannot be expressed
as a manipulation of the joinlist. That's not to say that I am 100%
confident that everything everyone could reasonably want to do can be
expressed this way; in fact, I don't think that's true at all. But I
_think_ that all of the things that I know about that people are
actually doing _could_ be expressed this way, but for the
join-direction and hard-failulre problems I mentioned in my earlier
reply.
--
Robert Haas
EDB: http://www.enterprisedb.com
On Wed, Aug 28, 2024 at 10:58 AM Robert Haas <robertmhaas@gmail.com> wrote:
Ever since I read
https://15721.courses.cs.cmu.edu/spring2020/papers/22-costmodels/p204-leis.pdf
I have believed that the cardinality misestimate leading to nested
loop plans is just because we're doing something dumb.
We don't even have an option to turn off that kind of join, and we
could probably avoid a lot of pain if we did. This, too, is mostly
separate from the topic of this thread, but I just can't believe we've
chosen to do literally nothing about this given that we all know this
specific thing hoses everybody, everywhere, all the time.
I couldn't agree more. I was really annoyed when your proposal was shot down.
It's an unusually clear-cut issue. Tying it to much broader and much
more complicated questions about how we model risk was a mistake.
--
Peter Geoghegan
On Tue, 2024-08-27 at 15:11 -0400, Robert Haas wrote:
- control over scan methods
- control over index selection
- control over join methods
- control over join order
I suggest we split join order into "commutative" and "associative".
The former is both useful and seems relatively easy -- A JOIN B or B
JOIN A (though there's some nuance about when you try to make that
decision).
The latter requires controlling an explosion of possibilities, and
would be an entirely different kind of hook.
Regards,
Jeff Davis
On Wed, Aug 28, 2024 at 3:23 PM Jeff Davis <pgsql@j-davis.com> wrote:
On Tue, 2024-08-27 at 15:11 -0400, Robert Haas wrote:
- control over scan methods
- control over index selection
- control over join methods
- control over join orderI suggest we split join order into "commutative" and "associative".
The former is both useful and seems relatively easy -- A JOIN B or B
JOIN A (though there's some nuance about when you try to make that
decision).The latter requires controlling an explosion of possibilities, and
would be an entirely different kind of hook.
My proposal in /messages/by-id/CA+TgmoZQyVxnRU--4g2bJonJ8RyJqNi2CHpy-=nwwBTNpAj71A@mail.gmail.com
seems like it can cover both cases.
--
Robert Haas
EDB: http://www.enterprisedb.com
On Mon, 2024-08-26 at 12:32 -0400, Robert Haas wrote:
I think there are two basic approaches that are possible here. If
someone sees a third option, let me know. First, we could allow users
to hook add_path() and add_partial_path().
...
The other possible approach is to allow extensions to feed some
information into the planner before path generation and let that
influence which paths are generated.
Preserving a path for the right amount of time seems like the primary
challenge for most of the use cases you raised (removing paths is
easier than resurrecting one that was pruned too early). If we try to
keep a path around, that implies that we need to keep parent paths
around too, which leads to an explosion if we aren't careful.
But we already solved all of that for pathkeys. We keep the paths
around if there's a reason to (a useful pathkey) and there's not some
other cheaper path that also satisfies the same reason.
Idea: generalize the idea of "pathkeys" to work for other reasons to
preserve a path.
Mechanically, a hint to use an index could work very similarly: come up
with a custom reason to keep a path around, such as "a hint suggests we
use index foo_idx for table foo", and assign it a unique number. If
there's another hint that says we should also use index bar_idx for
table bar, then that reason would get a different unique reason number.
(In other words, the number of reasons would not be fixed; there could
be one reason for each hint specified in the query, kind of like there
could be many interesting pathkeys for a query.)
Each Path would have a "preserve_for_these_reasons" bitmapset holding
all of the non-cost reasons we are preserving that path. If two paths
have exactly the same set of reasons, then add_path() would only keep
the cheaper one.
We could get fancy and have a compare_reasons_hook that would allow you
to take two paths with the same reason and see if there are other
factors to consider that would cause both to still be preserved
(similar to pathkey length).
I suspect that we might see interesting applications of this mechanism
in core as well: for instance, track partition keys or other properties
relevant to parallelism. That could allow us to keep parallel-friendly
paths around and then decide later in the planning process whether to
actually parallelize them or not.
Once we've generalized the "reasons" mechnism, it would be easy enough
to have a hook to add reasons to a path as it's being generated to be
sure it's not lost. These hooks should probably be called in the
individual create_*_path() functions where there's enough context to
know what's happening. There could be many such hooks, but I suspect
only a handful of important ones.
This idea allows the extension author to preserve the right paths long
enough to use set_rel_pathlist_hook/set_join_pathlist_hook, which can
editorialize on costs or do its own pruning.
Regards,
Jeff Davis
On Wed, Aug 28, 2024 at 4:29 PM Jeff Davis <pgsql@j-davis.com> wrote:
Preserving a path for the right amount of time seems like the primary
challenge for most of the use cases you raised (removing paths is
easier than resurrecting one that was pruned too early). If we try to
keep a path around, that implies that we need to keep parent paths
around too, which leads to an explosion if we aren't careful.But we already solved all of that for pathkeys. We keep the paths
around if there's a reason to (a useful pathkey) and there's not some
other cheaper path that also satisfies the same reason.
But we've already solved it for this case, too. This is exactly what
incrementing disabled_nodes does. This very recently replaced what we
did previously, which was adding disable_cost to the cost of every
path. Either way, you just need a hook that lets you disable the paths
that you don't prefer. Once you do that, add_path() takes care of the
rest: disabled paths lose to non-disabled paths, and disabled paths
lose to more expensive disabled paths.
--
Robert Haas
EDB: http://www.enterprisedb.com
On Wed, 2024-08-28 at 16:35 -0400, Robert Haas wrote:
On Wed, Aug 28, 2024 at 4:29 PM Jeff Davis <pgsql@j-davis.com> wrote:
Preserving a path for the right amount of time seems like the
primary
challenge for most of the use cases you raised (removing paths is
easier than resurrecting one that was pruned too early). If we try
to
keep a path around, that implies that we need to keep parent paths
around too, which leads to an explosion if we aren't careful.But we already solved all of that for pathkeys. We keep the paths
around if there's a reason to (a useful pathkey) and there's not
some
other cheaper path that also satisfies the same reason.But we've already solved it for this case, too. This is exactly what
incrementing disabled_nodes does.
Hints are often described as something positive: use this index, use a
hash join here, etc. Trying to force a positive thing by adding
negative attributes to everything else is awkward. We've all had the
experience where we disable one plan type hoping for a good plan, and
we end up getting a different crazy plan that we didn't expect, and
need to disable a few more plan types.
Beyond awkwardness, one case where it matters is the interaction
between an extension that provides hints and an extension that offers a
CustomScan. How is the hints extension supposed to disable a path it
doesn't know about?
Regards,
Jeff Davis
Jeff Davis <pgsql@j-davis.com> writes:
Beyond awkwardness, one case where it matters is the interaction
between an extension that provides hints and an extension that offers a
CustomScan. How is the hints extension supposed to disable a path it
doesn't know about?
This does not seem remarkably problematic to me, given Robert's
proposal of a bitmask of allowed plan types per RelOptInfo.
You just do something like
rel->allowed_plan_types = DESIRED_PLAN_TYPE;
The names of the bits you aren't setting are irrelevant to you.
regards, tom lane
On Wed, Aug 28, 2024 at 07:25:59PM -0400, Tom Lane wrote:
Jeff Davis <pgsql@j-davis.com> writes:
Beyond awkwardness, one case where it matters is the interaction
between an extension that provides hints and an extension that offers a
CustomScan. How is the hints extension supposed to disable a path it
doesn't know about?
pg_hint_plan documents its hints here:
https://pg-hint-plan.readthedocs.io/en/master/hint_list.html#hint-list
Hmm. I think that we should be careful to check that this works
correctly with pg_hint_plan, at least. The module goes through a lot
of tweaks and is a can of worms in terms of plan adjustments because
we can only rely on the planner hook to do the whole work. This leads
to a lot of frustration for users because each feature somebody asks
for leads to just more tweaks to apply on the paths.
The bullet list sent here sounds pretty good to me:
/messages/by-id/3131957.1724789735@sss.pgh.pa.us
This does not seem remarkably problematic to me, given Robert's
proposal of a bitmask of allowed plan types per RelOptInfo.
You just do something likerel->allowed_plan_types = DESIRED_PLAN_TYPE;
The names of the bits you aren't setting are irrelevant to you.
For the types of scans to use, that would be OK. The module has a
feature where one can also have a regex to match for an index, and
the module is very funky with inheritance and partitioned tables.
How does that help if using a Leading hint to force a join order?
That's something people like a lot. But perhaps that's just the part
of upthread where we'd need a extra hook? I am not completely sure to
get the portion of the proposal for that. add_paths_to_joinrel() has
been mentioned, and there is set_join_pathlist_hook already there.
--
Michael
On Wed, 2024-08-28 at 19:25 -0400, Tom Lane wrote:
This does not seem remarkably problematic to me, given Robert's
proposal of a bitmask of allowed plan types per RelOptInfo.
You just do something likerel->allowed_plan_types = DESIRED_PLAN_TYPE;
The names of the bits you aren't setting are irrelevant to you.
I don't see that in the code yet, so I assume you are referring to the
comment at [1]/messages/by-id/CA+TgmoZQyVxnRU--4g2bJonJ8RyJqNi2CHpy-=nwwBTNpAj71A@mail.gmail.com?
I still like my idea to generalize the pathkey infrastructure, and
Robert asked for other approaches to consider. It would allow us to
hold onto multiple paths for longer, similar to pathkeys, which might
offer some benefits or simplifications.
Regards,
Jeff Davis
[1]: /messages/by-id/CA+TgmoZQyVxnRU--4g2bJonJ8RyJqNi2CHpy-=nwwBTNpAj71A@mail.gmail.com
/messages/by-id/CA+TgmoZQyVxnRU--4g2bJonJ8RyJqNi2CHpy-=nwwBTNpAj71A@mail.gmail.com
On Thu, Aug 29, 2024 at 6:49 PM Jeff Davis <pgsql@j-davis.com> wrote:
I don't see that in the code yet, so I assume you are referring to the
comment at [1]?
FYI, I'm hacking on a revised approach but it's not ready to show to
other people yet.
I still like my idea to generalize the pathkey infrastructure, and
Robert asked for other approaches to consider. It would allow us to
hold onto multiple paths for longer, similar to pathkeys, which might
offer some benefits or simplifications.
This is a fair point. I dislike the fact that add_path() is a thicket
of if-statements that's actually quite hard to understand and easy to
screw up when you're making modifications. But I feel like it would be
difficult to generalize the infrastructure without making it
substantially slower, which would probably cause too much of an
increase in planning time to be acceptable. So my guess is that this
is a dead end, unless there's a clever idea that I'm not seeing.
--
Robert Haas
EDB: http://www.enterprisedb.com
On Fri, 2024-08-30 at 07:33 -0400, Robert Haas wrote:
This is a fair point. I dislike the fact that add_path() is a thicket
of if-statements that's actually quite hard to understand and easy to
screw up when you're making modifications. But I feel like it would
be
difficult to generalize the infrastructure without making it
substantially slower, which would probably cause too much of an
increase in planning time to be acceptable. So my guess is that this
is a dead end, unless there's a clever idea that I'm not seeing.
As far as performance goes, I'm only looking at branch in add_path()
that calls compare_pathkeys(). Do you have some example queries which
would be a worst case for that path?
In general if you can post some details about how you are measuring,
that would be helpful.
Regards,
Jeff Davis
On Fri, Aug 30, 2024 at 1:42 PM Jeff Davis <pgsql@j-davis.com> wrote:
As far as performance goes, I'm only looking at branch in add_path()
that calls compare_pathkeys(). Do you have some example queries which
would be a worst case for that path?
I think we must be talking past each other somehow. It seems to me
that your scheme would require replacing that branch with something
more complicated or generalized. If it doesn't, then I don't
understand the proposal. If it does, then that seems like it could be
a problem.
In general if you can post some details about how you are measuring,
that would be helpful.
I'm not really measuring anything at this point, just recalling the
many previous times when add_path() has been discussed as a pain
point.
--
Robert Haas
EDB: http://www.enterprisedb.com
On Tue, Aug 27, 2024 at 4:07 PM Robert Haas <robertmhaas@gmail.com> wrote:
I think that the beginning of add_paths_to_joinrel() looks like a
useful spot to get control. You could, for example, have a hook there
which returns a bitmask indicating which of merge-join, nested-loop,
and hash join will be allowable for this call; that hook would then
allow for control over the join method and the join order, and the
join order control is strong enough that you can implement either of
the two interpretations above. This idea theorizes that 0001 was wrong
to make the path mask a per-RelOptInfo value, because there could be
many calls to add_paths_to_joinrel() for a single RelOptInfo and, in
this idea, every one of those can enforce a different mask.
Here is an implementation of this idea. I think this is significantly
more elegant than the previous patch. Functionally, it does a better
job allowing for control over join planning than the previous patch,
because you can control both the join method and the join order. It
does not attempt to provide control over scan or appendrel methods; I
could build similar machinery for those cases, but let's talk about
this case, first. As non-for-commit proofs-of-concept, I've included
two sample contrib modules here, one called alphabet_join and one
called hint_via_alias. alphabet_join forces the join to be done in
alphabetical order by table alias. Consider this test query:
SELECT COUNT(*) FROM pgbench_accounts THE
INNER JOIN pgbench_accounts QUICK on THE.aid = QUICK.aid
INNER JOIN pgbench_accounts BROWN on THE.aid = BROWN.aid
INNER JOIN pgbench_accounts FOX on THE.aid = FOX.aid
INNER JOIN pgbench_accounts JUMPED on THE.aid = JUMPED.aid
INNER JOIN pgbench_accounts OVER on THE.aid = OVER.aid
INNER JOIN pgbench_accounts LAZY on THE.aid = LAZY.aid
INNER JOIN pgbench_accounts DOG on THE.aid = DOG.aid;
When you just execute this normally, the join order matches the order
you enter it: THE QUICK BROWN FOX JUMPED OVER LAZY DOG. But if you
load alphabet_join, then the join order becomes BROWN DOG FOX JUMPED
LAZY OVER QUICK THE. It is unlikely that anyone wants their join order
to be determined by strict alphabetical order, but again, this is just
intended to show that the hook works.
hint_via_alias whose table alias starts with mj_, hj_, or nl_ using a
merge-join, hash-join, or nested loop, respectively. Here again, I
don't think that passing hints through the table alias names is
probably the best thing from a UI perspective, but unlike the previous
one which is clearly a toy, I can imagine someone actually trying to
use this one on a real server. If we want anything in contrib at all
it should probably be something much better than this, but again at
this stage I'm just trying to showcase the hook.
Potentially, such a hook could return additional information, either
by using more bits of the bitmask or by returning other information
via some other data type. For instance, I still believe that
distinguishing between parameterized-nestloops and
unparameterized-nestloops would be real darn useful, so we could have
separate bits for each; or you could have a bit to control whether
foreign-join paths get disabled (or are considered at all), or you
could have separate bits for merge joins that involve 0, 1, or 2
sorts. Whether we need or want any or all of that is certainly
debatable, but the point is that if you did want some of that, or
something else, it doesn't look difficult to feed that information
through to the places where you would need it to be available.
I spent a lot of time thinking about what should and should not be in
scope for this hook and decided against both of the ideas above.
They're not necessarily bad ideas but they feel like examples of
arbitrary policy that you could want to implement, and I don't think
it's viable to have every arbitrary policy that someone happens to
favor in the core code. If we want extensions to be able to achieve
these kinds of results, I think we're going to need a hook at either
initial_cost_XXX time that would be free to make arbitrary decisions
about cost and disabled nodes for each possible path we might
generate, or a hook at final_cost_XXX time that could make paths more
disabled (but not less) or more costly (but not less costly unless it
also makes them more disabled). For now, I have not done that, because
I think the hook that I've added to add_paths_to_joinrel is fairly
powerful and significantly cheaper than a hook that has to fire for
every possible generated path. Also, even if we do add that, I bet
it's useful to let this hook pass some state through to that hook, as
a way of avoiding recomputation.
However, although I didn't end up including either of the policies
mentioned above in this patch, I still did end up subdividing the
"merge join" strategy according to whether or not a Materialize node
is used, and the "nested loop" strategy according to whether we use
Materialize, Memoize, or neither. At least according to my reading of
the code, the planner really does consider these to be separate
sub-strategies: it thinks about whether to use a nested loop without a
materialize node, and it thinks about whether to do a nested loop with
a materialize node, and there's separate code for those things. So
this doesn't feel like an arbitrary distinction to me. In contrast,
the parameterized-vs-unparameterized nested loop thing is just a
question of whether the outer path that we happen to choose happens to
satisfy some part of the parameterization of the inner path we happen
to choose; the code neither knows nor cares whether that will occur.
There is also a pragmatic reason to make sure that the hook allows for
control over use of these sub-strategies: pg_hint_plan has Memoize and
NoMemoize hints, and if whatever hook we add here can't replace what
pg_hint_plan is already doing, then it's clearly not up to the mark.
I also spent some time thinking about what behavior this hook does and
does not allow you to control. As noted, it allows you to control the
join method and the join order, including which table ends up on which
side of the join. But, is that good enough to reproduce a very
specific plan, say one that you saw before and liked? Let's imagine
that you've arranged to disable every path in outerrel and innerrel
other than the ones that you want to be chosen, either using some
hackery or some future patch not included here. Now, you want to use
this patch to make sure that those are joined in the way that you want
them to be joined. Can you do that? I think the answer is "mostly".
You'll be able to get the join method you want used, and you'll be
able to get Memoize and/or Materialize nodes if you want them or avoid
them if you don't. Also, join_path_setup_hook will see
JOIN_UNIQUE_INNER or JOIN_UNIQUE_OUTER so if we're thinking of
implementing a semijoin via uniquify+join, the hook will be able to
encourage or discourage that approach if it wants. However, it *won't*
be able to force the uniquification to happen using hashing rather
than sorting or vice versa, or at least not without doing something
pretty kludgy. Also, it won't be able to force a merge join produced
by sort_inner_and_outer() to use the sort keys that it prefers. Merge
joins produced by match_unsorted_outer() are entirely a function of
the input paths, but those produced by sort_inner_and_outer() are not.
Aside from these two cases, I found no other gaps.
AFAICS, the only way to close the gap around unique-ification strategy
would be with some piece of bespoke infrastructure that just does
exactly that. The inability to control the sort order selected by
sort_inner_and_outer() could, I believe, be closed by a hook at
initial or final cost time. As noted above, such a hook is also useful
when you're not trying to arrive at a specific plan, but rather have
some policy around what kind of plan you want to end up with and wish
to penalize plans that don't comply with your policy. So maybe this is
an argument for adding that hook. That said, even without that, this
might be close enough for government work. If the planner chooses the
correct input paths and if it also chooses a merge join, how likely is
it that it will now choose the wrong pathkeys to perform that merge
join? I bet it's quite unlikely, because I think the effect of the
logic we have will probably be to just do the smallest possible amount
of sorting, and that's also probably the answer the user wants. So it
might be OK in practice to just not worry about this. On the other
hand, a leading cause of disasters is people assuming that certain
things would not go wrong and then having those exact things go wrong,
so it's probably unwise to be confident that the attached patch is all
we'll ever need.
Still, I think it's a pretty useful starting point. It is mostly
enough to give you control over join planning, and if combined with
similar work for scan planning, I think it would be enough for
pg_hint_plan. If we also got control over appendrel and agg planning,
then you could do a bunch more cool things.
Comments?
--
Robert Haas
EDB: http://www.enterprisedb.com
Attachments:
v3-0003-New-contrib-module-hint_via_alias.patchapplication/octet-stream; name=v3-0003-New-contrib-module-hint_via_alias.patchDownload
From 2e612e565918608e192125830b821df5d1aa3c84 Mon Sep 17 00:00:00 2001
From: Robert Haas <rhaas@postgresql.org>
Date: Wed, 18 Sep 2024 10:13:19 -0400
Subject: [PATCH v3 3/3] New contrib module: hint_via_alias.
This forces a table to be merge joined, hash joined, or joined via a nested
loop if the table alias starts with mj_, hj_, or nl_, respectively. It
demonstrates that join_path_setup_hook is sufficient to control the join
method, and is not intended for commit.
---
contrib/Makefile | 1 +
contrib/hint_via_alias/Makefile | 17 ++++
contrib/hint_via_alias/hint_via_alias.c | 114 ++++++++++++++++++++++++
contrib/hint_via_alias/meson.build | 12 +++
contrib/meson.build | 1 +
5 files changed, 145 insertions(+)
create mode 100644 contrib/hint_via_alias/Makefile
create mode 100644 contrib/hint_via_alias/hint_via_alias.c
create mode 100644 contrib/hint_via_alias/meson.build
diff --git a/contrib/Makefile b/contrib/Makefile
index b3422616698..2b47095ce18 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -22,6 +22,7 @@ SUBDIRS = \
earthdistance \
file_fdw \
fuzzystrmatch \
+ hint_via_alias \
hstore \
intagg \
intarray \
diff --git a/contrib/hint_via_alias/Makefile b/contrib/hint_via_alias/Makefile
new file mode 100644
index 00000000000..2e0e540d352
--- /dev/null
+++ b/contrib/hint_via_alias/Makefile
@@ -0,0 +1,17 @@
+# contrib/hint_via_alias/Makefile
+
+MODULE_big = hint_via_alias
+OBJS = \
+ $(WIN32RES) \
+ hint_via_alias.o
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/hint_via_alias
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/hint_via_alias/hint_via_alias.c b/contrib/hint_via_alias/hint_via_alias.c
new file mode 100644
index 00000000000..fc0fbd85c22
--- /dev/null
+++ b/contrib/hint_via_alias/hint_via_alias.c
@@ -0,0 +1,114 @@
+/*-------------------------------------------------------------------------
+ *
+ * hint_via_alias.c
+ * force tables to be joined in using a nestedloop, mergejoin, or hash
+ * join if their alias name begins with nl_, mj_, or hj_.
+ *
+ * Copyright (c) 2016-2024, PostgreSQL Global Development Group
+ *
+ * contrib/hint_via_alias/hint_via_alias.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "fmgr.h"
+#include "optimizer/paths.h"
+#include "parser/parsetree.h"
+
+typedef enum
+{
+ HVA_UNSPECIFIED,
+ HVA_HASHJOIN,
+ HVA_MERGEJOIN,
+ HVA_NESTLOOP
+} hva_hint;
+
+static void hva_join_path_setup_hook(PlannerInfo *root,
+ RelOptInfo *joinrel,
+ RelOptInfo *outerrel,
+ RelOptInfo *innerrel,
+ JoinType jointype,
+ JoinPathExtraData *extra);
+static hva_hint get_hint(PlannerInfo *root, Index relid);
+
+static join_path_setup_hook_type prev_join_path_setup_hook = NULL;
+
+PG_MODULE_MAGIC;
+
+void
+_PG_init(void)
+{
+ prev_join_path_setup_hook = join_path_setup_hook;
+ join_path_setup_hook = hva_join_path_setup_hook;
+}
+
+static void
+hva_join_path_setup_hook(PlannerInfo *root, RelOptInfo *joinrel,
+ RelOptInfo *outerrel, RelOptInfo *innerrel,
+ JoinType jointype, JoinPathExtraData *extra)
+{
+ hva_hint outerhint = HVA_UNSPECIFIED;
+ hva_hint innerhint = HVA_UNSPECIFIED;
+ hva_hint hint;
+
+ if (outerrel->reloptkind == RELOPT_BASEREL ||
+ outerrel->reloptkind == RELOPT_OTHER_MEMBER_REL)
+ outerhint = get_hint(root, outerrel->relid);
+
+ if (innerrel->reloptkind == RELOPT_BASEREL ||
+ innerrel->reloptkind == RELOPT_OTHER_MEMBER_REL)
+ innerhint = get_hint(root, innerrel->relid);
+
+ /*
+ * If the hints conflict, that's not necessarily an indication of user error.
+ * For example, if the user joins A to B and supplies different join method hints
+ * for A and B, we will end up using a disabled path. However, if they are joining
+ * A, B, and C and supply different join method hints for A and B, we could
+ * potentially respect both hints by avoiding a direct A-B join altogether. Even if
+ * it does turn out that we can't respect all the hints, we don't need any special
+ * handling for that here: the planner will just return a disabled path.
+ */
+ if (outerhint != HVA_UNSPECIFIED && innerhint != HVA_UNSPECIFIED &&
+ outerhint != innerhint)
+ {
+ extra->jsa_mask = 0;
+ return;
+ }
+
+ if (outerhint != HVA_UNSPECIFIED)
+ hint = outerhint;
+ else
+ hint = innerhint;
+
+ switch (hint)
+ {
+ case HVA_UNSPECIFIED:
+ break;
+ case HVA_NESTLOOP:
+ extra->jsa_mask &= JSA_NESTLOOP_ANY;
+ break;
+ case HVA_HASHJOIN:
+ extra->jsa_mask &= JSA_HASHJOIN;
+ break;
+ case HVA_MERGEJOIN:
+ extra->jsa_mask &= JSA_MERGEJOIN_ANY;
+ break;
+ }
+}
+
+static hva_hint
+get_hint(PlannerInfo *root, Index relid)
+{
+ RangeTblEntry *rte = planner_rt_fetch(relid, root);
+
+ Assert(rte->eref != NULL && rte->eref->aliasname != NULL);
+
+ if (strncmp(rte->eref->aliasname, "nl_", 3) == 0)
+ return HVA_NESTLOOP;
+ else if (strncmp(rte->eref->aliasname, "hj_", 3) == 0)
+ return HVA_HASHJOIN;
+ else if (strncmp(rte->eref->aliasname, "mj_", 3) == 0)
+ return HVA_MERGEJOIN;
+ else
+ return HVA_UNSPECIFIED;
+}
diff --git a/contrib/hint_via_alias/meson.build b/contrib/hint_via_alias/meson.build
new file mode 100644
index 00000000000..7e42c5783ab
--- /dev/null
+++ b/contrib/hint_via_alias/meson.build
@@ -0,0 +1,12 @@
+# Copyright (c) 2022-2024, PostgreSQL Global Development Group
+
+hint_via_alias_sources = files(
+ 'hint_via_alias.c',
+)
+
+hint_via_alias = shared_module('hint_via_alias',
+ hint_via_alias_sources,
+ kwargs: contrib_mod_args,
+)
+
+contrib_targets += hint_via_alias
diff --git a/contrib/meson.build b/contrib/meson.build
index 4372242c8f3..261e4c480e2 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -30,6 +30,7 @@ subdir('dict_xsyn')
subdir('earthdistance')
subdir('file_fdw')
subdir('fuzzystrmatch')
+subdir('hint_via_alias')
subdir('hstore')
subdir('hstore_plperl')
subdir('hstore_plpython')
--
2.39.3 (Apple Git-145)
v3-0002-New-contrib-module-alphabet_join.patchapplication/octet-stream; name=v3-0002-New-contrib-module-alphabet_join.patchDownload
From 3845a51afdd2d197ff61368f45a5178e46d5fded Mon Sep 17 00:00:00 2001
From: Robert Haas <rhaas@postgresql.org>
Date: Fri, 30 Aug 2024 10:27:31 -0400
Subject: [PATCH v3 2/3] New contrib module: alphabet_join.
This forces joins to be done alphabetically by alias name. It demonstrates
that join_path_setup_hook is sufficient to control the join order, and
is not intended for commit.
---
contrib/Makefile | 1 +
contrib/alphabet_join/Makefile | 17 ++++++
contrib/alphabet_join/alphabet_join.c | 74 +++++++++++++++++++++++++++
contrib/alphabet_join/meson.build | 12 +++++
contrib/meson.build | 1 +
5 files changed, 105 insertions(+)
create mode 100644 contrib/alphabet_join/Makefile
create mode 100644 contrib/alphabet_join/alphabet_join.c
create mode 100644 contrib/alphabet_join/meson.build
diff --git a/contrib/Makefile b/contrib/Makefile
index abd780f2774..b3422616698 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -5,6 +5,7 @@ top_builddir = ..
include $(top_builddir)/src/Makefile.global
SUBDIRS = \
+ alphabet_join \
amcheck \
auth_delay \
auto_explain \
diff --git a/contrib/alphabet_join/Makefile b/contrib/alphabet_join/Makefile
new file mode 100644
index 00000000000..204bc35b3d4
--- /dev/null
+++ b/contrib/alphabet_join/Makefile
@@ -0,0 +1,17 @@
+# contrib/alphabet_join/Makefile
+
+MODULE_big = alphabet_join
+OBJS = \
+ $(WIN32RES) \
+ alphabet_join.o
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/alphabet_join
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/alphabet_join/alphabet_join.c b/contrib/alphabet_join/alphabet_join.c
new file mode 100644
index 00000000000..6794bded047
--- /dev/null
+++ b/contrib/alphabet_join/alphabet_join.c
@@ -0,0 +1,74 @@
+/*-------------------------------------------------------------------------
+ *
+ * alphabet_join.c
+ * force tables to be joined in alphabetical order by alias name.
+ * this is just a demonstration, so we don't worry about collation here.
+ *
+ * Copyright (c) 2016-2024, PostgreSQL Global Development Group
+ *
+ * contrib/alphabet_join/alphabet_join.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "fmgr.h"
+#include "optimizer/paths.h"
+#include "parser/parsetree.h"
+
+static void aj_join_path_setup_hook(PlannerInfo *root,
+ RelOptInfo *joinrel,
+ RelOptInfo *outerrel,
+ RelOptInfo *innerrel,
+ JoinType jointype,
+ JoinPathExtraData *extra);
+
+static join_path_setup_hook_type prev_join_path_setup_hook = NULL;
+
+PG_MODULE_MAGIC;
+
+void
+_PG_init(void)
+{
+ prev_join_path_setup_hook = join_path_setup_hook;
+ join_path_setup_hook = aj_join_path_setup_hook;
+}
+
+static void
+aj_join_path_setup_hook(PlannerInfo *root, RelOptInfo *joinrel,
+ RelOptInfo *outerrel, RelOptInfo *innerrel,
+ JoinType jointype, JoinPathExtraData *extra)
+{
+ int relid;
+ char *outerrel_last = NULL;
+
+ /* Find the alphabetically last outerrel. */
+ relid = -1;
+ while ((relid = bms_next_member(outerrel->relids, relid)) >= 0)
+ {
+ RangeTblEntry *rte = planner_rt_fetch(relid, root);
+
+ Assert(rte->eref != NULL && rte->eref->aliasname != NULL);
+
+ if (outerrel_last == NULL ||
+ strcmp(outerrel_last, rte->eref->aliasname) < 0)
+ outerrel_last = rte->eref->aliasname;
+ }
+
+ /*
+ * If any innerrel is alphabetically before the last outerrel, then
+ * this join order is not alphabetical and should be rejected.
+ */
+ relid = -1;
+ while ((relid = bms_next_member(innerrel->relids, relid)) >= 0)
+ {
+ RangeTblEntry *rte = planner_rt_fetch(relid, root);
+
+ Assert(rte->eref != NULL && rte->eref->aliasname != NULL);
+
+ if (strcmp(rte->eref->aliasname, outerrel_last) < 0)
+ {
+ extra->jsa_mask = 0;
+ return;
+ }
+ }
+}
diff --git a/contrib/alphabet_join/meson.build b/contrib/alphabet_join/meson.build
new file mode 100644
index 00000000000..437cb14af58
--- /dev/null
+++ b/contrib/alphabet_join/meson.build
@@ -0,0 +1,12 @@
+# Copyright (c) 2022-2024, PostgreSQL Global Development Group
+
+alphabet_join_sources = files(
+ 'alphabet_join.c',
+)
+
+alphabet_join = shared_module('alphabet_join',
+ alphabet_join_sources,
+ kwargs: contrib_mod_args,
+)
+
+contrib_targets += alphabet_join
diff --git a/contrib/meson.build b/contrib/meson.build
index 14a89068650..4372242c8f3 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -12,6 +12,7 @@ contrib_doc_args = {
'install_dir': contrib_doc_dir,
}
+subdir('alphabet_join')
subdir('amcheck')
subdir('auth_delay')
subdir('auto_explain')
--
2.39.3 (Apple Git-145)
v3-0001-Allow-extensions-to-control-join-strategy.patchapplication/octet-stream; name=v3-0001-Allow-extensions-to-control-join-strategy.patchDownload
From 815bd68324ce90e71e0ab40d7d9a08dea940423a Mon Sep 17 00:00:00 2001
From: Robert Haas <rhaas@postgresql.org>
Date: Thu, 29 Aug 2024 15:38:58 -0400
Subject: [PATCH v3 1/3] Allow extensions to control join strategy.
At the start of join planning, we build a bitmask called jsa_mask based on
the values of the various enable_* planner GUCs, indicating which join
strategies are allowed. Extensions can override this mask on a plan-wide
basis using join_search_hook, or, generaly more usefully, they can change
the mask for each call to add_paths_to_joinrel using a new hook called
join_path_setup_hook. This is sufficient to allow an extension to force
the use of particular join strategies either in general or for specific
joins, and it is also sufficient to allow an extension to force the join
order.
There are a number of things that this patch doesn't let you do.
First, it won't help if you want to implement some policy where the
allowable join methods might differ for each individual combination of
input paths (e.g. disable nested loops only when the inner side's
parameterization is not even partially satisfied by the outer side's
paramaeterization). Second, it doesn't allow you to control the
uniquification strategy for a particular joinrel. Third, it doesn't
give you any control over aspects of planning other than join planning.
Future patches may close some of these gaps.
---
src/backend/optimizer/geqo/geqo_eval.c | 22 +++--
src/backend/optimizer/geqo/geqo_main.c | 10 ++-
src/backend/optimizer/geqo/geqo_pool.c | 4 +-
src/backend/optimizer/path/allpaths.c | 38 +++++++--
src/backend/optimizer/path/costsize.c | 59 +++++++++-----
src/backend/optimizer/path/joinpath.c | 104 +++++++++++++++++-------
src/backend/optimizer/path/joinrels.c | 79 ++++++++++--------
src/backend/optimizer/plan/createplan.c | 1 +
src/backend/optimizer/util/pathnode.c | 6 +-
src/backend/optimizer/util/relnode.c | 17 ++--
src/include/nodes/pathnodes.h | 2 +
src/include/optimizer/cost.h | 4 +-
src/include/optimizer/geqo.h | 9 +-
src/include/optimizer/geqo_pool.h | 2 +-
src/include/optimizer/pathnode.h | 9 +-
src/include/optimizer/paths.h | 56 +++++++++++--
16 files changed, 292 insertions(+), 130 deletions(-)
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
index d2f7f4e5f3c..6b1d8df3ff6 100644
--- a/src/backend/optimizer/geqo/geqo_eval.c
+++ b/src/backend/optimizer/geqo/geqo_eval.c
@@ -40,7 +40,7 @@ typedef struct
} Clump;
static List *merge_clump(PlannerInfo *root, List *clumps, Clump *new_clump,
- int num_gene, bool force);
+ int num_gene, bool force, unsigned jsa_mask);
static bool desirable_join(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel);
@@ -54,7 +54,7 @@ static bool desirable_join(PlannerInfo *root,
* returns DBL_MAX.
*/
Cost
-geqo_eval(PlannerInfo *root, Gene *tour, int num_gene)
+geqo_eval(PlannerInfo *root, Gene *tour, int num_gene, unsigned jsa_mask)
{
MemoryContext mycontext;
MemoryContext oldcxt;
@@ -99,7 +99,7 @@ geqo_eval(PlannerInfo *root, Gene *tour, int num_gene)
root->join_rel_hash = NULL;
/* construct the best path for the given combination of relations */
- joinrel = gimme_tree(root, tour, num_gene);
+ joinrel = gimme_tree(root, tour, num_gene, jsa_mask);
/*
* compute fitness, if we found a valid join
@@ -160,7 +160,7 @@ geqo_eval(PlannerInfo *root, Gene *tour, int num_gene)
* since there's no provision for un-clumping, this must lead to failure.)
*/
RelOptInfo *
-gimme_tree(PlannerInfo *root, Gene *tour, int num_gene)
+gimme_tree(PlannerInfo *root, Gene *tour, int num_gene, unsigned jsa_mask)
{
GeqoPrivateData *private = (GeqoPrivateData *) root->join_search_private;
List *clumps;
@@ -196,7 +196,8 @@ gimme_tree(PlannerInfo *root, Gene *tour, int num_gene)
cur_clump->size = 1;
/* Merge it into the clumps list, using only desirable joins */
- clumps = merge_clump(root, clumps, cur_clump, num_gene, false);
+ clumps = merge_clump(root, clumps, cur_clump, num_gene, false,
+ jsa_mask);
}
if (list_length(clumps) > 1)
@@ -210,7 +211,8 @@ gimme_tree(PlannerInfo *root, Gene *tour, int num_gene)
{
Clump *clump = (Clump *) lfirst(lc);
- fclumps = merge_clump(root, fclumps, clump, num_gene, true);
+ fclumps = merge_clump(root, fclumps, clump, num_gene, true,
+ jsa_mask);
}
clumps = fclumps;
}
@@ -236,7 +238,7 @@ gimme_tree(PlannerInfo *root, Gene *tour, int num_gene)
*/
static List *
merge_clump(PlannerInfo *root, List *clumps, Clump *new_clump, int num_gene,
- bool force)
+ bool force, unsigned jsa_mask)
{
ListCell *lc;
int pos;
@@ -259,7 +261,8 @@ merge_clump(PlannerInfo *root, List *clumps, Clump *new_clump, int num_gene,
*/
joinrel = make_join_rel(root,
old_clump->joinrel,
- new_clump->joinrel);
+ new_clump->joinrel,
+ jsa_mask);
/* Keep searching if join order is not valid */
if (joinrel)
@@ -292,7 +295,8 @@ merge_clump(PlannerInfo *root, List *clumps, Clump *new_clump, int num_gene,
* others. When no further merge is possible, we'll reinsert
* it into the list.
*/
- return merge_clump(root, clumps, old_clump, num_gene, force);
+ return merge_clump(root, clumps, old_clump, num_gene, force,
+ jsa_mask);
}
}
}
diff --git a/src/backend/optimizer/geqo/geqo_main.c b/src/backend/optimizer/geqo/geqo_main.c
index 0c5540e2af4..c69b60d2e95 100644
--- a/src/backend/optimizer/geqo/geqo_main.c
+++ b/src/backend/optimizer/geqo/geqo_main.c
@@ -69,7 +69,8 @@ static int gimme_number_generations(int pool_size);
*/
RelOptInfo *
-geqo(PlannerInfo *root, int number_of_rels, List *initial_rels)
+geqo(PlannerInfo *root, int number_of_rels, List *initial_rels,
+ unsigned jsa_mask)
{
GeqoPrivateData private;
int generation;
@@ -116,7 +117,7 @@ geqo(PlannerInfo *root, int number_of_rels, List *initial_rels)
pool = alloc_pool(root, pool_size, number_of_rels);
/* random initialization of the pool */
- random_init_pool(root, pool);
+ random_init_pool(root, pool, jsa_mask);
/* sort the pool according to cheapest path as fitness */
sort_pool(root, pool); /* we have to do it only one time, since all
@@ -218,7 +219,8 @@ geqo(PlannerInfo *root, int number_of_rels, List *initial_rels)
/* EVALUATE FITNESS */
- kid->worth = geqo_eval(root, kid->string, pool->string_length);
+ kid->worth = geqo_eval(root, kid->string, pool->string_length,
+ jsa_mask);
/* push the kid into the wilderness of life according to its worth */
spread_chromo(root, kid, pool);
@@ -269,7 +271,7 @@ geqo(PlannerInfo *root, int number_of_rels, List *initial_rels)
*/
best_tour = (Gene *) pool->data[0].string;
- best_rel = gimme_tree(root, best_tour, pool->string_length);
+ best_rel = gimme_tree(root, best_tour, pool->string_length, jsa_mask);
if (best_rel == NULL)
elog(ERROR, "geqo failed to make a valid plan");
diff --git a/src/backend/optimizer/geqo/geqo_pool.c b/src/backend/optimizer/geqo/geqo_pool.c
index 0ec97d5a3f1..dbec3c50943 100644
--- a/src/backend/optimizer/geqo/geqo_pool.c
+++ b/src/backend/optimizer/geqo/geqo_pool.c
@@ -88,7 +88,7 @@ free_pool(PlannerInfo *root, Pool *pool)
* initialize genetic pool
*/
void
-random_init_pool(PlannerInfo *root, Pool *pool)
+random_init_pool(PlannerInfo *root, Pool *pool, unsigned jsa_mask)
{
Chromosome *chromo = (Chromosome *) pool->data;
int i;
@@ -107,7 +107,7 @@ random_init_pool(PlannerInfo *root, Pool *pool)
{
init_tour(root, chromo[i].string, pool->string_length);
pool->data[i].worth = geqo_eval(root, chromo[i].string,
- pool->string_length);
+ pool->string_length, jsa_mask);
if (pool->data[i].worth < DBL_MAX)
i++;
else
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 172edb643a4..61863482346 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -898,7 +898,7 @@ set_tablesample_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *
bms_membership(root->all_query_rels) != BMS_SINGLETON) &&
!(GetTsmRoutine(rte->tablesample->tsmhandler)->repeatable_across_scans))
{
- path = (Path *) create_material_path(rel, path);
+ path = (Path *) create_material_path(rel, path, enable_material);
}
add_path(rel, path);
@@ -3371,6 +3371,29 @@ make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
}
else
{
+ unsigned jsa_mask;
+
+ /* Compute the initial join strategy advice mask. */
+ jsa_mask = JSA_FOREIGN;
+ if (enable_hashjoin)
+ jsa_mask |= JSA_HASHJOIN;
+ if (enable_mergejoin)
+ {
+ jsa_mask |= JSA_MERGEJOIN_PLAIN;
+ if (enable_material)
+ jsa_mask |= JSA_MERGEJOIN_MATERIALIZE;
+ }
+ if (enable_nestloop)
+ {
+ jsa_mask |= JSA_NESTLOOP_PLAIN;
+ if (enable_material)
+ jsa_mask |= JSA_NESTLOOP_MATERIALIZE;
+ if (enable_memoize)
+ jsa_mask |= JSA_NESTLOOP_MEMOIZE;
+ }
+ if (enable_partitionwise_join)
+ jsa_mask |= JSA_PARTITIONWISE;
+
/*
* Consider the different orders in which we could join the rels,
* using a plugin, GEQO, or the regular join search code.
@@ -3381,11 +3404,13 @@ make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
root->initial_rels = initial_rels;
if (join_search_hook)
- return (*join_search_hook) (root, levels_needed, initial_rels);
+ return (*join_search_hook) (root, levels_needed, initial_rels,
+ jsa_mask);
else if (enable_geqo && levels_needed >= geqo_threshold)
- return geqo(root, levels_needed, initial_rels);
+ return geqo(root, levels_needed, initial_rels, jsa_mask);
else
- return standard_join_search(root, levels_needed, initial_rels);
+ return standard_join_search(root, levels_needed, initial_rels,
+ jsa_mask);
}
}
@@ -3419,7 +3444,8 @@ make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
* original states of those data structures. See geqo_eval() for an example.
*/
RelOptInfo *
-standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
+standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels,
+ unsigned jsa_mask)
{
int lev;
RelOptInfo *rel;
@@ -3454,7 +3480,7 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
* level, and build paths for making each one from every available
* pair of lower-level relations.
*/
- join_search_one_level(root, lev);
+ join_search_one_level(root, lev, jsa_mask);
/*
* Run generate_partitionwise_join_paths() and
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index e1523d15df1..0700c634bf4 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -2481,7 +2481,7 @@ cost_merge_append(Path *path, PlannerInfo *root,
*/
void
cost_material(Path *path,
- int input_disabled_nodes,
+ bool enabled, int input_disabled_nodes,
Cost input_startup_cost, Cost input_total_cost,
double tuples, int width)
{
@@ -2519,7 +2519,7 @@ cost_material(Path *path,
run_cost += seq_page_cost * npages;
}
- path->disabled_nodes = input_disabled_nodes + (enable_material ? 0 : 1);
+ path->disabled_nodes = input_disabled_nodes + (enabled ? 0 : 1);
path->startup_cost = startup_cost;
path->total_cost = startup_cost + run_cost;
}
@@ -3266,7 +3266,7 @@ cost_group(Path *path, PlannerInfo *root,
*/
void
initial_cost_nestloop(PlannerInfo *root, JoinCostWorkspace *workspace,
- JoinType jointype,
+ JoinType jointype, unsigned nestloop_subtype,
Path *outer_path, Path *inner_path,
JoinPathExtraData *extra)
{
@@ -3280,7 +3280,7 @@ initial_cost_nestloop(PlannerInfo *root, JoinCostWorkspace *workspace,
Cost inner_rescan_run_cost;
/* Count up disabled nodes. */
- disabled_nodes = enable_nestloop ? 0 : 1;
+ disabled_nodes = (extra->jsa_mask & nestloop_subtype) == 0 ? 1 : 0;
disabled_nodes += inner_path->disabled_nodes;
disabled_nodes += outer_path->disabled_nodes;
@@ -3676,7 +3676,12 @@ initial_cost_mergejoin(PlannerInfo *root, JoinCostWorkspace *workspace,
Assert(outerstartsel <= outerendsel);
Assert(innerstartsel <= innerendsel);
- disabled_nodes = enable_mergejoin ? 0 : 1;
+ /*
+ * Assume for now that this node is not itself disabled. We'll sort out
+ * whether that's really the case in final_cost_mergejoin(); here, we'll
+ * just account for any disabled child nodes.
+ */
+ disabled_nodes = 0;
/* cost of source data */
@@ -3814,9 +3819,6 @@ final_cost_mergejoin(PlannerInfo *root, MergePath *path,
rescannedtuples;
double rescanratio;
- /* Set the number of disabled nodes. */
- path->jpath.path.disabled_nodes = workspace->disabled_nodes;
-
/* Protect some assumptions below that rowcounts aren't zero */
if (inner_path_rows <= 0)
inner_path_rows = 1;
@@ -3943,16 +3945,20 @@ final_cost_mergejoin(PlannerInfo *root, MergePath *path,
path->materialize_inner = false;
/*
- * Prefer materializing if it looks cheaper, unless the user has asked to
- * suppress materialization.
+ * If merge joins with materialization are enabled, then choose
+ * materialization if either (a) it looks cheaper or (b) merge joins
+ * without materialization are disabled.
*/
- else if (enable_material && mat_inner_cost < bare_inner_cost)
+ else if ((extra->jsa_mask & JSA_MERGEJOIN_MATERIALIZE) != 0 &&
+ (mat_inner_cost < bare_inner_cost ||
+ (extra->jsa_mask & JSA_MERGEJOIN_PLAIN) == 0))
path->materialize_inner = true;
/*
- * Even if materializing doesn't look cheaper, we *must* do it if the
- * inner path is to be used directly (without sorting) and it doesn't
- * support mark/restore.
+ * Regardless of what plan shapes are enabled and what the costs seem
+ * to be, we *must* materialize it if the inner path is to be used directly
+ * (without sorting) and it doesn't support mark/restore. Planner failure
+ * is not an option!
*
* Since the inner side must be ordered, and only Sorts and IndexScans can
* create order to begin with, and they both support mark/restore, you
@@ -3960,10 +3966,6 @@ final_cost_mergejoin(PlannerInfo *root, MergePath *path,
* merge joins can *preserve* the order of their inputs, so they can be
* selected as the input of a mergejoin, and they don't support
* mark/restore at present.
- *
- * We don't test the value of enable_material here, because
- * materialization is required for correctness in this case, and turning
- * it off does not entitle us to deliver an invalid plan.
*/
else if (innersortkeys == NIL &&
!ExecSupportsMarkRestore(inner_path))
@@ -3980,7 +3982,8 @@ final_cost_mergejoin(PlannerInfo *root, MergePath *path,
* rather than necessary for correctness, we skip it if enable_material is
* off.
*/
- else if (enable_material && innersortkeys != NIL &&
+ else if ((extra->jsa_mask & JSA_MERGEJOIN_MATERIALIZE) != 0 &&
+ innersortkeys != NIL &&
relation_byte_size(inner_path_rows,
inner_path->pathtarget->width) >
(work_mem * 1024L))
@@ -3988,11 +3991,25 @@ final_cost_mergejoin(PlannerInfo *root, MergePath *path,
else
path->materialize_inner = false;
- /* Charge the right incremental cost for the chosen case */
+ /* Get the number of disabled nodes, not yet including this one. */
+ path->jpath.path.disabled_nodes = workspace->disabled_nodes;
+
+ /*
+ * Charge the right incremental cost for the chosen case, and increment
+ * disabled_nodes if appropriate.
+ */
if (path->materialize_inner)
+ {
run_cost += mat_inner_cost;
+ if ((extra->jsa_mask & JSA_MERGEJOIN_MATERIALIZE) == 0)
+ ++path->jpath.path.disabled_nodes;
+ }
else
+ {
run_cost += bare_inner_cost;
+ if ((extra->jsa_mask & JSA_MERGEJOIN_PLAIN) == 0)
+ ++path->jpath.path.disabled_nodes;
+ }
/* CPU costs */
@@ -4132,7 +4149,7 @@ initial_cost_hashjoin(PlannerInfo *root, JoinCostWorkspace *workspace,
size_t space_allowed; /* unused */
/* Count up disabled nodes. */
- disabled_nodes = enable_hashjoin ? 0 : 1;
+ disabled_nodes = (extra->jsa_mask & JSA_HASHJOIN) == 0 ? 1 : 0;
disabled_nodes += inner_path->disabled_nodes;
disabled_nodes += outer_path->disabled_nodes;
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index a244300463c..90f3829d04d 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -28,8 +28,9 @@
#include "utils/lsyscache.h"
#include "utils/typcache.h"
-/* Hook for plugins to get control in add_paths_to_joinrel() */
+/* Hooks for plugins to get control in add_paths_to_joinrel() */
set_join_pathlist_hook_type set_join_pathlist_hook = NULL;
+join_path_setup_hook_type join_path_setup_hook = NULL;
/*
* Paths parameterized by a parent rel can be considered to be parameterized
@@ -129,7 +130,8 @@ add_paths_to_joinrel(PlannerInfo *root,
RelOptInfo *innerrel,
JoinType jointype,
SpecialJoinInfo *sjinfo,
- List *restrictlist)
+ List *restrictlist,
+ unsigned jsa_mask)
{
JoinPathExtraData extra;
bool mergejoin_allowed = true;
@@ -152,6 +154,7 @@ add_paths_to_joinrel(PlannerInfo *root,
extra.mergeclause_list = NIL;
extra.sjinfo = sjinfo;
extra.param_source_rels = NULL;
+ extra.jsa_mask = jsa_mask;
/*
* See if the inner relation is provably unique for this outer rel.
@@ -203,13 +206,39 @@ add_paths_to_joinrel(PlannerInfo *root,
break;
}
+ /*
+ * Give extensions a chance to take control. In particular, an extension
+ * might want to modify extra.jsa_mask so as to provide join strategy
+ * advice. An extension can also override jsa_mask on a query-wide basis
+ * by using join_search_hook, but extensions that want to provide
+ * different advice when joining different rels, or even different advice
+ * for the same joinrel based on the choice of innerrel and outerrel, need
+ * to use this hook.
+ *
+ * A very simple way for an extension to use this hook is to set
+ * extra.jsa_mask = 0, if it simply doesn't want any of the paths
+ * generated by this call to add_paths_to_joinrel() to be selected. An
+ * extension could use this technique to constrain the join order, since
+ * it could thereby arrange to reject all paths from join orders that it
+ * does not like. An extension can also selectively clear bits from
+ * extra.jsa_mask to rule out specific techniques for specific joins, or
+ * even replace the mask entirely.
+ *
+ * NB: Below this point, this function should be careful to reference only
+ * extra.jsa_mask, and not jsa_mask directly, to avoid disregarding any
+ * changes made by the hook we're about to call.
+ */
+ if (join_path_setup_hook)
+ join_path_setup_hook(root, joinrel, outerrel, innerrel,
+ jointype, &extra);
+
/*
* Find potential mergejoin clauses. We can skip this if we are not
* interested in doing a mergejoin. However, mergejoin may be our only
- * way of implementing a full outer join, so override enable_mergejoin if
- * it's a full join.
+ * way of implementing a full outer join, so in that case we don't care
+ * whether mergejoins are disabled.
*/
- if (enable_mergejoin || jointype == JOIN_FULL)
+ if ((extra.jsa_mask & JSA_MERGEJOIN_ANY) != 0 || jointype == JOIN_FULL)
extra.mergeclause_list = select_mergejoin_clauses(root,
joinrel,
outerrel,
@@ -317,10 +346,10 @@ add_paths_to_joinrel(PlannerInfo *root,
/*
* 4. Consider paths where both outer and inner relations must be hashed
- * before being joined. As above, disregard enable_hashjoin for full
- * joins, because there may be no other alternative.
+ * before being joined. As above, when it's a full join, we must try this
+ * even when the path type is disabled, because it may be our only option.
*/
- if (enable_hashjoin || jointype == JOIN_FULL)
+ if ((extra.jsa_mask & JSA_HASHJOIN) != 0 || jointype == JOIN_FULL)
hash_inner_and_outer(root, joinrel, outerrel, innerrel,
jointype, &extra);
@@ -329,7 +358,7 @@ add_paths_to_joinrel(PlannerInfo *root,
* to the same server and assigned to the same user to check access
* permissions as, give the FDW a chance to push down joins.
*/
- if (joinrel->fdwroutine &&
+ if ((extra.jsa_mask & JSA_FOREIGN) != 0 && joinrel->fdwroutine &&
joinrel->fdwroutine->GetForeignJoinPaths)
joinrel->fdwroutine->GetForeignJoinPaths(root, joinrel,
outerrel, innerrel,
@@ -338,8 +367,13 @@ add_paths_to_joinrel(PlannerInfo *root,
/*
* 6. Finally, give extensions a chance to manipulate the path list. They
* could add new paths (such as CustomPaths) by calling add_path(), or
- * add_partial_path() if parallel aware. They could also delete or modify
- * paths added by the core code.
+ * add_partial_path() if parallel aware.
+ *
+ * In theory, extensions could also use this hook to delete or modify
+ * paths added by the core code, but in practice this is difficult to make
+ * work, since it's too late to get back any paths that have already been
+ * discarded by add_path() or add_partial_path(). If you're trying to
+ * suppress paths, consider using join_path_setup_hook instead.
*/
if (set_join_pathlist_hook)
set_join_pathlist_hook(root, joinrel, outerrel, innerrel,
@@ -829,6 +863,7 @@ try_nestloop_path(PlannerInfo *root,
Path *inner_path,
List *pathkeys,
JoinType jointype,
+ unsigned nestloop_subtype,
JoinPathExtraData *extra)
{
Relids required_outer;
@@ -913,7 +948,7 @@ try_nestloop_path(PlannerInfo *root,
* The latter two steps are expensive enough to make this two-phase
* methodology worthwhile.
*/
- initial_cost_nestloop(root, &workspace, jointype,
+ initial_cost_nestloop(root, &workspace, jointype, nestloop_subtype,
outer_path, inner_path, extra);
if (add_path_precheck(joinrel, workspace.disabled_nodes,
@@ -951,6 +986,7 @@ try_partial_nestloop_path(PlannerInfo *root,
Path *inner_path,
List *pathkeys,
JoinType jointype,
+ unsigned nestloop_subtype,
JoinPathExtraData *extra)
{
JoinCostWorkspace workspace;
@@ -998,7 +1034,7 @@ try_partial_nestloop_path(PlannerInfo *root,
* Before creating a path, get a quick lower bound on what it is likely to
* cost. Bail out right away if it looks terrible.
*/
- initial_cost_nestloop(root, &workspace, jointype,
+ initial_cost_nestloop(root, &workspace, jointype, nestloop_subtype,
outer_path, inner_path, extra);
if (!add_partial_path_precheck(joinrel, workspace.disabled_nodes,
workspace.total_cost, pathkeys))
@@ -1893,20 +1929,21 @@ match_unsorted_outer(PlannerInfo *root,
if (inner_cheapest_total == NULL)
return;
inner_cheapest_total = (Path *)
- create_unique_path(root, innerrel, inner_cheapest_total, extra->sjinfo);
+ create_unique_path(root, innerrel, inner_cheapest_total,
+ extra->sjinfo);
Assert(inner_cheapest_total);
}
else if (nestjoinOK)
{
/*
- * Consider materializing the cheapest inner path, unless
- * enable_material is off or the path in question materializes its
- * output anyway.
+ * Consider materializing the cheapest inner path, unless that is
+ * disabled or the path in question materializes its output anyway.
*/
- if (enable_material && inner_cheapest_total != NULL &&
+ if ((extra->jsa_mask & JSA_NESTLOOP_MATERIALIZE) != 0 &&
+ inner_cheapest_total != NULL &&
!ExecMaterializesOutput(inner_cheapest_total->pathtype))
matpath = (Path *)
- create_material_path(innerrel, inner_cheapest_total);
+ create_material_path(innerrel, inner_cheapest_total, true);
}
foreach(lc1, outerrel->pathlist)
@@ -1954,6 +1991,7 @@ match_unsorted_outer(PlannerInfo *root,
inner_cheapest_total,
merge_pathkeys,
jointype,
+ JSA_NESTLOOP_PLAIN,
extra);
}
else if (nestjoinOK)
@@ -1977,6 +2015,7 @@ match_unsorted_outer(PlannerInfo *root,
innerpath,
merge_pathkeys,
jointype,
+ JSA_NESTLOOP_PLAIN,
extra);
/*
@@ -1993,6 +2032,7 @@ match_unsorted_outer(PlannerInfo *root,
mpath,
merge_pathkeys,
jointype,
+ JSA_NESTLOOP_MEMOIZE,
extra);
}
@@ -2004,6 +2044,7 @@ match_unsorted_outer(PlannerInfo *root,
matpath,
merge_pathkeys,
jointype,
+ JSA_NESTLOOP_MATERIALIZE,
extra);
}
@@ -2135,20 +2176,18 @@ consider_parallel_nestloop(PlannerInfo *root,
/*
* Consider materializing the cheapest inner path, unless: 1) we're doing
* JOIN_UNIQUE_INNER, because in this case we have to unique-ify the
- * cheapest inner path, 2) enable_material is off, 3) the cheapest inner
- * path is not parallel-safe, 4) the cheapest inner path is parameterized
- * by the outer rel, or 5) the cheapest inner path materializes its output
- * anyway.
+ * cheapest inner path, 2) materialization is disabled here, 3) the
+ * cheapest inner path is not parallel-safe, 4) the cheapest inner path is
+ * parameterized by the outer rel, or 5) the cheapest inner path
+ * materializes its output anyway.
*/
if (save_jointype != JOIN_UNIQUE_INNER &&
- enable_material && inner_cheapest_total->parallel_safe &&
+ (extra->jsa_mask & JSA_NESTLOOP_MATERIALIZE) != 0 &&
+ inner_cheapest_total->parallel_safe &&
!PATH_PARAM_BY_REL(inner_cheapest_total, outerrel) &&
!ExecMaterializesOutput(inner_cheapest_total->pathtype))
- {
matpath = (Path *)
- create_material_path(innerrel, inner_cheapest_total);
- Assert(matpath->parallel_safe);
- }
+ create_material_path(innerrel, inner_cheapest_total, true);
foreach(lc1, outerrel->partial_pathlist)
{
@@ -2193,7 +2232,8 @@ consider_parallel_nestloop(PlannerInfo *root,
}
try_partial_nestloop_path(root, joinrel, outerpath, innerpath,
- pathkeys, jointype, extra);
+ pathkeys, jointype, JSA_NESTLOOP_PLAIN,
+ extra);
/*
* Try generating a memoize path and see if that makes the nested
@@ -2204,13 +2244,15 @@ consider_parallel_nestloop(PlannerInfo *root,
extra);
if (mpath != NULL)
try_partial_nestloop_path(root, joinrel, outerpath, mpath,
- pathkeys, jointype, extra);
+ pathkeys, jointype,
+ JSA_NESTLOOP_MEMOIZE, extra);
}
/* Also consider materialized form of the cheapest inner path */
if (matpath != NULL)
try_partial_nestloop_path(root, joinrel, outerpath, matpath,
- pathkeys, jointype, extra);
+ pathkeys, jointype,
+ JSA_NESTLOOP_MATERIALIZE, extra);
}
}
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 7db5e30eef8..dad5c555365 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -26,10 +26,12 @@
static void make_rels_by_clause_joins(PlannerInfo *root,
RelOptInfo *old_rel,
List *other_rels,
- int first_rel_idx);
+ int first_rel_idx,
+ unsigned jsa_mask);
static void make_rels_by_clauseless_joins(PlannerInfo *root,
RelOptInfo *old_rel,
- List *other_rels);
+ List *other_rels,
+ unsigned jsa_mask);
static bool has_join_restriction(PlannerInfo *root, RelOptInfo *rel);
static bool has_legal_joinclause(PlannerInfo *root, RelOptInfo *rel);
static bool restriction_is_constant_false(List *restrictlist,
@@ -37,11 +39,13 @@ static bool restriction_is_constant_false(List *restrictlist,
bool only_pushed_down);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
- SpecialJoinInfo *sjinfo, List *restrictlist);
+ SpecialJoinInfo *sjinfo,
+ List *restrictlist, unsigned jsa_mask);
static void try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *parent_sjinfo,
- List *parent_restrictlist);
+ List *parent_restrictlist,
+ unsigned jsa_mask);
static SpecialJoinInfo *build_child_join_sjinfo(PlannerInfo *root,
SpecialJoinInfo *parent_sjinfo,
Relids left_relids, Relids right_relids);
@@ -69,7 +73,7 @@ static void get_matching_part_pairs(PlannerInfo *root, RelOptInfo *joinrel,
* The result is returned in root->join_rel_level[level].
*/
void
-join_search_one_level(PlannerInfo *root, int level)
+join_search_one_level(PlannerInfo *root, int level, unsigned jsa_mask)
{
List **joinrels = root->join_rel_level;
ListCell *r;
@@ -114,7 +118,8 @@ join_search_one_level(PlannerInfo *root, int level)
else
first_rel = 0;
- make_rels_by_clause_joins(root, old_rel, joinrels[1], first_rel);
+ make_rels_by_clause_joins(root, old_rel, joinrels[1], first_rel,
+ jsa_mask);
}
else
{
@@ -132,7 +137,8 @@ join_search_one_level(PlannerInfo *root, int level)
*/
make_rels_by_clauseless_joins(root,
old_rel,
- joinrels[1]);
+ joinrels[1],
+ jsa_mask);
}
}
@@ -189,7 +195,7 @@ join_search_one_level(PlannerInfo *root, int level)
if (have_relevant_joinclause(root, old_rel, new_rel) ||
have_join_order_restriction(root, old_rel, new_rel))
{
- (void) make_join_rel(root, old_rel, new_rel);
+ (void) make_join_rel(root, old_rel, new_rel, jsa_mask);
}
}
}
@@ -227,7 +233,8 @@ join_search_one_level(PlannerInfo *root, int level)
make_rels_by_clauseless_joins(root,
old_rel,
- joinrels[1]);
+ joinrels[1],
+ jsa_mask);
}
/*----------
@@ -279,7 +286,8 @@ static void
make_rels_by_clause_joins(PlannerInfo *root,
RelOptInfo *old_rel,
List *other_rels,
- int first_rel_idx)
+ int first_rel_idx,
+ unsigned jsa_mask)
{
ListCell *l;
@@ -291,7 +299,7 @@ make_rels_by_clause_joins(PlannerInfo *root,
(have_relevant_joinclause(root, old_rel, other_rel) ||
have_join_order_restriction(root, old_rel, other_rel)))
{
- (void) make_join_rel(root, old_rel, other_rel);
+ (void) make_join_rel(root, old_rel, other_rel, jsa_mask);
}
}
}
@@ -312,7 +320,8 @@ make_rels_by_clause_joins(PlannerInfo *root,
static void
make_rels_by_clauseless_joins(PlannerInfo *root,
RelOptInfo *old_rel,
- List *other_rels)
+ List *other_rels,
+ unsigned jsa_mask)
{
ListCell *l;
@@ -322,7 +331,7 @@ make_rels_by_clauseless_joins(PlannerInfo *root,
if (!bms_overlap(other_rel->relids, old_rel->relids))
{
- (void) make_join_rel(root, old_rel, other_rel);
+ (void) make_join_rel(root, old_rel, other_rel, jsa_mask);
}
}
}
@@ -701,7 +710,8 @@ init_dummy_sjinfo(SpecialJoinInfo *sjinfo, Relids left_relids,
* turned into joins.
*/
RelOptInfo *
-make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
+ unsigned jsa_mask)
{
Relids joinrelids;
SpecialJoinInfo *sjinfo;
@@ -759,7 +769,7 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
*/
joinrel = build_join_rel(root, joinrelids, rel1, rel2,
sjinfo, pushed_down_joins,
- &restrictlist);
+ &restrictlist, jsa_mask);
/*
* If we've already proven this join is empty, we needn't consider any
@@ -773,7 +783,7 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
/* Add paths to the join relation. */
populate_joinrel_with_paths(root, rel1, rel2, joinrel, sjinfo,
- restrictlist);
+ restrictlist, jsa_mask);
bms_free(joinrelids);
@@ -892,7 +902,8 @@ add_outer_joins_to_relids(PlannerInfo *root, Relids input_relids,
static void
populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
- SpecialJoinInfo *sjinfo, List *restrictlist)
+ SpecialJoinInfo *sjinfo, List *restrictlist,
+ unsigned jsa_mask)
{
/*
* Consider paths using each rel as both outer and inner. Depending on
@@ -923,10 +934,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_INNER, sjinfo,
- restrictlist);
+ restrictlist, jsa_mask);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_INNER, sjinfo,
- restrictlist);
+ restrictlist, jsa_mask);
break;
case JOIN_LEFT:
if (is_dummy_rel(rel1) ||
@@ -940,10 +951,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_LEFT, sjinfo,
- restrictlist);
+ restrictlist, jsa_mask);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_RIGHT, sjinfo,
- restrictlist);
+ restrictlist, jsa_mask);
break;
case JOIN_FULL:
if ((is_dummy_rel(rel1) && is_dummy_rel(rel2)) ||
@@ -954,10 +965,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_FULL, sjinfo,
- restrictlist);
+ restrictlist, jsa_mask);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_FULL, sjinfo,
- restrictlist);
+ restrictlist, jsa_mask);
/*
* If there are join quals that aren't mergeable or hashable, we
@@ -990,10 +1001,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_SEMI, sjinfo,
- restrictlist);
+ restrictlist, jsa_mask);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_RIGHT_SEMI, sjinfo,
- restrictlist);
+ restrictlist, jsa_mask);
}
/*
@@ -1016,10 +1027,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_UNIQUE_INNER, sjinfo,
- restrictlist);
+ restrictlist, jsa_mask);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_UNIQUE_OUTER, sjinfo,
- restrictlist);
+ restrictlist, jsa_mask);
}
break;
case JOIN_ANTI:
@@ -1034,10 +1045,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_ANTI, sjinfo,
- restrictlist);
+ restrictlist, jsa_mask);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_RIGHT_ANTI, sjinfo,
- restrictlist);
+ restrictlist, jsa_mask);
break;
default:
/* other values not expected here */
@@ -1046,7 +1057,8 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
}
/* Apply partitionwise join technique, if possible. */
- try_partitionwise_join(root, rel1, rel2, joinrel, sjinfo, restrictlist);
+ try_partitionwise_join(root, rel1, rel2, joinrel, sjinfo, restrictlist,
+ jsa_mask);
}
@@ -1480,7 +1492,7 @@ restriction_is_constant_false(List *restrictlist,
static void
try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
RelOptInfo *joinrel, SpecialJoinInfo *parent_sjinfo,
- List *parent_restrictlist)
+ List *parent_restrictlist, unsigned jsa_mask)
{
bool rel1_is_simple = IS_SIMPLE_REL(rel1);
bool rel2_is_simple = IS_SIMPLE_REL(rel2);
@@ -1662,7 +1674,8 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
{
child_joinrel = build_child_join_rel(root, child_rel1, child_rel2,
joinrel, child_restrictlist,
- child_sjinfo, nappinfos, appinfos);
+ child_sjinfo, nappinfos,
+ appinfos, jsa_mask);
joinrel->part_rels[cnt_parts] = child_joinrel;
joinrel->live_parts = bms_add_member(joinrel->live_parts, cnt_parts);
joinrel->all_partrels = bms_add_members(joinrel->all_partrels,
@@ -1677,7 +1690,7 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
/* And make paths for the child join */
populate_joinrel_with_paths(root, child_rel1, child_rel2,
child_joinrel, child_sjinfo,
- child_restrictlist);
+ child_restrictlist, jsa_mask);
/*
* When there are thousands of partitions involved, this loop will
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index bb45ef318fb..14fd480fa89 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -6550,6 +6550,7 @@ materialize_finished_plan(Plan *subplan)
/* Set cost data */
cost_material(&matpath,
+ enable_material,
subplan->disabled_nodes,
subplan->startup_cost,
subplan->total_cost,
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index fc97bf6ee26..a704d0cd7bb 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -1631,7 +1631,7 @@ create_group_result_path(PlannerInfo *root, RelOptInfo *rel,
* pathnode.
*/
MaterialPath *
-create_material_path(RelOptInfo *rel, Path *subpath)
+create_material_path(RelOptInfo *rel, Path *subpath, bool enabled)
{
MaterialPath *pathnode = makeNode(MaterialPath);
@@ -1650,6 +1650,7 @@ create_material_path(RelOptInfo *rel, Path *subpath)
pathnode->subpath = subpath;
cost_material(&pathnode->path,
+ enabled,
subpath->disabled_nodes,
subpath->startup_cost,
subpath->total_cost,
@@ -4158,7 +4159,8 @@ reparameterize_path(PlannerInfo *root, Path *path,
loop_count);
if (spath == NULL)
return NULL;
- return (Path *) create_material_path(rel, spath);
+ return (Path *) create_material_path(rel, spath,
+ enable_material);
}
case T_Memoize:
{
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index d7266e4cdba..9e328c5ac7c 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -69,7 +69,8 @@ static void build_joinrel_partition_info(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
- List *restrictlist);
+ List *restrictlist,
+ unsigned jsa_mask);
static bool have_partkey_equi_join(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *rel1, RelOptInfo *rel2,
JoinType jointype, List *restrictlist);
@@ -668,7 +669,8 @@ build_join_rel(PlannerInfo *root,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
List *pushed_down_joins,
- List **restrictlist_ptr)
+ List **restrictlist_ptr,
+ unsigned jsa_mask)
{
RelOptInfo *joinrel;
List *restrictlist;
@@ -817,7 +819,7 @@ build_join_rel(PlannerInfo *root,
/* Store the partition information. */
build_joinrel_partition_info(root, joinrel, outer_rel, inner_rel, sjinfo,
- restrictlist);
+ restrictlist, jsa_mask);
/*
* Set estimates of the joinrel's size.
@@ -882,7 +884,8 @@ RelOptInfo *
build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
RelOptInfo *inner_rel, RelOptInfo *parent_joinrel,
List *restrictlist, SpecialJoinInfo *sjinfo,
- int nappinfos, AppendRelInfo **appinfos)
+ int nappinfos, AppendRelInfo **appinfos,
+ unsigned jsa_mask)
{
RelOptInfo *joinrel = makeNode(RelOptInfo);
@@ -981,7 +984,7 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
/* Is the join between partitions itself partitioned? */
build_joinrel_partition_info(root, joinrel, outer_rel, inner_rel, sjinfo,
- restrictlist);
+ restrictlist, jsa_mask);
/* Child joinrel is parallel safe if parent is parallel safe. */
joinrel->consider_parallel = parent_joinrel->consider_parallel;
@@ -2005,12 +2008,12 @@ static void
build_joinrel_partition_info(PlannerInfo *root,
RelOptInfo *joinrel, RelOptInfo *outer_rel,
RelOptInfo *inner_rel, SpecialJoinInfo *sjinfo,
- List *restrictlist)
+ List *restrictlist, unsigned jsa_mask)
{
PartitionScheme part_scheme;
/* Nothing to do if partitionwise join technique is disabled. */
- if (!enable_partitionwise_join)
+ if ((jsa_mask & JSA_PARTITIONWISE) == 0)
{
Assert(!IS_PARTITIONED_REL(joinrel));
return;
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 07e2415398e..4470c7817da 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -3231,6 +3231,7 @@ typedef struct SemiAntiJoinFactors
* sjinfo is extra info about special joins for selectivity estimation
* semifactors is as shown above (only valid for SEMI/ANTI/inner_unique joins)
* param_source_rels are OK targets for parameterization of result paths
+ * jsa_mask is a bitmask of JSA_* constants to direct the join strategy
*/
typedef struct JoinPathExtraData
{
@@ -3240,6 +3241,7 @@ typedef struct JoinPathExtraData
SpecialJoinInfo *sjinfo;
SemiAntiJoinFactors semifactors;
Relids param_source_rels;
+ unsigned jsa_mask;
} JoinPathExtraData;
/*
diff --git a/src/include/optimizer/cost.h b/src/include/optimizer/cost.h
index 854a782944a..071a8749cfa 100644
--- a/src/include/optimizer/cost.h
+++ b/src/include/optimizer/cost.h
@@ -125,7 +125,7 @@ extern void cost_merge_append(Path *path, PlannerInfo *root,
Cost input_startup_cost, Cost input_total_cost,
double tuples);
extern void cost_material(Path *path,
- int input_disabled_nodes,
+ bool enabled, int input_disabled_nodes,
Cost input_startup_cost, Cost input_total_cost,
double tuples, int width);
extern void cost_agg(Path *path, PlannerInfo *root,
@@ -148,7 +148,7 @@ extern void cost_group(Path *path, PlannerInfo *root,
double input_tuples);
extern void initial_cost_nestloop(PlannerInfo *root,
JoinCostWorkspace *workspace,
- JoinType jointype,
+ JoinType jointype, unsigned nestloop_subtype,
Path *outer_path, Path *inner_path,
JoinPathExtraData *extra);
extern void final_cost_nestloop(PlannerInfo *root, NestPath *path,
diff --git a/src/include/optimizer/geqo.h b/src/include/optimizer/geqo.h
index c52906d0916..a9d14b07aa1 100644
--- a/src/include/optimizer/geqo.h
+++ b/src/include/optimizer/geqo.h
@@ -81,10 +81,13 @@ typedef struct
/* routines in geqo_main.c */
extern RelOptInfo *geqo(PlannerInfo *root,
- int number_of_rels, List *initial_rels);
+ int number_of_rels, List *initial_rels,
+ unsigned jsa_mask);
/* routines in geqo_eval.c */
-extern Cost geqo_eval(PlannerInfo *root, Gene *tour, int num_gene);
-extern RelOptInfo *gimme_tree(PlannerInfo *root, Gene *tour, int num_gene);
+extern Cost geqo_eval(PlannerInfo *root, Gene *tour, int num_gene,
+ unsigned jsa_mask);
+extern RelOptInfo *gimme_tree(PlannerInfo *root, Gene *tour, int num_gene,
+ unsigned jsa_mask);
#endif /* GEQO_H */
diff --git a/src/include/optimizer/geqo_pool.h b/src/include/optimizer/geqo_pool.h
index b5e80554724..bd1ed152907 100644
--- a/src/include/optimizer/geqo_pool.h
+++ b/src/include/optimizer/geqo_pool.h
@@ -29,7 +29,7 @@
extern Pool *alloc_pool(PlannerInfo *root, int pool_size, int string_length);
extern void free_pool(PlannerInfo *root, Pool *pool);
-extern void random_init_pool(PlannerInfo *root, Pool *pool);
+extern void random_init_pool(PlannerInfo *root, Pool *pool, unsigned jsa_mask);
extern Chromosome *alloc_chromo(PlannerInfo *root, int string_length);
extern void free_chromo(PlannerInfo *root, Chromosome *chromo);
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 1035e6560c1..ed2e3a1c8b8 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -82,7 +82,8 @@ extern GroupResultPath *create_group_result_path(PlannerInfo *root,
RelOptInfo *rel,
PathTarget *target,
List *havingqual);
-extern MaterialPath *create_material_path(RelOptInfo *rel, Path *subpath);
+extern MaterialPath *create_material_path(RelOptInfo *rel, Path *subpath,
+ bool enabled);
extern MemoizePath *create_memoize_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
@@ -324,7 +325,8 @@ extern RelOptInfo *build_join_rel(PlannerInfo *root,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
List *pushed_down_joins,
- List **restrictlist_ptr);
+ List **restrictlist_ptr,
+ unsigned jsa_mask);
extern Relids min_join_parameterization(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
@@ -351,6 +353,7 @@ extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
SpecialJoinInfo *sjinfo,
- int nappinfos, AppendRelInfo **appinfos);
+ int nappinfos, AppendRelInfo **appinfos,
+ unsigned jsa_mask);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
index a78e90610fc..9ce925dc292 100644
--- a/src/include/optimizer/paths.h
+++ b/src/include/optimizer/paths.h
@@ -16,10 +16,44 @@
#include "nodes/pathnodes.h"
+/*
+ * Join strategy advice.
+ *
+ * Paths that don't match the join strategy advice will be either be disabled
+ * or will not be generated in the first place. It's only permissible to skip
+ * generating a path if doing so can't result in planner failure. The initial
+ * mask is computed on the basis of the various enable_* GUCs, and can be
+ * overriden by hooks.
+ *
+ * We have five main join strategies: a foreign join (when supported by the
+ * relevant FDW), a merge join, a nested loop, a hash join, and a partitionwise
+ * join. Merge joins are further subdivided based on whether the inner side
+ * is materialized, and nested loops are further subdivided based on whether
+ * the inner side is materialized, memoized, or neither. "Plain" means a
+ * strategy where neither materialization nor memoization is used.
+ *
+ * If you don't care whether materialization or memoization is used, set all
+ * the bits for the relevant major join strategy. If you do care, just set the
+ * subset of bits that correspond to the cases you want to allow.
+ */
+#define JSA_FOREIGN 0x0001
+#define JSA_MERGEJOIN_PLAIN 0x0002
+#define JSA_MERGEJOIN_MATERIALIZE 0x0004
+#define JSA_NESTLOOP_PLAIN 0x0008
+#define JSA_NESTLOOP_MATERIALIZE 0x0010
+#define JSA_NESTLOOP_MEMOIZE 0x0020
+#define JSA_HASHJOIN 0x0040
+#define JSA_PARTITIONWISE 0x0080
+
+#define JSA_MERGEJOIN_ANY \
+ (JSA_MERGEJOIN_PLAIN | JSA_MERGEJOIN_MATERIALIZE)
+#define JSA_NESTLOOP_ANY \
+ (JSA_NESTLOOP_PLAIN | JSA_NESTLOOP_MATERIALIZE | JSA_NESTLOOP_MEMOIZE)
/*
* allpaths.c
*/
+
extern PGDLLIMPORT bool enable_geqo;
extern PGDLLIMPORT int geqo_threshold;
extern PGDLLIMPORT int min_parallel_table_scan_size;
@@ -33,7 +67,14 @@ typedef void (*set_rel_pathlist_hook_type) (PlannerInfo *root,
RangeTblEntry *rte);
extern PGDLLIMPORT set_rel_pathlist_hook_type set_rel_pathlist_hook;
-/* Hook for plugins to get control in add_paths_to_joinrel() */
+/* Hooks for plugins to get control in add_paths_to_joinrel() */
+typedef void (*join_path_setup_hook_type) (PlannerInfo *root,
+ RelOptInfo *joinrel,
+ RelOptInfo *outerrel,
+ RelOptInfo *innerrel,
+ JoinType jointype,
+ JoinPathExtraData *extra);
+extern PGDLLIMPORT join_path_setup_hook_type join_path_setup_hook;
typedef void (*set_join_pathlist_hook_type) (PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
@@ -45,13 +86,14 @@ extern PGDLLIMPORT set_join_pathlist_hook_type set_join_pathlist_hook;
/* Hook for plugins to replace standard_join_search() */
typedef RelOptInfo *(*join_search_hook_type) (PlannerInfo *root,
int levels_needed,
- List *initial_rels);
+ List *initial_rels,
+ unsigned jsa_mask);
extern PGDLLIMPORT join_search_hook_type join_search_hook;
extern RelOptInfo *make_one_rel(PlannerInfo *root, List *joinlist);
extern RelOptInfo *standard_join_search(PlannerInfo *root, int levels_needed,
- List *initial_rels);
+ List *initial_rels, unsigned jsa_mask);
extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel,
bool override_rows);
@@ -92,15 +134,17 @@ extern bool create_tidscan_paths(PlannerInfo *root, RelOptInfo *rel);
extern void add_paths_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, SpecialJoinInfo *sjinfo,
- List *restrictlist);
+ List *restrictlist, unsigned jsa_mask);
/*
* joinrels.c
* routines to determine which relations to join
*/
-extern void join_search_one_level(PlannerInfo *root, int level);
+extern void join_search_one_level(PlannerInfo *root, int level,
+ unsigned jsa_mask);
extern RelOptInfo *make_join_rel(PlannerInfo *root,
- RelOptInfo *rel1, RelOptInfo *rel2);
+ RelOptInfo *rel1, RelOptInfo *rel2,
+ unsigned jsa_mask);
extern Relids add_outer_joins_to_relids(PlannerInfo *root, Relids input_relids,
SpecialJoinInfo *sjinfo,
List **pushed_down_joins);
--
2.39.3 (Apple Git-145)
On 18/9/2024 17:48, Robert Haas wrote:
Comments?
Let me share my personal experience on path management in the planner.
The main thing important for extensions is flexibility - I would discuss
a decision that is not limited by join ordering but could be applied to
implement an index picking strategy, Memoize/Material choice versus a
non-cached one, choice of custom paths, etc.
The most flexible way I have found to this moment is a collaboration
between the get_relation_info_hook and add_path hook. In
get_relation_info, we have enough context and can add some information
to RelOptInfo - I added an extra list to this structure where extensions
can add helpful info. Using extensible nodes, we can tackle interference
between extensions.
The add_path hook can analyse new and old paths and also look into the
extensible data inside RelOptInfo. The issue with lots of calls eases by
quick return on the out-of-scope paths: usually extensions manage some
specific path type or relation and quick test of RelOptInfo::extra_list
allow to sort out unnecessary cases.
Being flexible, this approach is less invasive. Now, I use it to
implement heuristics demanded by clients for cases when the estimator
predicts only one row - usually, it means that the optimiser
underestimates cardinality. For example, in-place switch-off of NestLoop
if it uses too many clauses, or rules to pick up index scan if we have
alternative scans, each of them predicts only one tuple.
Positive outcomes includes: we don't alter path costs; extension may be
sure that core doesn't remove path from the list if the extension
forbids it.
In attachment - hooks for add_path and add_partial_path. As you can see,
because of differences in these routines hooks also implemented
differently. Also the compare_path_costs_fuzzily is exported, but it is
really good stuff for an extension.
--
regards, Andrei Lepikhov
Attachments:
0001-Introduce-compare_path_hook.patchtext/plain; charset=UTF-8; name=0001-Introduce-compare_path_hook.patchDownload
From af8f5bd65e33c819723231ce433dcd51438b7ef0 Mon Sep 17 00:00:00 2001
From: "Andrei V. Lepikhov" <lepihov@gmail.com>
Date: Thu, 26 Sep 2024 10:34:08 +0200
Subject: [PATCH 1/2] Introduce compare_path_hook.
The add_path function is the only interface to safely add and remove paths in
the pathlist. Postgres core provides a few optimisation hooks that an extension
can use to offer additional paths. However, it is still uncertain whether such
a path will be replaced until the end of the path population process.
This hook allows the extension to control the comparison of a new path with
each pathlist's path and denote the optimiser which path is better.
It doesn't point to the optimiser directly about what exactly to do with the
incoming path and paths, already existed in pathlist.
Tag: optimizer.
caused by: PGPRO-10445, PGPRO-11017.
---
src/backend/optimizer/util/pathnode.c | 21 +++++++++++----------
src/include/optimizer/pathnode.h | 17 +++++++++++++++++
2 files changed, 28 insertions(+), 10 deletions(-)
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index fc97bf6ee2..ae57932862 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -31,13 +31,6 @@
#include "utils/memutils.h"
#include "utils/selfuncs.h"
-typedef enum
-{
- COSTS_EQUAL, /* path costs are fuzzily equal */
- COSTS_BETTER1, /* first path is cheaper than second */
- COSTS_BETTER2, /* second path is cheaper than first */
- COSTS_DIFFERENT, /* neither path dominates the other on cost */
-} PathCostComparison;
/*
* STD_FUZZ_FACTOR is the normal fuzz factor for compare_path_costs_fuzzily.
@@ -46,6 +39,9 @@ typedef enum
*/
#define STD_FUZZ_FACTOR 1.01
+/* Hook for plugins to get control over the add_path decision */
+compare_path_hook_type compare_path_hook = NULL;
+
static List *translate_sub_tlist(List *tlist, int relid);
static int append_total_cost_compare(const ListCell *a, const ListCell *b);
static int append_startup_cost_compare(const ListCell *a, const ListCell *b);
@@ -178,7 +174,7 @@ compare_fractional_path_costs(Path *path1, Path *path2,
* (But if total costs are fuzzily equal, we compare startup costs anyway,
* in hopes of eliminating one path or the other.)
*/
-static PathCostComparison
+PathCostComparison
compare_path_costs_fuzzily(Path *path1, Path *path2, double fuzz_factor)
{
#define CONSIDER_PATH_STARTUP_COST(p) \
@@ -490,8 +486,13 @@ add_path(RelOptInfo *parent_rel, Path *new_path)
/*
* Do a fuzzy cost comparison with standard fuzziness limit.
*/
- costcmp = compare_path_costs_fuzzily(new_path, old_path,
- STD_FUZZ_FACTOR);
+
+ if (compare_path_hook)
+ costcmp = (*compare_path_hook) (parent_rel, new_path, old_path,
+ STD_FUZZ_FACTOR);
+ else
+ costcmp = compare_path_costs_fuzzily(new_path, old_path,
+ STD_FUZZ_FACTOR);
/*
* If the two paths compare differently for startup and total cost,
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 1035e6560c..4b95d85e1a 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -18,11 +18,28 @@
#include "nodes/pathnodes.h"
+typedef enum
+{
+ COSTS_EQUAL, /* path costs are fuzzily equal */
+ COSTS_BETTER1, /* first path is cheaper than second */
+ COSTS_BETTER2, /* second path is cheaper than first */
+ COSTS_DIFFERENT, /* neither path dominates the other on cost */
+} PathCostComparison;
+
+/* Hook for plugins to get control when grouping_planner() plans upper rels */
+typedef PathCostComparison (*compare_path_hook_type) (RelOptInfo *parent_rel,
+ Path *new_path,
+ Path *old_path,
+ double fuzz_factor);
+extern PGDLLIMPORT compare_path_hook_type compare_path_hook;
+
/*
* prototypes for pathnode.c
*/
extern int compare_path_costs(Path *path1, Path *path2,
CostSelector criterion);
+extern PathCostComparison
+compare_path_costs_fuzzily(Path *path1, Path *path2, double fuzz_factor);
extern int compare_fractional_path_costs(Path *path1, Path *path2,
double fraction);
extern void set_cheapest(RelOptInfo *parent_rel);
--
2.46.2
0002-Introduce-compare_partial_path_hook.patchtext/plain; charset=UTF-8; name=0002-Introduce-compare_partial_path_hook.patchDownload
From f63c93a404554f7e48a01ed14661a119ae10b604 Mon Sep 17 00:00:00 2001
From: "Andrei V. Lepikhov" <lepihov@gmail.com>
Date: Mon, 30 Sep 2024 13:32:43 +0700
Subject: [PATCH 2/2] Introduce compare_partial_path_hook.
By analogy of compare_path_hook, let an extension to alter decisions on
accepting new and removing old path. The add_partial_path() routine has
different logic. Hence, the hook also differs from non-partial case.
Also, move hooks and newly exported functions to paths.h. Do we need to
revert it and allow to stay in the pathnode.h?
---
src/backend/optimizer/util/pathnode.c | 5 +++++
src/include/optimizer/pathnode.h | 17 -----------------
src/include/optimizer/paths.h | 24 ++++++++++++++++++++++++
3 files changed, 29 insertions(+), 17 deletions(-)
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index ae57932862..5f91ead226 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -41,6 +41,7 @@
/* Hook for plugins to get control over the add_path decision */
compare_path_hook_type compare_path_hook = NULL;
+compare_partial_path_hook_type compare_partial_path_hook = NULL;
static List *translate_sub_tlist(List *tlist, int relid);
static int append_total_cost_compare(const ListCell *a, const ListCell *b);
@@ -870,6 +871,10 @@ add_partial_path(RelOptInfo *parent_rel, Path *new_path)
}
}
+ if (compare_partial_path_hook)
+ (*compare_partial_path_hook)(parent_rel, new_path, old_path,
+ &accept_new, &remove_old);
+
/*
* Remove current element from partial_pathlist if dominated by new.
*/
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 4b95d85e1a..1035e6560c 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -18,28 +18,11 @@
#include "nodes/pathnodes.h"
-typedef enum
-{
- COSTS_EQUAL, /* path costs are fuzzily equal */
- COSTS_BETTER1, /* first path is cheaper than second */
- COSTS_BETTER2, /* second path is cheaper than first */
- COSTS_DIFFERENT, /* neither path dominates the other on cost */
-} PathCostComparison;
-
-/* Hook for plugins to get control when grouping_planner() plans upper rels */
-typedef PathCostComparison (*compare_path_hook_type) (RelOptInfo *parent_rel,
- Path *new_path,
- Path *old_path,
- double fuzz_factor);
-extern PGDLLIMPORT compare_path_hook_type compare_path_hook;
-
/*
* prototypes for pathnode.c
*/
extern int compare_path_costs(Path *path1, Path *path2,
CostSelector criterion);
-extern PathCostComparison
-compare_path_costs_fuzzily(Path *path1, Path *path2, double fuzz_factor);
extern int compare_fractional_path_costs(Path *path1, Path *path2,
double fraction);
extern void set_cheapest(RelOptInfo *parent_rel);
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
index 54869d4401..47ae26033c 100644
--- a/src/include/optimizer/paths.h
+++ b/src/include/optimizer/paths.h
@@ -271,4 +271,28 @@ extern PathKey *make_canonical_pathkey(PlannerInfo *root,
extern void add_paths_to_append_rel(PlannerInfo *root, RelOptInfo *rel,
List *live_childrels);
+
+typedef enum
+{
+ COSTS_EQUAL, /* path costs are fuzzily equal */
+ COSTS_BETTER1, /* first path is cheaper than second */
+ COSTS_BETTER2, /* second path is cheaper than first */
+ COSTS_DIFFERENT, /* neither path dominates the other on cost */
+} PathCostComparison;
+
+extern PathCostComparison compare_path_costs_fuzzily(Path *path1, Path *path2,
+ double fuzz_factor);
+/* Hook for plugins to get control when grouping_planner() plans upper rels */
+typedef PathCostComparison (*compare_path_hook_type) (RelOptInfo *rel,
+ Path *new_path,
+ Path *old_path,
+ double fuzz_factor);
+typedef void (*compare_partial_path_hook_type) (RelOptInfo *rel,
+ Path *new_path,
+ Path *old_path,
+ bool *accept_new,
+ bool *remove_old);
+extern PGDLLIMPORT compare_path_hook_type compare_path_hook;
+extern PGDLLIMPORT compare_partial_path_hook_type compare_partial_path_hook;
+
#endif /* PATHS_H */
--
2.46.2
On Mon, Sep 30, 2024 at 5:50 AM Andrei Lepikhov <lepihov@gmail.com> wrote:
Being flexible, this approach is less invasive. Now, I use it to
implement heuristics demanded by clients for cases when the estimator
predicts only one row - usually, it means that the optimiser
underestimates cardinality. For example, in-place switch-off of NestLoop
if it uses too many clauses, or rules to pick up index scan if we have
alternative scans, each of them predicts only one tuple.Positive outcomes includes: we don't alter path costs; extension may be
sure that core doesn't remove path from the list if the extension
forbids it.In attachment - hooks for add_path and add_partial_path. As you can see,
because of differences in these routines hooks also implemented
differently. Also the compare_path_costs_fuzzily is exported, but it is
really good stuff for an extension.
I agree that this is more flexible, but it also seems like it would be
a lot more expensive. For every add_path() or add_partial_path() call,
you'll have to examine the input path and decide what you want to do
with it. If you want to do something like avoid nested loops with
materialization, you'll need to first check the top-level node, and
then if it's a nested loop, you have to check the inner subpath to see
if it's a Materialize node.
I'm not completely against having something like this; I think there
are cases where something along these lines is the only way to achieve
some desired objective. But I don't think this kind of hook should be
the primary way for extensions to control the planner; it seems too
low-level to me.
--
Robert Haas
EDB: http://www.enterprisedb.com
On 10/4/24 01:35, Robert Haas wrote:
On Mon, Sep 30, 2024 at 5:50 AM Andrei Lepikhov <lepihov@gmail.com> wrote:
In attachment - hooks for add_path and add_partial_path. As you can see,
because of differences in these routines hooks also implemented
differently. Also the compare_path_costs_fuzzily is exported, but it is
really good stuff for an extension.I agree that this is more flexible, but it also seems like it would be
a lot more expensive. For every add_path() or add_partial_path() call,
you'll have to examine the input path and decide what you want to do
with it. If you want to do something like avoid nested loops with
materialization, you'll need to first check the top-level node, and
then if it's a nested loop, you have to check the inner subpath to see
if it's a Materialize node.
I agree, and as you can see, the first simple test on a NestLoop already
avoids further checks for lots of calls for baserels, upper rels, Hash-
and MergeJoins, relieving the issue.
I'm not completely against having something like this; I think there
are cases where something along these lines is the only way to achieve
some desired objective. But I don't think this kind of hook should be
the primary way for extensions to control the planner; it seems too
low-level to me.
It is a fact. Maybe I was unclear, but I usually use it as an addition
to planner_hook or pathlist hooks to have some guarantee or, at least,
have a chance to do something if another path is much better and is
going to displace my path. Sometimes it is just a way to remove a path
from the pathlist without playing games with costs of my path.
I spent some time discovering the pg_hint_plan extension to apprehend
when extensions need to make massive core code copying and IMO, Michael
has the most to say here.
--
regards, Andrei Lepikhov
On Wed, Sep 18, 2024 at 11:48 AM Robert Haas <robertmhaas@gmail.com> wrote:
Still, I think it's a pretty useful starting point. It is mostly
enough to give you control over join planning, and if combined with
similar work for scan planning, I think it would be enough for
pg_hint_plan. If we also got control over appendrel and agg planning,
then you could do a bunch more cool things.
Here's a new set of patches where I added a similar mechanism for scan
type control. See the commit message for some notes on limitations of
this approach. In the absence of better ideas, I'd like to proceed
with something along the lines of 0001 and 0002.
I upgraded the hint_via_alias contrib module (which is not intended
for commit, it's just a demo) so that it can hint either scan type or
join type. I think this is sufficient to demonstrate that it's quite
easy to use hooks to leverage this new infrastructure. In fact, the
biggest thing I'm unhappy about right now is the difficulty of
providing the hooks with any sort of useful information. I don't think
it should be the job of this patch set to solve that problem, but I do
think we should do something about it. The problem exists on two
levels:
1. If you want to specify in-query hints using comments, how does your
extension get access to the comments? I realize that a lot of people
hate in-query hints and hope they die in a fire, but pg_hint_plan does
implement them, among other things, and the way it does that is to
have a complete copy of the backend lexer so that it can re-lex the
query text and pull out the comments, which it can then parse. The
fact that someone was willing to do that is impressive, but it's
pretty ugly. Still, it's not clear what other approach you could
adopt. We could make the core system able to extract and pass through
comments to extensions. We could add new syntax so that instead of
saying SELECT ... FROM foo AS bar you can say SELECT ... FROM foo AS
bar ADVICE 'anything you want goes here' and arrange to pass that
string through to extensions. We could also spend a lot of time
ranting about how this is a terrible idea on principle and therefore
we shouldn't care about supporting it, but the horse is already out of
the barn, so I'm not very impressed by that approach.
2. If you want a certain base relation or join relation to be treated
in a certain way, how do you identify it? You might think that this is
easy because, even when a query contains multiple references to a
relation with the same name, or identical aliases in different parts
of the query, EXPLAIN renames them so they have disjoint names. What
would be nice is if you could feed those names back into your
extension and use them as a way of specifying what behavior you want
where. But that doesn't work, because what actually happens is that
the plan can contain duplicated aliases, and then when EXPLAIN
deparses it using ruleutils.c, that's when we rename things so that
they're distinct. This means that, at the time we're planning, we
don't yet know what name EXPLAIN will end up giving to any relation
yet, which means we can't use the names that EXPLAIN produced for an
earlier plan for the same query to associate behaviors with relations.
I wonder if we could think about reversing the order of operations
here and making it so that we do the distinct-ification during parse
analysis or maybe early in planning, so that the name you see EXPLAIN
print out is the real name of that thing and not just a value invented
for display purposes.
This second problem affects practically any use of the mechanism added
by this patch, as well as things like pg_hint_plan and EDB's own
internal implementation of planner hints. As far as I know, nobody has
a good solution, and everybody just punts. Hints get specified by
table name or table alias and then you hope that things match in the
right places. This is sort of workable if hints are what you're trying
to implement, but AFAICS it's a complete disaster if what you want to
do is recreate automatically a plan you saw before. If you're hinting
your queries and the hints aren't applying in quite the right places
because of some name collisions, you can maybe adjust the query to
avoid the collisions and still win. But if you are trying to recreate
a previous plan, you really need to look at what happened last time
and then make the same things happen in the same places this time, and
how are you supposed to do that if there's no unique key that you can
use to reliably identify the rels involved in the query? Before
somebody says it, I do realize that these patches as proposed aren't
enough to ensure reliably recreating a plan even if we had a perfect
solution to this problem, but you have to start someplace. It seems
fundamentally reasonable to me to say "hey, if I want to modify the
planner behavior, I need a way to say which part of the query plan
should get modified," and right now it appears to me that we don't
have that.
So again, I am definitely not saying that these patches get us all the
way to where we should be -- not in terms of the ability to control
the plan, and definitely not in terms of giving extensions all the
information they need to be effective. But if we insist on a perfect
solution before doing anything, we will never get anywhere, and I
personally believe these are going in a useful direction.
Comments, preferably constructive ones, welcome.
--
Robert Haas
EDB: http://www.enterprisedb.com
Attachments:
v4-0003-New-contrib-module-alphabet_join.patchapplication/octet-stream; name=v4-0003-New-contrib-module-alphabet_join.patchDownload
From 2a6614c7ed49683d034616d22446f1616fb6cb19 Mon Sep 17 00:00:00 2001
From: Robert Haas <rhaas@postgresql.org>
Date: Fri, 30 Aug 2024 10:27:31 -0400
Subject: [PATCH v4 3/4] New contrib module: alphabet_join.
This forces joins to be done alphabetically by alias name. It demonstrates
that join_path_setup_hook is sufficient to control the join order, and
is not intended for commit.
---
contrib/Makefile | 1 +
contrib/alphabet_join/Makefile | 17 ++++++
contrib/alphabet_join/alphabet_join.c | 74 +++++++++++++++++++++++++++
contrib/alphabet_join/meson.build | 12 +++++
contrib/meson.build | 1 +
5 files changed, 105 insertions(+)
create mode 100644 contrib/alphabet_join/Makefile
create mode 100644 contrib/alphabet_join/alphabet_join.c
create mode 100644 contrib/alphabet_join/meson.build
diff --git a/contrib/Makefile b/contrib/Makefile
index abd780f2774..b3422616698 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -5,6 +5,7 @@ top_builddir = ..
include $(top_builddir)/src/Makefile.global
SUBDIRS = \
+ alphabet_join \
amcheck \
auth_delay \
auto_explain \
diff --git a/contrib/alphabet_join/Makefile b/contrib/alphabet_join/Makefile
new file mode 100644
index 00000000000..204bc35b3d4
--- /dev/null
+++ b/contrib/alphabet_join/Makefile
@@ -0,0 +1,17 @@
+# contrib/alphabet_join/Makefile
+
+MODULE_big = alphabet_join
+OBJS = \
+ $(WIN32RES) \
+ alphabet_join.o
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/alphabet_join
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/alphabet_join/alphabet_join.c b/contrib/alphabet_join/alphabet_join.c
new file mode 100644
index 00000000000..ae712141522
--- /dev/null
+++ b/contrib/alphabet_join/alphabet_join.c
@@ -0,0 +1,74 @@
+/*-------------------------------------------------------------------------
+ *
+ * alphabet_join.c
+ * force tables to be joined in alphabetical order by alias name.
+ * this is just a demonstration, so we don't worry about collation here.
+ *
+ * Copyright (c) 2016-2024, PostgreSQL Global Development Group
+ *
+ * contrib/alphabet_join/alphabet_join.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "fmgr.h"
+#include "optimizer/paths.h"
+#include "parser/parsetree.h"
+
+static void aj_join_path_setup_hook(PlannerInfo *root,
+ RelOptInfo *joinrel,
+ RelOptInfo *outerrel,
+ RelOptInfo *innerrel,
+ JoinType jointype,
+ JoinPathExtraData *extra);
+
+static join_path_setup_hook_type prev_join_path_setup_hook = NULL;
+
+PG_MODULE_MAGIC;
+
+void
+_PG_init(void)
+{
+ prev_join_path_setup_hook = join_path_setup_hook;
+ join_path_setup_hook = aj_join_path_setup_hook;
+}
+
+static void
+aj_join_path_setup_hook(PlannerInfo *root, RelOptInfo *joinrel,
+ RelOptInfo *outerrel, RelOptInfo *innerrel,
+ JoinType jointype, JoinPathExtraData *extra)
+{
+ int relid;
+ char *outerrel_last = NULL;
+
+ /* Find the alphabetically last outerrel. */
+ relid = -1;
+ while ((relid = bms_next_member(outerrel->relids, relid)) >= 0)
+ {
+ RangeTblEntry *rte = planner_rt_fetch(relid, root);
+
+ Assert(rte->eref != NULL && rte->eref->aliasname != NULL);
+
+ if (outerrel_last == NULL ||
+ strcmp(outerrel_last, rte->eref->aliasname) < 0)
+ outerrel_last = rte->eref->aliasname;
+ }
+
+ /*
+ * If any innerrel is alphabetically before the last outerrel, then this
+ * join order is not alphabetical and should be rejected.
+ */
+ relid = -1;
+ while ((relid = bms_next_member(innerrel->relids, relid)) >= 0)
+ {
+ RangeTblEntry *rte = planner_rt_fetch(relid, root);
+
+ Assert(rte->eref != NULL && rte->eref->aliasname != NULL);
+
+ if (strcmp(rte->eref->aliasname, outerrel_last) < 0)
+ {
+ extra->jsa_mask = 0;
+ return;
+ }
+ }
+}
diff --git a/contrib/alphabet_join/meson.build b/contrib/alphabet_join/meson.build
new file mode 100644
index 00000000000..437cb14af58
--- /dev/null
+++ b/contrib/alphabet_join/meson.build
@@ -0,0 +1,12 @@
+# Copyright (c) 2022-2024, PostgreSQL Global Development Group
+
+alphabet_join_sources = files(
+ 'alphabet_join.c',
+)
+
+alphabet_join = shared_module('alphabet_join',
+ alphabet_join_sources,
+ kwargs: contrib_mod_args,
+)
+
+contrib_targets += alphabet_join
diff --git a/contrib/meson.build b/contrib/meson.build
index 14a89068650..4372242c8f3 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -12,6 +12,7 @@ contrib_doc_args = {
'install_dir': contrib_doc_dir,
}
+subdir('alphabet_join')
subdir('amcheck')
subdir('auth_delay')
subdir('auto_explain')
--
2.39.3 (Apple Git-145)
v4-0004-New-contrib-module-hint_via_alias.patchapplication/octet-stream; name=v4-0004-New-contrib-module-hint_via_alias.patchDownload
From 1d7feacea79ea27d7159e3889f8d527b67601591 Mon Sep 17 00:00:00 2001
From: Robert Haas <rhaas@postgresql.org>
Date: Thu, 10 Oct 2024 11:58:22 -0400
Subject: [PATCH v4 4/4] New contrib module: hint_via_alias.
This forces a table to be merge joined, hash joined, or joined via a nested
loop if the table alias starts with mj_, hj_, or nl_, respectively. This
demonstrates that join_path_setup_hook is sufficient to control the join
method.
It forces a table to be sequential scanned, index scanned, index only
scanned, bitmap scanned, or tid scanned if the table alias starts with
ss_, is_, ios_, bs_, o ts_, respectively. This demonstrates that, with
the patch, get_relation_info_hook is sufficient to control the scan
method.
This is not intended for commit.
---
contrib/Makefile | 1 +
contrib/hint_via_alias/Makefile | 17 +++
contrib/hint_via_alias/hint_via_alias.c | 168 ++++++++++++++++++++++++
contrib/hint_via_alias/meson.build | 12 ++
contrib/meson.build | 1 +
src/tools/pgindent/typedefs.list | 1 +
6 files changed, 200 insertions(+)
create mode 100644 contrib/hint_via_alias/Makefile
create mode 100644 contrib/hint_via_alias/hint_via_alias.c
create mode 100644 contrib/hint_via_alias/meson.build
diff --git a/contrib/Makefile b/contrib/Makefile
index b3422616698..2b47095ce18 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -22,6 +22,7 @@ SUBDIRS = \
earthdistance \
file_fdw \
fuzzystrmatch \
+ hint_via_alias \
hstore \
intagg \
intarray \
diff --git a/contrib/hint_via_alias/Makefile b/contrib/hint_via_alias/Makefile
new file mode 100644
index 00000000000..2e0e540d352
--- /dev/null
+++ b/contrib/hint_via_alias/Makefile
@@ -0,0 +1,17 @@
+# contrib/hint_via_alias/Makefile
+
+MODULE_big = hint_via_alias
+OBJS = \
+ $(WIN32RES) \
+ hint_via_alias.o
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/hint_via_alias
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/hint_via_alias/hint_via_alias.c b/contrib/hint_via_alias/hint_via_alias.c
new file mode 100644
index 00000000000..695d346f3af
--- /dev/null
+++ b/contrib/hint_via_alias/hint_via_alias.c
@@ -0,0 +1,168 @@
+/*-------------------------------------------------------------------------
+ *
+ * hint_via_alias.c
+ * force tables to be joined in using a nestedloop, mergejoin, or hash
+ * join if their alias name begins with nl_, mj_, or hj_.
+ *
+ * forces tables to be sequential scanned, index scanned, index only
+ * scanned, bitmap scanned, or tid scanned if their alias name begins
+ * with ss_, is_, ios_, bs_, or ts_.
+ *
+ * Copyright (c) 2016-2024, PostgreSQL Global Development Group
+ *
+ * contrib/hint_via_alias/hint_via_alias.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "fmgr.h"
+#include "optimizer/paths.h"
+#include "optimizer/plancat.h"
+#include "parser/parsetree.h"
+
+typedef enum
+{
+ HVAJ_UNSPECIFIED,
+ HVAJ_HASHJOIN,
+ HVAJ_MERGEJOIN,
+ HVAJ_NESTLOOP
+} hvaj_hint;
+
+static void hva_get_relation_info(PlannerInfo *root, Oid relationObjectId,
+ bool inhparent, RelOptInfo *rel);
+static void hva_join_path_setup(PlannerInfo *root,
+ RelOptInfo *joinrel,
+ RelOptInfo *outerrel,
+ RelOptInfo *innerrel,
+ JoinType jointype,
+ JoinPathExtraData *extra);
+static hvaj_hint get_join_hint(PlannerInfo *root, Index relid);
+static uint32 get_scan_hint(PlannerInfo *root, Index relid);
+
+static get_relation_info_hook_type prev_get_relation_info_hook = NULL;
+static join_path_setup_hook_type prev_join_path_setup_hook = NULL;
+
+PG_MODULE_MAGIC;
+
+void
+_PG_init(void)
+{
+ prev_get_relation_info_hook = get_relation_info_hook;
+ get_relation_info_hook = hva_get_relation_info;
+ prev_join_path_setup_hook = join_path_setup_hook;
+ join_path_setup_hook = hva_join_path_setup;
+}
+
+static void
+hva_get_relation_info(PlannerInfo *root, Oid relationObjectId,
+ bool inhparent, RelOptInfo *rel)
+{
+ uint32 hint = get_scan_hint(root, rel->relid);
+
+ if (hint != 0)
+ rel->ssa_mask &= hint;
+
+ if (prev_get_relation_info_hook != NULL)
+ prev_get_relation_info_hook(root, relationObjectId, inhparent, rel);
+}
+
+static void
+hva_join_path_setup(PlannerInfo *root, RelOptInfo *joinrel,
+ RelOptInfo *outerrel, RelOptInfo *innerrel,
+ JoinType jointype, JoinPathExtraData *extra)
+{
+ hvaj_hint outerhint = HVAJ_UNSPECIFIED;
+ hvaj_hint innerhint = HVAJ_UNSPECIFIED;
+ hvaj_hint hint;
+
+ if (outerrel->reloptkind == RELOPT_BASEREL ||
+ outerrel->reloptkind == RELOPT_OTHER_MEMBER_REL)
+ outerhint = get_join_hint(root, outerrel->relid);
+
+ if (innerrel->reloptkind == RELOPT_BASEREL ||
+ innerrel->reloptkind == RELOPT_OTHER_MEMBER_REL)
+ innerhint = get_join_hint(root, innerrel->relid);
+
+ /*
+ * If the hints conflict, that's not necessarily an indication of user
+ * error. For example, if the user joins A to B and supplies different
+ * join method hints for A and B, we will end up using a disabled path.
+ * However, if they are joining A, B, and C and supply different join
+ * method hints for A and B, we could potentially respect both hints by
+ * avoiding a direct A-B join altogether. Even if it does turn out that we
+ * can't respect all the hints, we don't need any special handling for
+ * that here: the planner will just return a disabled path.
+ */
+ if (outerhint != HVAJ_UNSPECIFIED && innerhint != HVAJ_UNSPECIFIED &&
+ outerhint != innerhint)
+ {
+ extra->jsa_mask = 0;
+ return;
+ }
+
+ if (outerhint != HVAJ_UNSPECIFIED)
+ hint = outerhint;
+ else
+ hint = innerhint;
+
+ switch (hint)
+ {
+ case HVAJ_UNSPECIFIED:
+ break;
+ case HVAJ_NESTLOOP:
+ extra->jsa_mask &= JSA_NESTLOOP_ANY;
+ break;
+ case HVAJ_HASHJOIN:
+ extra->jsa_mask &= JSA_HASHJOIN;
+ break;
+ case HVAJ_MERGEJOIN:
+ extra->jsa_mask &= JSA_MERGEJOIN_ANY;
+ break;
+ }
+
+ if (prev_join_path_setup_hook != NULL)
+ prev_join_path_setup_hook(root, joinrel, outerrel, innerrel, jointype,
+ extra);
+}
+
+static hvaj_hint
+get_join_hint(PlannerInfo *root, Index relid)
+{
+ RangeTblEntry *rte = planner_rt_fetch(relid, root);
+
+ Assert(rte->eref != NULL && rte->eref->aliasname != NULL);
+
+ if (strncmp(rte->eref->aliasname, "nl_", 3) == 0)
+ return HVAJ_NESTLOOP;
+ else if (strncmp(rte->eref->aliasname, "hj_", 3) == 0)
+ return HVAJ_HASHJOIN;
+ else if (strncmp(rte->eref->aliasname, "mj_", 3) == 0)
+ return HVAJ_MERGEJOIN;
+ else
+ return HVAJ_UNSPECIFIED;
+}
+
+static uint32
+get_scan_hint(PlannerInfo *root, Index relid)
+{
+ RangeTblEntry *rte = planner_rt_fetch(relid, root);
+
+ /* happens if CREATE INDEX is used without an index name, at least */
+ if (rte->eref == NULL)
+ return 0;
+
+ Assert(rte->eref->aliasname != NULL);
+
+ if (strncmp(rte->eref->aliasname, "ts_", 3) == 0)
+ return SSA_TIDSCAN;
+ else if (strncmp(rte->eref->aliasname, "ss_", 3) == 0)
+ return SSA_SEQSCAN;
+ else if (strncmp(rte->eref->aliasname, "is_", 3) == 0)
+ return SSA_INDEXSCAN;
+ else if (strncmp(rte->eref->aliasname, "ios_", 3) == 0)
+ return SSA_INDEXONLYSCAN | SSA_CONSIDER_INDEXONLY;
+ else if (strncmp(rte->eref->aliasname, "bs_", 3) == 0)
+ return SSA_BITMAPSCAN;
+ else
+ return 0;
+}
diff --git a/contrib/hint_via_alias/meson.build b/contrib/hint_via_alias/meson.build
new file mode 100644
index 00000000000..7e42c5783ab
--- /dev/null
+++ b/contrib/hint_via_alias/meson.build
@@ -0,0 +1,12 @@
+# Copyright (c) 2022-2024, PostgreSQL Global Development Group
+
+hint_via_alias_sources = files(
+ 'hint_via_alias.c',
+)
+
+hint_via_alias = shared_module('hint_via_alias',
+ hint_via_alias_sources,
+ kwargs: contrib_mod_args,
+)
+
+contrib_targets += hint_via_alias
diff --git a/contrib/meson.build b/contrib/meson.build
index 4372242c8f3..261e4c480e2 100644
--- a/contrib/meson.build
+++ b/contrib/meson.build
@@ -30,6 +30,7 @@ subdir('dict_xsyn')
subdir('earthdistance')
subdir('file_fdw')
subdir('fuzzystrmatch')
+subdir('hint_via_alias')
subdir('hstore')
subdir('hstore_plperl')
subdir('hstore_plpython')
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index a65e1c07c5d..8a919214690 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -4179,3 +4179,4 @@ yyscan_t
z_stream
z_streamp
zic_t
+hvaj_hint
--
2.39.3 (Apple Git-145)
v4-0002-Allow-extensions-to-control-scan-strategy.patchapplication/octet-stream; name=v4-0002-Allow-extensions-to-control-scan-strategy.patchDownload
From 8d4640c7f9722ff4bf3d1af918f5afa61a7de3b7 Mon Sep 17 00:00:00 2001
From: Robert Haas <rhaas@postgresql.org>
Date: Mon, 7 Oct 2024 15:41:51 -0400
Subject: [PATCH v4 2/4] Allow extensions to control scan strategy.
At the start of planning, we build a bitmask of allowable scan
strategies beased on the value of the various enable_* planner GUCs.
Extensions can override the behavior on a per-rel basis using
get_relation_info_hook.
As with the join strategy advice, this isn't sufficient for all
needs. If you want to control which index is used, the same hook,
get_relation_info_hook, that you use to set scan strategy can also
editorialize on the index list. However, that doesn't appear to be
sufficient to fully control the shape of bitmap plans.
Another gap is that it's not clear what to do if you want to
encourage parallel plans or non-parallel plans. It is not entirely
clear to me whether that is a problem that is specific to the scan
level or whether it is something more general.
---
src/backend/optimizer/path/costsize.c | 25 +++++++++++++---------
src/backend/optimizer/path/indxpath.c | 4 ++--
src/backend/optimizer/path/tidpath.c | 7 ++++---
src/backend/optimizer/plan/planner.c | 22 ++++++++++++++++++++
src/backend/optimizer/util/plancat.c | 3 +++
src/backend/optimizer/util/relnode.c | 7 +++++++
src/include/nodes/pathnodes.h | 5 +++++
src/include/optimizer/paths.h | 30 +++++++++++++++++++++++++++
8 files changed, 88 insertions(+), 15 deletions(-)
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index 5888ecac65d..3a0f30a1a6d 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -354,7 +354,7 @@ cost_seqscan(Path *path, PlannerInfo *root,
path->rows = clamp_row_est(path->rows / parallel_divisor);
}
- path->disabled_nodes = enable_seqscan ? 0 : 1;
+ path->disabled_nodes = (baserel->ssa_mask & SSA_SEQSCAN) != 0 ? 0 : 1;
path->startup_cost = startup_cost;
path->total_cost = startup_cost + cpu_run_cost + disk_run_cost;
}
@@ -583,6 +583,7 @@ cost_index(IndexPath *path, PlannerInfo *root, double loop_count,
double pages_fetched;
double rand_heap_pages;
double index_pages;
+ bool enabled;
/* Should only be applied to base relations */
Assert(IsA(baserel, RelOptInfo) &&
@@ -614,8 +615,12 @@ cost_index(IndexPath *path, PlannerInfo *root, double loop_count,
path->indexclauses);
}
- /* we don't need to check enable_indexonlyscan; indxpath.c does that */
- path->path.disabled_nodes = enable_indexscan ? 0 : 1;
+ /* is this scan type disabled? */
+ if (indexonly)
+ enabled = (baserel->ssa_mask & SSA_INDEXONLYSCAN) ? 1 : 0;
+ else
+ enabled = (baserel->ssa_mask & SSA_INDEXSCAN) ? 1 : 0;
+ path->path.disabled_nodes = enabled ? 0 : 1;
/*
* Call index-access-method-specific code to estimate the processing cost
@@ -1109,7 +1114,7 @@ cost_bitmap_heap_scan(Path *path, PlannerInfo *root, RelOptInfo *baserel,
startup_cost += path->pathtarget->cost.startup;
run_cost += path->pathtarget->cost.per_tuple * path->rows;
- path->disabled_nodes = enable_bitmapscan ? 0 : 1;
+ path->disabled_nodes = (baserel->ssa_mask & SSA_BITMAPSCAN) != 0 ? 0 : 1;
path->startup_cost = startup_cost;
path->total_cost = startup_cost + run_cost;
}
@@ -1287,10 +1292,10 @@ cost_tidscan(Path *path, PlannerInfo *root,
/*
* We must use a TID scan for CurrentOfExpr; in any other case, we
- * should be generating a TID scan only if enable_tidscan=true. Also,
+ * should be generating a TID scan only if TID scans are allowed. Also,
* if CurrentOfExpr is the qual, there should be only one.
*/
- Assert(enable_tidscan || IsA(qual, CurrentOfExpr));
+ Assert((baserel->ssa_mask & SSA_TIDSCAN) != 0 || IsA(qual, CurrentOfExpr));
Assert(list_length(tidquals) == 1 || !IsA(qual, CurrentOfExpr));
if (IsA(qual, ScalarArrayOpExpr))
@@ -1342,8 +1347,8 @@ cost_tidscan(Path *path, PlannerInfo *root,
/*
* There are assertions above verifying that we only reach this function
- * either when enable_tidscan=true or when the TID scan is the only legal
- * path, so it's safe to set disabled_nodes to zero here.
+ * either when baserel->ssa_mask includes SSA_TIDSCAN or when the TID scan
+ * is the only legal path, so it's safe to set disabled_nodes to zero here.
*/
path->disabled_nodes = 0;
path->startup_cost = startup_cost;
@@ -1438,8 +1443,8 @@ cost_tidrangescan(Path *path, PlannerInfo *root,
startup_cost += path->pathtarget->cost.startup;
run_cost += path->pathtarget->cost.per_tuple * path->rows;
- /* we should not generate this path type when enable_tidscan=false */
- Assert(enable_tidscan);
+ /* we should not generate this path type when TID scans are disabled */
+ Assert((baserel->ssa_mask & SSA_TIDSCAN) != 0);
path->disabled_nodes = 0;
path->startup_cost = startup_cost;
path->total_cost = startup_cost + run_cost;
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
index c0fcc7d78df..a42dbc38251 100644
--- a/src/backend/optimizer/path/indxpath.c
+++ b/src/backend/optimizer/path/indxpath.c
@@ -1735,8 +1735,8 @@ check_index_only(RelOptInfo *rel, IndexOptInfo *index)
ListCell *lc;
int i;
- /* Index-only scans must be enabled */
- if (!enable_indexonlyscan)
+ /* If we're not allowed to consider index-only scans, give up now */
+ if ((rel->ssa_mask & SSA_CONSIDER_INDEXONLY) == 0)
return false;
/*
diff --git a/src/backend/optimizer/path/tidpath.c b/src/backend/optimizer/path/tidpath.c
index b0323b26eca..efe569457fa 100644
--- a/src/backend/optimizer/path/tidpath.c
+++ b/src/backend/optimizer/path/tidpath.c
@@ -500,18 +500,19 @@ create_tidscan_paths(PlannerInfo *root, RelOptInfo *rel)
List *tidquals;
List *tidrangequals;
bool isCurrentOf;
+ bool enabled = (rel->ssa_mask & SSA_TIDSCAN) != 0;
/*
* If any suitable quals exist in the rel's baserestrict list, generate a
* plain (unparameterized) TidPath with them.
*
- * We skip this when enable_tidscan = false, except when the qual is
+ * We skip this when TID scans are disabled, except when the qual is
* CurrentOfExpr. In that case, a TID scan is the only correct path.
*/
tidquals = TidQualFromRestrictInfoList(root, rel->baserestrictinfo, rel,
&isCurrentOf);
- if (tidquals != NIL && (enable_tidscan || isCurrentOf))
+ if (tidquals != NIL && (enabled || isCurrentOf))
{
/*
* This path uses no join clauses, but it could still have required
@@ -533,7 +534,7 @@ create_tidscan_paths(PlannerInfo *root, RelOptInfo *rel)
}
/* Skip the rest if TID scans are disabled. */
- if (!enable_tidscan)
+ if (!enabled)
return false;
/*
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 7c1000879ec..8b0a507f2d9 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -419,6 +419,28 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions,
tuple_fraction = 0.0;
}
+ /*
+ * Compute the initial scan strategy advice mask.
+ *
+ * It may seem surprising that enable_indexscan sets both SSA_INDEXSCAN
+ * and SSA_INDEXONLYSCAN. However, the historical behavior of this GUC
+ * corresponds to this exactly: enable_indexscan=off disables both
+ * index-scan and index-only scan paths, whereas enable_indexonlyscan=off
+ * converts the index-only scan paths that we would have considered into
+ * index scan paths.
+ */
+ glob->default_ssa_mask = 0;
+ if (enable_tidscan)
+ glob->default_ssa_mask |= SSA_TIDSCAN;
+ if (enable_seqscan)
+ glob->default_ssa_mask |= SSA_SEQSCAN;
+ if (enable_indexscan)
+ glob->default_ssa_mask |= SSA_INDEXSCAN | SSA_INDEXONLYSCAN;
+ if (enable_indexonlyscan)
+ glob->default_ssa_mask |= SSA_CONSIDER_INDEXONLY;
+ if (enable_bitmapscan)
+ glob->default_ssa_mask |= SSA_BITMAPSCAN;
+
/* Compute the initial join strategy advice mask. */
glob->default_jsa_mask = JSA_FOREIGN;
if (enable_hashjoin)
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index b913f91ff03..97066ec1f86 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -570,6 +570,9 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
* Allow a plugin to editorialize on the info we obtained from the
* catalogs. Actions might include altering the assumed relation size,
* removing an index, or adding a hypothetical index to the indexlist.
+ *
+ * An extension can also modify rel->ssa_mask here to control the scan
+ * strategy.
*/
if (get_relation_info_hook)
(*get_relation_info_hook) (root, relationObjectId, inhparent, rel);
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 9e328c5ac7c..8d56f5717e4 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -321,6 +321,12 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
rel->direct_lateral_relids = parent->direct_lateral_relids;
rel->lateral_relids = parent->lateral_relids;
rel->lateral_referencers = parent->lateral_referencers;
+
+ /*
+ * By default, a parent's scan strategy advice is preserved for each
+ * inheritance child.
+ */
+ rel->ssa_mask = parent->ssa_mask;
}
else
{
@@ -331,6 +337,7 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
rel->direct_lateral_relids = NULL;
rel->lateral_relids = NULL;
rel->lateral_referencers = NULL;
+ rel->ssa_mask = root->glob->default_ssa_mask;
}
/* Check type of rtable entry */
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index ca328d6edad..8f13f2c80e8 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -161,6 +161,9 @@ typedef struct PlannerGlobal
/* worst PROPARALLEL hazard level */
char maxParallelHazard;
+ /* default scan strategy advice, except where overrriden by hooks */
+ uint32 default_ssa_mask;
+
/* default join strategy advice, except where overrriden by hooks */
uint32 default_jsa_mask;
@@ -931,6 +934,8 @@ typedef struct RelOptInfo
Relids *attr_needed pg_node_attr(read_write_ignore);
/* array indexed [min_attr .. max_attr] */
int32 *attr_widths pg_node_attr(read_write_ignore);
+ /* scan strategy advice */
+ uint32 ssa_mask;
/*
* Zero-based set containing attnums of NOT NULL columns. Not populated
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
index f9f346f86a1..4ee0344e5e1 100644
--- a/src/include/optimizer/paths.h
+++ b/src/include/optimizer/paths.h
@@ -16,6 +16,36 @@
#include "nodes/pathnodes.h"
+/*
+ * Scan strategy advice.
+ *
+ * If SSA_CONSIDER_INDEXONLY is not set, index-only scan paths will not even
+ * be generated, and we'll generated index-scan paths for the same cases
+ * instead. If any other bit is not set, paths of that type will still be
+ * generated but will be marked as disabled.
+ *
+ * So, if you want to avoid an index-only scan, you can either unset
+ * SSA_CONSIDER_INDEXONLY (in which case you'll get an index-scan instead,
+ * which may end up disabled if you also unset SSA_INDEXSCAN) or you can
+ * unset SSA_INDEXONLYSCAN (in which the index-only scan will be disabled
+ * and the cheapest non-disabled alternative, if any, will be chosen, but
+ * no corresponding index scan will be considered). If, on the other hand,
+ * you want to encourage an index-only scan, you can set just SSA_INDEXONLYSCAN
+ * and SSA_CONSIDER_INDEXONLY and clear all of the other bits.
+ *
+ * A default scan strategy advice mask is stored in the PlannerGlobal object
+ * based on the values of the various enable_* GUCs. This value is propagted
+ * into each RelOptInfo for a baserel, and from baserels to their inheritance
+ * children when partitions are expanded. In either case, the value can be
+ * usefully changed in get_relation_info_hook.
+ */
+#define SSA_TIDSCAN 0x0001
+#define SSA_SEQSCAN 0x0002
+#define SSA_INDEXSCAN 0x0004
+#define SSA_INDEXONLYSCAN 0x0008
+#define SSA_BITMAPSCAN 0x0010
+#define SSA_CONSIDER_INDEXONLY 0x0020
+
/*
* Join strategy advice.
*
--
2.39.3 (Apple Git-145)
v4-0001-Allow-extensions-to-control-join-strategy.patchapplication/octet-stream; name=v4-0001-Allow-extensions-to-control-join-strategy.patchDownload
From 6200809f38d9c39c1d3dd096c43461ee696c4384 Mon Sep 17 00:00:00 2001
From: Robert Haas <rhaas@postgresql.org>
Date: Thu, 29 Aug 2024 15:38:58 -0400
Subject: [PATCH v4 1/4] Allow extensions to control join strategy.
At the start of planning, we build a bitmask of allowable join
strategies based on the values of the various enable_* planner GUCs,
indicating which join strategies are allowed. Extensions can override
this mask for an entire subquery using join_search_hook, or, generaly
more usefully, they can change the mask for each call to
add_paths_to_joinrel using a new hook called join_path_setup_hook.
This is sufficient to allow an extension to force the use of
particular join strategies either in general or for specific joins,
and it is also sufficient to allow an extension to force the join
order, including which relation appears on which side of a given join.
There are a number of things that this patch doesn't let you do.
First, it won't help if you want to implement some policy where the
allowable join methods might differ for each individual combination of
input paths (e.g. disable nested loops only when the inner side's
parameterization is not even partially satisfied by the outer side's
paramaeterization). Second, it doesn't allow you to control the
uniquification strategy for a particular joinrel. Third, it doesn't
give you any control over aspects of planning other than join planning.
Future patches may close some of these gaps.
---
src/backend/optimizer/geqo/geqo_eval.c | 22 +++--
src/backend/optimizer/geqo/geqo_main.c | 10 ++-
src/backend/optimizer/geqo/geqo_pool.c | 4 +-
src/backend/optimizer/path/allpaths.c | 17 ++--
src/backend/optimizer/path/costsize.c | 59 +++++++++-----
src/backend/optimizer/path/joinpath.c | 104 +++++++++++++++++-------
src/backend/optimizer/path/joinrels.c | 79 ++++++++++--------
src/backend/optimizer/plan/createplan.c | 1 +
src/backend/optimizer/plan/planner.c | 21 +++++
src/backend/optimizer/util/pathnode.c | 6 +-
src/backend/optimizer/util/relnode.c | 17 ++--
src/include/nodes/pathnodes.h | 5 ++
src/include/optimizer/cost.h | 4 +-
src/include/optimizer/geqo.h | 9 +-
src/include/optimizer/geqo_pool.h | 2 +-
src/include/optimizer/pathnode.h | 9 +-
src/include/optimizer/paths.h | 56 +++++++++++--
17 files changed, 295 insertions(+), 130 deletions(-)
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
index d2f7f4e5f3c..6b1d8df3ff6 100644
--- a/src/backend/optimizer/geqo/geqo_eval.c
+++ b/src/backend/optimizer/geqo/geqo_eval.c
@@ -40,7 +40,7 @@ typedef struct
} Clump;
static List *merge_clump(PlannerInfo *root, List *clumps, Clump *new_clump,
- int num_gene, bool force);
+ int num_gene, bool force, unsigned jsa_mask);
static bool desirable_join(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel);
@@ -54,7 +54,7 @@ static bool desirable_join(PlannerInfo *root,
* returns DBL_MAX.
*/
Cost
-geqo_eval(PlannerInfo *root, Gene *tour, int num_gene)
+geqo_eval(PlannerInfo *root, Gene *tour, int num_gene, unsigned jsa_mask)
{
MemoryContext mycontext;
MemoryContext oldcxt;
@@ -99,7 +99,7 @@ geqo_eval(PlannerInfo *root, Gene *tour, int num_gene)
root->join_rel_hash = NULL;
/* construct the best path for the given combination of relations */
- joinrel = gimme_tree(root, tour, num_gene);
+ joinrel = gimme_tree(root, tour, num_gene, jsa_mask);
/*
* compute fitness, if we found a valid join
@@ -160,7 +160,7 @@ geqo_eval(PlannerInfo *root, Gene *tour, int num_gene)
* since there's no provision for un-clumping, this must lead to failure.)
*/
RelOptInfo *
-gimme_tree(PlannerInfo *root, Gene *tour, int num_gene)
+gimme_tree(PlannerInfo *root, Gene *tour, int num_gene, unsigned jsa_mask)
{
GeqoPrivateData *private = (GeqoPrivateData *) root->join_search_private;
List *clumps;
@@ -196,7 +196,8 @@ gimme_tree(PlannerInfo *root, Gene *tour, int num_gene)
cur_clump->size = 1;
/* Merge it into the clumps list, using only desirable joins */
- clumps = merge_clump(root, clumps, cur_clump, num_gene, false);
+ clumps = merge_clump(root, clumps, cur_clump, num_gene, false,
+ jsa_mask);
}
if (list_length(clumps) > 1)
@@ -210,7 +211,8 @@ gimme_tree(PlannerInfo *root, Gene *tour, int num_gene)
{
Clump *clump = (Clump *) lfirst(lc);
- fclumps = merge_clump(root, fclumps, clump, num_gene, true);
+ fclumps = merge_clump(root, fclumps, clump, num_gene, true,
+ jsa_mask);
}
clumps = fclumps;
}
@@ -236,7 +238,7 @@ gimme_tree(PlannerInfo *root, Gene *tour, int num_gene)
*/
static List *
merge_clump(PlannerInfo *root, List *clumps, Clump *new_clump, int num_gene,
- bool force)
+ bool force, unsigned jsa_mask)
{
ListCell *lc;
int pos;
@@ -259,7 +261,8 @@ merge_clump(PlannerInfo *root, List *clumps, Clump *new_clump, int num_gene,
*/
joinrel = make_join_rel(root,
old_clump->joinrel,
- new_clump->joinrel);
+ new_clump->joinrel,
+ jsa_mask);
/* Keep searching if join order is not valid */
if (joinrel)
@@ -292,7 +295,8 @@ merge_clump(PlannerInfo *root, List *clumps, Clump *new_clump, int num_gene,
* others. When no further merge is possible, we'll reinsert
* it into the list.
*/
- return merge_clump(root, clumps, old_clump, num_gene, force);
+ return merge_clump(root, clumps, old_clump, num_gene, force,
+ jsa_mask);
}
}
}
diff --git a/src/backend/optimizer/geqo/geqo_main.c b/src/backend/optimizer/geqo/geqo_main.c
index 0c5540e2af4..c69b60d2e95 100644
--- a/src/backend/optimizer/geqo/geqo_main.c
+++ b/src/backend/optimizer/geqo/geqo_main.c
@@ -69,7 +69,8 @@ static int gimme_number_generations(int pool_size);
*/
RelOptInfo *
-geqo(PlannerInfo *root, int number_of_rels, List *initial_rels)
+geqo(PlannerInfo *root, int number_of_rels, List *initial_rels,
+ unsigned jsa_mask)
{
GeqoPrivateData private;
int generation;
@@ -116,7 +117,7 @@ geqo(PlannerInfo *root, int number_of_rels, List *initial_rels)
pool = alloc_pool(root, pool_size, number_of_rels);
/* random initialization of the pool */
- random_init_pool(root, pool);
+ random_init_pool(root, pool, jsa_mask);
/* sort the pool according to cheapest path as fitness */
sort_pool(root, pool); /* we have to do it only one time, since all
@@ -218,7 +219,8 @@ geqo(PlannerInfo *root, int number_of_rels, List *initial_rels)
/* EVALUATE FITNESS */
- kid->worth = geqo_eval(root, kid->string, pool->string_length);
+ kid->worth = geqo_eval(root, kid->string, pool->string_length,
+ jsa_mask);
/* push the kid into the wilderness of life according to its worth */
spread_chromo(root, kid, pool);
@@ -269,7 +271,7 @@ geqo(PlannerInfo *root, int number_of_rels, List *initial_rels)
*/
best_tour = (Gene *) pool->data[0].string;
- best_rel = gimme_tree(root, best_tour, pool->string_length);
+ best_rel = gimme_tree(root, best_tour, pool->string_length, jsa_mask);
if (best_rel == NULL)
elog(ERROR, "geqo failed to make a valid plan");
diff --git a/src/backend/optimizer/geqo/geqo_pool.c b/src/backend/optimizer/geqo/geqo_pool.c
index 0ec97d5a3f1..dbec3c50943 100644
--- a/src/backend/optimizer/geqo/geqo_pool.c
+++ b/src/backend/optimizer/geqo/geqo_pool.c
@@ -88,7 +88,7 @@ free_pool(PlannerInfo *root, Pool *pool)
* initialize genetic pool
*/
void
-random_init_pool(PlannerInfo *root, Pool *pool)
+random_init_pool(PlannerInfo *root, Pool *pool, unsigned jsa_mask)
{
Chromosome *chromo = (Chromosome *) pool->data;
int i;
@@ -107,7 +107,7 @@ random_init_pool(PlannerInfo *root, Pool *pool)
{
init_tour(root, chromo[i].string, pool->string_length);
pool->data[i].worth = geqo_eval(root, chromo[i].string,
- pool->string_length);
+ pool->string_length, jsa_mask);
if (pool->data[i].worth < DBL_MAX)
i++;
else
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 172edb643a4..5fe326c8de4 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -898,7 +898,7 @@ set_tablesample_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *
bms_membership(root->all_query_rels) != BMS_SINGLETON) &&
!(GetTsmRoutine(rte->tablesample->tsmhandler)->repeatable_across_scans))
{
- path = (Path *) create_material_path(rel, path);
+ path = (Path *) create_material_path(rel, path, enable_material);
}
add_path(rel, path);
@@ -3371,6 +3371,7 @@ make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
}
else
{
+
/*
* Consider the different orders in which we could join the rels,
* using a plugin, GEQO, or the regular join search code.
@@ -3381,11 +3382,14 @@ make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
root->initial_rels = initial_rels;
if (join_search_hook)
- return (*join_search_hook) (root, levels_needed, initial_rels);
+ return (*join_search_hook) (root, levels_needed, initial_rels,
+ root->glob->default_jsa_mask);
else if (enable_geqo && levels_needed >= geqo_threshold)
- return geqo(root, levels_needed, initial_rels);
+ return geqo(root, levels_needed, initial_rels,
+ root->glob->default_jsa_mask);
else
- return standard_join_search(root, levels_needed, initial_rels);
+ return standard_join_search(root, levels_needed, initial_rels,
+ root->glob->default_jsa_mask);
}
}
@@ -3419,7 +3423,8 @@ make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
* original states of those data structures. See geqo_eval() for an example.
*/
RelOptInfo *
-standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
+standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels,
+ unsigned jsa_mask)
{
int lev;
RelOptInfo *rel;
@@ -3454,7 +3459,7 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
* level, and build paths for making each one from every available
* pair of lower-level relations.
*/
- join_search_one_level(root, lev);
+ join_search_one_level(root, lev, jsa_mask);
/*
* Run generate_partitionwise_join_paths() and
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index 2bb6db1df77..5888ecac65d 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -2481,7 +2481,7 @@ cost_merge_append(Path *path, PlannerInfo *root,
*/
void
cost_material(Path *path,
- int input_disabled_nodes,
+ bool enabled, int input_disabled_nodes,
Cost input_startup_cost, Cost input_total_cost,
double tuples, int width)
{
@@ -2519,7 +2519,7 @@ cost_material(Path *path,
run_cost += seq_page_cost * npages;
}
- path->disabled_nodes = input_disabled_nodes + (enable_material ? 0 : 1);
+ path->disabled_nodes = input_disabled_nodes + (enabled ? 0 : 1);
path->startup_cost = startup_cost;
path->total_cost = startup_cost + run_cost;
}
@@ -3266,7 +3266,7 @@ cost_group(Path *path, PlannerInfo *root,
*/
void
initial_cost_nestloop(PlannerInfo *root, JoinCostWorkspace *workspace,
- JoinType jointype,
+ JoinType jointype, unsigned nestloop_subtype,
Path *outer_path, Path *inner_path,
JoinPathExtraData *extra)
{
@@ -3280,7 +3280,7 @@ initial_cost_nestloop(PlannerInfo *root, JoinCostWorkspace *workspace,
Cost inner_rescan_run_cost;
/* Count up disabled nodes. */
- disabled_nodes = enable_nestloop ? 0 : 1;
+ disabled_nodes = (extra->jsa_mask & nestloop_subtype) == 0 ? 1 : 0;
disabled_nodes += inner_path->disabled_nodes;
disabled_nodes += outer_path->disabled_nodes;
@@ -3678,7 +3678,12 @@ initial_cost_mergejoin(PlannerInfo *root, JoinCostWorkspace *workspace,
Assert(outerstartsel <= outerendsel);
Assert(innerstartsel <= innerendsel);
- disabled_nodes = enable_mergejoin ? 0 : 1;
+ /*
+ * Assume for now that this node is not itself disabled. We'll sort out
+ * whether that's really the case in final_cost_mergejoin(); here, we'll
+ * just account for any disabled child nodes.
+ */
+ disabled_nodes = 0;
/* cost of source data */
@@ -3859,9 +3864,6 @@ final_cost_mergejoin(PlannerInfo *root, MergePath *path,
rescannedtuples;
double rescanratio;
- /* Set the number of disabled nodes. */
- path->jpath.path.disabled_nodes = workspace->disabled_nodes;
-
/* Protect some assumptions below that rowcounts aren't zero */
if (inner_path_rows <= 0)
inner_path_rows = 1;
@@ -3988,16 +3990,20 @@ final_cost_mergejoin(PlannerInfo *root, MergePath *path,
path->materialize_inner = false;
/*
- * Prefer materializing if it looks cheaper, unless the user has asked to
- * suppress materialization.
+ * If merge joins with materialization are enabled, then choose
+ * materialization if either (a) it looks cheaper or (b) merge joins
+ * without materialization are disabled.
*/
- else if (enable_material && mat_inner_cost < bare_inner_cost)
+ else if ((extra->jsa_mask & JSA_MERGEJOIN_MATERIALIZE) != 0 &&
+ (mat_inner_cost < bare_inner_cost ||
+ (extra->jsa_mask & JSA_MERGEJOIN_PLAIN) == 0))
path->materialize_inner = true;
/*
- * Even if materializing doesn't look cheaper, we *must* do it if the
- * inner path is to be used directly (without sorting) and it doesn't
- * support mark/restore.
+ * Regardless of what plan shapes are enabled and what the costs seem
+ * to be, we *must* materialize it if the inner path is to be used directly
+ * (without sorting) and it doesn't support mark/restore. Planner failure
+ * is not an option!
*
* Since the inner side must be ordered, and only Sorts and IndexScans can
* create order to begin with, and they both support mark/restore, you
@@ -4005,10 +4011,6 @@ final_cost_mergejoin(PlannerInfo *root, MergePath *path,
* merge joins can *preserve* the order of their inputs, so they can be
* selected as the input of a mergejoin, and they don't support
* mark/restore at present.
- *
- * We don't test the value of enable_material here, because
- * materialization is required for correctness in this case, and turning
- * it off does not entitle us to deliver an invalid plan.
*/
else if (innersortkeys == NIL &&
!ExecSupportsMarkRestore(inner_path))
@@ -4025,7 +4027,8 @@ final_cost_mergejoin(PlannerInfo *root, MergePath *path,
* rather than necessary for correctness, we skip it if enable_material is
* off.
*/
- else if (enable_material && innersortkeys != NIL &&
+ else if ((extra->jsa_mask & JSA_MERGEJOIN_MATERIALIZE) != 0 &&
+ innersortkeys != NIL &&
relation_byte_size(inner_path_rows,
inner_path->pathtarget->width) >
(work_mem * 1024L))
@@ -4033,11 +4036,25 @@ final_cost_mergejoin(PlannerInfo *root, MergePath *path,
else
path->materialize_inner = false;
- /* Charge the right incremental cost for the chosen case */
+ /* Get the number of disabled nodes, not yet including this one. */
+ path->jpath.path.disabled_nodes = workspace->disabled_nodes;
+
+ /*
+ * Charge the right incremental cost for the chosen case, and increment
+ * disabled_nodes if appropriate.
+ */
if (path->materialize_inner)
+ {
run_cost += mat_inner_cost;
+ if ((extra->jsa_mask & JSA_MERGEJOIN_MATERIALIZE) == 0)
+ ++path->jpath.path.disabled_nodes;
+ }
else
+ {
run_cost += bare_inner_cost;
+ if ((extra->jsa_mask & JSA_MERGEJOIN_PLAIN) == 0)
+ ++path->jpath.path.disabled_nodes;
+ }
/* CPU costs */
@@ -4177,7 +4194,7 @@ initial_cost_hashjoin(PlannerInfo *root, JoinCostWorkspace *workspace,
size_t space_allowed; /* unused */
/* Count up disabled nodes. */
- disabled_nodes = enable_hashjoin ? 0 : 1;
+ disabled_nodes = (extra->jsa_mask & JSA_HASHJOIN) == 0 ? 1 : 0;
disabled_nodes += inner_path->disabled_nodes;
disabled_nodes += outer_path->disabled_nodes;
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index a244300463c..90f3829d04d 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -28,8 +28,9 @@
#include "utils/lsyscache.h"
#include "utils/typcache.h"
-/* Hook for plugins to get control in add_paths_to_joinrel() */
+/* Hooks for plugins to get control in add_paths_to_joinrel() */
set_join_pathlist_hook_type set_join_pathlist_hook = NULL;
+join_path_setup_hook_type join_path_setup_hook = NULL;
/*
* Paths parameterized by a parent rel can be considered to be parameterized
@@ -129,7 +130,8 @@ add_paths_to_joinrel(PlannerInfo *root,
RelOptInfo *innerrel,
JoinType jointype,
SpecialJoinInfo *sjinfo,
- List *restrictlist)
+ List *restrictlist,
+ unsigned jsa_mask)
{
JoinPathExtraData extra;
bool mergejoin_allowed = true;
@@ -152,6 +154,7 @@ add_paths_to_joinrel(PlannerInfo *root,
extra.mergeclause_list = NIL;
extra.sjinfo = sjinfo;
extra.param_source_rels = NULL;
+ extra.jsa_mask = jsa_mask;
/*
* See if the inner relation is provably unique for this outer rel.
@@ -203,13 +206,39 @@ add_paths_to_joinrel(PlannerInfo *root,
break;
}
+ /*
+ * Give extensions a chance to take control. In particular, an extension
+ * might want to modify extra.jsa_mask so as to provide join strategy
+ * advice. An extension can also override jsa_mask on a query-wide basis
+ * by using join_search_hook, but extensions that want to provide
+ * different advice when joining different rels, or even different advice
+ * for the same joinrel based on the choice of innerrel and outerrel, need
+ * to use this hook.
+ *
+ * A very simple way for an extension to use this hook is to set
+ * extra.jsa_mask = 0, if it simply doesn't want any of the paths
+ * generated by this call to add_paths_to_joinrel() to be selected. An
+ * extension could use this technique to constrain the join order, since
+ * it could thereby arrange to reject all paths from join orders that it
+ * does not like. An extension can also selectively clear bits from
+ * extra.jsa_mask to rule out specific techniques for specific joins, or
+ * even replace the mask entirely.
+ *
+ * NB: Below this point, this function should be careful to reference only
+ * extra.jsa_mask, and not jsa_mask directly, to avoid disregarding any
+ * changes made by the hook we're about to call.
+ */
+ if (join_path_setup_hook)
+ join_path_setup_hook(root, joinrel, outerrel, innerrel,
+ jointype, &extra);
+
/*
* Find potential mergejoin clauses. We can skip this if we are not
* interested in doing a mergejoin. However, mergejoin may be our only
- * way of implementing a full outer join, so override enable_mergejoin if
- * it's a full join.
+ * way of implementing a full outer join, so in that case we don't care
+ * whether mergejoins are disabled.
*/
- if (enable_mergejoin || jointype == JOIN_FULL)
+ if ((extra.jsa_mask & JSA_MERGEJOIN_ANY) != 0 || jointype == JOIN_FULL)
extra.mergeclause_list = select_mergejoin_clauses(root,
joinrel,
outerrel,
@@ -317,10 +346,10 @@ add_paths_to_joinrel(PlannerInfo *root,
/*
* 4. Consider paths where both outer and inner relations must be hashed
- * before being joined. As above, disregard enable_hashjoin for full
- * joins, because there may be no other alternative.
+ * before being joined. As above, when it's a full join, we must try this
+ * even when the path type is disabled, because it may be our only option.
*/
- if (enable_hashjoin || jointype == JOIN_FULL)
+ if ((extra.jsa_mask & JSA_HASHJOIN) != 0 || jointype == JOIN_FULL)
hash_inner_and_outer(root, joinrel, outerrel, innerrel,
jointype, &extra);
@@ -329,7 +358,7 @@ add_paths_to_joinrel(PlannerInfo *root,
* to the same server and assigned to the same user to check access
* permissions as, give the FDW a chance to push down joins.
*/
- if (joinrel->fdwroutine &&
+ if ((extra.jsa_mask & JSA_FOREIGN) != 0 && joinrel->fdwroutine &&
joinrel->fdwroutine->GetForeignJoinPaths)
joinrel->fdwroutine->GetForeignJoinPaths(root, joinrel,
outerrel, innerrel,
@@ -338,8 +367,13 @@ add_paths_to_joinrel(PlannerInfo *root,
/*
* 6. Finally, give extensions a chance to manipulate the path list. They
* could add new paths (such as CustomPaths) by calling add_path(), or
- * add_partial_path() if parallel aware. They could also delete or modify
- * paths added by the core code.
+ * add_partial_path() if parallel aware.
+ *
+ * In theory, extensions could also use this hook to delete or modify
+ * paths added by the core code, but in practice this is difficult to make
+ * work, since it's too late to get back any paths that have already been
+ * discarded by add_path() or add_partial_path(). If you're trying to
+ * suppress paths, consider using join_path_setup_hook instead.
*/
if (set_join_pathlist_hook)
set_join_pathlist_hook(root, joinrel, outerrel, innerrel,
@@ -829,6 +863,7 @@ try_nestloop_path(PlannerInfo *root,
Path *inner_path,
List *pathkeys,
JoinType jointype,
+ unsigned nestloop_subtype,
JoinPathExtraData *extra)
{
Relids required_outer;
@@ -913,7 +948,7 @@ try_nestloop_path(PlannerInfo *root,
* The latter two steps are expensive enough to make this two-phase
* methodology worthwhile.
*/
- initial_cost_nestloop(root, &workspace, jointype,
+ initial_cost_nestloop(root, &workspace, jointype, nestloop_subtype,
outer_path, inner_path, extra);
if (add_path_precheck(joinrel, workspace.disabled_nodes,
@@ -951,6 +986,7 @@ try_partial_nestloop_path(PlannerInfo *root,
Path *inner_path,
List *pathkeys,
JoinType jointype,
+ unsigned nestloop_subtype,
JoinPathExtraData *extra)
{
JoinCostWorkspace workspace;
@@ -998,7 +1034,7 @@ try_partial_nestloop_path(PlannerInfo *root,
* Before creating a path, get a quick lower bound on what it is likely to
* cost. Bail out right away if it looks terrible.
*/
- initial_cost_nestloop(root, &workspace, jointype,
+ initial_cost_nestloop(root, &workspace, jointype, nestloop_subtype,
outer_path, inner_path, extra);
if (!add_partial_path_precheck(joinrel, workspace.disabled_nodes,
workspace.total_cost, pathkeys))
@@ -1893,20 +1929,21 @@ match_unsorted_outer(PlannerInfo *root,
if (inner_cheapest_total == NULL)
return;
inner_cheapest_total = (Path *)
- create_unique_path(root, innerrel, inner_cheapest_total, extra->sjinfo);
+ create_unique_path(root, innerrel, inner_cheapest_total,
+ extra->sjinfo);
Assert(inner_cheapest_total);
}
else if (nestjoinOK)
{
/*
- * Consider materializing the cheapest inner path, unless
- * enable_material is off or the path in question materializes its
- * output anyway.
+ * Consider materializing the cheapest inner path, unless that is
+ * disabled or the path in question materializes its output anyway.
*/
- if (enable_material && inner_cheapest_total != NULL &&
+ if ((extra->jsa_mask & JSA_NESTLOOP_MATERIALIZE) != 0 &&
+ inner_cheapest_total != NULL &&
!ExecMaterializesOutput(inner_cheapest_total->pathtype))
matpath = (Path *)
- create_material_path(innerrel, inner_cheapest_total);
+ create_material_path(innerrel, inner_cheapest_total, true);
}
foreach(lc1, outerrel->pathlist)
@@ -1954,6 +1991,7 @@ match_unsorted_outer(PlannerInfo *root,
inner_cheapest_total,
merge_pathkeys,
jointype,
+ JSA_NESTLOOP_PLAIN,
extra);
}
else if (nestjoinOK)
@@ -1977,6 +2015,7 @@ match_unsorted_outer(PlannerInfo *root,
innerpath,
merge_pathkeys,
jointype,
+ JSA_NESTLOOP_PLAIN,
extra);
/*
@@ -1993,6 +2032,7 @@ match_unsorted_outer(PlannerInfo *root,
mpath,
merge_pathkeys,
jointype,
+ JSA_NESTLOOP_MEMOIZE,
extra);
}
@@ -2004,6 +2044,7 @@ match_unsorted_outer(PlannerInfo *root,
matpath,
merge_pathkeys,
jointype,
+ JSA_NESTLOOP_MATERIALIZE,
extra);
}
@@ -2135,20 +2176,18 @@ consider_parallel_nestloop(PlannerInfo *root,
/*
* Consider materializing the cheapest inner path, unless: 1) we're doing
* JOIN_UNIQUE_INNER, because in this case we have to unique-ify the
- * cheapest inner path, 2) enable_material is off, 3) the cheapest inner
- * path is not parallel-safe, 4) the cheapest inner path is parameterized
- * by the outer rel, or 5) the cheapest inner path materializes its output
- * anyway.
+ * cheapest inner path, 2) materialization is disabled here, 3) the
+ * cheapest inner path is not parallel-safe, 4) the cheapest inner path is
+ * parameterized by the outer rel, or 5) the cheapest inner path
+ * materializes its output anyway.
*/
if (save_jointype != JOIN_UNIQUE_INNER &&
- enable_material && inner_cheapest_total->parallel_safe &&
+ (extra->jsa_mask & JSA_NESTLOOP_MATERIALIZE) != 0 &&
+ inner_cheapest_total->parallel_safe &&
!PATH_PARAM_BY_REL(inner_cheapest_total, outerrel) &&
!ExecMaterializesOutput(inner_cheapest_total->pathtype))
- {
matpath = (Path *)
- create_material_path(innerrel, inner_cheapest_total);
- Assert(matpath->parallel_safe);
- }
+ create_material_path(innerrel, inner_cheapest_total, true);
foreach(lc1, outerrel->partial_pathlist)
{
@@ -2193,7 +2232,8 @@ consider_parallel_nestloop(PlannerInfo *root,
}
try_partial_nestloop_path(root, joinrel, outerpath, innerpath,
- pathkeys, jointype, extra);
+ pathkeys, jointype, JSA_NESTLOOP_PLAIN,
+ extra);
/*
* Try generating a memoize path and see if that makes the nested
@@ -2204,13 +2244,15 @@ consider_parallel_nestloop(PlannerInfo *root,
extra);
if (mpath != NULL)
try_partial_nestloop_path(root, joinrel, outerpath, mpath,
- pathkeys, jointype, extra);
+ pathkeys, jointype,
+ JSA_NESTLOOP_MEMOIZE, extra);
}
/* Also consider materialized form of the cheapest inner path */
if (matpath != NULL)
try_partial_nestloop_path(root, joinrel, outerpath, matpath,
- pathkeys, jointype, extra);
+ pathkeys, jointype,
+ JSA_NESTLOOP_MATERIALIZE, extra);
}
}
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 7db5e30eef8..dad5c555365 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -26,10 +26,12 @@
static void make_rels_by_clause_joins(PlannerInfo *root,
RelOptInfo *old_rel,
List *other_rels,
- int first_rel_idx);
+ int first_rel_idx,
+ unsigned jsa_mask);
static void make_rels_by_clauseless_joins(PlannerInfo *root,
RelOptInfo *old_rel,
- List *other_rels);
+ List *other_rels,
+ unsigned jsa_mask);
static bool has_join_restriction(PlannerInfo *root, RelOptInfo *rel);
static bool has_legal_joinclause(PlannerInfo *root, RelOptInfo *rel);
static bool restriction_is_constant_false(List *restrictlist,
@@ -37,11 +39,13 @@ static bool restriction_is_constant_false(List *restrictlist,
bool only_pushed_down);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
- SpecialJoinInfo *sjinfo, List *restrictlist);
+ SpecialJoinInfo *sjinfo,
+ List *restrictlist, unsigned jsa_mask);
static void try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *parent_sjinfo,
- List *parent_restrictlist);
+ List *parent_restrictlist,
+ unsigned jsa_mask);
static SpecialJoinInfo *build_child_join_sjinfo(PlannerInfo *root,
SpecialJoinInfo *parent_sjinfo,
Relids left_relids, Relids right_relids);
@@ -69,7 +73,7 @@ static void get_matching_part_pairs(PlannerInfo *root, RelOptInfo *joinrel,
* The result is returned in root->join_rel_level[level].
*/
void
-join_search_one_level(PlannerInfo *root, int level)
+join_search_one_level(PlannerInfo *root, int level, unsigned jsa_mask)
{
List **joinrels = root->join_rel_level;
ListCell *r;
@@ -114,7 +118,8 @@ join_search_one_level(PlannerInfo *root, int level)
else
first_rel = 0;
- make_rels_by_clause_joins(root, old_rel, joinrels[1], first_rel);
+ make_rels_by_clause_joins(root, old_rel, joinrels[1], first_rel,
+ jsa_mask);
}
else
{
@@ -132,7 +137,8 @@ join_search_one_level(PlannerInfo *root, int level)
*/
make_rels_by_clauseless_joins(root,
old_rel,
- joinrels[1]);
+ joinrels[1],
+ jsa_mask);
}
}
@@ -189,7 +195,7 @@ join_search_one_level(PlannerInfo *root, int level)
if (have_relevant_joinclause(root, old_rel, new_rel) ||
have_join_order_restriction(root, old_rel, new_rel))
{
- (void) make_join_rel(root, old_rel, new_rel);
+ (void) make_join_rel(root, old_rel, new_rel, jsa_mask);
}
}
}
@@ -227,7 +233,8 @@ join_search_one_level(PlannerInfo *root, int level)
make_rels_by_clauseless_joins(root,
old_rel,
- joinrels[1]);
+ joinrels[1],
+ jsa_mask);
}
/*----------
@@ -279,7 +286,8 @@ static void
make_rels_by_clause_joins(PlannerInfo *root,
RelOptInfo *old_rel,
List *other_rels,
- int first_rel_idx)
+ int first_rel_idx,
+ unsigned jsa_mask)
{
ListCell *l;
@@ -291,7 +299,7 @@ make_rels_by_clause_joins(PlannerInfo *root,
(have_relevant_joinclause(root, old_rel, other_rel) ||
have_join_order_restriction(root, old_rel, other_rel)))
{
- (void) make_join_rel(root, old_rel, other_rel);
+ (void) make_join_rel(root, old_rel, other_rel, jsa_mask);
}
}
}
@@ -312,7 +320,8 @@ make_rels_by_clause_joins(PlannerInfo *root,
static void
make_rels_by_clauseless_joins(PlannerInfo *root,
RelOptInfo *old_rel,
- List *other_rels)
+ List *other_rels,
+ unsigned jsa_mask)
{
ListCell *l;
@@ -322,7 +331,7 @@ make_rels_by_clauseless_joins(PlannerInfo *root,
if (!bms_overlap(other_rel->relids, old_rel->relids))
{
- (void) make_join_rel(root, old_rel, other_rel);
+ (void) make_join_rel(root, old_rel, other_rel, jsa_mask);
}
}
}
@@ -701,7 +710,8 @@ init_dummy_sjinfo(SpecialJoinInfo *sjinfo, Relids left_relids,
* turned into joins.
*/
RelOptInfo *
-make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
+ unsigned jsa_mask)
{
Relids joinrelids;
SpecialJoinInfo *sjinfo;
@@ -759,7 +769,7 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
*/
joinrel = build_join_rel(root, joinrelids, rel1, rel2,
sjinfo, pushed_down_joins,
- &restrictlist);
+ &restrictlist, jsa_mask);
/*
* If we've already proven this join is empty, we needn't consider any
@@ -773,7 +783,7 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
/* Add paths to the join relation. */
populate_joinrel_with_paths(root, rel1, rel2, joinrel, sjinfo,
- restrictlist);
+ restrictlist, jsa_mask);
bms_free(joinrelids);
@@ -892,7 +902,8 @@ add_outer_joins_to_relids(PlannerInfo *root, Relids input_relids,
static void
populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
- SpecialJoinInfo *sjinfo, List *restrictlist)
+ SpecialJoinInfo *sjinfo, List *restrictlist,
+ unsigned jsa_mask)
{
/*
* Consider paths using each rel as both outer and inner. Depending on
@@ -923,10 +934,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_INNER, sjinfo,
- restrictlist);
+ restrictlist, jsa_mask);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_INNER, sjinfo,
- restrictlist);
+ restrictlist, jsa_mask);
break;
case JOIN_LEFT:
if (is_dummy_rel(rel1) ||
@@ -940,10 +951,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_LEFT, sjinfo,
- restrictlist);
+ restrictlist, jsa_mask);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_RIGHT, sjinfo,
- restrictlist);
+ restrictlist, jsa_mask);
break;
case JOIN_FULL:
if ((is_dummy_rel(rel1) && is_dummy_rel(rel2)) ||
@@ -954,10 +965,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_FULL, sjinfo,
- restrictlist);
+ restrictlist, jsa_mask);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_FULL, sjinfo,
- restrictlist);
+ restrictlist, jsa_mask);
/*
* If there are join quals that aren't mergeable or hashable, we
@@ -990,10 +1001,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_SEMI, sjinfo,
- restrictlist);
+ restrictlist, jsa_mask);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_RIGHT_SEMI, sjinfo,
- restrictlist);
+ restrictlist, jsa_mask);
}
/*
@@ -1016,10 +1027,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_UNIQUE_INNER, sjinfo,
- restrictlist);
+ restrictlist, jsa_mask);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_UNIQUE_OUTER, sjinfo,
- restrictlist);
+ restrictlist, jsa_mask);
}
break;
case JOIN_ANTI:
@@ -1034,10 +1045,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_ANTI, sjinfo,
- restrictlist);
+ restrictlist, jsa_mask);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_RIGHT_ANTI, sjinfo,
- restrictlist);
+ restrictlist, jsa_mask);
break;
default:
/* other values not expected here */
@@ -1046,7 +1057,8 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
}
/* Apply partitionwise join technique, if possible. */
- try_partitionwise_join(root, rel1, rel2, joinrel, sjinfo, restrictlist);
+ try_partitionwise_join(root, rel1, rel2, joinrel, sjinfo, restrictlist,
+ jsa_mask);
}
@@ -1480,7 +1492,7 @@ restriction_is_constant_false(List *restrictlist,
static void
try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
RelOptInfo *joinrel, SpecialJoinInfo *parent_sjinfo,
- List *parent_restrictlist)
+ List *parent_restrictlist, unsigned jsa_mask)
{
bool rel1_is_simple = IS_SIMPLE_REL(rel1);
bool rel2_is_simple = IS_SIMPLE_REL(rel2);
@@ -1662,7 +1674,8 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
{
child_joinrel = build_child_join_rel(root, child_rel1, child_rel2,
joinrel, child_restrictlist,
- child_sjinfo, nappinfos, appinfos);
+ child_sjinfo, nappinfos,
+ appinfos, jsa_mask);
joinrel->part_rels[cnt_parts] = child_joinrel;
joinrel->live_parts = bms_add_member(joinrel->live_parts, cnt_parts);
joinrel->all_partrels = bms_add_members(joinrel->all_partrels,
@@ -1677,7 +1690,7 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
/* And make paths for the child join */
populate_joinrel_with_paths(root, child_rel1, child_rel2,
child_joinrel, child_sjinfo,
- child_restrictlist);
+ child_restrictlist, jsa_mask);
/*
* When there are thousands of partitions involved, this loop will
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 0d195a07ffc..1669e4b8b03 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -6623,6 +6623,7 @@ materialize_finished_plan(Plan *subplan)
/* Set cost data */
cost_material(&matpath,
+ enable_material,
subplan->disabled_nodes,
subplan->startup_cost,
subplan->total_cost,
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 0f423e96847..7c1000879ec 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -419,6 +419,27 @@ standard_planner(Query *parse, const char *query_string, int cursorOptions,
tuple_fraction = 0.0;
}
+ /* Compute the initial join strategy advice mask. */
+ glob->default_jsa_mask = JSA_FOREIGN;
+ if (enable_hashjoin)
+ glob->default_jsa_mask |= JSA_HASHJOIN;
+ if (enable_mergejoin)
+ {
+ glob->default_jsa_mask |= JSA_MERGEJOIN_PLAIN;
+ if (enable_material)
+ glob->default_jsa_mask |= JSA_MERGEJOIN_MATERIALIZE;
+ }
+ if (enable_nestloop)
+ {
+ glob->default_jsa_mask |= JSA_NESTLOOP_PLAIN;
+ if (enable_material)
+ glob->default_jsa_mask |= JSA_NESTLOOP_MATERIALIZE;
+ if (enable_memoize)
+ glob->default_jsa_mask |= JSA_NESTLOOP_MEMOIZE;
+ }
+ if (enable_partitionwise_join)
+ glob->default_jsa_mask |= JSA_PARTITIONWISE;
+
/* primary planning entry point (may recurse for subqueries) */
root = subquery_planner(glob, parse, NULL, false, tuple_fraction, NULL);
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index fc97bf6ee26..a704d0cd7bb 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -1631,7 +1631,7 @@ create_group_result_path(PlannerInfo *root, RelOptInfo *rel,
* pathnode.
*/
MaterialPath *
-create_material_path(RelOptInfo *rel, Path *subpath)
+create_material_path(RelOptInfo *rel, Path *subpath, bool enabled)
{
MaterialPath *pathnode = makeNode(MaterialPath);
@@ -1650,6 +1650,7 @@ create_material_path(RelOptInfo *rel, Path *subpath)
pathnode->subpath = subpath;
cost_material(&pathnode->path,
+ enabled,
subpath->disabled_nodes,
subpath->startup_cost,
subpath->total_cost,
@@ -4158,7 +4159,8 @@ reparameterize_path(PlannerInfo *root, Path *path,
loop_count);
if (spath == NULL)
return NULL;
- return (Path *) create_material_path(rel, spath);
+ return (Path *) create_material_path(rel, spath,
+ enable_material);
}
case T_Memoize:
{
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index d7266e4cdba..9e328c5ac7c 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -69,7 +69,8 @@ static void build_joinrel_partition_info(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
- List *restrictlist);
+ List *restrictlist,
+ unsigned jsa_mask);
static bool have_partkey_equi_join(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *rel1, RelOptInfo *rel2,
JoinType jointype, List *restrictlist);
@@ -668,7 +669,8 @@ build_join_rel(PlannerInfo *root,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
List *pushed_down_joins,
- List **restrictlist_ptr)
+ List **restrictlist_ptr,
+ unsigned jsa_mask)
{
RelOptInfo *joinrel;
List *restrictlist;
@@ -817,7 +819,7 @@ build_join_rel(PlannerInfo *root,
/* Store the partition information. */
build_joinrel_partition_info(root, joinrel, outer_rel, inner_rel, sjinfo,
- restrictlist);
+ restrictlist, jsa_mask);
/*
* Set estimates of the joinrel's size.
@@ -882,7 +884,8 @@ RelOptInfo *
build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
RelOptInfo *inner_rel, RelOptInfo *parent_joinrel,
List *restrictlist, SpecialJoinInfo *sjinfo,
- int nappinfos, AppendRelInfo **appinfos)
+ int nappinfos, AppendRelInfo **appinfos,
+ unsigned jsa_mask)
{
RelOptInfo *joinrel = makeNode(RelOptInfo);
@@ -981,7 +984,7 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
/* Is the join between partitions itself partitioned? */
build_joinrel_partition_info(root, joinrel, outer_rel, inner_rel, sjinfo,
- restrictlist);
+ restrictlist, jsa_mask);
/* Child joinrel is parallel safe if parent is parallel safe. */
joinrel->consider_parallel = parent_joinrel->consider_parallel;
@@ -2005,12 +2008,12 @@ static void
build_joinrel_partition_info(PlannerInfo *root,
RelOptInfo *joinrel, RelOptInfo *outer_rel,
RelOptInfo *inner_rel, SpecialJoinInfo *sjinfo,
- List *restrictlist)
+ List *restrictlist, unsigned jsa_mask)
{
PartitionScheme part_scheme;
/* Nothing to do if partitionwise join technique is disabled. */
- if (!enable_partitionwise_join)
+ if ((jsa_mask & JSA_PARTITIONWISE) == 0)
{
Assert(!IS_PARTITIONED_REL(joinrel));
return;
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 07e2415398e..ca328d6edad 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -161,6 +161,9 @@ typedef struct PlannerGlobal
/* worst PROPARALLEL hazard level */
char maxParallelHazard;
+ /* default join strategy advice, except where overrriden by hooks */
+ uint32 default_jsa_mask;
+
/* partition descriptors */
PartitionDirectory partition_directory pg_node_attr(read_write_ignore);
} PlannerGlobal;
@@ -3231,6 +3234,7 @@ typedef struct SemiAntiJoinFactors
* sjinfo is extra info about special joins for selectivity estimation
* semifactors is as shown above (only valid for SEMI/ANTI/inner_unique joins)
* param_source_rels are OK targets for parameterization of result paths
+ * jsa_mask is a bitmask of JSA_* constants to direct the join strategy
*/
typedef struct JoinPathExtraData
{
@@ -3240,6 +3244,7 @@ typedef struct JoinPathExtraData
SpecialJoinInfo *sjinfo;
SemiAntiJoinFactors semifactors;
Relids param_source_rels;
+ unsigned jsa_mask;
} JoinPathExtraData;
/*
diff --git a/src/include/optimizer/cost.h b/src/include/optimizer/cost.h
index 854a782944a..071a8749cfa 100644
--- a/src/include/optimizer/cost.h
+++ b/src/include/optimizer/cost.h
@@ -125,7 +125,7 @@ extern void cost_merge_append(Path *path, PlannerInfo *root,
Cost input_startup_cost, Cost input_total_cost,
double tuples);
extern void cost_material(Path *path,
- int input_disabled_nodes,
+ bool enabled, int input_disabled_nodes,
Cost input_startup_cost, Cost input_total_cost,
double tuples, int width);
extern void cost_agg(Path *path, PlannerInfo *root,
@@ -148,7 +148,7 @@ extern void cost_group(Path *path, PlannerInfo *root,
double input_tuples);
extern void initial_cost_nestloop(PlannerInfo *root,
JoinCostWorkspace *workspace,
- JoinType jointype,
+ JoinType jointype, unsigned nestloop_subtype,
Path *outer_path, Path *inner_path,
JoinPathExtraData *extra);
extern void final_cost_nestloop(PlannerInfo *root, NestPath *path,
diff --git a/src/include/optimizer/geqo.h b/src/include/optimizer/geqo.h
index c52906d0916..a9d14b07aa1 100644
--- a/src/include/optimizer/geqo.h
+++ b/src/include/optimizer/geqo.h
@@ -81,10 +81,13 @@ typedef struct
/* routines in geqo_main.c */
extern RelOptInfo *geqo(PlannerInfo *root,
- int number_of_rels, List *initial_rels);
+ int number_of_rels, List *initial_rels,
+ unsigned jsa_mask);
/* routines in geqo_eval.c */
-extern Cost geqo_eval(PlannerInfo *root, Gene *tour, int num_gene);
-extern RelOptInfo *gimme_tree(PlannerInfo *root, Gene *tour, int num_gene);
+extern Cost geqo_eval(PlannerInfo *root, Gene *tour, int num_gene,
+ unsigned jsa_mask);
+extern RelOptInfo *gimme_tree(PlannerInfo *root, Gene *tour, int num_gene,
+ unsigned jsa_mask);
#endif /* GEQO_H */
diff --git a/src/include/optimizer/geqo_pool.h b/src/include/optimizer/geqo_pool.h
index b5e80554724..bd1ed152907 100644
--- a/src/include/optimizer/geqo_pool.h
+++ b/src/include/optimizer/geqo_pool.h
@@ -29,7 +29,7 @@
extern Pool *alloc_pool(PlannerInfo *root, int pool_size, int string_length);
extern void free_pool(PlannerInfo *root, Pool *pool);
-extern void random_init_pool(PlannerInfo *root, Pool *pool);
+extern void random_init_pool(PlannerInfo *root, Pool *pool, unsigned jsa_mask);
extern Chromosome *alloc_chromo(PlannerInfo *root, int string_length);
extern void free_chromo(PlannerInfo *root, Chromosome *chromo);
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 1035e6560c1..ed2e3a1c8b8 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -82,7 +82,8 @@ extern GroupResultPath *create_group_result_path(PlannerInfo *root,
RelOptInfo *rel,
PathTarget *target,
List *havingqual);
-extern MaterialPath *create_material_path(RelOptInfo *rel, Path *subpath);
+extern MaterialPath *create_material_path(RelOptInfo *rel, Path *subpath,
+ bool enabled);
extern MemoizePath *create_memoize_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
@@ -324,7 +325,8 @@ extern RelOptInfo *build_join_rel(PlannerInfo *root,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
List *pushed_down_joins,
- List **restrictlist_ptr);
+ List **restrictlist_ptr,
+ unsigned jsa_mask);
extern Relids min_join_parameterization(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
@@ -351,6 +353,7 @@ extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
SpecialJoinInfo *sjinfo,
- int nappinfos, AppendRelInfo **appinfos);
+ int nappinfos, AppendRelInfo **appinfos,
+ unsigned jsa_mask);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
index 54869d44013..f9f346f86a1 100644
--- a/src/include/optimizer/paths.h
+++ b/src/include/optimizer/paths.h
@@ -16,10 +16,44 @@
#include "nodes/pathnodes.h"
+/*
+ * Join strategy advice.
+ *
+ * Paths that don't match the join strategy advice will be either be disabled
+ * or will not be generated in the first place. It's only permissible to skip
+ * generating a path if doing so can't result in planner failure. The initial
+ * mask is computed on the basis of the various enable_* GUCs, and can be
+ * overriden by hooks.
+ *
+ * We have five main join strategies: a foreign join (when supported by the
+ * relevant FDW), a merge join, a nested loop, a hash join, and a partitionwise
+ * join. Merge joins are further subdivided based on whether the inner side
+ * is materialized, and nested loops are further subdivided based on whether
+ * the inner side is materialized, memoized, or neither. "Plain" means a
+ * strategy where neither materialization nor memoization is used.
+ *
+ * If you don't care whether materialization or memoization is used, set all
+ * the bits for the relevant major join strategy. If you do care, just set the
+ * subset of bits that correspond to the cases you want to allow.
+ */
+#define JSA_FOREIGN 0x0001
+#define JSA_MERGEJOIN_PLAIN 0x0002
+#define JSA_MERGEJOIN_MATERIALIZE 0x0004
+#define JSA_NESTLOOP_PLAIN 0x0008
+#define JSA_NESTLOOP_MATERIALIZE 0x0010
+#define JSA_NESTLOOP_MEMOIZE 0x0020
+#define JSA_HASHJOIN 0x0040
+#define JSA_PARTITIONWISE 0x0080
+
+#define JSA_MERGEJOIN_ANY \
+ (JSA_MERGEJOIN_PLAIN | JSA_MERGEJOIN_MATERIALIZE)
+#define JSA_NESTLOOP_ANY \
+ (JSA_NESTLOOP_PLAIN | JSA_NESTLOOP_MATERIALIZE | JSA_NESTLOOP_MEMOIZE)
/*
* allpaths.c
*/
+
extern PGDLLIMPORT bool enable_geqo;
extern PGDLLIMPORT int geqo_threshold;
extern PGDLLIMPORT int min_parallel_table_scan_size;
@@ -33,7 +67,14 @@ typedef void (*set_rel_pathlist_hook_type) (PlannerInfo *root,
RangeTblEntry *rte);
extern PGDLLIMPORT set_rel_pathlist_hook_type set_rel_pathlist_hook;
-/* Hook for plugins to get control in add_paths_to_joinrel() */
+/* Hooks for plugins to get control in add_paths_to_joinrel() */
+typedef void (*join_path_setup_hook_type) (PlannerInfo *root,
+ RelOptInfo *joinrel,
+ RelOptInfo *outerrel,
+ RelOptInfo *innerrel,
+ JoinType jointype,
+ JoinPathExtraData *extra);
+extern PGDLLIMPORT join_path_setup_hook_type join_path_setup_hook;
typedef void (*set_join_pathlist_hook_type) (PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
@@ -45,13 +86,14 @@ extern PGDLLIMPORT set_join_pathlist_hook_type set_join_pathlist_hook;
/* Hook for plugins to replace standard_join_search() */
typedef RelOptInfo *(*join_search_hook_type) (PlannerInfo *root,
int levels_needed,
- List *initial_rels);
+ List *initial_rels,
+ unsigned jsa_mask);
extern PGDLLIMPORT join_search_hook_type join_search_hook;
extern RelOptInfo *make_one_rel(PlannerInfo *root, List *joinlist);
extern RelOptInfo *standard_join_search(PlannerInfo *root, int levels_needed,
- List *initial_rels);
+ List *initial_rels, unsigned jsa_mask);
extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel,
bool override_rows);
@@ -92,15 +134,17 @@ extern bool create_tidscan_paths(PlannerInfo *root, RelOptInfo *rel);
extern void add_paths_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, SpecialJoinInfo *sjinfo,
- List *restrictlist);
+ List *restrictlist, unsigned jsa_mask);
/*
* joinrels.c
* routines to determine which relations to join
*/
-extern void join_search_one_level(PlannerInfo *root, int level);
+extern void join_search_one_level(PlannerInfo *root, int level,
+ unsigned jsa_mask);
extern RelOptInfo *make_join_rel(PlannerInfo *root,
- RelOptInfo *rel1, RelOptInfo *rel2);
+ RelOptInfo *rel1, RelOptInfo *rel2,
+ unsigned jsa_mask);
extern Relids add_outer_joins_to_relids(PlannerInfo *root, Relids input_relids,
SpecialJoinInfo *sjinfo,
List **pushed_down_joins);
--
2.39.3 (Apple Git-145)
On 10/10/24 23:51, Robert Haas wrote:
On Wed, Sep 18, 2024 at 11:48 AM Robert Haas <robertmhaas@gmail.com> wrote:
1. If you want to specify in-query hints using comments, how does your
extension get access to the comments?
Having designed two features [1,2] that do the stuff mostly similar to
pg_hint_plan but based on real cardinalities earned from previous
executions, I can say the most annoying problem is hinting subqueries &
CTEs. Sometimes, you want to hint at the top query and not touch the
subquery and vice versa. Sometimes, users get stuck in accidents when a
flattened subquery influences your hint. So, the key property to invent
in advance should be an identification system.
I have chosen hash-based identification when each RelOptInfo has a hash
created over values of relids, restrictions and hashes of underlying
RelOptInfos. At the core, it seems inappropriate. But anyway, some
additional info must be pushed into the subquery to allow it to identify
itself and decide which hint can be used.
[1]: https://github.com/postgrespro/aqo
[2]: https://postgrespro.com/docs/enterprise/16/realtime-query-replanning
--
regards, Andrei Lepikhov
Hi Robert,
On Thu, Oct 10, 2024 at 6:52 PM Robert Haas <robertmhaas@gmail.com> wrote:
On Wed, Sep 18, 2024 at 11:48 AM Robert Haas <robertmhaas@gmail.com>
wrote:Still, I think it's a pretty useful starting point. It is mostly
enough to give you control over join planning, and if combined with
similar work for scan planning, I think it would be enough for
pg_hint_plan. If we also got control over appendrel and agg planning,
then you could do a bunch more cool things.Here's a new set of patches where I added a similar mechanism for scan
type control. See the commit message for some notes on limitations of
this approach. In the absence of better ideas, I'd like to proceed
with something along the lines of 0001 and 0002.I upgraded the hint_via_alias contrib module (which is not intended
for commit, it's just a demo) so that it can hint either scan type or
join type. I think this is sufficient to demonstrate that it's quite
easy to use hooks to leverage this new infrastructure.
Thank You! I've played a little and IMHO this is a step in a good direction
after playing a tiny bit with 'hint_via_alias'.
In fact, the
biggest thing I'm unhappy about right now is the difficulty of
providing the hooks with any sort of useful information. I don't think
it should be the job of this patch set to solve that problem, but I do
think we should do something about it. The problem exists on two
levels:1. If you want to specify in-query hints using comments, how does your
extension get access to the comments? [..]Still, it's not clear what other
approach you could
adopt.
No, I don't think the ability to influence optimizers should be tied to SQL
comments as a "vehicle" to transfer some information, so -1 for going into
any discussion about it and wasting Your time on this. Rationale: as you
note such "interface" is quirky, and troublesome even for users. IMHO they
are mostly just used as a way to run experiments, but noone with sense of
touch with reality would ever embed query with hints in the comment section
inside the production application if had other choices, and that's for
multiple reasons (e.g. it's better to have control about it in the DB as
performance is function of time [and pgversion], ORM might be even unable
to do it in the 1st place, people do not have access to the source code).
You could just assume we have "SET
extension.influence_optimizer='SeqScan(t)'" and be done with it as far as
the production-troubleshooting goes. It's better because it's easier to use
(one does not need to even launch an editor to modify the query) during
those experiments. E.g. I would find it much faster to iterate in psql with
a loop of: SET + `\i query.sql` rather than often having dozens of KBs to
re-edit. And there's even a logon trigger today, so it could be (ab)used to
SET that GUC with just some specific conditions (e.g. only for specific
application_name and that could be even forced by e.g. pgjdbc driver --
jdbcurl?ApplicationName=enable_this_workaround_just_here).
Well the issue is however how do you bind such influence to just one
specific query without changing the app in the long run. My take is that we
should utilize compute_query_id (hash) and then extension should allow
doing something along of the lines of mapping (queryId <->
influence_optimizer), or even it could provide `ALTER SQL <queryId> SET
influence_optimizer='SeqScan(t)'`. Users could take that hash from the %Q
in the log_line_prefix.
2. If you want a certain base relation or join relation to be treated
in a certain way, how do you identify it? You might think that this is
easy because, even when a query contains multiple references to a
relation with the same name, or identical aliases in different parts
of the query, EXPLAIN renames them so they have disjoint names. What
would be nice is if you could feed those names back into your
extension and use them as a way of specifying what behavior you want
where. But that doesn't work, because what actually happens is that
the plan can contain duplicated aliases, and then when EXPLAIN
deparses it using ruleutils.c, that's when we rename things so that
they're distinct. This means that, at the time we're planning, we
don't yet know what name EXPLAIN will end up giving to any relation
yet, which means we can't use the names that EXPLAIN produced for an
earlier plan for the same query to associate behaviors with relations.
I wonder if we could think about reversing the order of operations
here and making it so that we do the distinct-ification during parse
analysis or maybe early in planning, so that the name you see EXPLAIN
print out is the real name of that thing and not just a value invented
for display purposes.
This if that's possible?, or simply some counter and numbering the plan
operation? or Andrei's response/idea of using hashing??
So again, I am definitely not saying that these patches get us all the
way to where we should be -- not in terms of the ability to control
the plan, and definitely not in terms of giving extensions all the
information they need to be effective. But if we insist on a perfect
solution before doing anything, we will never get anywhere, and I
personally believe these are going in a useful direction.
Even as it stands today, the v4-0002 would be better to have than nothing
(well other than pg_hint_plan), as the it looks to me that the most
frequent workaround for optimizer issues is to just throw 'enable_nestloop
= no' into the mix quite often (so having the ability to just throw
fixproblem.so into session_preload_libraries with just strstr()/regex() -
to match on specific query - and disable it just there seems to be
completely achievable and has much better granularity when targeting whole
sessions with SET issued there).
-Jakub Wartak.
On Mon, Oct 14, 2024 at 6:02 AM Jakub Wartak
<jakub.wartak@enterprisedb.com> wrote:
I wonder if we could think about reversing the order of operations
here and making it so that we do the distinct-ification during parse
analysis or maybe early in planning, so that the name you see EXPLAIN
print out is the real name of that thing and not just a value invented
for display purposes.This if that's possible?, or simply some counter and numbering the plan operation? or Andrei's response/idea of using hashing??
I spent some time looking at this. I think everything that we might
ever want to do here is possible, but what I unfortunately couldn't
find is a place to do it where it would be more-or-less free.
In parse analysis, we do detect certain kinds of namespace collisions.
For example, "SELECT * FROM x, x" or "SELECT * FROM x, y x" will fail
with ERROR: table name "x" specified more than once. Similarly,
"SELECT * FROM generate_series(1,2), x generate_series" will fail with
ERROR: table name "generate_series" specified more than once, even
though generate_series is not, strictly speaking, a table name. From
this you might infer that each alias can be used only once, but it
turns out that's not entirely correct. If you say "SELECT * FROM x,
q.x" that is totally fine even though both relations end up with the
alias "x". There's a specific exception in the code to allow this
case, when the aliases are the same and but they refer to different
relation OIDs, and this is justified by reference to the SQL standard.
Furthermore, "SELECT * FROM (x JOIN y ON true) x, y" is totally fine
even though there are two x's and two y's. The fact that the
parenthesized part has its own alias makes everything inside the
parentheses a separate scope from the names outside the parentheses,
so there's no name conflict. If you delete the "x" just after the
closing parenthesis then it's all a single scope and you get a
complaint about "y" being used twice. In this example, we're allowed
to have collisions even though there are no subqueries, but it is also
true that each subquery level gets its own scope e.g. "SELECT * FROM
(SELECT * FROM x) x" is permitted.
What I think all this means is that piggy-backing on the existing
logic here doesn't look especially promising. If we're trying to
identify a specific usage of a specific relation inside of a query, we
want global uniqueness across the whole query, but this isn't even
local uniqueness within a certain query level -- it's uniqueness
within a part of a query level with a special carve-out for same-named
tables in different schemas. Just to drive the point home, the
duplicate checks are done with two nested for loops, a good
old-fashioned O(n^2) algorithm, instead of something like entering
everything into a hash table and complaining if the entry already
exists. We're basically relying on the fact that there won't be very
many items at a single level of a FROM-list for this to have
reasonable performance; and scaling it even as far as the whole query
sounds like it might be dubious. Maybe it's possible to rewrite this
somehow to use a hash table and distinguish the cases where it
shouldn't complain, but it doesn't look like a ton of fun.
Once we get past parse analysis, there's really nothing that cares
about duplicate names, as far as I can see. The planner is perfectly
happy to work on multiple instances of the same table or of different
tables with the same alias, and it really has no reason to care about
what anything would look like if it were deparsed and printed out. I
suppose this is why the code works the way it does: by postponing
renaming until somebody actually does EXPLAIN or some other deparsing
operation, we avoid doing it when it isn't "really needed". Aside from
this nudge-the-planner use case there's no real reason to do anything
else.
In terms of solving the problem, nothing prevents us from installing a
hash table in PlannerGlobal and using that to label every
RangeTblEntry (except unnamed joins, probably) with a unique alias. My
tentative idea is to have the hash table key be a string. We'd check
whether the table alias is of the form
string+'_'+positive_integer_without_a_leading_zero. If so, we'd look
up the string in the hash table, else the entire alias. The hash table
entry would tell us which integers were already used for that string,
so then we could arrange to use one that isn't used yet, essentially
re-aliasing tables wherever collisions occur. However, in the "normal"
case where the query plan is not being printed and no
query-plan-frobbing extensions are in use, this effort is all wasted.
Maybe there's a way to do this kind of work only on demand, but it
seems to be somewhat complicated by the way we handle flattening the
range table: initially, every subquery level gets its own range table,
and at the end of planning, those range tables are all collapsed into
one. This means that depending on WHEN we get asked for information
about the re-aliased table names, the information is in a different
place. Uggh.
One could also argue that this sort of functionality doesn't belong in
core at all - that it's the problem of an extension that wants to frob
the planner behavior to figure out how to identify the rels. But I
think that is short-sighted. I think what people will want to do is
have the output of EXPLAIN tell them how they can identify a rel for
frobbing purposes. If not that, then what?
Even as it stands today, the v4-0002 would be better to have than nothing (well other than pg_hint_plan), as the it looks to me that the most frequent workaround for optimizer issues is to just throw 'enable_nestloop = no' into the mix quite often (so having the ability to just throw fixproblem.so into session_preload_libraries with just strstr()/regex() - to match on specific query - and disable it just there seems to be completely achievable and has much better granularity when targeting whole sessions with SET issued there).
Thanks for the +1.
--
Robert Haas
EDB: http://www.enterprisedb.com
Hi Andrei,
On Fri, Oct 11, 2024 at 8:21 AM Andrei Lepikhov <lepihov@gmail.com> wrote:
On 10/10/24 23:51, Robert Haas wrote:
On Wed, Sep 18, 2024 at 11:48 AM Robert Haas <robertmhaas@gmail.com>
wrote:
1. If you want to specify in-query hints using comments, how does your
extension get access to the comments?Having designed two features [1,2] that do the stuff mostly similar to
pg_hint_plan but based on real cardinalities earned from previous
executions, I can say the most annoying problem is hinting subqueries &
CTEs. Sometimes, you want to hint at the top query and not touch the
subquery and vice versa. Sometimes, users get stuck in accidents when a
flattened subquery influences your hint. So, the key property to invent
in advance should be an identification system.
I have chosen hash-based identification when each RelOptInfo has a hash
created over values of relids, restrictions and hashes of underlying
RelOptInfos. At the core, it seems inappropriate.
out of curiosity, why do You think it would be inappropriate to do so in
the core? Maybe it Is something similiar to compute_query_id that can be
computed externally too. I remember Tom arguing that query_id should be
computed externally by extension for some reasons because each extension
may want differently to fingerprint the queries, but I cannot find link to
discussion now or bring more details)
-J.
On 18/10/2024 14:56, Jakub Wartak wrote:
Hi Andrei,
On Fri, Oct 11, 2024 at 8:21 AM Andrei Lepikhov <lepihov@gmail.com
<mailto:lepihov@gmail.com>> wrote:
out of curiosity, why do You think it would be inappropriate to do so in
the core? Maybe it Is something similiar to compute_query_id that can be
computed externally too.
Generally, a hash value doesn't 100% guarantee the uniqueness of a node
identification. Also, RelOptInfo corresponds to a subtree in the final
plan, and sometimes, it takes work to find which node in the partially
executed plan corresponds to this specific estimation on row number
during selectivity estimation. Remember parameterised paths - you should
attach some signature for each path. So, it is not fully strict method.
If you are interested, I can perhaps explain the method a little bit
more at some meetup.
I remember Tom arguing that query_id should be
computed externally by extension for some reasons because each extension
may want differently to fingerprint the queries, but I cannot find link
to discussion now or bring more details)
I argued a lot about allowing extensions to provide separate query_ids
and came up with the patch a year ago. But it needs more debate on use
cases - my use cases are primarily from enterprise-grade closed
extensions, and it doesn't convince anyone. Also, I think any query_id
should be generated, likewise the core-made one. Does it seem like an
extensible Perl script?
--
regards, Andrei Lepikhov
On Sat, Oct 19, 2024 at 6:00 AM Andrei Lepikhov <lepihov@gmail.com> wrote:
Generally, a hash value doesn't 100% guarantee the uniqueness of a node
identification. Also, RelOptInfo corresponds to a subtree in the final
plan, and sometimes, it takes work to find which node in the partially
executed plan corresponds to this specific estimation on row number
during selectivity estimation. Remember parameterised paths - you should
attach some signature for each path. So, it is not fully strict method.
If you are interested, I can perhaps explain the method a little bit
more at some meetup.
Yeah, I agree that this is not the best method. While it's true that
you could get a false match in case of a hash value collision, IMHO
the bigger problem is that it seems like an expensive way of
determining something that we really should know already. If the user
types the same query, mentioning the same relations, in the same
order, with the same constructs around them, it's hard to believe that
hashing is the cheapest way of matching up the old and new ones. I'm
not sure exactly what we should do instead, but it feels like we more
or less have this information during parsing and then we lose track of
it as the query goes through the rewrite and planning phases.
--
Robert Haas
EDB: http://www.enterprisedb.com
On 10/23/24 15:05, Robert Haas wrote:
On Sat, Oct 19, 2024 at 6:00 AM Andrei Lepikhov <lepihov@gmail.com> wrote:
Generally, a hash value doesn't 100% guarantee the uniqueness of a node
identification. Also, RelOptInfo corresponds to a subtree in the final
plan, and sometimes, it takes work to find which node in the partially
executed plan corresponds to this specific estimation on row number
during selectivity estimation. Remember parameterised paths - you should
attach some signature for each path. So, it is not fully strict method.
If you are interested, I can perhaps explain the method a little bit
more at some meetup.Yeah, I agree that this is not the best method. While it's true that
you could get a false match in case of a hash value collision, IMHO
the bigger problem is that it seems like an expensive way of
determining something that we really should know already. If the user
types the same query, mentioning the same relations, in the same
order, with the same constructs around them, it's hard to believe that
hashing is the cheapest way of matching up the old and new ones. I'm
not sure exactly what we should do instead, but it feels like we more
or less have this information during parsing and then we lose track of
it as the query goes through the rewrite and planning phases.
Parse tree may be implemented with multiple execution plans. Even
clauses can be transformed during optimisation (Remember OR -> ANY).
Also, the cardinality of a middle-tree join depends on the inner and
outer subtrees. Because of that, having a hash on RelOptInfo's relids
and restrictions + hashes of child RelOptInfos and carrying it through
all other stages up to the end of execution is the most stable approach
I know.
--
regards, Andrei Lepikhov
On Wed, Oct 23, 2024 at 11:51 AM Andrei Lepikhov <lepihov@gmail.com> wrote:
Parse tree may be implemented with multiple execution plans. Even
clauses can be transformed during optimisation (Remember OR -> ANY).
Also, the cardinality of a middle-tree join depends on the inner and
outer subtrees. Because of that, having a hash on RelOptInfo's relids
and restrictions + hashes of child RelOptInfos and carrying it through
all other stages up to the end of execution is the most stable approach
I know.
I'm not saying there's a better way to do it today. I'm saying I think
there SHOULD be a better way to do it.
--
Robert Haas
EDB: http://www.enterprisedb.com