Retiring is_pushed_down
Started by Richard Guoover 2 years ago8 messages
When forming an outer join's joinrel, we have the is_pushed_down flag in
RestrictInfo nodes to distinguish those quals that are in that join's
JOIN/ON condition from those that were pushed down to the joinrel and
thus act as filter quals. Since now we have the outer-join-aware-Var
infrastructure, I think we can check to see whether a qual clause's
required_relids reference the outer join(s) being formed, in order to
tell if it's a join or filter clause. This seems like a more principled
way. (Interesting that optimizer/README actually describes this way in
section 'Relation Identification and Qual Clause Placement'.)
So I give it a try to retire is_pushed_down as attached. But there are
several points that may need more thoughts.
* When we form an outer join, it's possible that more than one outer
join relid is added to the join's relid set, if there are any pushed
down outer joins per identity 3. And it's also possible that no outer
join relid is added, for an outer join that has been pushed down. So
instead of checking if a qual clause's required_relids include the outer
join's relid, I think we should check if its required_relids overlap
the outer join relids that are being formed, which means that we should
use bms_overlap(rinfo->required_relids, ojrelids) rather than
bms_is_member(ojrelid, rinfo->required_relids). And we should do this
check only for outer joins.
* This patch calculates the outer join relids that are being formed
generally in this way:
bms_difference(joinrelids, bms_union(outerrelids, innerrelids))
Of course this can only be used after the outer join relids has been
added by add_outer_joins_to_relids(). This calculation is performed
multiple times during planning. I'm not sure if this has performance
issues. Maybe we can calculate it only once and store the result in
some place (such as in JoinPath)?
* ANTI joins are treated as outer joins but sometimes they do not have
rtindex (such as ANTI joins derived from SEMI). This would be a problem
with this new check. As an example, consider query
select * from a where not exists (select 1 from b where a.i = b.i) and
CURRENT_USER = SESSION_USER;
The pseudoconstant clause 'CURRENT_USER = SESSION_USER' is supposed to
be treated as a filter clause but the new check would treat it as a join
clause because the outer join relid set being formed is empty since the
ANTI join here does not have an rtindex. To solve this problem, this
patch manually adds a RTE for a ANTI join derived from SEMI.
Any thoughts?
Thanks
Richard
Attachments:
v1-0001-Retiring-is_pushed_down.patchapplication/octet-stream; name=v1-0001-Retiring-is_pushed_down.patchDownload
From 7c4f5b4625a23e69f650cfd206ad6b33adced460 Mon Sep 17 00:00:00 2001
From: pgsql-guo <richard.guo@openpie.com>
Date: Thu, 23 Mar 2023 11:53:40 +0800
Subject: [PATCH v1] Retiring is_pushed_down
When forming an outer join's joinrel, we have the is_pushed_down flag in
RestrictInfo nodes to distinguish those quals that are in that join's
JOIN/ON condition from those that were pushed down to the joinrel and
thus act as filter quals. Since now we have the outer-join-aware-Var
infrastructure, we can check to see whether a qual clause's
required_relids reference the outer join(s) being formed, in order to
tell if it's a join or filter clause. This seems like a more principled
way.
This patch is an attempt to retire the is_pushed_down flag.
---
contrib/postgres_fdw/postgres_fdw.c | 3 +-
src/backend/optimizer/path/costsize.c | 13 +++--
src/backend/optimizer/path/equivclass.c | 4 --
src/backend/optimizer/path/joinpath.c | 14 +++--
src/backend/optimizer/path/joinrels.c | 25 +++++----
src/backend/optimizer/plan/analyzejoins.c | 13 +++--
src/backend/optimizer/plan/createplan.c | 18 +++++++
src/backend/optimizer/plan/initsplan.c | 62 ++++-------------------
src/backend/optimizer/plan/subselect.c | 28 +++++++++-
src/backend/optimizer/util/inherit.c | 2 -
src/backend/optimizer/util/orclauses.c | 1 -
src/backend/optimizer/util/relnode.c | 23 +++++++--
src/backend/optimizer/util/restrictinfo.c | 31 ++++--------
src/include/nodes/pathnodes.h | 53 +++++++++----------
src/include/optimizer/cost.h | 2 +-
src/include/optimizer/restrictinfo.h | 4 +-
16 files changed, 155 insertions(+), 141 deletions(-)
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index c5cada55fb..60159e58d0 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -5798,7 +5798,7 @@ foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel, JoinType jointype,
rinfo->clause);
if (IS_OUTER_JOIN(jointype) &&
- !RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ !RINFO_IS_PUSHED_DOWN(rinfo, extra->ojrelids, joinrel->relids))
{
if (!is_remote_clause)
return false;
@@ -6519,7 +6519,6 @@ foreign_grouping_ok(PlannerInfo *root, RelOptInfo *grouped_rel,
Assert(!IsA(expr, RestrictInfo));
rinfo = make_restrictinfo(root,
expr,
- true,
false,
false,
false,
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index ef475d95a1..be80a8b327 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -4738,7 +4738,7 @@ compute_semi_anti_join_factors(PlannerInfo *root,
JoinType jointype,
SpecialJoinInfo *sjinfo,
List *restrictlist,
- SemiAntiJoinFactors *semifactors)
+ JoinPathExtraData *extra)
{
Selectivity jselec;
Selectivity nselec;
@@ -4761,7 +4761,7 @@ compute_semi_anti_join_factors(PlannerInfo *root,
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
- if (!RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ if (!RINFO_IS_PUSHED_DOWN(rinfo, extra->ojrelids, joinrel->relids))
joinquals = lappend(joinquals, rinfo);
}
}
@@ -4829,8 +4829,8 @@ compute_semi_anti_join_factors(PlannerInfo *root,
else
avgmatch = 1.0;
- semifactors->outer_match_frac = jselec;
- semifactors->match_count = avgmatch;
+ extra->semifactors.outer_match_frac = jselec;
+ extra->semifactors.match_count = avgmatch;
}
/*
@@ -5195,13 +5195,16 @@ calc_joinrel_size_estimate(PlannerInfo *root,
List *joinquals = NIL;
List *pushedquals = NIL;
ListCell *l;
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ outer_rel->relids,
+ inner_rel->relids);
/* Grovel through the clauses to separate into two lists */
foreach(l, restrictlist)
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
- if (RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ if (RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrel->relids))
pushedquals = lappend(pushedquals, rinfo);
else
joinquals = lappend(joinquals, rinfo);
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 7fa502d6e2..96a22f1389 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -196,7 +196,6 @@ process_equivalence(PlannerInfo *root,
*p_restrictinfo =
make_restrictinfo(root,
(Expr *) ntest,
- restrictinfo->is_pushed_down,
restrictinfo->has_clone,
restrictinfo->is_clone,
restrictinfo->pseudoconstant,
@@ -2005,7 +2004,6 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* throw back a dummy replacement clause (see notes above) */
rinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(true, false),
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
false, /* pseudoconstant */
@@ -2033,7 +2031,6 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* throw back a dummy replacement clause (see notes above) */
rinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(true, false),
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
false, /* pseudoconstant */
@@ -2061,7 +2058,6 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* throw back a dummy replacement clause (see notes above) */
rinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(true, false),
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
false, /* pseudoconstant */
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index f047ad9ba4..01f2e4fbb2 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -83,6 +83,7 @@ static List *select_mergejoin_clauses(PlannerInfo *root,
RelOptInfo *innerrel,
List *restrictlist,
JoinType jointype,
+ JoinPathExtraData *extra,
bool *mergejoin_allowed);
static void generate_mergejoin_paths(PlannerInfo *root,
RelOptInfo *joinrel,
@@ -149,6 +150,9 @@ add_paths_to_joinrel(PlannerInfo *root,
extra.mergeclause_list = NIL;
extra.sjinfo = sjinfo;
extra.param_source_rels = NULL;
+ extra.ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ outerrel->relids,
+ innerrel->relids);
/*
* See if the inner relation is provably unique for this outer rel.
@@ -213,6 +217,7 @@ add_paths_to_joinrel(PlannerInfo *root,
innerrel,
restrictlist,
jointype,
+ &extra,
&mergejoin_allowed);
/*
@@ -222,7 +227,7 @@ add_paths_to_joinrel(PlannerInfo *root,
if (jointype == JOIN_SEMI || jointype == JOIN_ANTI || extra.inner_unique)
compute_semi_anti_join_factors(root, joinrel, outerrel, innerrel,
jointype, sjinfo, restrictlist,
- &extra.semifactors);
+ &extra);
/*
* Decide whether it's sensible to generate parameterized paths for this
@@ -2090,7 +2095,8 @@ hash_inner_and_outer(PlannerInfo *root,
* If processing an outer join, only use its own join clauses for
* hashing. For inner joins we need not be so picky.
*/
- if (isouterjoin && RINFO_IS_PUSHED_DOWN(restrictinfo, joinrel->relids))
+ if (isouterjoin &&
+ RINFO_IS_PUSHED_DOWN(restrictinfo, extra->ojrelids, joinrel->relids))
continue;
if (!restrictinfo->can_join ||
@@ -2322,6 +2328,7 @@ select_mergejoin_clauses(PlannerInfo *root,
RelOptInfo *innerrel,
List *restrictlist,
JoinType jointype,
+ JoinPathExtraData *extra,
bool *mergejoin_allowed)
{
List *result_list = NIL;
@@ -2339,7 +2346,8 @@ select_mergejoin_clauses(PlannerInfo *root,
* we don't set have_nonmergeable_joinclause here because pushed-down
* clauses will become otherquals not joinquals.)
*/
- if (isouterjoin && RINFO_IS_PUSHED_DOWN(restrictinfo, joinrel->relids))
+ if (isouterjoin &&
+ RINFO_IS_PUSHED_DOWN(restrictinfo, extra->ojrelids, joinrel->relids))
continue;
/* Check that clause is a mergeable operator clause */
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 015a0b3cbe..b283d13bf0 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -34,6 +34,7 @@ static bool has_join_restriction(PlannerInfo *root, RelOptInfo *rel);
static bool has_legal_joinclause(PlannerInfo *root, RelOptInfo *rel);
static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
+ Relids ojrelids,
bool only_pushed_down);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
@@ -895,6 +896,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *sjinfo, List *restrictlist)
{
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ rel1->relids,
+ rel2->relids);
+
/*
* Consider paths using each rel as both outer and inner. Depending on
* the join type, a provably empty outer or inner rel might mean the join
@@ -917,7 +922,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
{
case JOIN_INNER:
if (is_dummy_rel(rel1) || is_dummy_rel(rel2) ||
- restriction_is_constant_false(restrictlist, joinrel, false))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, false))
{
mark_dummy_rel(joinrel);
break;
@@ -931,12 +936,12 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
break;
case JOIN_LEFT:
if (is_dummy_rel(rel1) ||
- restriction_is_constant_false(restrictlist, joinrel, true))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, true))
{
mark_dummy_rel(joinrel);
break;
}
- if (restriction_is_constant_false(restrictlist, joinrel, false) &&
+ if (restriction_is_constant_false(restrictlist, joinrel, ojrelids, false) &&
bms_is_subset(rel2->relids, sjinfo->syn_righthand))
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
@@ -948,7 +953,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
break;
case JOIN_FULL:
if ((is_dummy_rel(rel1) && is_dummy_rel(rel2)) ||
- restriction_is_constant_false(restrictlist, joinrel, true))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, true))
{
mark_dummy_rel(joinrel);
break;
@@ -984,7 +989,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
bms_is_subset(sjinfo->min_righthand, rel2->relids))
{
if (is_dummy_rel(rel1) || is_dummy_rel(rel2) ||
- restriction_is_constant_false(restrictlist, joinrel, false))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, false))
{
mark_dummy_rel(joinrel);
break;
@@ -1007,7 +1012,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
sjinfo) != NULL)
{
if (is_dummy_rel(rel1) || is_dummy_rel(rel2) ||
- restriction_is_constant_false(restrictlist, joinrel, false))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, false))
{
mark_dummy_rel(joinrel);
break;
@@ -1022,12 +1027,12 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
break;
case JOIN_ANTI:
if (is_dummy_rel(rel1) ||
- restriction_is_constant_false(restrictlist, joinrel, true))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, true))
{
mark_dummy_rel(joinrel);
break;
}
- if (restriction_is_constant_false(restrictlist, joinrel, false) &&
+ if (restriction_is_constant_false(restrictlist, joinrel, ojrelids, false) &&
bms_is_subset(rel2->relids, sjinfo->syn_righthand))
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
@@ -1424,6 +1429,7 @@ mark_dummy_rel(RelOptInfo *rel)
static bool
restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
+ Relids ojrelids,
bool only_pushed_down)
{
ListCell *lc;
@@ -1438,7 +1444,8 @@ restriction_is_constant_false(List *restrictlist,
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, lc);
- if (only_pushed_down && !RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ if (only_pushed_down &&
+ !RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrel->relids))
continue;
if (rinfo->clause && IsA(rinfo->clause, Const))
diff --git a/src/backend/optimizer/plan/analyzejoins.c b/src/backend/optimizer/plan/analyzejoins.c
index 5f3cce873a..4c8c5d672b 100644
--- a/src/backend/optimizer/plan/analyzejoins.c
+++ b/src/backend/optimizer/plan/analyzejoins.c
@@ -284,7 +284,9 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
* above the outer join, even if it references no other rels (it might
* be from WHERE, for example).
*/
- if (RINFO_IS_PUSHED_DOWN(restrictinfo, joinrelids))
+ if (RINFO_IS_PUSHED_DOWN(restrictinfo,
+ bms_make_singleton(sjinfo->ojrelid),
+ joinrelids))
continue; /* ignore; not useful here */
/* Ignore if it's not a mergejoinable clause */
@@ -484,7 +486,9 @@ remove_rel_from_query(PlannerInfo *root, int relid, SpecialJoinInfo *sjinfo)
remove_join_clause_from_rels(root, rinfo, rinfo->required_relids);
- if (RINFO_IS_PUSHED_DOWN(rinfo, join_plus_commute))
+ if (RINFO_IS_PUSHED_DOWN(rinfo,
+ bms_make_singleton(sjinfo->ojrelid),
+ join_plus_commute))
{
/*
* There might be references to relid or ojrelid in the
@@ -1286,6 +1290,9 @@ is_innerrel_unique_for(PlannerInfo *root,
{
List *clause_list = NIL;
ListCell *lc;
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrelids,
+ outerrelids,
+ innerrel->relids);
/*
* Search for mergejoinable clauses that constrain the inner rel against
@@ -1303,7 +1310,7 @@ is_innerrel_unique_for(PlannerInfo *root,
* join, we can't use it.
*/
if (IS_OUTER_JOIN(jointype) &&
- RINFO_IS_PUSHED_DOWN(restrictinfo, joinrelids))
+ RINFO_IS_PUSHED_DOWN(restrictinfo, ojrelids, joinrelids))
continue;
/* Ignore if it's not a mergejoinable clause */
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index af48109058..13ab1d3eeb 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -4347,8 +4347,14 @@ create_nestloop_plan(PlannerInfo *root,
/* Any pseudoconstant clauses are ignored here */
if (IS_OUTER_JOIN(best_path->jpath.jointype))
{
+ Relids ojrelids =
+ CALC_OUTER_JOIN_RELIDS(best_path->jpath.path.parent->relids,
+ best_path->jpath.outerjoinpath->parent->relids,
+ best_path->jpath.innerjoinpath->parent->relids);
+
extract_actual_join_clauses(joinrestrictclauses,
best_path->jpath.path.parent->relids,
+ ojrelids,
&joinclauses, &otherclauses);
}
else
@@ -4435,8 +4441,14 @@ create_mergejoin_plan(PlannerInfo *root,
/* Any pseudoconstant clauses are ignored here */
if (IS_OUTER_JOIN(best_path->jpath.jointype))
{
+ Relids ojrelids =
+ CALC_OUTER_JOIN_RELIDS(best_path->jpath.path.parent->relids,
+ best_path->jpath.outerjoinpath->parent->relids,
+ best_path->jpath.innerjoinpath->parent->relids);
+
extract_actual_join_clauses(joinclauses,
best_path->jpath.path.parent->relids,
+ ojrelids,
&joinclauses, &otherclauses);
}
else
@@ -4737,8 +4749,14 @@ create_hashjoin_plan(PlannerInfo *root,
/* Any pseudoconstant clauses are ignored here */
if (IS_OUTER_JOIN(best_path->jpath.jointype))
{
+ Relids ojrelids =
+ CALC_OUTER_JOIN_RELIDS(best_path->jpath.path.parent->relids,
+ best_path->jpath.outerjoinpath->parent->relids,
+ best_path->jpath.innerjoinpath->parent->relids);
+
extract_actual_join_clauses(joinclauses,
best_path->jpath.path.parent->relids,
+ ojrelids,
&joinclauses, &otherclauses);
}
else
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index b31d892121..c7e0369905 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -963,19 +963,16 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
child_domain->jd_relids);
jtitem->qualscope = bms_union(left_item->qualscope,
right_item->qualscope);
- /* caution: ANTI join derived from SEMI will lack rtindex */
- if (j->rtindex != 0)
- {
- parent_domain->jd_relids =
- bms_add_member(parent_domain->jd_relids,
- j->rtindex);
- jtitem->qualscope = bms_add_member(jtitem->qualscope,
+ Assert(j->rtindex != 0);
+ parent_domain->jd_relids =
+ bms_add_member(parent_domain->jd_relids,
+ j->rtindex);
+ jtitem->qualscope = bms_add_member(jtitem->qualscope,
+ j->rtindex);
+ root->outer_join_rels = bms_add_member(root->outer_join_rels,
j->rtindex);
- root->outer_join_rels = bms_add_member(root->outer_join_rels,
- j->rtindex);
- mark_rels_nulled_by_join(root, j->rtindex,
- right_item->qualscope);
- }
+ mark_rels_nulled_by_join(root, j->rtindex,
+ right_item->qualscope);
jtitem->inner_join_rels = bms_union(left_item->inner_join_rels,
right_item->inner_join_rels);
jtitem->left_rels = left_item->qualscope;
@@ -2209,7 +2206,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
List **postponed_oj_qual_list)
{
Relids relids;
- bool is_pushed_down;
bool pseudoconstant = false;
bool maybe_equivalence;
bool maybe_outer_join;
@@ -2319,37 +2315,9 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
}
}
- /*----------
+ /*
* Check to see if clause application must be delayed by outer-join
* considerations.
- *
- * A word about is_pushed_down: we mark the qual as "pushed down" if
- * it is (potentially) applicable at a level different from its original
- * syntactic level. This flag is used to distinguish OUTER JOIN ON quals
- * from other quals pushed down to the same joinrel. The rules are:
- * WHERE quals and INNER JOIN quals: is_pushed_down = true.
- * Non-degenerate OUTER JOIN quals: is_pushed_down = false.
- * Degenerate OUTER JOIN quals: is_pushed_down = true.
- * A "degenerate" OUTER JOIN qual is one that doesn't mention the
- * non-nullable side, and hence can be pushed down into the nullable side
- * without changing the join result. It is correct to treat it as a
- * regular filter condition at the level where it is evaluated.
- *
- * Note: it is not immediately obvious that a simple boolean is enough
- * for this: if for some reason we were to attach a degenerate qual to
- * its original join level, it would need to be treated as an outer join
- * qual there. However, this cannot happen, because all the rels the
- * clause mentions must be in the outer join's min_righthand, therefore
- * the join it needs must be formed before the outer join; and we always
- * attach quals to the lowest level where they can be evaluated. But
- * if we were ever to re-introduce a mechanism for delaying evaluation
- * of "expensive" quals, this area would need work.
- *
- * Note: generally, use of is_pushed_down has to go through the macro
- * RINFO_IS_PUSHED_DOWN, because that flag alone is not always sufficient
- * to tell whether a clause must be treated as pushed-down in context.
- * This seems like another reason why it should perhaps be rethought.
- *----------
*/
if (bms_overlap(relids, outerjoin_nonnullable))
{
@@ -2373,7 +2341,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
* deductions, if it is mergejoinable. So consider adding it to the
* lists of set-aside outer-join clauses.
*/
- is_pushed_down = false;
maybe_equivalence = false;
maybe_outer_join = true;
@@ -2390,12 +2357,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
}
else
{
- /*
- * Normal qual clause or degenerate outer-join clause. Either way, we
- * can mark it as pushed-down.
- */
- is_pushed_down = true;
-
/*
* It's possible that this is an IS NULL clause that's redundant with
* a lower antijoin; if so we can just discard it. We need not test
@@ -2420,7 +2381,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
*/
restrictinfo = make_restrictinfo(root,
(Expr *) clause,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -2792,7 +2752,6 @@ process_implied_equality(PlannerInfo *root,
*/
restrictinfo = make_restrictinfo(root,
(Expr *) clause,
- true, /* is_pushed_down */
false, /* !has_clone */
false, /* !is_clone */
pseudoconstant,
@@ -2886,7 +2845,6 @@ build_implied_join_equality(PlannerInfo *root,
*/
restrictinfo = make_restrictinfo(root,
clause,
- true, /* is_pushed_down */
false, /* !has_clone */
false, /* !is_clone */
false, /* pseudoconstant */
diff --git a/src/backend/optimizer/plan/subselect.c b/src/backend/optimizer/plan/subselect.c
index 7a9fe88fec..f63afec871 100644
--- a/src/backend/optimizer/plan/subselect.c
+++ b/src/backend/optimizer/plan/subselect.c
@@ -1519,7 +1519,33 @@ convert_EXISTS_sublink_to_join(PlannerInfo *root, SubLink *sublink,
result->join_using_alias = NULL;
result->quals = whereClause;
result->alias = NULL;
- result->rtindex = 0; /* we don't need an RTE for it */
+ result->rtindex = 0; /* we don't need an RTE for JOIN_SEMI */
+
+ /*
+ * Add a RTE for JOIN_ANTI
+ */
+ if (result->jointype == JOIN_ANTI)
+ {
+ ParseNamespaceItem *jnsitem;
+ ParseState *pstate;
+
+ /* Create a dummy ParseState for addRangeTableEntryForJoin */
+ pstate = make_parsestate(NULL);
+
+ jnsitem = addRangeTableEntryForJoin(pstate,
+ NULL,
+ NULL,
+ JOIN_ANTI,
+ 0,
+ NULL,
+ NIL,
+ NIL,
+ NULL,
+ NULL,
+ false);
+ parse->rtable = lappend(parse->rtable, jnsitem->p_rte);
+ result->rtindex = list_length(parse->rtable);
+ }
return result;
}
diff --git a/src/backend/optimizer/util/inherit.c b/src/backend/optimizer/util/inherit.c
index 94de855a22..46ce5692cf 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -893,7 +893,6 @@ apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
childquals = lappend(childquals,
make_restrictinfo(root,
(Expr *) onecq,
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
pseudoconstant,
@@ -931,7 +930,6 @@ apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
/* not likely that we'd see constants here, so no check */
childquals = lappend(childquals,
make_restrictinfo(root, qual,
- true,
false, false,
false,
security_level,
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index 6ef9d14b90..bc0ce171cc 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -265,7 +265,6 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
*/
or_rinfo = make_restrictinfo(root,
orclause,
- true,
false,
false,
false,
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 76dad17e33..9542954f0c 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -57,6 +57,7 @@ static List *subbuild_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *input_rel,
Relids both_input_relids,
+ SpecialJoinInfo *sjinfo,
List *new_restrictlist);
static List *subbuild_joinrel_joinlist(RelOptInfo *joinrel,
List *joininfo_list,
@@ -1286,9 +1287,9 @@ build_joinrel_restrictlist(PlannerInfo *root,
* same clauses arriving from both input relations).
*/
result = subbuild_joinrel_restrictlist(root, joinrel, outer_rel,
- both_input_relids, NIL);
+ both_input_relids, sjinfo, NIL);
result = subbuild_joinrel_restrictlist(root, joinrel, inner_rel,
- both_input_relids, result);
+ both_input_relids, sjinfo, result);
/*
* Add on any clauses derived from EquivalenceClasses. These cannot be
@@ -1328,6 +1329,7 @@ subbuild_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *input_rel,
Relids both_input_relids,
+ SpecialJoinInfo *sjinfo,
List *new_restrictlist)
{
ListCell *l;
@@ -1349,11 +1351,15 @@ subbuild_joinrel_restrictlist(PlannerInfo *root,
*/
if (rinfo->has_clone || rinfo->is_clone)
{
- Assert(!RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids));
if (!bms_is_subset(rinfo->required_relids, both_input_relids))
continue;
if (bms_overlap(rinfo->incompatible_relids, both_input_relids))
continue;
+
+ Assert(!RINFO_IS_PUSHED_DOWN(rinfo,
+ bms_difference(joinrel->relids,
+ both_input_relids),
+ joinrel->relids));
}
else
{
@@ -1364,7 +1370,11 @@ subbuild_joinrel_restrictlist(PlannerInfo *root,
* (There is little point in checking incompatible_relids,
* because it'll be NULL.)
*/
- Assert(RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids) ||
+ Assert(sjinfo->ojrelid == 0 ||
+ RINFO_IS_PUSHED_DOWN(rinfo,
+ bms_difference(joinrel->relids,
+ both_input_relids),
+ joinrel->relids) ||
bms_is_subset(rinfo->required_relids,
both_input_relids));
}
@@ -2059,6 +2069,9 @@ have_partkey_equi_join(PlannerInfo *root, RelOptInfo *joinrel,
int cnt_pks;
bool pk_has_clause[PARTITION_MAX_KEYS];
bool strict_op;
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ rel1->relids,
+ rel2->relids);
/*
* This function must only be called when the joined relations have same
@@ -2079,7 +2092,7 @@ have_partkey_equi_join(PlannerInfo *root, RelOptInfo *joinrel,
/* If processing an outer join, only use its own join clauses. */
if (IS_OUTER_JOIN(jointype) &&
- RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrel->relids))
continue;
/* Skip clauses which can not be used for a join. */
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index d6d26a2b51..75945c3ab9 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -24,7 +24,6 @@
static RestrictInfo *make_restrictinfo_internal(PlannerInfo *root,
Expr *clause,
Expr *orclause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -34,7 +33,6 @@ static RestrictInfo *make_restrictinfo_internal(PlannerInfo *root,
Relids outer_relids);
static Expr *make_sub_restrictinfos(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -49,11 +47,11 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
*
* Build a RestrictInfo node containing the given subexpression.
*
- * The is_pushed_down, has_clone, is_clone, and pseudoconstant flags for the
- * RestrictInfo must be supplied by the caller, as well as the correct values
- * for security_level, incompatible_relids, and outer_relids.
- * required_relids can be NULL, in which case it defaults to the actual clause
- * contents (i.e., clause_relids).
+ * The has_clone, is_clone, and pseudoconstant flags for the RestrictInfo
+ * must be supplied by the caller, as well as the correct values for
+ * security_level, incompatible_relids, and outer_relids. required_relids
+ * can be NULL, in which case it defaults to the actual clause contents
+ * (i.e., clause_relids).
*
* We initialize fields that depend only on the given subexpression, leaving
* others that depend on context (or may never be needed at all) to be filled
@@ -62,7 +60,6 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
RestrictInfo *
make_restrictinfo(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -78,7 +75,6 @@ make_restrictinfo(PlannerInfo *root,
if (is_orclause(clause))
return (RestrictInfo *) make_sub_restrictinfos(root,
clause,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -93,7 +89,6 @@ make_restrictinfo(PlannerInfo *root,
return make_restrictinfo_internal(root,
clause,
NULL,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -112,7 +107,6 @@ static RestrictInfo *
make_restrictinfo_internal(PlannerInfo *root,
Expr *clause,
Expr *orclause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -126,7 +120,6 @@ make_restrictinfo_internal(PlannerInfo *root,
restrictinfo->clause = clause;
restrictinfo->orclause = orclause;
- restrictinfo->is_pushed_down = is_pushed_down;
restrictinfo->pseudoconstant = pseudoconstant;
restrictinfo->has_clone = has_clone;
restrictinfo->is_clone = is_clone;
@@ -259,9 +252,9 @@ make_restrictinfo_internal(PlannerInfo *root,
* implicit-AND lists at top level of RestrictInfo lists. Only ORs and
* simple clauses are valid RestrictInfos.
*
- * The same is_pushed_down, has_clone, is_clone, and pseudoconstant flag
- * values can be applied to all RestrictInfo nodes in the result. Likewise
- * for security_level, incompatible_relids, and outer_relids.
+ * The same has_clone, is_clone, and pseudoconstant flag values can be
+ * applied to all RestrictInfo nodes in the result. Likewise for
+ * security_level, incompatible_relids, and outer_relids.
*
* The given required_relids are attached to our top-level output,
* but any OR-clause constituents are allowed to default to just the
@@ -270,7 +263,6 @@ make_restrictinfo_internal(PlannerInfo *root,
static Expr *
make_sub_restrictinfos(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -288,7 +280,6 @@ make_sub_restrictinfos(PlannerInfo *root,
orlist = lappend(orlist,
make_sub_restrictinfos(root,
lfirst(temp),
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -299,7 +290,6 @@ make_sub_restrictinfos(PlannerInfo *root,
return (Expr *) make_restrictinfo_internal(root,
clause,
make_orclause(orlist),
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -317,7 +307,6 @@ make_sub_restrictinfos(PlannerInfo *root,
andlist = lappend(andlist,
make_sub_restrictinfos(root,
lfirst(temp),
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -331,7 +320,6 @@ make_sub_restrictinfos(PlannerInfo *root,
return (Expr *) make_restrictinfo_internal(root,
clause,
NULL,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -521,6 +509,7 @@ extract_actual_clauses(List *restrictinfo_list,
void
extract_actual_join_clauses(List *restrictinfo_list,
Relids joinrelids,
+ Relids ojrelids,
List **joinquals,
List **otherquals)
{
@@ -533,7 +522,7 @@ extract_actual_join_clauses(List *restrictinfo_list,
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
- if (RINFO_IS_PUSHED_DOWN(rinfo, joinrelids))
+ if (RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrelids))
{
if (!rinfo->pseudoconstant &&
!rinfo_is_constant_true(rinfo))
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index c17b53f7ad..d1dc27be6f 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -2408,27 +2408,16 @@ typedef struct LimitPath
* level of the outer join, which is true except when it is a "degenerate"
* condition that references only Vars from the nullable side of the join.
*
- * RestrictInfo nodes contain a flag to indicate whether a qual has been
- * pushed down to a lower level than its original syntactic placement in the
- * join tree would suggest. If an outer join prevents us from pushing a qual
- * down to its "natural" semantic level (the level associated with just the
- * base rels used in the qual) then we mark the qual with a "required_relids"
- * value including more than just the base rels it actually uses. By
- * pretending that the qual references all the rels required to form the outer
- * join, we prevent it from being evaluated below the outer join's joinrel.
- * When we do form the outer join's joinrel, we still need to distinguish
- * those quals that are actually in that join's JOIN/ON condition from those
- * that appeared elsewhere in the tree and were pushed down to the join rel
- * because they used no other rels. That's what the is_pushed_down flag is
- * for; it tells us that a qual is not an OUTER JOIN qual for the set of base
- * rels listed in required_relids. A clause that originally came from WHERE
- * or an INNER JOIN condition will *always* have its is_pushed_down flag set.
- * It's possible for an OUTER JOIN clause to be marked is_pushed_down too,
- * if we decide that it can be pushed down into the nullable side of the join.
- * In that case it acts as a plain filter qual for wherever it gets evaluated.
- * (In short, is_pushed_down is only false for non-degenerate outer join
- * conditions. Possibly we should rename it to reflect that meaning? But
- * see also the comments for RINFO_IS_PUSHED_DOWN, below.)
+ * If an outer join prevents us from pushing a qual down to its "natural"
+ * semantic level (the level associated with just the base rels used in the
+ * qual) then we mark the qual with a "required_relids" value including more
+ * than just the base rels it actually uses. By pretending that the qual
+ * references all the rels required to form the outer join, we prevent it from
+ * being evaluated below the outer join's joinrel. When we do form the outer
+ * join's joinrel, we still need to distinguish those quals that are actually
+ * in that join's JOIN/ON condition from those that appeared elsewhere in the
+ * tree and were pushed down to the join rel because they used no other rels.
+ * See the comments for RINFO_IS_PUSHED_DOWN for how we do that.
*
* There is also an incompatible_relids field, which is a set of outer-join
* relids above which we cannot evaluate the clause (because they might null
@@ -2515,9 +2504,6 @@ typedef struct RestrictInfo
/* the represented clause of WHERE or JOIN */
Expr *clause;
- /* true if clause was pushed down in level */
- bool is_pushed_down;
-
/* see comment above */
bool can_join pg_node_attr(equal_ignore);
@@ -2661,16 +2647,20 @@ typedef struct RestrictInfo
* This macro embodies the correct way to test whether a RestrictInfo is
* "pushed down" to a given outer join, that is, should be treated as a filter
* clause rather than a join clause at that outer join. This is certainly so
- * if is_pushed_down is true; but examining that is not sufficient anymore,
- * because outer-join clauses will get pushed down to lower outer joins when
- * we generate a path for the lower outer join that is parameterized by the
- * LHS of the upper one. We can detect such a clause by noting that its
+ * if the outer join clause references that outer join in any varnullingrels or
+ * phnullingrels set; but examining that is not sufficient anymore, because
+ * outer-join clauses will get pushed down to lower outer joins when we
+ * generate a path for the lower outer join that is parameterized by the LHS of
+ * the upper one. We can detect such a clause by noting that its
* required_relids exceed the scope of the join.
*/
-#define RINFO_IS_PUSHED_DOWN(rinfo, joinrelids) \
- ((rinfo)->is_pushed_down || \
+#define RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrelids) \
+ (bms_overlap((rinfo)->required_relids, (ojrelids)) || \
!bms_is_subset((rinfo)->required_relids, joinrelids))
+#define CALC_OUTER_JOIN_RELIDS(joinrelids, outerrelids, innerrelids) \
+ (bms_difference((joinrelids), bms_union((outerrelids), (innerrelids))))
+
/*
* Since mergejoinscansel() is a relatively expensive function, and would
* otherwise be invoked many times while planning a large join tree,
@@ -3170,6 +3160,8 @@ typedef struct SemiAntiJoinFactors
* sjinfo is extra info about special joins for selectivity estimation
* semifactors is as shown above (only valid for SEMI/ANTI/inner_unique joins)
* param_source_rels are OK targets for parameterization of result paths
+ * ojrelids is the relid set of any outer joins that will be calculated at this
+ * join, or NULL for inner join
*/
typedef struct JoinPathExtraData
{
@@ -3179,6 +3171,7 @@ typedef struct JoinPathExtraData
SpecialJoinInfo *sjinfo;
SemiAntiJoinFactors semifactors;
Relids param_source_rels;
+ Relids ojrelids;
} JoinPathExtraData;
/*
diff --git a/src/include/optimizer/cost.h b/src/include/optimizer/cost.h
index 6cf49705d3..11bdd120c2 100644
--- a/src/include/optimizer/cost.h
+++ b/src/include/optimizer/cost.h
@@ -183,7 +183,7 @@ extern void compute_semi_anti_join_factors(PlannerInfo *root,
JoinType jointype,
SpecialJoinInfo *sjinfo,
List *restrictlist,
- SemiAntiJoinFactors *semifactors);
+ JoinPathExtraData *extra);
extern void set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel);
extern double get_parameterized_baserel_size(PlannerInfo *root,
RelOptInfo *rel,
diff --git a/src/include/optimizer/restrictinfo.h b/src/include/optimizer/restrictinfo.h
index e140e619ac..04658f0067 100644
--- a/src/include/optimizer/restrictinfo.h
+++ b/src/include/optimizer/restrictinfo.h
@@ -19,12 +19,11 @@
/* Convenience macro for the common case of a valid-everywhere qual */
#define make_simple_restrictinfo(root, clause) \
- make_restrictinfo(root, clause, true, false, false, false, 0, \
+ make_restrictinfo(root, clause, false, false, false, 0, \
NULL, NULL, NULL)
extern RestrictInfo *make_restrictinfo(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -41,6 +40,7 @@ extern List *extract_actual_clauses(List *restrictinfo_list,
bool pseudoconstant);
extern void extract_actual_join_clauses(List *restrictinfo_list,
Relids joinrelids,
+ Relids ojrelids,
List **joinquals,
List **otherquals);
extern bool join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel);
--
2.32.0
Re: Retiring is_pushed_down
On Tue, Jul 25, 2023 at 3:39 PM Richard Guo <guofenglinux@gmail.com> wrote:
* This patch calculates the outer join relids that are being formed
generally in this way:bms_difference(joinrelids, bms_union(outerrelids, innerrelids))
Of course this can only be used after the outer join relids has been
added by add_outer_joins_to_relids(). This calculation is performed
multiple times during planning. I'm not sure if this has performance
issues. Maybe we can calculate it only once and store the result in
some place (such as in JoinPath)?
In the v2 patch, I added a member in JoinPath to store the relid set of
any outer joins that will be calculated at this join, and this would
avoid repeating this calculation when creating nestloop/merge/hash join
plan nodes. Also fixed a comment in v2.
Thanks
Richard
Attachments:
v2-0001-Retiring-is_pushed_down.patchapplication/octet-stream; name=v2-0001-Retiring-is_pushed_down.patchDownload
From 6aaa608a748b9c396ac13dae77125230a5b3c367 Mon Sep 17 00:00:00 2001
From: Richard Guo <guofenglinux@gmail.com>
Date: Wed, 26 Jul 2023 16:16:49 +0800
Subject: [PATCH v2] Retiring is_pushed_down
When forming an outer join's joinrel, we have the is_pushed_down flag in
RestrictInfo nodes to distinguish those quals that are in that join's
JOIN/ON condition from those that were pushed down to the joinrel and
thus act as filter quals. Since now we have the outer-join-aware-Var
infrastructure, we can check to see whether a qual clause's
required_relids reference the outer join(s) being formed, in order to
tell if it's a join or filter clause. This seems like a more principled
way.
This patch is an attempt to retire the is_pushed_down flag.
---
contrib/postgres_fdw/postgres_fdw.c | 3 +-
src/backend/optimizer/path/costsize.c | 15 +++---
src/backend/optimizer/path/equivclass.c | 4 --
src/backend/optimizer/path/joinpath.c | 14 +++--
src/backend/optimizer/path/joinrels.c | 25 +++++----
src/backend/optimizer/plan/analyzejoins.c | 13 +++--
src/backend/optimizer/plan/createplan.c | 3 ++
src/backend/optimizer/plan/initsplan.c | 62 ++++-------------------
src/backend/optimizer/plan/subselect.c | 28 +++++++++-
src/backend/optimizer/util/inherit.c | 2 -
src/backend/optimizer/util/orclauses.c | 1 -
src/backend/optimizer/util/pathnode.c | 3 ++
src/backend/optimizer/util/relnode.c | 23 +++++++--
src/backend/optimizer/util/restrictinfo.c | 31 ++++--------
src/include/nodes/pathnodes.h | 57 ++++++++++-----------
src/include/optimizer/cost.h | 2 +-
src/include/optimizer/restrictinfo.h | 4 +-
17 files changed, 148 insertions(+), 142 deletions(-)
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index c5cada55fb..60159e58d0 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -5798,7 +5798,7 @@ foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel, JoinType jointype,
rinfo->clause);
if (IS_OUTER_JOIN(jointype) &&
- !RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ !RINFO_IS_PUSHED_DOWN(rinfo, extra->ojrelids, joinrel->relids))
{
if (!is_remote_clause)
return false;
@@ -6519,7 +6519,6 @@ foreign_grouping_ok(PlannerInfo *root, RelOptInfo *grouped_rel,
Assert(!IsA(expr, RestrictInfo));
rinfo = make_restrictinfo(root,
expr,
- true,
false,
false,
false,
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index ef475d95a1..a446cb265a 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -4728,7 +4728,7 @@ get_restriction_qual_cost(PlannerInfo *root, RelOptInfo *baserel,
* sjinfo: SpecialJoinInfo relevant to this join
* restrictlist: join quals
* Output parameters:
- * *semifactors is filled in (see pathnodes.h for field definitions)
+ * extra->semifactors is filled in (see pathnodes.h for field definitions)
*/
void
compute_semi_anti_join_factors(PlannerInfo *root,
@@ -4738,7 +4738,7 @@ compute_semi_anti_join_factors(PlannerInfo *root,
JoinType jointype,
SpecialJoinInfo *sjinfo,
List *restrictlist,
- SemiAntiJoinFactors *semifactors)
+ JoinPathExtraData *extra)
{
Selectivity jselec;
Selectivity nselec;
@@ -4761,7 +4761,7 @@ compute_semi_anti_join_factors(PlannerInfo *root,
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
- if (!RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ if (!RINFO_IS_PUSHED_DOWN(rinfo, extra->ojrelids, joinrel->relids))
joinquals = lappend(joinquals, rinfo);
}
}
@@ -4829,8 +4829,8 @@ compute_semi_anti_join_factors(PlannerInfo *root,
else
avgmatch = 1.0;
- semifactors->outer_match_frac = jselec;
- semifactors->match_count = avgmatch;
+ extra->semifactors.outer_match_frac = jselec;
+ extra->semifactors.match_count = avgmatch;
}
/*
@@ -5195,13 +5195,16 @@ calc_joinrel_size_estimate(PlannerInfo *root,
List *joinquals = NIL;
List *pushedquals = NIL;
ListCell *l;
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ outer_rel->relids,
+ inner_rel->relids);
/* Grovel through the clauses to separate into two lists */
foreach(l, restrictlist)
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
- if (RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ if (RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrel->relids))
pushedquals = lappend(pushedquals, rinfo);
else
joinquals = lappend(joinquals, rinfo);
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 7fa502d6e2..96a22f1389 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -196,7 +196,6 @@ process_equivalence(PlannerInfo *root,
*p_restrictinfo =
make_restrictinfo(root,
(Expr *) ntest,
- restrictinfo->is_pushed_down,
restrictinfo->has_clone,
restrictinfo->is_clone,
restrictinfo->pseudoconstant,
@@ -2005,7 +2004,6 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* throw back a dummy replacement clause (see notes above) */
rinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(true, false),
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
false, /* pseudoconstant */
@@ -2033,7 +2031,6 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* throw back a dummy replacement clause (see notes above) */
rinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(true, false),
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
false, /* pseudoconstant */
@@ -2061,7 +2058,6 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* throw back a dummy replacement clause (see notes above) */
rinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(true, false),
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
false, /* pseudoconstant */
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index f047ad9ba4..01f2e4fbb2 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -83,6 +83,7 @@ static List *select_mergejoin_clauses(PlannerInfo *root,
RelOptInfo *innerrel,
List *restrictlist,
JoinType jointype,
+ JoinPathExtraData *extra,
bool *mergejoin_allowed);
static void generate_mergejoin_paths(PlannerInfo *root,
RelOptInfo *joinrel,
@@ -149,6 +150,9 @@ add_paths_to_joinrel(PlannerInfo *root,
extra.mergeclause_list = NIL;
extra.sjinfo = sjinfo;
extra.param_source_rels = NULL;
+ extra.ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ outerrel->relids,
+ innerrel->relids);
/*
* See if the inner relation is provably unique for this outer rel.
@@ -213,6 +217,7 @@ add_paths_to_joinrel(PlannerInfo *root,
innerrel,
restrictlist,
jointype,
+ &extra,
&mergejoin_allowed);
/*
@@ -222,7 +227,7 @@ add_paths_to_joinrel(PlannerInfo *root,
if (jointype == JOIN_SEMI || jointype == JOIN_ANTI || extra.inner_unique)
compute_semi_anti_join_factors(root, joinrel, outerrel, innerrel,
jointype, sjinfo, restrictlist,
- &extra.semifactors);
+ &extra);
/*
* Decide whether it's sensible to generate parameterized paths for this
@@ -2090,7 +2095,8 @@ hash_inner_and_outer(PlannerInfo *root,
* If processing an outer join, only use its own join clauses for
* hashing. For inner joins we need not be so picky.
*/
- if (isouterjoin && RINFO_IS_PUSHED_DOWN(restrictinfo, joinrel->relids))
+ if (isouterjoin &&
+ RINFO_IS_PUSHED_DOWN(restrictinfo, extra->ojrelids, joinrel->relids))
continue;
if (!restrictinfo->can_join ||
@@ -2322,6 +2328,7 @@ select_mergejoin_clauses(PlannerInfo *root,
RelOptInfo *innerrel,
List *restrictlist,
JoinType jointype,
+ JoinPathExtraData *extra,
bool *mergejoin_allowed)
{
List *result_list = NIL;
@@ -2339,7 +2346,8 @@ select_mergejoin_clauses(PlannerInfo *root,
* we don't set have_nonmergeable_joinclause here because pushed-down
* clauses will become otherquals not joinquals.)
*/
- if (isouterjoin && RINFO_IS_PUSHED_DOWN(restrictinfo, joinrel->relids))
+ if (isouterjoin &&
+ RINFO_IS_PUSHED_DOWN(restrictinfo, extra->ojrelids, joinrel->relids))
continue;
/* Check that clause is a mergeable operator clause */
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 015a0b3cbe..b283d13bf0 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -34,6 +34,7 @@ static bool has_join_restriction(PlannerInfo *root, RelOptInfo *rel);
static bool has_legal_joinclause(PlannerInfo *root, RelOptInfo *rel);
static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
+ Relids ojrelids,
bool only_pushed_down);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
@@ -895,6 +896,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *sjinfo, List *restrictlist)
{
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ rel1->relids,
+ rel2->relids);
+
/*
* Consider paths using each rel as both outer and inner. Depending on
* the join type, a provably empty outer or inner rel might mean the join
@@ -917,7 +922,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
{
case JOIN_INNER:
if (is_dummy_rel(rel1) || is_dummy_rel(rel2) ||
- restriction_is_constant_false(restrictlist, joinrel, false))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, false))
{
mark_dummy_rel(joinrel);
break;
@@ -931,12 +936,12 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
break;
case JOIN_LEFT:
if (is_dummy_rel(rel1) ||
- restriction_is_constant_false(restrictlist, joinrel, true))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, true))
{
mark_dummy_rel(joinrel);
break;
}
- if (restriction_is_constant_false(restrictlist, joinrel, false) &&
+ if (restriction_is_constant_false(restrictlist, joinrel, ojrelids, false) &&
bms_is_subset(rel2->relids, sjinfo->syn_righthand))
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
@@ -948,7 +953,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
break;
case JOIN_FULL:
if ((is_dummy_rel(rel1) && is_dummy_rel(rel2)) ||
- restriction_is_constant_false(restrictlist, joinrel, true))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, true))
{
mark_dummy_rel(joinrel);
break;
@@ -984,7 +989,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
bms_is_subset(sjinfo->min_righthand, rel2->relids))
{
if (is_dummy_rel(rel1) || is_dummy_rel(rel2) ||
- restriction_is_constant_false(restrictlist, joinrel, false))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, false))
{
mark_dummy_rel(joinrel);
break;
@@ -1007,7 +1012,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
sjinfo) != NULL)
{
if (is_dummy_rel(rel1) || is_dummy_rel(rel2) ||
- restriction_is_constant_false(restrictlist, joinrel, false))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, false))
{
mark_dummy_rel(joinrel);
break;
@@ -1022,12 +1027,12 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
break;
case JOIN_ANTI:
if (is_dummy_rel(rel1) ||
- restriction_is_constant_false(restrictlist, joinrel, true))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, true))
{
mark_dummy_rel(joinrel);
break;
}
- if (restriction_is_constant_false(restrictlist, joinrel, false) &&
+ if (restriction_is_constant_false(restrictlist, joinrel, ojrelids, false) &&
bms_is_subset(rel2->relids, sjinfo->syn_righthand))
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
@@ -1424,6 +1429,7 @@ mark_dummy_rel(RelOptInfo *rel)
static bool
restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
+ Relids ojrelids,
bool only_pushed_down)
{
ListCell *lc;
@@ -1438,7 +1444,8 @@ restriction_is_constant_false(List *restrictlist,
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, lc);
- if (only_pushed_down && !RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ if (only_pushed_down &&
+ !RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrel->relids))
continue;
if (rinfo->clause && IsA(rinfo->clause, Const))
diff --git a/src/backend/optimizer/plan/analyzejoins.c b/src/backend/optimizer/plan/analyzejoins.c
index 5f3cce873a..4c8c5d672b 100644
--- a/src/backend/optimizer/plan/analyzejoins.c
+++ b/src/backend/optimizer/plan/analyzejoins.c
@@ -284,7 +284,9 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
* above the outer join, even if it references no other rels (it might
* be from WHERE, for example).
*/
- if (RINFO_IS_PUSHED_DOWN(restrictinfo, joinrelids))
+ if (RINFO_IS_PUSHED_DOWN(restrictinfo,
+ bms_make_singleton(sjinfo->ojrelid),
+ joinrelids))
continue; /* ignore; not useful here */
/* Ignore if it's not a mergejoinable clause */
@@ -484,7 +486,9 @@ remove_rel_from_query(PlannerInfo *root, int relid, SpecialJoinInfo *sjinfo)
remove_join_clause_from_rels(root, rinfo, rinfo->required_relids);
- if (RINFO_IS_PUSHED_DOWN(rinfo, join_plus_commute))
+ if (RINFO_IS_PUSHED_DOWN(rinfo,
+ bms_make_singleton(sjinfo->ojrelid),
+ join_plus_commute))
{
/*
* There might be references to relid or ojrelid in the
@@ -1286,6 +1290,9 @@ is_innerrel_unique_for(PlannerInfo *root,
{
List *clause_list = NIL;
ListCell *lc;
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrelids,
+ outerrelids,
+ innerrel->relids);
/*
* Search for mergejoinable clauses that constrain the inner rel against
@@ -1303,7 +1310,7 @@ is_innerrel_unique_for(PlannerInfo *root,
* join, we can't use it.
*/
if (IS_OUTER_JOIN(jointype) &&
- RINFO_IS_PUSHED_DOWN(restrictinfo, joinrelids))
+ RINFO_IS_PUSHED_DOWN(restrictinfo, ojrelids, joinrelids))
continue;
/* Ignore if it's not a mergejoinable clause */
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index af48109058..3eeea338fe 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -4349,6 +4349,7 @@ create_nestloop_plan(PlannerInfo *root,
{
extract_actual_join_clauses(joinrestrictclauses,
best_path->jpath.path.parent->relids,
+ best_path->jpath.ojrelids,
&joinclauses, &otherclauses);
}
else
@@ -4437,6 +4438,7 @@ create_mergejoin_plan(PlannerInfo *root,
{
extract_actual_join_clauses(joinclauses,
best_path->jpath.path.parent->relids,
+ best_path->jpath.ojrelids,
&joinclauses, &otherclauses);
}
else
@@ -4739,6 +4741,7 @@ create_hashjoin_plan(PlannerInfo *root,
{
extract_actual_join_clauses(joinclauses,
best_path->jpath.path.parent->relids,
+ best_path->jpath.ojrelids,
&joinclauses, &otherclauses);
}
else
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index b31d892121..c7e0369905 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -963,19 +963,16 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
child_domain->jd_relids);
jtitem->qualscope = bms_union(left_item->qualscope,
right_item->qualscope);
- /* caution: ANTI join derived from SEMI will lack rtindex */
- if (j->rtindex != 0)
- {
- parent_domain->jd_relids =
- bms_add_member(parent_domain->jd_relids,
- j->rtindex);
- jtitem->qualscope = bms_add_member(jtitem->qualscope,
+ Assert(j->rtindex != 0);
+ parent_domain->jd_relids =
+ bms_add_member(parent_domain->jd_relids,
+ j->rtindex);
+ jtitem->qualscope = bms_add_member(jtitem->qualscope,
+ j->rtindex);
+ root->outer_join_rels = bms_add_member(root->outer_join_rels,
j->rtindex);
- root->outer_join_rels = bms_add_member(root->outer_join_rels,
- j->rtindex);
- mark_rels_nulled_by_join(root, j->rtindex,
- right_item->qualscope);
- }
+ mark_rels_nulled_by_join(root, j->rtindex,
+ right_item->qualscope);
jtitem->inner_join_rels = bms_union(left_item->inner_join_rels,
right_item->inner_join_rels);
jtitem->left_rels = left_item->qualscope;
@@ -2209,7 +2206,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
List **postponed_oj_qual_list)
{
Relids relids;
- bool is_pushed_down;
bool pseudoconstant = false;
bool maybe_equivalence;
bool maybe_outer_join;
@@ -2319,37 +2315,9 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
}
}
- /*----------
+ /*
* Check to see if clause application must be delayed by outer-join
* considerations.
- *
- * A word about is_pushed_down: we mark the qual as "pushed down" if
- * it is (potentially) applicable at a level different from its original
- * syntactic level. This flag is used to distinguish OUTER JOIN ON quals
- * from other quals pushed down to the same joinrel. The rules are:
- * WHERE quals and INNER JOIN quals: is_pushed_down = true.
- * Non-degenerate OUTER JOIN quals: is_pushed_down = false.
- * Degenerate OUTER JOIN quals: is_pushed_down = true.
- * A "degenerate" OUTER JOIN qual is one that doesn't mention the
- * non-nullable side, and hence can be pushed down into the nullable side
- * without changing the join result. It is correct to treat it as a
- * regular filter condition at the level where it is evaluated.
- *
- * Note: it is not immediately obvious that a simple boolean is enough
- * for this: if for some reason we were to attach a degenerate qual to
- * its original join level, it would need to be treated as an outer join
- * qual there. However, this cannot happen, because all the rels the
- * clause mentions must be in the outer join's min_righthand, therefore
- * the join it needs must be formed before the outer join; and we always
- * attach quals to the lowest level where they can be evaluated. But
- * if we were ever to re-introduce a mechanism for delaying evaluation
- * of "expensive" quals, this area would need work.
- *
- * Note: generally, use of is_pushed_down has to go through the macro
- * RINFO_IS_PUSHED_DOWN, because that flag alone is not always sufficient
- * to tell whether a clause must be treated as pushed-down in context.
- * This seems like another reason why it should perhaps be rethought.
- *----------
*/
if (bms_overlap(relids, outerjoin_nonnullable))
{
@@ -2373,7 +2341,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
* deductions, if it is mergejoinable. So consider adding it to the
* lists of set-aside outer-join clauses.
*/
- is_pushed_down = false;
maybe_equivalence = false;
maybe_outer_join = true;
@@ -2390,12 +2357,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
}
else
{
- /*
- * Normal qual clause or degenerate outer-join clause. Either way, we
- * can mark it as pushed-down.
- */
- is_pushed_down = true;
-
/*
* It's possible that this is an IS NULL clause that's redundant with
* a lower antijoin; if so we can just discard it. We need not test
@@ -2420,7 +2381,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
*/
restrictinfo = make_restrictinfo(root,
(Expr *) clause,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -2792,7 +2752,6 @@ process_implied_equality(PlannerInfo *root,
*/
restrictinfo = make_restrictinfo(root,
(Expr *) clause,
- true, /* is_pushed_down */
false, /* !has_clone */
false, /* !is_clone */
pseudoconstant,
@@ -2886,7 +2845,6 @@ build_implied_join_equality(PlannerInfo *root,
*/
restrictinfo = make_restrictinfo(root,
clause,
- true, /* is_pushed_down */
false, /* !has_clone */
false, /* !is_clone */
false, /* pseudoconstant */
diff --git a/src/backend/optimizer/plan/subselect.c b/src/backend/optimizer/plan/subselect.c
index 7a9fe88fec..f63afec871 100644
--- a/src/backend/optimizer/plan/subselect.c
+++ b/src/backend/optimizer/plan/subselect.c
@@ -1519,7 +1519,33 @@ convert_EXISTS_sublink_to_join(PlannerInfo *root, SubLink *sublink,
result->join_using_alias = NULL;
result->quals = whereClause;
result->alias = NULL;
- result->rtindex = 0; /* we don't need an RTE for it */
+ result->rtindex = 0; /* we don't need an RTE for JOIN_SEMI */
+
+ /*
+ * Add a RTE for JOIN_ANTI
+ */
+ if (result->jointype == JOIN_ANTI)
+ {
+ ParseNamespaceItem *jnsitem;
+ ParseState *pstate;
+
+ /* Create a dummy ParseState for addRangeTableEntryForJoin */
+ pstate = make_parsestate(NULL);
+
+ jnsitem = addRangeTableEntryForJoin(pstate,
+ NULL,
+ NULL,
+ JOIN_ANTI,
+ 0,
+ NULL,
+ NIL,
+ NIL,
+ NULL,
+ NULL,
+ false);
+ parse->rtable = lappend(parse->rtable, jnsitem->p_rte);
+ result->rtindex = list_length(parse->rtable);
+ }
return result;
}
diff --git a/src/backend/optimizer/util/inherit.c b/src/backend/optimizer/util/inherit.c
index 94de855a22..46ce5692cf 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -893,7 +893,6 @@ apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
childquals = lappend(childquals,
make_restrictinfo(root,
(Expr *) onecq,
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
pseudoconstant,
@@ -931,7 +930,6 @@ apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
/* not likely that we'd see constants here, so no check */
childquals = lappend(childquals,
make_restrictinfo(root, qual,
- true,
false, false,
false,
security_level,
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index 6ef9d14b90..bc0ce171cc 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -265,7 +265,6 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
*/
or_rinfo = make_restrictinfo(root,
orclause,
- true,
false,
false,
false,
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index f123fcb41e..3ade50c0c5 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -2477,6 +2477,7 @@ create_nestloop_path(PlannerInfo *root,
pathnode->jpath.outerjoinpath = outer_path;
pathnode->jpath.innerjoinpath = inner_path;
pathnode->jpath.joinrestrictinfo = restrict_clauses;
+ pathnode->jpath.ojrelids = extra->ojrelids;
final_cost_nestloop(root, pathnode, workspace, extra);
@@ -2541,6 +2542,7 @@ create_mergejoin_path(PlannerInfo *root,
pathnode->jpath.outerjoinpath = outer_path;
pathnode->jpath.innerjoinpath = inner_path;
pathnode->jpath.joinrestrictinfo = restrict_clauses;
+ pathnode->jpath.ojrelids = extra->ojrelids;
pathnode->path_mergeclauses = mergeclauses;
pathnode->outersortkeys = outersortkeys;
pathnode->innersortkeys = innersortkeys;
@@ -2618,6 +2620,7 @@ create_hashjoin_path(PlannerInfo *root,
pathnode->jpath.outerjoinpath = outer_path;
pathnode->jpath.innerjoinpath = inner_path;
pathnode->jpath.joinrestrictinfo = restrict_clauses;
+ pathnode->jpath.ojrelids = extra->ojrelids;
pathnode->path_hashclauses = hashclauses;
/* final_cost_hashjoin will fill in pathnode->num_batches */
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 76dad17e33..9542954f0c 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -57,6 +57,7 @@ static List *subbuild_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *input_rel,
Relids both_input_relids,
+ SpecialJoinInfo *sjinfo,
List *new_restrictlist);
static List *subbuild_joinrel_joinlist(RelOptInfo *joinrel,
List *joininfo_list,
@@ -1286,9 +1287,9 @@ build_joinrel_restrictlist(PlannerInfo *root,
* same clauses arriving from both input relations).
*/
result = subbuild_joinrel_restrictlist(root, joinrel, outer_rel,
- both_input_relids, NIL);
+ both_input_relids, sjinfo, NIL);
result = subbuild_joinrel_restrictlist(root, joinrel, inner_rel,
- both_input_relids, result);
+ both_input_relids, sjinfo, result);
/*
* Add on any clauses derived from EquivalenceClasses. These cannot be
@@ -1328,6 +1329,7 @@ subbuild_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *input_rel,
Relids both_input_relids,
+ SpecialJoinInfo *sjinfo,
List *new_restrictlist)
{
ListCell *l;
@@ -1349,11 +1351,15 @@ subbuild_joinrel_restrictlist(PlannerInfo *root,
*/
if (rinfo->has_clone || rinfo->is_clone)
{
- Assert(!RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids));
if (!bms_is_subset(rinfo->required_relids, both_input_relids))
continue;
if (bms_overlap(rinfo->incompatible_relids, both_input_relids))
continue;
+
+ Assert(!RINFO_IS_PUSHED_DOWN(rinfo,
+ bms_difference(joinrel->relids,
+ both_input_relids),
+ joinrel->relids));
}
else
{
@@ -1364,7 +1370,11 @@ subbuild_joinrel_restrictlist(PlannerInfo *root,
* (There is little point in checking incompatible_relids,
* because it'll be NULL.)
*/
- Assert(RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids) ||
+ Assert(sjinfo->ojrelid == 0 ||
+ RINFO_IS_PUSHED_DOWN(rinfo,
+ bms_difference(joinrel->relids,
+ both_input_relids),
+ joinrel->relids) ||
bms_is_subset(rinfo->required_relids,
both_input_relids));
}
@@ -2059,6 +2069,9 @@ have_partkey_equi_join(PlannerInfo *root, RelOptInfo *joinrel,
int cnt_pks;
bool pk_has_clause[PARTITION_MAX_KEYS];
bool strict_op;
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ rel1->relids,
+ rel2->relids);
/*
* This function must only be called when the joined relations have same
@@ -2079,7 +2092,7 @@ have_partkey_equi_join(PlannerInfo *root, RelOptInfo *joinrel,
/* If processing an outer join, only use its own join clauses. */
if (IS_OUTER_JOIN(jointype) &&
- RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrel->relids))
continue;
/* Skip clauses which can not be used for a join. */
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index d6d26a2b51..75945c3ab9 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -24,7 +24,6 @@
static RestrictInfo *make_restrictinfo_internal(PlannerInfo *root,
Expr *clause,
Expr *orclause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -34,7 +33,6 @@ static RestrictInfo *make_restrictinfo_internal(PlannerInfo *root,
Relids outer_relids);
static Expr *make_sub_restrictinfos(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -49,11 +47,11 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
*
* Build a RestrictInfo node containing the given subexpression.
*
- * The is_pushed_down, has_clone, is_clone, and pseudoconstant flags for the
- * RestrictInfo must be supplied by the caller, as well as the correct values
- * for security_level, incompatible_relids, and outer_relids.
- * required_relids can be NULL, in which case it defaults to the actual clause
- * contents (i.e., clause_relids).
+ * The has_clone, is_clone, and pseudoconstant flags for the RestrictInfo
+ * must be supplied by the caller, as well as the correct values for
+ * security_level, incompatible_relids, and outer_relids. required_relids
+ * can be NULL, in which case it defaults to the actual clause contents
+ * (i.e., clause_relids).
*
* We initialize fields that depend only on the given subexpression, leaving
* others that depend on context (or may never be needed at all) to be filled
@@ -62,7 +60,6 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
RestrictInfo *
make_restrictinfo(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -78,7 +75,6 @@ make_restrictinfo(PlannerInfo *root,
if (is_orclause(clause))
return (RestrictInfo *) make_sub_restrictinfos(root,
clause,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -93,7 +89,6 @@ make_restrictinfo(PlannerInfo *root,
return make_restrictinfo_internal(root,
clause,
NULL,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -112,7 +107,6 @@ static RestrictInfo *
make_restrictinfo_internal(PlannerInfo *root,
Expr *clause,
Expr *orclause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -126,7 +120,6 @@ make_restrictinfo_internal(PlannerInfo *root,
restrictinfo->clause = clause;
restrictinfo->orclause = orclause;
- restrictinfo->is_pushed_down = is_pushed_down;
restrictinfo->pseudoconstant = pseudoconstant;
restrictinfo->has_clone = has_clone;
restrictinfo->is_clone = is_clone;
@@ -259,9 +252,9 @@ make_restrictinfo_internal(PlannerInfo *root,
* implicit-AND lists at top level of RestrictInfo lists. Only ORs and
* simple clauses are valid RestrictInfos.
*
- * The same is_pushed_down, has_clone, is_clone, and pseudoconstant flag
- * values can be applied to all RestrictInfo nodes in the result. Likewise
- * for security_level, incompatible_relids, and outer_relids.
+ * The same has_clone, is_clone, and pseudoconstant flag values can be
+ * applied to all RestrictInfo nodes in the result. Likewise for
+ * security_level, incompatible_relids, and outer_relids.
*
* The given required_relids are attached to our top-level output,
* but any OR-clause constituents are allowed to default to just the
@@ -270,7 +263,6 @@ make_restrictinfo_internal(PlannerInfo *root,
static Expr *
make_sub_restrictinfos(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -288,7 +280,6 @@ make_sub_restrictinfos(PlannerInfo *root,
orlist = lappend(orlist,
make_sub_restrictinfos(root,
lfirst(temp),
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -299,7 +290,6 @@ make_sub_restrictinfos(PlannerInfo *root,
return (Expr *) make_restrictinfo_internal(root,
clause,
make_orclause(orlist),
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -317,7 +307,6 @@ make_sub_restrictinfos(PlannerInfo *root,
andlist = lappend(andlist,
make_sub_restrictinfos(root,
lfirst(temp),
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -331,7 +320,6 @@ make_sub_restrictinfos(PlannerInfo *root,
return (Expr *) make_restrictinfo_internal(root,
clause,
NULL,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -521,6 +509,7 @@ extract_actual_clauses(List *restrictinfo_list,
void
extract_actual_join_clauses(List *restrictinfo_list,
Relids joinrelids,
+ Relids ojrelids,
List **joinquals,
List **otherquals)
{
@@ -533,7 +522,7 @@ extract_actual_join_clauses(List *restrictinfo_list,
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
- if (RINFO_IS_PUSHED_DOWN(rinfo, joinrelids))
+ if (RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrelids))
{
if (!rinfo->pseudoconstant &&
!rinfo_is_constant_true(rinfo))
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index c17b53f7ad..c08f9551c3 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -2036,6 +2036,10 @@ typedef struct JoinPath
* joinrestrictinfo is needed in JoinPath, and can't be merged into the
* parent RelOptInfo.
*/
+
+ Relids ojrelids; /* the relid set of any outer joins that will be
+ * calculated at this join for outer join, or NULL
+ * for inner join */
} JoinPath;
/*
@@ -2408,27 +2412,16 @@ typedef struct LimitPath
* level of the outer join, which is true except when it is a "degenerate"
* condition that references only Vars from the nullable side of the join.
*
- * RestrictInfo nodes contain a flag to indicate whether a qual has been
- * pushed down to a lower level than its original syntactic placement in the
- * join tree would suggest. If an outer join prevents us from pushing a qual
- * down to its "natural" semantic level (the level associated with just the
- * base rels used in the qual) then we mark the qual with a "required_relids"
- * value including more than just the base rels it actually uses. By
- * pretending that the qual references all the rels required to form the outer
- * join, we prevent it from being evaluated below the outer join's joinrel.
- * When we do form the outer join's joinrel, we still need to distinguish
- * those quals that are actually in that join's JOIN/ON condition from those
- * that appeared elsewhere in the tree and were pushed down to the join rel
- * because they used no other rels. That's what the is_pushed_down flag is
- * for; it tells us that a qual is not an OUTER JOIN qual for the set of base
- * rels listed in required_relids. A clause that originally came from WHERE
- * or an INNER JOIN condition will *always* have its is_pushed_down flag set.
- * It's possible for an OUTER JOIN clause to be marked is_pushed_down too,
- * if we decide that it can be pushed down into the nullable side of the join.
- * In that case it acts as a plain filter qual for wherever it gets evaluated.
- * (In short, is_pushed_down is only false for non-degenerate outer join
- * conditions. Possibly we should rename it to reflect that meaning? But
- * see also the comments for RINFO_IS_PUSHED_DOWN, below.)
+ * If an outer join prevents us from pushing a qual down to its "natural"
+ * semantic level (the level associated with just the base rels used in the
+ * qual) then we mark the qual with a "required_relids" value including more
+ * than just the base rels it actually uses. By pretending that the qual
+ * references all the rels required to form the outer join, we prevent it from
+ * being evaluated below the outer join's joinrel. When we do form the outer
+ * join's joinrel, we still need to distinguish those quals that are actually
+ * in that join's JOIN/ON condition from those that appeared elsewhere in the
+ * tree and were pushed down to the join rel because they used no other rels.
+ * See the comments for RINFO_IS_PUSHED_DOWN for how we do that.
*
* There is also an incompatible_relids field, which is a set of outer-join
* relids above which we cannot evaluate the clause (because they might null
@@ -2515,9 +2508,6 @@ typedef struct RestrictInfo
/* the represented clause of WHERE or JOIN */
Expr *clause;
- /* true if clause was pushed down in level */
- bool is_pushed_down;
-
/* see comment above */
bool can_join pg_node_attr(equal_ignore);
@@ -2661,16 +2651,20 @@ typedef struct RestrictInfo
* This macro embodies the correct way to test whether a RestrictInfo is
* "pushed down" to a given outer join, that is, should be treated as a filter
* clause rather than a join clause at that outer join. This is certainly so
- * if is_pushed_down is true; but examining that is not sufficient anymore,
- * because outer-join clauses will get pushed down to lower outer joins when
- * we generate a path for the lower outer join that is parameterized by the
- * LHS of the upper one. We can detect such a clause by noting that its
+ * if the outer join clause references that outer join in any varnullingrels or
+ * phnullingrels set; but examining that is not sufficient anymore, because
+ * outer-join clauses will get pushed down to lower outer joins when we
+ * generate a path for the lower outer join that is parameterized by the LHS of
+ * the upper one. We can detect such a clause by noting that its
* required_relids exceed the scope of the join.
*/
-#define RINFO_IS_PUSHED_DOWN(rinfo, joinrelids) \
- ((rinfo)->is_pushed_down || \
+#define RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrelids) \
+ (bms_overlap((rinfo)->required_relids, (ojrelids)) || \
!bms_is_subset((rinfo)->required_relids, joinrelids))
+#define CALC_OUTER_JOIN_RELIDS(joinrelids, outerrelids, innerrelids) \
+ (bms_difference((joinrelids), bms_union((outerrelids), (innerrelids))))
+
/*
* Since mergejoinscansel() is a relatively expensive function, and would
* otherwise be invoked many times while planning a large join tree,
@@ -3170,6 +3164,8 @@ typedef struct SemiAntiJoinFactors
* sjinfo is extra info about special joins for selectivity estimation
* semifactors is as shown above (only valid for SEMI/ANTI/inner_unique joins)
* param_source_rels are OK targets for parameterization of result paths
+ * ojrelids is the relid set of any outer joins that will be calculated at this
+ * join, or NULL for inner join
*/
typedef struct JoinPathExtraData
{
@@ -3179,6 +3175,7 @@ typedef struct JoinPathExtraData
SpecialJoinInfo *sjinfo;
SemiAntiJoinFactors semifactors;
Relids param_source_rels;
+ Relids ojrelids;
} JoinPathExtraData;
/*
diff --git a/src/include/optimizer/cost.h b/src/include/optimizer/cost.h
index 6cf49705d3..11bdd120c2 100644
--- a/src/include/optimizer/cost.h
+++ b/src/include/optimizer/cost.h
@@ -183,7 +183,7 @@ extern void compute_semi_anti_join_factors(PlannerInfo *root,
JoinType jointype,
SpecialJoinInfo *sjinfo,
List *restrictlist,
- SemiAntiJoinFactors *semifactors);
+ JoinPathExtraData *extra);
extern void set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel);
extern double get_parameterized_baserel_size(PlannerInfo *root,
RelOptInfo *rel,
diff --git a/src/include/optimizer/restrictinfo.h b/src/include/optimizer/restrictinfo.h
index e140e619ac..04658f0067 100644
--- a/src/include/optimizer/restrictinfo.h
+++ b/src/include/optimizer/restrictinfo.h
@@ -19,12 +19,11 @@
/* Convenience macro for the common case of a valid-everywhere qual */
#define make_simple_restrictinfo(root, clause) \
- make_restrictinfo(root, clause, true, false, false, false, 0, \
+ make_restrictinfo(root, clause, false, false, false, 0, \
NULL, NULL, NULL)
extern RestrictInfo *make_restrictinfo(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -41,6 +40,7 @@ extern List *extract_actual_clauses(List *restrictinfo_list,
bool pseudoconstant);
extern void extract_actual_join_clauses(List *restrictinfo_list,
Relids joinrelids,
+ Relids ojrelids,
List **joinquals,
List **otherquals);
extern bool join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel);
--
2.31.0
Re: Retiring is_pushed_down
On Thu, 27 Jul 2023 at 08:25, Richard Guo <guofenglinux@gmail.com> wrote:
On Tue, Jul 25, 2023 at 3:39 PM Richard Guo <guofenglinux@gmail.com> wrote:
* This patch calculates the outer join relids that are being formed
generally in this way:bms_difference(joinrelids, bms_union(outerrelids, innerrelids))
Of course this can only be used after the outer join relids has been
added by add_outer_joins_to_relids(). This calculation is performed
multiple times during planning. I'm not sure if this has performance
issues. Maybe we can calculate it only once and store the result in
some place (such as in JoinPath)?In the v2 patch, I added a member in JoinPath to store the relid set of
any outer joins that will be calculated at this join, and this would
avoid repeating this calculation when creating nestloop/merge/hash join
plan nodes. Also fixed a comment in v2.
I'm seeing that there has been no activity in this thread for nearly 6
months, I'm planning to close this in the current commitfest unless
someone is planning to take it forward. It can be opened again when
there is more interest.
Regards,
Vignesh
Re: Retiring is_pushed_down
On Sun, Jan 21, 2024 at 8:37 PM vignesh C <vignesh21@gmail.com> wrote:
I'm seeing that there has been no activity in this thread for nearly 6
months, I'm planning to close this in the current commitfest unless
someone is planning to take it forward. It can be opened again when
there is more interest.
I'm planning to take it forward. The v2 patch does not compile any
more. Attached is an updated patch that is rebased over master and
fixes the compiling issues. Nothing else has changed.
Thanks
Richard
Attachments:
v3-0001-Retiring-is_pushed_down.patchapplication/octet-stream; name=v3-0001-Retiring-is_pushed_down.patchDownload
From 642b89bdbca7cf1fbc4f54363f9ef8af72bced80 Mon Sep 17 00:00:00 2001
From: Richard Guo <guofenglinux@gmail.com>
Date: Wed, 26 Jul 2023 16:16:49 +0800
Subject: [PATCH v3] Retiring is_pushed_down
When forming an outer join's joinrel, we have the is_pushed_down flag in
RestrictInfo nodes to distinguish those quals that are in that join's
JOIN/ON condition from those that were pushed down to the joinrel and
thus act as filter quals. Since now we have the outer-join-aware-Var
infrastructure, we can check to see whether a qual clause's
required_relids reference the outer join(s) being formed, in order to
tell if it's a join or filter clause. This seems like a more principled
way.
This patch is an attempt to retire the is_pushed_down flag.
---
contrib/postgres_fdw/postgres_fdw.c | 3 +-
src/backend/optimizer/path/costsize.c | 15 +++---
src/backend/optimizer/path/equivclass.c | 4 --
src/backend/optimizer/path/joinpath.c | 14 +++--
src/backend/optimizer/path/joinrels.c | 25 +++++----
src/backend/optimizer/plan/analyzejoins.c | 13 +++--
src/backend/optimizer/plan/createplan.c | 3 ++
src/backend/optimizer/plan/initsplan.c | 63 ++++-------------------
src/backend/optimizer/plan/subselect.c | 28 +++++++++-
src/backend/optimizer/util/inherit.c | 2 -
src/backend/optimizer/util/joininfo.c | 1 -
src/backend/optimizer/util/orclauses.c | 1 -
src/backend/optimizer/util/pathnode.c | 3 ++
src/backend/optimizer/util/relnode.c | 23 +++++++--
src/backend/optimizer/util/restrictinfo.c | 31 ++++-------
src/include/nodes/pathnodes.h | 57 ++++++++++----------
src/include/optimizer/cost.h | 2 +-
src/include/optimizer/restrictinfo.h | 4 +-
18 files changed, 148 insertions(+), 144 deletions(-)
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 142dcfc995..3fafc2760e 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -5847,7 +5847,7 @@ foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel, JoinType jointype,
rinfo->clause);
if (IS_OUTER_JOIN(jointype) &&
- !RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ !RINFO_IS_PUSHED_DOWN(rinfo, extra->ojrelids, joinrel->relids))
{
if (!is_remote_clause)
return false;
@@ -6609,7 +6609,6 @@ foreign_grouping_ok(PlannerInfo *root, RelOptInfo *grouped_rel,
Assert(!IsA(expr, RestrictInfo));
rinfo = make_restrictinfo(root,
expr,
- true,
false,
false,
false,
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index 8b76e98529..32212e9e77 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -4998,7 +4998,7 @@ get_restriction_qual_cost(PlannerInfo *root, RelOptInfo *baserel,
* sjinfo: SpecialJoinInfo relevant to this join
* restrictlist: join quals
* Output parameters:
- * *semifactors is filled in (see pathnodes.h for field definitions)
+ * extra->semifactors is filled in (see pathnodes.h for field definitions)
*/
void
compute_semi_anti_join_factors(PlannerInfo *root,
@@ -5008,7 +5008,7 @@ compute_semi_anti_join_factors(PlannerInfo *root,
JoinType jointype,
SpecialJoinInfo *sjinfo,
List *restrictlist,
- SemiAntiJoinFactors *semifactors)
+ JoinPathExtraData *extra)
{
Selectivity jselec;
Selectivity nselec;
@@ -5031,7 +5031,7 @@ compute_semi_anti_join_factors(PlannerInfo *root,
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
- if (!RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ if (!RINFO_IS_PUSHED_DOWN(rinfo, extra->ojrelids, joinrel->relids))
joinquals = lappend(joinquals, rinfo);
}
}
@@ -5099,8 +5099,8 @@ compute_semi_anti_join_factors(PlannerInfo *root,
else
avgmatch = 1.0;
- semifactors->outer_match_frac = jselec;
- semifactors->match_count = avgmatch;
+ extra->semifactors.outer_match_frac = jselec;
+ extra->semifactors.match_count = avgmatch;
}
/*
@@ -5465,13 +5465,16 @@ calc_joinrel_size_estimate(PlannerInfo *root,
List *joinquals = NIL;
List *pushedquals = NIL;
ListCell *l;
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ outer_rel->relids,
+ inner_rel->relids);
/* Grovel through the clauses to separate into two lists */
foreach(l, restrictlist)
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
- if (RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ if (RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrel->relids))
pushedquals = lappend(pushedquals, rinfo);
else
joinquals = lappend(joinquals, rinfo);
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 4bd60a09c6..74fe422f49 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -196,7 +196,6 @@ process_equivalence(PlannerInfo *root,
*p_restrictinfo =
make_restrictinfo(root,
(Expr *) ntest,
- restrictinfo->is_pushed_down,
restrictinfo->has_clone,
restrictinfo->is_clone,
restrictinfo->pseudoconstant,
@@ -2016,7 +2015,6 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* throw back a dummy replacement clause (see notes above) */
rinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(true, false),
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
false, /* pseudoconstant */
@@ -2044,7 +2042,6 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* throw back a dummy replacement clause (see notes above) */
rinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(true, false),
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
false, /* pseudoconstant */
@@ -2072,7 +2069,6 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* throw back a dummy replacement clause (see notes above) */
rinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(true, false),
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
false, /* pseudoconstant */
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index 6aca66f196..5cdee43d90 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -83,6 +83,7 @@ static List *select_mergejoin_clauses(PlannerInfo *root,
RelOptInfo *innerrel,
List *restrictlist,
JoinType jointype,
+ JoinPathExtraData *extra,
bool *mergejoin_allowed);
static void generate_mergejoin_paths(PlannerInfo *root,
RelOptInfo *joinrel,
@@ -149,6 +150,9 @@ add_paths_to_joinrel(PlannerInfo *root,
extra.mergeclause_list = NIL;
extra.sjinfo = sjinfo;
extra.param_source_rels = NULL;
+ extra.ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ outerrel->relids,
+ innerrel->relids);
/*
* See if the inner relation is provably unique for this outer rel.
@@ -213,6 +217,7 @@ add_paths_to_joinrel(PlannerInfo *root,
innerrel,
restrictlist,
jointype,
+ &extra,
&mergejoin_allowed);
/*
@@ -222,7 +227,7 @@ add_paths_to_joinrel(PlannerInfo *root,
if (jointype == JOIN_SEMI || jointype == JOIN_ANTI || extra.inner_unique)
compute_semi_anti_join_factors(root, joinrel, outerrel, innerrel,
jointype, sjinfo, restrictlist,
- &extra.semifactors);
+ &extra);
/*
* Decide whether it's sensible to generate parameterized paths for this
@@ -2129,7 +2134,8 @@ hash_inner_and_outer(PlannerInfo *root,
* If processing an outer join, only use its own join clauses for
* hashing. For inner joins we need not be so picky.
*/
- if (isouterjoin && RINFO_IS_PUSHED_DOWN(restrictinfo, joinrel->relids))
+ if (isouterjoin &&
+ RINFO_IS_PUSHED_DOWN(restrictinfo, extra->ojrelids, joinrel->relids))
continue;
if (!restrictinfo->can_join ||
@@ -2361,6 +2367,7 @@ select_mergejoin_clauses(PlannerInfo *root,
RelOptInfo *innerrel,
List *restrictlist,
JoinType jointype,
+ JoinPathExtraData *extra,
bool *mergejoin_allowed)
{
List *result_list = NIL;
@@ -2378,7 +2385,8 @@ select_mergejoin_clauses(PlannerInfo *root,
* we don't set have_nonmergeable_joinclause here because pushed-down
* clauses will become otherquals not joinquals.)
*/
- if (isouterjoin && RINFO_IS_PUSHED_DOWN(restrictinfo, joinrel->relids))
+ if (isouterjoin &&
+ RINFO_IS_PUSHED_DOWN(restrictinfo, extra->ojrelids, joinrel->relids))
continue;
/* Check that clause is a mergeable operator clause */
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 4750579b0a..5cc277e190 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -34,6 +34,7 @@ static bool has_join_restriction(PlannerInfo *root, RelOptInfo *rel);
static bool has_legal_joinclause(PlannerInfo *root, RelOptInfo *rel);
static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
+ Relids ojrelids,
bool only_pushed_down);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
@@ -876,6 +877,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *sjinfo, List *restrictlist)
{
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ rel1->relids,
+ rel2->relids);
+
/*
* Consider paths using each rel as both outer and inner. Depending on
* the join type, a provably empty outer or inner rel might mean the join
@@ -898,7 +903,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
{
case JOIN_INNER:
if (is_dummy_rel(rel1) || is_dummy_rel(rel2) ||
- restriction_is_constant_false(restrictlist, joinrel, false))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, false))
{
mark_dummy_rel(joinrel);
break;
@@ -912,12 +917,12 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
break;
case JOIN_LEFT:
if (is_dummy_rel(rel1) ||
- restriction_is_constant_false(restrictlist, joinrel, true))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, true))
{
mark_dummy_rel(joinrel);
break;
}
- if (restriction_is_constant_false(restrictlist, joinrel, false) &&
+ if (restriction_is_constant_false(restrictlist, joinrel, ojrelids, false) &&
bms_is_subset(rel2->relids, sjinfo->syn_righthand))
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
@@ -929,7 +934,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
break;
case JOIN_FULL:
if ((is_dummy_rel(rel1) && is_dummy_rel(rel2)) ||
- restriction_is_constant_false(restrictlist, joinrel, true))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, true))
{
mark_dummy_rel(joinrel);
break;
@@ -965,7 +970,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
bms_is_subset(sjinfo->min_righthand, rel2->relids))
{
if (is_dummy_rel(rel1) || is_dummy_rel(rel2) ||
- restriction_is_constant_false(restrictlist, joinrel, false))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, false))
{
mark_dummy_rel(joinrel);
break;
@@ -988,7 +993,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
sjinfo) != NULL)
{
if (is_dummy_rel(rel1) || is_dummy_rel(rel2) ||
- restriction_is_constant_false(restrictlist, joinrel, false))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, false))
{
mark_dummy_rel(joinrel);
break;
@@ -1003,12 +1008,12 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
break;
case JOIN_ANTI:
if (is_dummy_rel(rel1) ||
- restriction_is_constant_false(restrictlist, joinrel, true))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, true))
{
mark_dummy_rel(joinrel);
break;
}
- if (restriction_is_constant_false(restrictlist, joinrel, false) &&
+ if (restriction_is_constant_false(restrictlist, joinrel, ojrelids, false) &&
bms_is_subset(rel2->relids, sjinfo->syn_righthand))
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
@@ -1405,6 +1410,7 @@ mark_dummy_rel(RelOptInfo *rel)
static bool
restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
+ Relids ojrelids,
bool only_pushed_down)
{
ListCell *lc;
@@ -1419,7 +1425,8 @@ restriction_is_constant_false(List *restrictlist,
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, lc);
- if (only_pushed_down && !RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ if (only_pushed_down &&
+ !RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrel->relids))
continue;
if (rinfo->clause && IsA(rinfo->clause, Const))
diff --git a/src/backend/optimizer/plan/analyzejoins.c b/src/backend/optimizer/plan/analyzejoins.c
index 7dcb74572a..2cd44539a0 100644
--- a/src/backend/optimizer/plan/analyzejoins.c
+++ b/src/backend/optimizer/plan/analyzejoins.c
@@ -312,7 +312,9 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
* above the outer join, even if it references no other rels (it might
* be from WHERE, for example).
*/
- if (RINFO_IS_PUSHED_DOWN(restrictinfo, joinrelids))
+ if (RINFO_IS_PUSHED_DOWN(restrictinfo,
+ bms_make_singleton(sjinfo->ojrelid),
+ joinrelids))
continue; /* ignore; not useful here */
/* Ignore if it's not a mergejoinable clause */
@@ -542,7 +544,9 @@ remove_leftjoinrel_from_query(PlannerInfo *root, int relid,
remove_join_clause_from_rels(root, rinfo, rinfo->required_relids);
- if (RINFO_IS_PUSHED_DOWN(rinfo, join_plus_commute))
+ if (RINFO_IS_PUSHED_DOWN(rinfo,
+ bms_make_singleton(sjinfo->ojrelid),
+ join_plus_commute))
{
/*
* There might be references to relid or ojrelid in the
@@ -1393,6 +1397,9 @@ is_innerrel_unique_for(PlannerInfo *root,
{
List *clause_list = NIL;
ListCell *lc;
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrelids,
+ outerrelids,
+ innerrel->relids);
/*
* Search for mergejoinable clauses that constrain the inner rel against
@@ -1410,7 +1417,7 @@ is_innerrel_unique_for(PlannerInfo *root,
* join, we can't use it.
*/
if (IS_OUTER_JOIN(jointype) &&
- RINFO_IS_PUSHED_DOWN(restrictinfo, joinrelids))
+ RINFO_IS_PUSHED_DOWN(restrictinfo, ojrelids, joinrelids))
continue;
/* Ignore if it's not a mergejoinable clause */
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 610f4a56d6..4510325739 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -4377,6 +4377,7 @@ create_nestloop_plan(PlannerInfo *root,
{
extract_actual_join_clauses(joinrestrictclauses,
best_path->jpath.path.parent->relids,
+ best_path->jpath.ojrelids,
&joinclauses, &otherclauses);
}
else
@@ -4465,6 +4466,7 @@ create_mergejoin_plan(PlannerInfo *root,
{
extract_actual_join_clauses(joinclauses,
best_path->jpath.path.parent->relids,
+ best_path->jpath.ojrelids,
&joinclauses, &otherclauses);
}
else
@@ -4767,6 +4769,7 @@ create_hashjoin_plan(PlannerInfo *root,
{
extract_actual_join_clauses(joinclauses,
best_path->jpath.path.parent->relids,
+ best_path->jpath.ojrelids,
&joinclauses, &otherclauses);
}
else
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index d4a9d77d7f..d6ccefdeff 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -963,19 +963,16 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
child_domain->jd_relids);
jtitem->qualscope = bms_union(left_item->qualscope,
right_item->qualscope);
- /* caution: ANTI join derived from SEMI will lack rtindex */
- if (j->rtindex != 0)
- {
- parent_domain->jd_relids =
- bms_add_member(parent_domain->jd_relids,
- j->rtindex);
- jtitem->qualscope = bms_add_member(jtitem->qualscope,
+ Assert(j->rtindex != 0);
+ parent_domain->jd_relids =
+ bms_add_member(parent_domain->jd_relids,
+ j->rtindex);
+ jtitem->qualscope = bms_add_member(jtitem->qualscope,
+ j->rtindex);
+ root->outer_join_rels = bms_add_member(root->outer_join_rels,
j->rtindex);
- root->outer_join_rels = bms_add_member(root->outer_join_rels,
- j->rtindex);
- mark_rels_nulled_by_join(root, j->rtindex,
- right_item->qualscope);
- }
+ mark_rels_nulled_by_join(root, j->rtindex,
+ right_item->qualscope);
jtitem->inner_join_rels = bms_union(left_item->inner_join_rels,
right_item->inner_join_rels);
jtitem->left_rels = left_item->qualscope;
@@ -2209,7 +2206,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
List **postponed_oj_qual_list)
{
Relids relids;
- bool is_pushed_down;
bool pseudoconstant = false;
bool maybe_equivalence;
bool maybe_outer_join;
@@ -2319,37 +2315,9 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
}
}
- /*----------
+ /*
* Check to see if clause application must be delayed by outer-join
* considerations.
- *
- * A word about is_pushed_down: we mark the qual as "pushed down" if
- * it is (potentially) applicable at a level different from its original
- * syntactic level. This flag is used to distinguish OUTER JOIN ON quals
- * from other quals pushed down to the same joinrel. The rules are:
- * WHERE quals and INNER JOIN quals: is_pushed_down = true.
- * Non-degenerate OUTER JOIN quals: is_pushed_down = false.
- * Degenerate OUTER JOIN quals: is_pushed_down = true.
- * A "degenerate" OUTER JOIN qual is one that doesn't mention the
- * non-nullable side, and hence can be pushed down into the nullable side
- * without changing the join result. It is correct to treat it as a
- * regular filter condition at the level where it is evaluated.
- *
- * Note: it is not immediately obvious that a simple boolean is enough
- * for this: if for some reason we were to attach a degenerate qual to
- * its original join level, it would need to be treated as an outer join
- * qual there. However, this cannot happen, because all the rels the
- * clause mentions must be in the outer join's min_righthand, therefore
- * the join it needs must be formed before the outer join; and we always
- * attach quals to the lowest level where they can be evaluated. But
- * if we were ever to re-introduce a mechanism for delaying evaluation
- * of "expensive" quals, this area would need work.
- *
- * Note: generally, use of is_pushed_down has to go through the macro
- * RINFO_IS_PUSHED_DOWN, because that flag alone is not always sufficient
- * to tell whether a clause must be treated as pushed-down in context.
- * This seems like another reason why it should perhaps be rethought.
- *----------
*/
if (bms_overlap(relids, outerjoin_nonnullable))
{
@@ -2373,7 +2341,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
* deductions, if it is mergejoinable. So consider adding it to the
* lists of set-aside outer-join clauses.
*/
- is_pushed_down = false;
maybe_equivalence = false;
maybe_outer_join = true;
@@ -2390,12 +2357,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
}
else
{
- /*
- * Normal qual clause or degenerate outer-join clause. Either way, we
- * can mark it as pushed-down.
- */
- is_pushed_down = true;
-
/*
* It's possible that this is an IS NULL clause that's redundant with
* a lower antijoin; if so we can just discard it. We need not test
@@ -2420,7 +2381,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
*/
restrictinfo = make_restrictinfo(root,
(Expr *) clause,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -2648,7 +2608,6 @@ add_base_clause_to_rel(PlannerInfo *root, Index relid,
restrictinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(false, false),
- restrictinfo->is_pushed_down,
restrictinfo->has_clone,
restrictinfo->is_clone,
restrictinfo->pseudoconstant,
@@ -2973,7 +2932,6 @@ process_implied_equality(PlannerInfo *root,
*/
restrictinfo = make_restrictinfo(root,
(Expr *) clause,
- true, /* is_pushed_down */
false, /* !has_clone */
false, /* !is_clone */
pseudoconstant,
@@ -3067,7 +3025,6 @@ build_implied_join_equality(PlannerInfo *root,
*/
restrictinfo = make_restrictinfo(root,
clause,
- true, /* is_pushed_down */
false, /* !has_clone */
false, /* !is_clone */
false, /* pseudoconstant */
diff --git a/src/backend/optimizer/plan/subselect.c b/src/backend/optimizer/plan/subselect.c
index 3115d79ad9..433f7ecf6f 100644
--- a/src/backend/optimizer/plan/subselect.c
+++ b/src/backend/optimizer/plan/subselect.c
@@ -1519,7 +1519,33 @@ convert_EXISTS_sublink_to_join(PlannerInfo *root, SubLink *sublink,
result->join_using_alias = NULL;
result->quals = whereClause;
result->alias = NULL;
- result->rtindex = 0; /* we don't need an RTE for it */
+ result->rtindex = 0; /* we don't need an RTE for JOIN_SEMI */
+
+ /*
+ * Add a RTE for JOIN_ANTI
+ */
+ if (result->jointype == JOIN_ANTI)
+ {
+ ParseNamespaceItem *jnsitem;
+ ParseState *pstate;
+
+ /* Create a dummy ParseState for addRangeTableEntryForJoin */
+ pstate = make_parsestate(NULL);
+
+ jnsitem = addRangeTableEntryForJoin(pstate,
+ NULL,
+ NULL,
+ JOIN_ANTI,
+ 0,
+ NULL,
+ NIL,
+ NIL,
+ NULL,
+ NULL,
+ false);
+ parse->rtable = lappend(parse->rtable, jnsitem->p_rte);
+ result->rtindex = list_length(parse->rtable);
+ }
return result;
}
diff --git a/src/backend/optimizer/util/inherit.c b/src/backend/optimizer/util/inherit.c
index 5c7acf8a90..9e718f9360 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -889,7 +889,6 @@ apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
childquals = lappend(childquals,
make_restrictinfo(root,
(Expr *) onecq,
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
pseudoconstant,
@@ -927,7 +926,6 @@ apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
/* not likely that we'd see constants here, so no check */
childquals = lappend(childquals,
make_restrictinfo(root, qual,
- true,
false, false,
false,
security_level,
diff --git a/src/backend/optimizer/util/joininfo.c b/src/backend/optimizer/util/joininfo.c
index 5fb0c17630..173aff761b 100644
--- a/src/backend/optimizer/util/joininfo.c
+++ b/src/backend/optimizer/util/joininfo.c
@@ -115,7 +115,6 @@ add_join_clause_to_rels(PlannerInfo *root,
restrictinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(false, false),
- restrictinfo->is_pushed_down,
restrictinfo->has_clone,
restrictinfo->is_clone,
restrictinfo->pseudoconstant,
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index 6a5234e2ce..c2e64e6cfa 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -265,7 +265,6 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
*/
or_rinfo = make_restrictinfo(root,
orclause,
- true,
false,
false,
false,
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 8dbf790e89..17b6675378 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -2505,6 +2505,7 @@ create_nestloop_path(PlannerInfo *root,
pathnode->jpath.outerjoinpath = outer_path;
pathnode->jpath.innerjoinpath = inner_path;
pathnode->jpath.joinrestrictinfo = restrict_clauses;
+ pathnode->jpath.ojrelids = extra->ojrelids;
final_cost_nestloop(root, pathnode, workspace, extra);
@@ -2569,6 +2570,7 @@ create_mergejoin_path(PlannerInfo *root,
pathnode->jpath.outerjoinpath = outer_path;
pathnode->jpath.innerjoinpath = inner_path;
pathnode->jpath.joinrestrictinfo = restrict_clauses;
+ pathnode->jpath.ojrelids = extra->ojrelids;
pathnode->path_mergeclauses = mergeclauses;
pathnode->outersortkeys = outersortkeys;
pathnode->innersortkeys = innersortkeys;
@@ -2646,6 +2648,7 @@ create_hashjoin_path(PlannerInfo *root,
pathnode->jpath.outerjoinpath = outer_path;
pathnode->jpath.innerjoinpath = inner_path;
pathnode->jpath.joinrestrictinfo = restrict_clauses;
+ pathnode->jpath.ojrelids = extra->ojrelids;
pathnode->path_hashclauses = hashclauses;
/* final_cost_hashjoin will fill in pathnode->num_batches */
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index e5f4062bfb..72562ba767 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -58,6 +58,7 @@ static List *subbuild_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *input_rel,
Relids both_input_relids,
+ SpecialJoinInfo *sjinfo,
List *new_restrictlist);
static List *subbuild_joinrel_joinlist(RelOptInfo *joinrel,
List *joininfo_list,
@@ -1304,9 +1305,9 @@ build_joinrel_restrictlist(PlannerInfo *root,
* same clauses arriving from both input relations).
*/
result = subbuild_joinrel_restrictlist(root, joinrel, outer_rel,
- both_input_relids, NIL);
+ both_input_relids, sjinfo, NIL);
result = subbuild_joinrel_restrictlist(root, joinrel, inner_rel,
- both_input_relids, result);
+ both_input_relids, sjinfo, result);
/*
* Add on any clauses derived from EquivalenceClasses. These cannot be
@@ -1346,6 +1347,7 @@ subbuild_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *input_rel,
Relids both_input_relids,
+ SpecialJoinInfo *sjinfo,
List *new_restrictlist)
{
ListCell *l;
@@ -1367,11 +1369,15 @@ subbuild_joinrel_restrictlist(PlannerInfo *root,
*/
if (rinfo->has_clone || rinfo->is_clone)
{
- Assert(!RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids));
if (!bms_is_subset(rinfo->required_relids, both_input_relids))
continue;
if (bms_overlap(rinfo->incompatible_relids, both_input_relids))
continue;
+
+ Assert(!RINFO_IS_PUSHED_DOWN(rinfo,
+ bms_difference(joinrel->relids,
+ both_input_relids),
+ joinrel->relids));
}
else
{
@@ -1382,7 +1388,11 @@ subbuild_joinrel_restrictlist(PlannerInfo *root,
* (There is little point in checking incompatible_relids,
* because it'll be NULL.)
*/
- Assert(RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids) ||
+ Assert(sjinfo->ojrelid == 0 ||
+ RINFO_IS_PUSHED_DOWN(rinfo,
+ bms_difference(joinrel->relids,
+ both_input_relids),
+ joinrel->relids) ||
bms_is_subset(rinfo->required_relids,
both_input_relids));
}
@@ -2077,6 +2087,9 @@ have_partkey_equi_join(PlannerInfo *root, RelOptInfo *joinrel,
int cnt_pks;
bool pk_has_clause[PARTITION_MAX_KEYS];
bool strict_op;
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ rel1->relids,
+ rel2->relids);
/*
* This function must only be called when the joined relations have same
@@ -2097,7 +2110,7 @@ have_partkey_equi_join(PlannerInfo *root, RelOptInfo *joinrel,
/* If processing an outer join, only use its own join clauses. */
if (IS_OUTER_JOIN(jointype) &&
- RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrel->relids))
continue;
/* Skip clauses which can not be used for a join. */
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index 0b406e9334..5d05287bf3 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -24,7 +24,6 @@
static RestrictInfo *make_restrictinfo_internal(PlannerInfo *root,
Expr *clause,
Expr *orclause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -34,7 +33,6 @@ static RestrictInfo *make_restrictinfo_internal(PlannerInfo *root,
Relids outer_relids);
static Expr *make_sub_restrictinfos(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -49,11 +47,11 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
*
* Build a RestrictInfo node containing the given subexpression.
*
- * The is_pushed_down, has_clone, is_clone, and pseudoconstant flags for the
- * RestrictInfo must be supplied by the caller, as well as the correct values
- * for security_level, incompatible_relids, and outer_relids.
- * required_relids can be NULL, in which case it defaults to the actual clause
- * contents (i.e., clause_relids).
+ * The has_clone, is_clone, and pseudoconstant flags for the RestrictInfo
+ * must be supplied by the caller, as well as the correct values for
+ * security_level, incompatible_relids, and outer_relids. required_relids
+ * can be NULL, in which case it defaults to the actual clause contents
+ * (i.e., clause_relids).
*
* We initialize fields that depend only on the given subexpression, leaving
* others that depend on context (or may never be needed at all) to be filled
@@ -62,7 +60,6 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
RestrictInfo *
make_restrictinfo(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -78,7 +75,6 @@ make_restrictinfo(PlannerInfo *root,
if (is_orclause(clause))
return (RestrictInfo *) make_sub_restrictinfos(root,
clause,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -93,7 +89,6 @@ make_restrictinfo(PlannerInfo *root,
return make_restrictinfo_internal(root,
clause,
NULL,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -112,7 +107,6 @@ static RestrictInfo *
make_restrictinfo_internal(PlannerInfo *root,
Expr *clause,
Expr *orclause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -126,7 +120,6 @@ make_restrictinfo_internal(PlannerInfo *root,
restrictinfo->clause = clause;
restrictinfo->orclause = orclause;
- restrictinfo->is_pushed_down = is_pushed_down;
restrictinfo->pseudoconstant = pseudoconstant;
restrictinfo->has_clone = has_clone;
restrictinfo->is_clone = is_clone;
@@ -259,9 +252,9 @@ make_restrictinfo_internal(PlannerInfo *root,
* implicit-AND lists at top level of RestrictInfo lists. Only ORs and
* simple clauses are valid RestrictInfos.
*
- * The same is_pushed_down, has_clone, is_clone, and pseudoconstant flag
- * values can be applied to all RestrictInfo nodes in the result. Likewise
- * for security_level, incompatible_relids, and outer_relids.
+ * The same has_clone, is_clone, and pseudoconstant flag values can be
+ * applied to all RestrictInfo nodes in the result. Likewise for
+ * security_level, incompatible_relids, and outer_relids.
*
* The given required_relids are attached to our top-level output,
* but any OR-clause constituents are allowed to default to just the
@@ -270,7 +263,6 @@ make_restrictinfo_internal(PlannerInfo *root,
static Expr *
make_sub_restrictinfos(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -288,7 +280,6 @@ make_sub_restrictinfos(PlannerInfo *root,
orlist = lappend(orlist,
make_sub_restrictinfos(root,
lfirst(temp),
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -299,7 +290,6 @@ make_sub_restrictinfos(PlannerInfo *root,
return (Expr *) make_restrictinfo_internal(root,
clause,
make_orclause(orlist),
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -317,7 +307,6 @@ make_sub_restrictinfos(PlannerInfo *root,
andlist = lappend(andlist,
make_sub_restrictinfos(root,
lfirst(temp),
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -331,7 +320,6 @@ make_sub_restrictinfos(PlannerInfo *root,
return (Expr *) make_restrictinfo_internal(root,
clause,
NULL,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -521,6 +509,7 @@ extract_actual_clauses(List *restrictinfo_list,
void
extract_actual_join_clauses(List *restrictinfo_list,
Relids joinrelids,
+ Relids ojrelids,
List **joinquals,
List **otherquals)
{
@@ -533,7 +522,7 @@ extract_actual_join_clauses(List *restrictinfo_list,
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
- if (RINFO_IS_PUSHED_DOWN(rinfo, joinrelids))
+ if (RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrelids))
{
if (!rinfo->pseudoconstant &&
!rinfo_is_constant_true(rinfo))
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 534692bee1..de1122d6e0 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -2061,6 +2061,10 @@ typedef struct JoinPath
* joinrestrictinfo is needed in JoinPath, and can't be merged into the
* parent RelOptInfo.
*/
+
+ Relids ojrelids; /* the relid set of any outer joins that will be
+ * calculated at this join for outer join, or NULL
+ * for inner join */
} JoinPath;
/*
@@ -2433,27 +2437,16 @@ typedef struct LimitPath
* level of the outer join, which is true except when it is a "degenerate"
* condition that references only Vars from the nullable side of the join.
*
- * RestrictInfo nodes contain a flag to indicate whether a qual has been
- * pushed down to a lower level than its original syntactic placement in the
- * join tree would suggest. If an outer join prevents us from pushing a qual
- * down to its "natural" semantic level (the level associated with just the
- * base rels used in the qual) then we mark the qual with a "required_relids"
- * value including more than just the base rels it actually uses. By
- * pretending that the qual references all the rels required to form the outer
- * join, we prevent it from being evaluated below the outer join's joinrel.
- * When we do form the outer join's joinrel, we still need to distinguish
- * those quals that are actually in that join's JOIN/ON condition from those
- * that appeared elsewhere in the tree and were pushed down to the join rel
- * because they used no other rels. That's what the is_pushed_down flag is
- * for; it tells us that a qual is not an OUTER JOIN qual for the set of base
- * rels listed in required_relids. A clause that originally came from WHERE
- * or an INNER JOIN condition will *always* have its is_pushed_down flag set.
- * It's possible for an OUTER JOIN clause to be marked is_pushed_down too,
- * if we decide that it can be pushed down into the nullable side of the join.
- * In that case it acts as a plain filter qual for wherever it gets evaluated.
- * (In short, is_pushed_down is only false for non-degenerate outer join
- * conditions. Possibly we should rename it to reflect that meaning? But
- * see also the comments for RINFO_IS_PUSHED_DOWN, below.)
+ * If an outer join prevents us from pushing a qual down to its "natural"
+ * semantic level (the level associated with just the base rels used in the
+ * qual) then we mark the qual with a "required_relids" value including more
+ * than just the base rels it actually uses. By pretending that the qual
+ * references all the rels required to form the outer join, we prevent it from
+ * being evaluated below the outer join's joinrel. When we do form the outer
+ * join's joinrel, we still need to distinguish those quals that are actually
+ * in that join's JOIN/ON condition from those that appeared elsewhere in the
+ * tree and were pushed down to the join rel because they used no other rels.
+ * See the comments for RINFO_IS_PUSHED_DOWN for how we do that.
*
* There is also an incompatible_relids field, which is a set of outer-join
* relids above which we cannot evaluate the clause (because they might null
@@ -2540,9 +2533,6 @@ typedef struct RestrictInfo
/* the represented clause of WHERE or JOIN */
Expr *clause;
- /* true if clause was pushed down in level */
- bool is_pushed_down;
-
/* see comment above */
bool can_join pg_node_attr(equal_ignore);
@@ -2689,16 +2679,20 @@ typedef struct RestrictInfo
* This macro embodies the correct way to test whether a RestrictInfo is
* "pushed down" to a given outer join, that is, should be treated as a filter
* clause rather than a join clause at that outer join. This is certainly so
- * if is_pushed_down is true; but examining that is not sufficient anymore,
- * because outer-join clauses will get pushed down to lower outer joins when
- * we generate a path for the lower outer join that is parameterized by the
- * LHS of the upper one. We can detect such a clause by noting that its
+ * if the outer join clause references that outer join in any varnullingrels or
+ * phnullingrels set; but examining that is not sufficient anymore, because
+ * outer-join clauses will get pushed down to lower outer joins when we
+ * generate a path for the lower outer join that is parameterized by the LHS of
+ * the upper one. We can detect such a clause by noting that its
* required_relids exceed the scope of the join.
*/
-#define RINFO_IS_PUSHED_DOWN(rinfo, joinrelids) \
- ((rinfo)->is_pushed_down || \
+#define RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrelids) \
+ (bms_overlap((rinfo)->required_relids, (ojrelids)) || \
!bms_is_subset((rinfo)->required_relids, joinrelids))
+#define CALC_OUTER_JOIN_RELIDS(joinrelids, outerrelids, innerrelids) \
+ (bms_difference((joinrelids), bms_union((outerrelids), (innerrelids))))
+
/*
* Since mergejoinscansel() is a relatively expensive function, and would
* otherwise be invoked many times while planning a large join tree,
@@ -3198,6 +3192,8 @@ typedef struct SemiAntiJoinFactors
* sjinfo is extra info about special joins for selectivity estimation
* semifactors is as shown above (only valid for SEMI/ANTI/inner_unique joins)
* param_source_rels are OK targets for parameterization of result paths
+ * ojrelids is the relid set of any outer joins that will be calculated at this
+ * join, or NULL for inner join
*/
typedef struct JoinPathExtraData
{
@@ -3207,6 +3203,7 @@ typedef struct JoinPathExtraData
SpecialJoinInfo *sjinfo;
SemiAntiJoinFactors semifactors;
Relids param_source_rels;
+ Relids ojrelids;
} JoinPathExtraData;
/*
diff --git a/src/include/optimizer/cost.h b/src/include/optimizer/cost.h
index b1c51a4e70..6b83499c62 100644
--- a/src/include/optimizer/cost.h
+++ b/src/include/optimizer/cost.h
@@ -183,7 +183,7 @@ extern void compute_semi_anti_join_factors(PlannerInfo *root,
JoinType jointype,
SpecialJoinInfo *sjinfo,
List *restrictlist,
- SemiAntiJoinFactors *semifactors);
+ JoinPathExtraData *extra);
extern void set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel);
extern double get_parameterized_baserel_size(PlannerInfo *root,
RelOptInfo *rel,
diff --git a/src/include/optimizer/restrictinfo.h b/src/include/optimizer/restrictinfo.h
index 1b42c832c5..295ce764d3 100644
--- a/src/include/optimizer/restrictinfo.h
+++ b/src/include/optimizer/restrictinfo.h
@@ -19,12 +19,11 @@
/* Convenience macro for the common case of a valid-everywhere qual */
#define make_simple_restrictinfo(root, clause) \
- make_restrictinfo(root, clause, true, false, false, false, 0, \
+ make_restrictinfo(root, clause, false, false, false, 0, \
NULL, NULL, NULL)
extern RestrictInfo *make_restrictinfo(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -41,6 +40,7 @@ extern List *extract_actual_clauses(List *restrictinfo_list,
bool pseudoconstant);
extern void extract_actual_join_clauses(List *restrictinfo_list,
Relids joinrelids,
+ Relids ojrelids,
List **joinquals,
List **otherquals);
extern bool join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel);
--
2.31.0
Re: Retiring is_pushed_down
Here is another rebase over 3af7040985. Nothing else has changed.
Thanks
Richard
Attachments:
v4-0001-Retiring-is_pushed_down.patchapplication/octet-stream; name=v4-0001-Retiring-is_pushed_down.patchDownload
From 3f2682d8c7b8116e5853e121f843d4c3ad8eacbe Mon Sep 17 00:00:00 2001
From: Richard Guo <guofenglinux@gmail.com>
Date: Wed, 26 Jul 2023 16:16:49 +0800
Subject: [PATCH v4] Retiring is_pushed_down
When forming an outer join's joinrel, we have the is_pushed_down flag in
RestrictInfo nodes to distinguish those quals that are in that join's
JOIN/ON condition from those that were pushed down to the joinrel and
thus act as filter quals. Since now we have the outer-join-aware-Var
infrastructure, we can check to see whether a qual clause's
required_relids reference the outer join(s) being formed, in order to
tell if it's a join or filter clause. This seems like a more principled
way.
This patch is an attempt to retire the is_pushed_down flag.
---
contrib/postgres_fdw/postgres_fdw.c | 3 +-
src/backend/optimizer/path/costsize.c | 15 +++---
src/backend/optimizer/path/equivclass.c | 4 --
src/backend/optimizer/path/joinpath.c | 14 +++--
src/backend/optimizer/path/joinrels.c | 25 +++++----
src/backend/optimizer/plan/analyzejoins.c | 13 +++--
src/backend/optimizer/plan/createplan.c | 3 ++
src/backend/optimizer/plan/initsplan.c | 63 ++++-------------------
src/backend/optimizer/plan/subselect.c | 28 +++++++++-
src/backend/optimizer/util/inherit.c | 2 -
src/backend/optimizer/util/joininfo.c | 1 -
src/backend/optimizer/util/orclauses.c | 1 -
src/backend/optimizer/util/pathnode.c | 3 ++
src/backend/optimizer/util/relnode.c | 23 +++++++--
src/backend/optimizer/util/restrictinfo.c | 31 ++++-------
src/include/nodes/pathnodes.h | 57 ++++++++++----------
src/include/optimizer/cost.h | 2 +-
src/include/optimizer/restrictinfo.h | 4 +-
18 files changed, 148 insertions(+), 144 deletions(-)
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 4053cd641c..635b5c7c95 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -5847,7 +5847,7 @@ foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel, JoinType jointype,
rinfo->clause);
if (IS_OUTER_JOIN(jointype) &&
- !RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ !RINFO_IS_PUSHED_DOWN(rinfo, extra->ojrelids, joinrel->relids))
{
if (!is_remote_clause)
return false;
@@ -6609,7 +6609,6 @@ foreign_grouping_ok(PlannerInfo *root, RelOptInfo *grouped_rel,
Assert(!IsA(expr, RestrictInfo));
rinfo = make_restrictinfo(root,
expr,
- true,
false,
false,
false,
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index ee23ed7835..625e500feb 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -4998,7 +4998,7 @@ get_restriction_qual_cost(PlannerInfo *root, RelOptInfo *baserel,
* sjinfo: SpecialJoinInfo relevant to this join
* restrictlist: join quals
* Output parameters:
- * *semifactors is filled in (see pathnodes.h for field definitions)
+ * extra->semifactors is filled in (see pathnodes.h for field definitions)
*/
void
compute_semi_anti_join_factors(PlannerInfo *root,
@@ -5008,7 +5008,7 @@ compute_semi_anti_join_factors(PlannerInfo *root,
JoinType jointype,
SpecialJoinInfo *sjinfo,
List *restrictlist,
- SemiAntiJoinFactors *semifactors)
+ JoinPathExtraData *extra)
{
Selectivity jselec;
Selectivity nselec;
@@ -5031,7 +5031,7 @@ compute_semi_anti_join_factors(PlannerInfo *root,
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
- if (!RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ if (!RINFO_IS_PUSHED_DOWN(rinfo, extra->ojrelids, joinrel->relids))
joinquals = lappend(joinquals, rinfo);
}
}
@@ -5083,8 +5083,8 @@ compute_semi_anti_join_factors(PlannerInfo *root,
else
avgmatch = 1.0;
- semifactors->outer_match_frac = jselec;
- semifactors->match_count = avgmatch;
+ extra->semifactors.outer_match_frac = jselec;
+ extra->semifactors.match_count = avgmatch;
}
/*
@@ -5434,13 +5434,16 @@ calc_joinrel_size_estimate(PlannerInfo *root,
List *joinquals = NIL;
List *pushedquals = NIL;
ListCell *l;
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ outer_rel->relids,
+ inner_rel->relids);
/* Grovel through the clauses to separate into two lists */
foreach(l, restrictlist)
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
- if (RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ if (RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrel->relids))
pushedquals = lappend(pushedquals, rinfo);
else
joinquals = lappend(joinquals, rinfo);
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 1d6bedb399..bb51bd97aa 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -196,7 +196,6 @@ process_equivalence(PlannerInfo *root,
*p_restrictinfo =
make_restrictinfo(root,
(Expr *) ntest,
- restrictinfo->is_pushed_down,
restrictinfo->has_clone,
restrictinfo->is_clone,
restrictinfo->pseudoconstant,
@@ -2031,7 +2030,6 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* throw back a dummy replacement clause (see notes above) */
rinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(true, false),
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
false, /* pseudoconstant */
@@ -2059,7 +2057,6 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* throw back a dummy replacement clause (see notes above) */
rinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(true, false),
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
false, /* pseudoconstant */
@@ -2087,7 +2084,6 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* throw back a dummy replacement clause (see notes above) */
rinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(true, false),
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
false, /* pseudoconstant */
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index 5be8da9e09..9809260d18 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -84,6 +84,7 @@ static List *select_mergejoin_clauses(PlannerInfo *root,
RelOptInfo *innerrel,
List *restrictlist,
JoinType jointype,
+ JoinPathExtraData *extra,
bool *mergejoin_allowed);
static void generate_mergejoin_paths(PlannerInfo *root,
RelOptInfo *joinrel,
@@ -150,6 +151,9 @@ add_paths_to_joinrel(PlannerInfo *root,
extra.mergeclause_list = NIL;
extra.sjinfo = sjinfo;
extra.param_source_rels = NULL;
+ extra.ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ outerrel->relids,
+ innerrel->relids);
/*
* See if the inner relation is provably unique for this outer rel.
@@ -214,6 +218,7 @@ add_paths_to_joinrel(PlannerInfo *root,
innerrel,
restrictlist,
jointype,
+ &extra,
&mergejoin_allowed);
/*
@@ -223,7 +228,7 @@ add_paths_to_joinrel(PlannerInfo *root,
if (jointype == JOIN_SEMI || jointype == JOIN_ANTI || extra.inner_unique)
compute_semi_anti_join_factors(root, joinrel, outerrel, innerrel,
jointype, sjinfo, restrictlist,
- &extra.semifactors);
+ &extra);
/*
* Decide whether it's sensible to generate parameterized paths for this
@@ -2118,7 +2123,8 @@ hash_inner_and_outer(PlannerInfo *root,
* If processing an outer join, only use its own join clauses for
* hashing. For inner joins we need not be so picky.
*/
- if (isouterjoin && RINFO_IS_PUSHED_DOWN(restrictinfo, joinrel->relids))
+ if (isouterjoin &&
+ RINFO_IS_PUSHED_DOWN(restrictinfo, extra->ojrelids, joinrel->relids))
continue;
if (!restrictinfo->can_join ||
@@ -2350,6 +2356,7 @@ select_mergejoin_clauses(PlannerInfo *root,
RelOptInfo *innerrel,
List *restrictlist,
JoinType jointype,
+ JoinPathExtraData *extra,
bool *mergejoin_allowed)
{
List *result_list = NIL;
@@ -2367,7 +2374,8 @@ select_mergejoin_clauses(PlannerInfo *root,
* we don't set have_nonmergeable_joinclause here because pushed-down
* clauses will become otherquals not joinquals.)
*/
- if (isouterjoin && RINFO_IS_PUSHED_DOWN(restrictinfo, joinrel->relids))
+ if (isouterjoin &&
+ RINFO_IS_PUSHED_DOWN(restrictinfo, extra->ojrelids, joinrel->relids))
continue;
/* Check that clause is a mergeable operator clause */
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index f3a9412d18..b27ad767c9 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -34,6 +34,7 @@ static bool has_join_restriction(PlannerInfo *root, RelOptInfo *rel);
static bool has_legal_joinclause(PlannerInfo *root, RelOptInfo *rel);
static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
+ Relids ojrelids,
bool only_pushed_down);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
@@ -894,6 +895,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *sjinfo, List *restrictlist)
{
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ rel1->relids,
+ rel2->relids);
+
/*
* Consider paths using each rel as both outer and inner. Depending on
* the join type, a provably empty outer or inner rel might mean the join
@@ -916,7 +921,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
{
case JOIN_INNER:
if (is_dummy_rel(rel1) || is_dummy_rel(rel2) ||
- restriction_is_constant_false(restrictlist, joinrel, false))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, false))
{
mark_dummy_rel(joinrel);
break;
@@ -930,12 +935,12 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
break;
case JOIN_LEFT:
if (is_dummy_rel(rel1) ||
- restriction_is_constant_false(restrictlist, joinrel, true))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, true))
{
mark_dummy_rel(joinrel);
break;
}
- if (restriction_is_constant_false(restrictlist, joinrel, false) &&
+ if (restriction_is_constant_false(restrictlist, joinrel, ojrelids, false) &&
bms_is_subset(rel2->relids, sjinfo->syn_righthand))
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
@@ -947,7 +952,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
break;
case JOIN_FULL:
if ((is_dummy_rel(rel1) && is_dummy_rel(rel2)) ||
- restriction_is_constant_false(restrictlist, joinrel, true))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, true))
{
mark_dummy_rel(joinrel);
break;
@@ -983,7 +988,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
bms_is_subset(sjinfo->min_righthand, rel2->relids))
{
if (is_dummy_rel(rel1) || is_dummy_rel(rel2) ||
- restriction_is_constant_false(restrictlist, joinrel, false))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, false))
{
mark_dummy_rel(joinrel);
break;
@@ -1006,7 +1011,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
sjinfo) != NULL)
{
if (is_dummy_rel(rel1) || is_dummy_rel(rel2) ||
- restriction_is_constant_false(restrictlist, joinrel, false))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, false))
{
mark_dummy_rel(joinrel);
break;
@@ -1021,12 +1026,12 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
break;
case JOIN_ANTI:
if (is_dummy_rel(rel1) ||
- restriction_is_constant_false(restrictlist, joinrel, true))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, true))
{
mark_dummy_rel(joinrel);
break;
}
- if (restriction_is_constant_false(restrictlist, joinrel, false) &&
+ if (restriction_is_constant_false(restrictlist, joinrel, ojrelids, false) &&
bms_is_subset(rel2->relids, sjinfo->syn_righthand))
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
@@ -1423,6 +1428,7 @@ mark_dummy_rel(RelOptInfo *rel)
static bool
restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
+ Relids ojrelids,
bool only_pushed_down)
{
ListCell *lc;
@@ -1437,7 +1443,8 @@ restriction_is_constant_false(List *restrictlist,
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, lc);
- if (only_pushed_down && !RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ if (only_pushed_down &&
+ !RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrel->relids))
continue;
if (rinfo->clause && IsA(rinfo->clause, Const))
diff --git a/src/backend/optimizer/plan/analyzejoins.c b/src/backend/optimizer/plan/analyzejoins.c
index 506fccd20c..91831ea3ae 100644
--- a/src/backend/optimizer/plan/analyzejoins.c
+++ b/src/backend/optimizer/plan/analyzejoins.c
@@ -299,7 +299,9 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
* above the outer join, even if it references no other rels (it might
* be from WHERE, for example).
*/
- if (RINFO_IS_PUSHED_DOWN(restrictinfo, joinrelids))
+ if (RINFO_IS_PUSHED_DOWN(restrictinfo,
+ bms_make_singleton(sjinfo->ojrelid),
+ joinrelids))
continue; /* ignore; not useful here */
/* Ignore if it's not a mergejoinable clause */
@@ -530,7 +532,9 @@ remove_leftjoinrel_from_query(PlannerInfo *root, int relid,
remove_join_clause_from_rels(root, rinfo, rinfo->required_relids);
- if (RINFO_IS_PUSHED_DOWN(rinfo, join_plus_commute))
+ if (RINFO_IS_PUSHED_DOWN(rinfo,
+ bms_make_singleton(sjinfo->ojrelid),
+ join_plus_commute))
{
/*
* There might be references to relid or ojrelid in the
@@ -1381,6 +1385,9 @@ is_innerrel_unique_for(PlannerInfo *root,
{
List *clause_list = NIL;
ListCell *lc;
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrelids,
+ outerrelids,
+ innerrel->relids);
/*
* Search for mergejoinable clauses that constrain the inner rel against
@@ -1398,7 +1405,7 @@ is_innerrel_unique_for(PlannerInfo *root,
* join, we can't use it.
*/
if (IS_OUTER_JOIN(jointype) &&
- RINFO_IS_PUSHED_DOWN(restrictinfo, joinrelids))
+ RINFO_IS_PUSHED_DOWN(restrictinfo, ojrelids, joinrelids))
continue;
/* Ignore if it's not a mergejoinable clause */
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 3b77886567..4de49cacdc 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -4396,6 +4396,7 @@ create_nestloop_plan(PlannerInfo *root,
{
extract_actual_join_clauses(joinrestrictclauses,
best_path->jpath.path.parent->relids,
+ best_path->jpath.ojrelids,
&joinclauses, &otherclauses);
}
else
@@ -4484,6 +4485,7 @@ create_mergejoin_plan(PlannerInfo *root,
{
extract_actual_join_clauses(joinclauses,
best_path->jpath.path.parent->relids,
+ best_path->jpath.ojrelids,
&joinclauses, &otherclauses);
}
else
@@ -4786,6 +4788,7 @@ create_hashjoin_plan(PlannerInfo *root,
{
extract_actual_join_clauses(joinclauses,
best_path->jpath.path.parent->relids,
+ best_path->jpath.ojrelids,
&joinclauses, &otherclauses);
}
else
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index e2c68fe6f9..d4353ff56a 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -962,19 +962,16 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
child_domain->jd_relids);
jtitem->qualscope = bms_union(left_item->qualscope,
right_item->qualscope);
- /* caution: ANTI join derived from SEMI will lack rtindex */
- if (j->rtindex != 0)
- {
- parent_domain->jd_relids =
- bms_add_member(parent_domain->jd_relids,
- j->rtindex);
- jtitem->qualscope = bms_add_member(jtitem->qualscope,
+ Assert(j->rtindex != 0);
+ parent_domain->jd_relids =
+ bms_add_member(parent_domain->jd_relids,
+ j->rtindex);
+ jtitem->qualscope = bms_add_member(jtitem->qualscope,
+ j->rtindex);
+ root->outer_join_rels = bms_add_member(root->outer_join_rels,
j->rtindex);
- root->outer_join_rels = bms_add_member(root->outer_join_rels,
- j->rtindex);
- mark_rels_nulled_by_join(root, j->rtindex,
- right_item->qualscope);
- }
+ mark_rels_nulled_by_join(root, j->rtindex,
+ right_item->qualscope);
jtitem->inner_join_rels = bms_union(left_item->inner_join_rels,
right_item->inner_join_rels);
jtitem->left_rels = left_item->qualscope;
@@ -2208,7 +2205,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
List **postponed_oj_qual_list)
{
Relids relids;
- bool is_pushed_down;
bool pseudoconstant = false;
bool maybe_equivalence;
bool maybe_outer_join;
@@ -2318,37 +2314,9 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
}
}
- /*----------
+ /*
* Check to see if clause application must be delayed by outer-join
* considerations.
- *
- * A word about is_pushed_down: we mark the qual as "pushed down" if
- * it is (potentially) applicable at a level different from its original
- * syntactic level. This flag is used to distinguish OUTER JOIN ON quals
- * from other quals pushed down to the same joinrel. The rules are:
- * WHERE quals and INNER JOIN quals: is_pushed_down = true.
- * Non-degenerate OUTER JOIN quals: is_pushed_down = false.
- * Degenerate OUTER JOIN quals: is_pushed_down = true.
- * A "degenerate" OUTER JOIN qual is one that doesn't mention the
- * non-nullable side, and hence can be pushed down into the nullable side
- * without changing the join result. It is correct to treat it as a
- * regular filter condition at the level where it is evaluated.
- *
- * Note: it is not immediately obvious that a simple boolean is enough
- * for this: if for some reason we were to attach a degenerate qual to
- * its original join level, it would need to be treated as an outer join
- * qual there. However, this cannot happen, because all the rels the
- * clause mentions must be in the outer join's min_righthand, therefore
- * the join it needs must be formed before the outer join; and we always
- * attach quals to the lowest level where they can be evaluated. But
- * if we were ever to re-introduce a mechanism for delaying evaluation
- * of "expensive" quals, this area would need work.
- *
- * Note: generally, use of is_pushed_down has to go through the macro
- * RINFO_IS_PUSHED_DOWN, because that flag alone is not always sufficient
- * to tell whether a clause must be treated as pushed-down in context.
- * This seems like another reason why it should perhaps be rethought.
- *----------
*/
if (bms_overlap(relids, outerjoin_nonnullable))
{
@@ -2372,7 +2340,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
* deductions, if it is mergejoinable. So consider adding it to the
* lists of set-aside outer-join clauses.
*/
- is_pushed_down = false;
maybe_equivalence = false;
maybe_outer_join = true;
@@ -2389,12 +2356,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
}
else
{
- /*
- * Normal qual clause or degenerate outer-join clause. Either way, we
- * can mark it as pushed-down.
- */
- is_pushed_down = true;
-
/*
* It's possible that this is an IS NULL clause that's redundant with
* a lower antijoin; if so we can just discard it. We need not test
@@ -2419,7 +2380,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
*/
restrictinfo = make_restrictinfo(root,
(Expr *) clause,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -2665,7 +2625,6 @@ add_base_clause_to_rel(PlannerInfo *root, Index relid,
restrictinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(false, false),
- restrictinfo->is_pushed_down,
restrictinfo->has_clone,
restrictinfo->is_clone,
restrictinfo->pseudoconstant,
@@ -2991,7 +2950,6 @@ process_implied_equality(PlannerInfo *root,
*/
restrictinfo = make_restrictinfo(root,
(Expr *) clause,
- true, /* is_pushed_down */
false, /* !has_clone */
false, /* !is_clone */
pseudoconstant,
@@ -3085,7 +3043,6 @@ build_implied_join_equality(PlannerInfo *root,
*/
restrictinfo = make_restrictinfo(root,
clause,
- true, /* is_pushed_down */
false, /* !has_clone */
false, /* !is_clone */
false, /* pseudoconstant */
diff --git a/src/backend/optimizer/plan/subselect.c b/src/backend/optimizer/plan/subselect.c
index e35ebea8b4..496d362bfd 100644
--- a/src/backend/optimizer/plan/subselect.c
+++ b/src/backend/optimizer/plan/subselect.c
@@ -1515,7 +1515,33 @@ convert_EXISTS_sublink_to_join(PlannerInfo *root, SubLink *sublink,
result->join_using_alias = NULL;
result->quals = whereClause;
result->alias = NULL;
- result->rtindex = 0; /* we don't need an RTE for it */
+ result->rtindex = 0; /* we don't need an RTE for JOIN_SEMI */
+
+ /*
+ * Add a RTE for JOIN_ANTI
+ */
+ if (result->jointype == JOIN_ANTI)
+ {
+ ParseNamespaceItem *jnsitem;
+ ParseState *pstate;
+
+ /* Create a dummy ParseState for addRangeTableEntryForJoin */
+ pstate = make_parsestate(NULL);
+
+ jnsitem = addRangeTableEntryForJoin(pstate,
+ NULL,
+ NULL,
+ JOIN_ANTI,
+ 0,
+ NULL,
+ NIL,
+ NIL,
+ NULL,
+ NULL,
+ false);
+ parse->rtable = lappend(parse->rtable, jnsitem->p_rte);
+ result->rtindex = list_length(parse->rtable);
+ }
return result;
}
diff --git a/src/backend/optimizer/util/inherit.c b/src/backend/optimizer/util/inherit.c
index 4797312ae5..3050bf5e37 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -891,7 +891,6 @@ apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
/* reconstitute RestrictInfo with appropriate properties */
childrinfo = make_restrictinfo(root,
(Expr *) onecq,
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
pseudoconstant,
@@ -938,7 +937,6 @@ apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
/* not likely that we'd see constants here, so no check */
childquals = lappend(childquals,
make_restrictinfo(root, qual,
- true,
false, false,
false,
security_level,
diff --git a/src/backend/optimizer/util/joininfo.c b/src/backend/optimizer/util/joininfo.c
index 5fb0c17630..173aff761b 100644
--- a/src/backend/optimizer/util/joininfo.c
+++ b/src/backend/optimizer/util/joininfo.c
@@ -115,7 +115,6 @@ add_join_clause_to_rels(PlannerInfo *root,
restrictinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(false, false),
- restrictinfo->is_pushed_down,
restrictinfo->has_clone,
restrictinfo->is_clone,
restrictinfo->pseudoconstant,
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index 5e2bf26ec4..35500e40b9 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -264,7 +264,6 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
*/
or_rinfo = make_restrictinfo(root,
orclause,
- true,
false,
false,
false,
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 3cf1dac087..0b392cba4f 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -2524,6 +2524,7 @@ create_nestloop_path(PlannerInfo *root,
pathnode->jpath.outerjoinpath = outer_path;
pathnode->jpath.innerjoinpath = inner_path;
pathnode->jpath.joinrestrictinfo = restrict_clauses;
+ pathnode->jpath.ojrelids = extra->ojrelids;
final_cost_nestloop(root, pathnode, workspace, extra);
@@ -2588,6 +2589,7 @@ create_mergejoin_path(PlannerInfo *root,
pathnode->jpath.outerjoinpath = outer_path;
pathnode->jpath.innerjoinpath = inner_path;
pathnode->jpath.joinrestrictinfo = restrict_clauses;
+ pathnode->jpath.ojrelids = extra->ojrelids;
pathnode->path_mergeclauses = mergeclauses;
pathnode->outersortkeys = outersortkeys;
pathnode->innersortkeys = innersortkeys;
@@ -2665,6 +2667,7 @@ create_hashjoin_path(PlannerInfo *root,
pathnode->jpath.outerjoinpath = outer_path;
pathnode->jpath.innerjoinpath = inner_path;
pathnode->jpath.joinrestrictinfo = restrict_clauses;
+ pathnode->jpath.ojrelids = extra->ojrelids;
pathnode->path_hashclauses = hashclauses;
/* final_cost_hashjoin will fill in pathnode->num_batches */
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index e05b21c884..f03c8c2fa2 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -58,6 +58,7 @@ static List *subbuild_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *input_rel,
Relids both_input_relids,
+ SpecialJoinInfo *sjinfo,
List *new_restrictlist);
static List *subbuild_joinrel_joinlist(RelOptInfo *joinrel,
List *joininfo_list,
@@ -1311,9 +1312,9 @@ build_joinrel_restrictlist(PlannerInfo *root,
* same clauses arriving from both input relations).
*/
result = subbuild_joinrel_restrictlist(root, joinrel, outer_rel,
- both_input_relids, NIL);
+ both_input_relids, sjinfo, NIL);
result = subbuild_joinrel_restrictlist(root, joinrel, inner_rel,
- both_input_relids, result);
+ both_input_relids, sjinfo, result);
/*
* Add on any clauses derived from EquivalenceClasses. These cannot be
@@ -1353,6 +1354,7 @@ subbuild_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *input_rel,
Relids both_input_relids,
+ SpecialJoinInfo *sjinfo,
List *new_restrictlist)
{
ListCell *l;
@@ -1374,11 +1376,15 @@ subbuild_joinrel_restrictlist(PlannerInfo *root,
*/
if (rinfo->has_clone || rinfo->is_clone)
{
- Assert(!RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids));
if (!bms_is_subset(rinfo->required_relids, both_input_relids))
continue;
if (bms_overlap(rinfo->incompatible_relids, both_input_relids))
continue;
+
+ Assert(!RINFO_IS_PUSHED_DOWN(rinfo,
+ bms_difference(joinrel->relids,
+ both_input_relids),
+ joinrel->relids));
}
else
{
@@ -1389,7 +1395,11 @@ subbuild_joinrel_restrictlist(PlannerInfo *root,
* (There is little point in checking incompatible_relids,
* because it'll be NULL.)
*/
- Assert(RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids) ||
+ Assert(sjinfo->ojrelid == 0 ||
+ RINFO_IS_PUSHED_DOWN(rinfo,
+ bms_difference(joinrel->relids,
+ both_input_relids),
+ joinrel->relids) ||
bms_is_subset(rinfo->required_relids,
both_input_relids));
}
@@ -2096,6 +2106,9 @@ have_partkey_equi_join(PlannerInfo *root, RelOptInfo *joinrel,
int cnt_pks;
bool pk_has_clause[PARTITION_MAX_KEYS];
bool strict_op;
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ rel1->relids,
+ rel2->relids);
/*
* This function must only be called when the joined relations have same
@@ -2116,7 +2129,7 @@ have_partkey_equi_join(PlannerInfo *root, RelOptInfo *joinrel,
/* If processing an outer join, only use its own join clauses. */
if (IS_OUTER_JOIN(jointype) &&
- RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrel->relids))
continue;
/* Skip clauses which can not be used for a join. */
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index 0b406e9334..5d05287bf3 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -24,7 +24,6 @@
static RestrictInfo *make_restrictinfo_internal(PlannerInfo *root,
Expr *clause,
Expr *orclause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -34,7 +33,6 @@ static RestrictInfo *make_restrictinfo_internal(PlannerInfo *root,
Relids outer_relids);
static Expr *make_sub_restrictinfos(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -49,11 +47,11 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
*
* Build a RestrictInfo node containing the given subexpression.
*
- * The is_pushed_down, has_clone, is_clone, and pseudoconstant flags for the
- * RestrictInfo must be supplied by the caller, as well as the correct values
- * for security_level, incompatible_relids, and outer_relids.
- * required_relids can be NULL, in which case it defaults to the actual clause
- * contents (i.e., clause_relids).
+ * The has_clone, is_clone, and pseudoconstant flags for the RestrictInfo
+ * must be supplied by the caller, as well as the correct values for
+ * security_level, incompatible_relids, and outer_relids. required_relids
+ * can be NULL, in which case it defaults to the actual clause contents
+ * (i.e., clause_relids).
*
* We initialize fields that depend only on the given subexpression, leaving
* others that depend on context (or may never be needed at all) to be filled
@@ -62,7 +60,6 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
RestrictInfo *
make_restrictinfo(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -78,7 +75,6 @@ make_restrictinfo(PlannerInfo *root,
if (is_orclause(clause))
return (RestrictInfo *) make_sub_restrictinfos(root,
clause,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -93,7 +89,6 @@ make_restrictinfo(PlannerInfo *root,
return make_restrictinfo_internal(root,
clause,
NULL,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -112,7 +107,6 @@ static RestrictInfo *
make_restrictinfo_internal(PlannerInfo *root,
Expr *clause,
Expr *orclause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -126,7 +120,6 @@ make_restrictinfo_internal(PlannerInfo *root,
restrictinfo->clause = clause;
restrictinfo->orclause = orclause;
- restrictinfo->is_pushed_down = is_pushed_down;
restrictinfo->pseudoconstant = pseudoconstant;
restrictinfo->has_clone = has_clone;
restrictinfo->is_clone = is_clone;
@@ -259,9 +252,9 @@ make_restrictinfo_internal(PlannerInfo *root,
* implicit-AND lists at top level of RestrictInfo lists. Only ORs and
* simple clauses are valid RestrictInfos.
*
- * The same is_pushed_down, has_clone, is_clone, and pseudoconstant flag
- * values can be applied to all RestrictInfo nodes in the result. Likewise
- * for security_level, incompatible_relids, and outer_relids.
+ * The same has_clone, is_clone, and pseudoconstant flag values can be
+ * applied to all RestrictInfo nodes in the result. Likewise for
+ * security_level, incompatible_relids, and outer_relids.
*
* The given required_relids are attached to our top-level output,
* but any OR-clause constituents are allowed to default to just the
@@ -270,7 +263,6 @@ make_restrictinfo_internal(PlannerInfo *root,
static Expr *
make_sub_restrictinfos(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -288,7 +280,6 @@ make_sub_restrictinfos(PlannerInfo *root,
orlist = lappend(orlist,
make_sub_restrictinfos(root,
lfirst(temp),
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -299,7 +290,6 @@ make_sub_restrictinfos(PlannerInfo *root,
return (Expr *) make_restrictinfo_internal(root,
clause,
make_orclause(orlist),
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -317,7 +307,6 @@ make_sub_restrictinfos(PlannerInfo *root,
andlist = lappend(andlist,
make_sub_restrictinfos(root,
lfirst(temp),
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -331,7 +320,6 @@ make_sub_restrictinfos(PlannerInfo *root,
return (Expr *) make_restrictinfo_internal(root,
clause,
NULL,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -521,6 +509,7 @@ extract_actual_clauses(List *restrictinfo_list,
void
extract_actual_join_clauses(List *restrictinfo_list,
Relids joinrelids,
+ Relids ojrelids,
List **joinquals,
List **otherquals)
{
@@ -533,7 +522,7 @@ extract_actual_join_clauses(List *restrictinfo_list,
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
- if (RINFO_IS_PUSHED_DOWN(rinfo, joinrelids))
+ if (RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrelids))
{
if (!rinfo->pseudoconstant &&
!rinfo_is_constant_true(rinfo))
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 6c71098f2d..5778cfa345 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -2070,6 +2070,10 @@ typedef struct JoinPath
* joinrestrictinfo is needed in JoinPath, and can't be merged into the
* parent RelOptInfo.
*/
+
+ Relids ojrelids; /* the relid set of any outer joins that will be
+ * calculated at this join for outer join, or NULL
+ * for inner join */
} JoinPath;
/*
@@ -2444,27 +2448,16 @@ typedef struct LimitPath
* level of the outer join, which is true except when it is a "degenerate"
* condition that references only Vars from the nullable side of the join.
*
- * RestrictInfo nodes contain a flag to indicate whether a qual has been
- * pushed down to a lower level than its original syntactic placement in the
- * join tree would suggest. If an outer join prevents us from pushing a qual
- * down to its "natural" semantic level (the level associated with just the
- * base rels used in the qual) then we mark the qual with a "required_relids"
- * value including more than just the base rels it actually uses. By
- * pretending that the qual references all the rels required to form the outer
- * join, we prevent it from being evaluated below the outer join's joinrel.
- * When we do form the outer join's joinrel, we still need to distinguish
- * those quals that are actually in that join's JOIN/ON condition from those
- * that appeared elsewhere in the tree and were pushed down to the join rel
- * because they used no other rels. That's what the is_pushed_down flag is
- * for; it tells us that a qual is not an OUTER JOIN qual for the set of base
- * rels listed in required_relids. A clause that originally came from WHERE
- * or an INNER JOIN condition will *always* have its is_pushed_down flag set.
- * It's possible for an OUTER JOIN clause to be marked is_pushed_down too,
- * if we decide that it can be pushed down into the nullable side of the join.
- * In that case it acts as a plain filter qual for wherever it gets evaluated.
- * (In short, is_pushed_down is only false for non-degenerate outer join
- * conditions. Possibly we should rename it to reflect that meaning? But
- * see also the comments for RINFO_IS_PUSHED_DOWN, below.)
+ * If an outer join prevents us from pushing a qual down to its "natural"
+ * semantic level (the level associated with just the base rels used in the
+ * qual) then we mark the qual with a "required_relids" value including more
+ * than just the base rels it actually uses. By pretending that the qual
+ * references all the rels required to form the outer join, we prevent it from
+ * being evaluated below the outer join's joinrel. When we do form the outer
+ * join's joinrel, we still need to distinguish those quals that are actually
+ * in that join's JOIN/ON condition from those that appeared elsewhere in the
+ * tree and were pushed down to the join rel because they used no other rels.
+ * See the comments for RINFO_IS_PUSHED_DOWN for how we do that.
*
* There is also an incompatible_relids field, which is a set of outer-join
* relids above which we cannot evaluate the clause (because they might null
@@ -2551,9 +2544,6 @@ typedef struct RestrictInfo
/* the represented clause of WHERE or JOIN */
Expr *clause;
- /* true if clause was pushed down in level */
- bool is_pushed_down;
-
/* see comment above */
bool can_join pg_node_attr(equal_ignore);
@@ -2700,16 +2690,20 @@ typedef struct RestrictInfo
* This macro embodies the correct way to test whether a RestrictInfo is
* "pushed down" to a given outer join, that is, should be treated as a filter
* clause rather than a join clause at that outer join. This is certainly so
- * if is_pushed_down is true; but examining that is not sufficient anymore,
- * because outer-join clauses will get pushed down to lower outer joins when
- * we generate a path for the lower outer join that is parameterized by the
- * LHS of the upper one. We can detect such a clause by noting that its
+ * if the outer join clause references that outer join in any varnullingrels or
+ * phnullingrels set; but examining that is not sufficient anymore, because
+ * outer-join clauses will get pushed down to lower outer joins when we
+ * generate a path for the lower outer join that is parameterized by the LHS of
+ * the upper one. We can detect such a clause by noting that its
* required_relids exceed the scope of the join.
*/
-#define RINFO_IS_PUSHED_DOWN(rinfo, joinrelids) \
- ((rinfo)->is_pushed_down || \
+#define RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrelids) \
+ (bms_overlap((rinfo)->required_relids, (ojrelids)) || \
!bms_is_subset((rinfo)->required_relids, joinrelids))
+#define CALC_OUTER_JOIN_RELIDS(joinrelids, outerrelids, innerrelids) \
+ (bms_difference((joinrelids), bms_union((outerrelids), (innerrelids))))
+
/*
* Since mergejoinscansel() is a relatively expensive function, and would
* otherwise be invoked many times while planning a large join tree,
@@ -3212,6 +3206,8 @@ typedef struct SemiAntiJoinFactors
* sjinfo is extra info about special joins for selectivity estimation
* semifactors is as shown above (only valid for SEMI/ANTI/inner_unique joins)
* param_source_rels are OK targets for parameterization of result paths
+ * ojrelids is the relid set of any outer joins that will be calculated at this
+ * join, or NULL for inner join
*/
typedef struct JoinPathExtraData
{
@@ -3221,6 +3217,7 @@ typedef struct JoinPathExtraData
SpecialJoinInfo *sjinfo;
SemiAntiJoinFactors semifactors;
Relids param_source_rels;
+ Relids ojrelids;
} JoinPathExtraData;
/*
diff --git a/src/include/optimizer/cost.h b/src/include/optimizer/cost.h
index b1c51a4e70..6b83499c62 100644
--- a/src/include/optimizer/cost.h
+++ b/src/include/optimizer/cost.h
@@ -183,7 +183,7 @@ extern void compute_semi_anti_join_factors(PlannerInfo *root,
JoinType jointype,
SpecialJoinInfo *sjinfo,
List *restrictlist,
- SemiAntiJoinFactors *semifactors);
+ JoinPathExtraData *extra);
extern void set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel);
extern double get_parameterized_baserel_size(PlannerInfo *root,
RelOptInfo *rel,
diff --git a/src/include/optimizer/restrictinfo.h b/src/include/optimizer/restrictinfo.h
index 1b42c832c5..295ce764d3 100644
--- a/src/include/optimizer/restrictinfo.h
+++ b/src/include/optimizer/restrictinfo.h
@@ -19,12 +19,11 @@
/* Convenience macro for the common case of a valid-everywhere qual */
#define make_simple_restrictinfo(root, clause) \
- make_restrictinfo(root, clause, true, false, false, false, 0, \
+ make_restrictinfo(root, clause, false, false, false, 0, \
NULL, NULL, NULL)
extern RestrictInfo *make_restrictinfo(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -41,6 +40,7 @@ extern List *extract_actual_clauses(List *restrictinfo_list,
bool pseudoconstant);
extern void extract_actual_join_clauses(List *restrictinfo_list,
Relids joinrelids,
+ Relids ojrelids,
List **joinquals,
List **otherquals);
extern bool join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel);
--
2.31.0
Re: Retiring is_pushed_down
I see following issue with the latest patch,
relnode.c:2122:32: error: use of undeclared identifier 'ojrelids'
RINFO_IS_PUSHED_DOWN(rinfo, ojrelids,
joinrel->relids))
On Thu, 18 Apr 2024 at 09:34, Richard Guo <guofenglinux@gmail.com> wrote:
Here is another rebase over 3af7040985. Nothing else has changed.
Thanks
Richard
--
Regards,
Rafia Sabih
Re: Retiring is_pushed_down
Richard Guo <guofenglinux@gmail.com> writes:
When forming an outer join's joinrel, we have the is_pushed_down flag in
RestrictInfo nodes to distinguish those quals that are in that join's
JOIN/ON condition from those that were pushed down to the joinrel and
thus act as filter quals. Since now we have the outer-join-aware-Var
infrastructure, I think we can check to see whether a qual clause's
required_relids reference the outer join(s) being formed, in order to
tell if it's a join or filter clause. This seems like a more principled
way. (Interesting that optimizer/README actually describes this way in
section 'Relation Identification and Qual Clause Placement'.)
Sorry for being so slow to look at this patch. The idea you're
following is one that I spent a fair amount of time on while working
on what became 2489d76c4 ("Make Vars be outer-join-aware"). I failed
to make it work though. Digging in my notes from the time:
-----
How about is_pushed_down?
Would really like to get rid of that, because it's ugly/sloppily defined,
and it's hard to determine the correct value for EquivClass-generated
clauses once we allow derivations from OJ clauses. However, my original
idea of checking for join's ojrelid present in clause's required_relids
has issues:
* fails if clause is not pushed as far down as it can possibly be (and
lateral refs mean that that's hard to do sometimes)
* getting the join's ojrelid to everywhere we need to check this is messy.
I'd tolerate the mess if it worked nicely, but ...
-----
So I'm worried that the point about lateral refs is still a problem
in your version. To be clear, the hazard is that if a WHERE clause
ends up getting placed at an outer join that's higher than any of
the OJs specifically listed in its required_relids, we'd misinterpret
it as being a join clause for that OJ although it should be a filter
clause.
The other thing I find in my old notes is speculation that we could
use the concept of JoinDomains to replace is_pushed_down. That is,
we'd have to label every RestrictInfo with the JoinDomain of its
syntactic source location, and then we could tell if the RI was
"pushed down" relative to a particular join by seeing if the JD was
above or below that join. This ought to be impervious to
not-pushed-down-all-the-way problems. The thing I'd not figured
out was how to make this work with quals of full joins: they don't
belong to either the upper JoinDomain or either of the lower ones.
We could possibly fix this by giving a full join its very own
JoinDomain that is understood to be a parent of both lower domains,
but I ran out of energy to pursue that.
If we went this route, we'd basically be replacing the is_pushed_down
field with a JoinDomain field, which is surely not simpler. But it
seems more crisply defined and perhaps more amenable to my long-term
desire to be able to use the EquivalenceClass machinery with outer
join clauses. (The idea being that an EC would describe equalities
that hold within a JoinDomain, but not necessarily elsewhere.)
regards, tom lane
Re: Retiring is_pushed_down
On Fri, Sep 27, 2024 at 5:06 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
So I'm worried that the point about lateral refs is still a problem
in your version. To be clear, the hazard is that if a WHERE clause
ends up getting placed at an outer join that's higher than any of
the OJs specifically listed in its required_relids, we'd misinterpret
it as being a join clause for that OJ although it should be a filter
clause.
Sorry it took me so long to get back to this thread.
I don't quite understand how this could happen. If a WHERE clause is
placed on an outer join but does not include the outer join's ojrelid
in its required_relids, then it must only refer to the non-nullable
side. In that case, we should be able to push this clause down to the
non-nullable side of the outer join.
Perhaps this issue could occur with a lateral join, but I wasn't able
to construct such a query. I wonder if you happen to have an example
on hand.
If we went this route, we'd basically be replacing the is_pushed_down
field with a JoinDomain field, which is surely not simpler. But it
seems more crisply defined and perhaps more amenable to my long-term
desire to be able to use the EquivalenceClass machinery with outer
join clauses. (The idea being that an EC would describe equalities
that hold within a JoinDomain, but not necessarily elsewhere.)
Exactly. At first, I thought that with JoinDomain, we could avoid
checking if a clause's required_relids exceeds the scope of the join
in RINFO_IS_PUSHED_DOWN, so RINFO_IS_PUSHED_DOWN can be reduced to:
#define RINFO_IS_PUSHED_DOWN(rinfo, joinrelids) \
(bms_is_subset(joinrelids, (rinfo)->jdomain->jd_relids))
However, if the outer join was commuted with another one according to
Identity 3, simply checking whether the JoinDomain is above or below
the outer join is not sufficient. We'll still need to check if the
clause's required_relids exceeds the scope of the join, as we do
currently.
But I agree that changing to use JoinDomain is more amenable to the
goal of using ECs with outer join clauses.
Thanks
Richard